Chaos-Enhanced Harris Hawks Optimizer for Cascade Reservoir Operation with Ecological Flow Similarity

Abstract

In the pursuit of sustainable development, optimizing water resources management while maintaining ecological balance is crucial. This study introduces a Chaos-enhanced Harris Hawks Optimizer (CEHHO) aimed at optimizing natural flow patterns in cascade reservoirs. First, an ecological scheduling model considering ensuring guaranteed output is established based on the similarity of ecological flows. Subsequently, the CEHHO algorithm is proposed, which uses tilted skew chaos mapping for population initialization, improving the quality of the initial population. In the exploration phase, an adaptive strategy enhances the efficiency of group search algorithms, enabling effective navigation of the complex solution space. A random difference mutation strategy, combined with the Q-learning algorithm, mitigates premature convergence and maintains algorithmic diversity. Comparative analysis with the existing technology under different typical hydrological frequency shows that the search accuracy and convergence efficiency of the proposed method are significantly improved. Under the guaranteed output limit of 1000 MW, the proposed method enhances the optimal, median, mean, and worst values by 293.92, 493.23, 422.14, and 381.15, respectively, compared to the HHO. Furthermore, the results of the multi-purpose guaranteed output scenario highlight the superior detection and exploitation capabilities of this algorithm. These findings highlight the great potential of the proposed method for practical engineering applications, providing a reliable tool for optimizing water resources management while maintaining ecological balance.

1. Introduction

Reservoirs enhance the efficiency of water resource utilization by capturing and storing water through the construction of dams on rivers or other water bodies [1]. During reservoir operation, river hydrology is influenced by changes in outflow magnitude, frequency, and duration [2,3]. Existing reservoir dispatching and operation methods primarily aim to maximize power generation benefits by optimizing water level management [4,5]. Often, discharge flow is adjusted to control turbine operation, leading to substantial changes in downstream river flow compared to pre-reservoir conditions, thereby impacting the river ecosystem and causing serious negative effects on the ecological environment. For instance, reducing discharge flow during the impoundment period can lead to downstream river desiccation, disrupting continuity and ecological integrity [6]. Discharging low-temperature water may delay fish spawning times and reduce spawning success, while releasing high-temperature water can degrade fish habitats and increase mortality rates [7]. Additionally, improper reservoir operation can result in eutrophication in slow-flow areas or tributary backwaters within the reservoir [8]. In recent years, growing environmental awareness has spurred global attention and research aimed at mitigating or eliminating the adverse effects of reservoirs on river health, which is of significant importance to the concept of sustainable development [9,10,11]. Li et al. [12] proposed the Dam Ecological Effect Index (DEEI) to quickly identify the ecological and environmental changes in dams over the past 36 years. Yan et al. [13] defined two ecological objectives, water quantity level (WQL) and hydrological alteration (HA), and developed an optimal operational strategy that balances economic and ecological benefits for hydropower stations in a certain river basin in China. Wang et al. [14] proposed a new ecological indicator, the reservoir outflow variations across different time scales (OVDTS), which helps to protect the ecological benefits of the downstream river basin. In addition, unreasonable water resource management measures also lead to insufficient guaranteed output of reservoirs, affecting the stable operation of reservoirs [15]. Therefore, the reservoir operation mode considering the natural river flow situation has become a prominent topic of current research.

Existing reservoir management typically involves the joint operation and cooperation of cascade reservoirs, presenting a complex multi-stage decision-making problem characterized by multiple constraints and spatiotemporal coupling [16]. The main solution methods can be summarized into two categories [17]. The first category involves traditional mathematical programming methods, including nonlinear programming [18], linear programming [19], and dynamic programming [20]. These methods typically require the optimization problem to have clear mathematical expressions and constraints, which often need to meet certain requirements such as concavity and continuity. However, many constraints in cascade operation and scheduling problems are challenging to describe with precise mathematical models, complicating the solution process. The second category encompasses heuristic optimization methods that have gained popularity in recent years [21,22,23]. These methods do not depend on the specific form of the problem but handle various types of optimization problems by simulating natural processes or human intelligent behavior [24]. However, heuristic algorithms can easily fall into local optima during the search process and struggle to balance exploration and exploitation effectively, posing challenges for practical applications. Thus, developing an algorithm with robust search capabilities and high convergence accuracy is crucial for identifying sustainable scheduling schemes for cascade reservoirs.

In recent years, due to the high efficiency and ease of implementation of heuristic algorithms, they have been designed and employed as powerful solvers for many reservoir operation problems. These methods mainly include the Whale Optimization Algorithm (WOA) [25], which is based on whale behavior; the Marine Predators Algorithm (MPA) [26], which simulates the behavior of predators in marine ecosystems; and the Harris Hawks Optimizer (HHO) [27], inspired by the hunting behavior of eagle groups. The HHO algorithm has garnered widespread attention from researchers across various fields and has been successfully applied to tasks such as feature selection [28], image segmentation [29], vehicle routing [30] and so on. Despite its excellent search performance, the HHO algorithm has certain limitations. For instance, the random generation of the initial population is influenced by hardware conditions, making it difficult to ensure population diversity. Additionally, there is an imbalance between exploration and exploitation in multiple local optimum optimization problems, which can impede accurately locating the global optimum or lead to search stagnation. Therefore, it is both important and meaningful to study and develop an improved version of the HHO algorithm to enhance reservoir operation optimization.

To address the shortcomings identified in the preceding analysis and research, this paper first develops an ecological scheduling model that accounts for the natural flow regime. Subsequently, a chaos-enhanced Harris Hawks Optimizer (CEHHO) is introduced to derive the optimal scheduling solution. The CEHHO incorporates a chaotic population initialization strategy, an exploration phase adaptive strategy, and a random difference mutation strategy. These enhancements improve population diversity and balance the exploration and exploitation capabilities of the algorithm. To verify the effectiveness of the proposed method, four cascade reservoirs in the Jinsha River Basin are used as research targets for practical exploration.

The main contributions of this paper are as follows:

• A new ecological scheduling model is proposed, which incorporates the natural flow regime to more accurately represent hydrological conditions under ecological demand.
• Utilizing the skew tent chaotic map for population initialization significantly improves both the quality and diversity of initial solutions.
• Introducing a random mutation strategy with Q-learning, along with incorporating an adaptive exploration strategy, enhances the ability of algorithm to navigate complex solution spaces, improve exploration efficiency, and avoid premature convergence.
• The method demonstrates superior search accuracy and convergence efficiency in optimizing four cascade reservoirs in the Jinsha River Basin, highlighting its strong potential for practical engineering applications.

The remainder of this article is organized as follows: Section 2 presents the mathematical model of reservoir ecological operation. Section 3 introduces the proposed CEHHO algorithm. Section 4 validates and analyzes the effectiveness of the CEHHO. The conclusions are presented in Section 5.

2. Problem Formulation

Reservoir construction often disrupts the integrity and functioning of basin ecosystems. The natural flow regime is considered the optimal state of the ecosystem for reservoir operation. To more accurately assess hydrological changes, the dynamic time warping (DTW) method is employed to measure the discrepancies between the current reservoir operations and the natural flow system. The following section provides a detailed explanation of how the DTW algorithm is applied to address this issue.

2.1. Dynamic Time Warping (DTW) Based on Sakoe-Chiba Band

Dynamic Time Warping (DTW) [31], as shown in Figure 1a, is an algorithm based on dynamic programming (DP) to measure the similarity between two time series. DTW has been widely used in various fields such as speech recognition [32], gesture recognition [33], and data mining [34]. The Sakoe-Chiba band constraint, as shown in Figure 1b, is a method to limit the degree of time warping in the DTW algorithm, which defines a band-shaped area on the distance matrix and only allows points within the area to be matched, thereby limiting the degree of time warping to improve the efficiency and accuracy of the algorithm [35]. The process of the DTW method based on the Sakoe-Chiba band constraint is as follows:

(1) Set the band width r of the Sakoe-Chiba band and assume two time series A = [a₁, a₂,…, a_l1] and B = [b₁, b₂,…, b_l2], l₁ and l₂ represent the length of time series A and B, separately.

(2) Construct an n × m distance matrix D_n×m and initialize it:

$$D_{i,j}={\left(A_i-B_j\right)}^2$$

(1)

where D_i,j represents the distance between the i-th element of series A and the j-th element of series B.

(3) Create a cumulative distance matrix C_n×m of the same size as the distance matrix D_n×m and initialize all elements to infinity except C_0,0 = 0.

(4) Then set the initial condition: C_1,1 = D_1,1, which indicates the cumulative distance from the starting point; For other elements in the first column, if |i − 1| < r, then let C_i,1 = D_i,1 + C_i−1,1; For other elements in the first column, if |j − 1| ≤ r, then let C_1,j = D_1,j + C_1,j−1.

(5) For the other elements in C in the cumulative matrix, the formula considering the Sakoe-Chiba band is used as follows:

$$C_{i,j}=D_{i,j}+min\left(C_{i-1,j},C_{i,j-1},C_{i-1,j-1}\right),if(\left|i-j\right|\le r)$$

(2)

(6) The dynamic programming algorithm is used to trace back the path and obtain the minimum cumulative distance as the difference value of the two series.

2.2. Objective Function

The objective function aims to minimize the variation in flow at the reservoir control section, particularly the difference between the natural flow and the regulated flow. Its primary function is to serve as a quantitative indicator to guide the optimization process. By minimizing the DTW distance between the natural flow sequence and the regulated flow sequence, this function ensures that the regulation strategy maintains flow conditions as close as possible to the natural state throughout the entire scheduling period, which is expressed as follows:

$$Obj=\min {\displaystyle \sum _{i=1}^l\frac{dist\left[C(F{n}_i,F{o}_i)\right]}{T}}$$

(3)

where Obj represents the difference between the natural flow regime of the controlled section and the reservoir-controlled flow regime; Fn_i and Fo_i represent the flow series of the natural flow state of the river in the i-th control section and the flow series after the reservoir is built, respectively. dist[·] denotes the DTW distance between sequences Fn_i and Fo_i, T represents the number of time periods in the scheduling cycle.

2.3. Operation Constraints

The following are the physical constraints that must be observed when operating a cascade reservoir system [16]:

(1) Reservoir water balance constraint:

$$\left\{\begin{array}{c}{V}_{i,t+1}=V_{i,t}+(I_{i,t}-O_{i,t})\cdot {\Delta }_t\\ I_{i,t}=q_{i,t}+{\displaystyle \sum _{j=1}^{U_i}{O}_{j,t}}\end{array}\right.$$

(4)

where V_i,t+1 denotes the water level of the i-th reservoir at time period t + 1; I_i,t, O_i,t and q_i,t symbolize the inflow, outflow and local flow of the i-th reservoir at time period t, respectively; U_i denotes the number of upstream reservoirs that are directly hydraulically connected to reservoir i.

(2) water level constraint:

$$Z_{i,t}^{\min }\le Z_{i,t}\le Z_{i,t}^{\max }$$

(5)

where $Z_{i,t}^{\min }$ and $Z_{i,t}^{\max }$ symbolize the minimal and maximal operating boundaries of the water level of the i-th reservoir at time period t, respectively.

(3) Reservoir discharge constraint:

$$O_{i,t}^{\min }\le O_{i,t}\le O_{i,t}^{\max }$$

(6)

where $O_{i,t}^{\min }$ and $O_{i,t}^{\max }$ symbolize the minimal and maximal operating boundaries of the outflow of the i-th reservoir at time period t, respectively.

(4) Outflow balance constraint

$$O_{i,t}=s_{i,t}+Q_{i,t}^d$$

(7)

where s_i,t and $Q_{i,t}^d$ symbolize the spillage flow and turbine flow of the i-th reservoir at time period t.

(5) Power output constraint:

$$P_{i,t}^{\min }\le P_{i,t}\le P_{i,t}^{\max }$$

(8)

where $P_{i,t}^{\min }$ and $P_{i,t}^{\max }$ symbolize the minimal and maximal boundaries of the output of the i-th reservoir at time period t, respectively.

(6) Output guarantee constraint:

$$P_{i,t}\ge P_{i,t}^G$$

(9)

where $P_{i,t}^G$ denotes guaranteed output operation limit of the i-th power station.

(7) Initial and terminal level constraint:

$$\left\{\begin{array}{c}{Z}_{i,0}=Z_i^{begin}\\ Z_{i,T}=Z_i^{end}\end{array}\right.$$

(10)

where $Z_i^{begin}$ and $Z_i^{end}$ symbolize the initial and terminal water level of reservoir i during the entire scheduling period, respectively.

3. Proposed Chaotic-Enhanced Harris Hawks Optimizer (CEHHO)

3.1. Harris Hawks Optimizer (HHO)

The Harris Hawks Optimizer (HHO) is a population-based, gradient-free optimization technique inspired by the heuristic cooperative predation behavior observed in Harris hawks groups, as shown in Figure 2, which includes prey exploration, surprise attacks, and various attack strategies [35,36,37]. In the HHO, the positions of different Harris hawks within the population are abstracted as potential candidate solutions to the optimization problem, with the optimal solution corresponding to potential location of the target. The operational mechanism of the HHO primarily encompasses three phases: exploration, transition, and exploitation. These phases are succinctly described below.

3.1.1. Exploration Phase

In the Harris Hawks Optimizer (HHO), Harris hawks employ two equal-opportunity strategies to wait, observe, monitor, and capture prey at relatively random locations. Simultaneously, Harris hawks can obtain possible prey location information based on the roosting positions of other group members. The mathematical expression is as follows:

$$X_i(k+1)=\left\{\begin{array}{ll}{X}_{rand}(k)-r_1\cdot \left|X_{rand}(k)-2r_2\cdot X_i(k)\right|\hfill & ,q\ge 0.5\hfill \\ \left[X_{prey}(k)-X_m(k)\right]-r_3\cdot \left[Lb+r_4\cdot (Ub-Lb)\right]\hfill & ,q<0.5\hfill \end{array}\right.$$

(11)

where X(k + 1) represents the position vector of the hawk during the (k + 1)-th iteration; X_rand(k) represents the position vector of the hawk randomly selected from the k-th iteration; r₁, r₂, r₃, r₄ and q represent different random numbers uniformly distributed in the interval [0, 1], respectively; X_prey(k) represents the position vector of the tracked prey during the (k + 1)-th iteration; X_m(k) represents the average position vector of the population at the k-th iteration, and its mathematical expression is as follows:

$$X_m(k)={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{X}_i}(k)$$

(12)

where N represents the number of hawks in the population.

3.1.2. Transition from Exploration to Exploitation

To facilitate the transition from exploration to exploitation, the HHO proposes a prey escape energy function model, E. Initially, a starting energy value is given. As the number of iterations increases, the escape energy of prey gradually decreases. When the escape energy |E| ≥ 1, the algorithm executes the exploration phase; otherwise, it executes the exploitation phase, the mathematical expression is as follows:

$$E=2E_0\cdot (1-{\displaystyle \frac{k}{K}})$$

(13)

where E₀ represents the initial escape energy; K represents the maximum number of iterations for the population.

3.1.3. Exploitation Phase

• (a). Soft besiege

When r ≥ 0.5 and |E| ≥ 0.5, the hawk population performs a soft besiege on the prey. At this point, the prey has enough energy and attempts to escape by making random misleading jumps. During these attempts, Harris hawks gradually surround the prey to exhaust it before pouncing. The mathematical expression is as follows:

$$\left\{\begin{array}{c}{X}_i(k+1)=\Delta X(k)-E\left|J\cdot X_{prey}(k)-X_i(k)\right|\\ \Delta X(k)=X_{prey}(k)-X_i(k)\\ J=2(1-r_5)\end{array}\right.$$

(14)

where ∆X(k) is the position vector difference between the prey and the current hawk group in the k-th iteration. r₅ is a random number in [0, 1], and J represents the jumping intensity of the prey in the random escape process in each iteration.

• (b). Hard besiege

When r ≥ 0.5 and |E| < 0.5, the hawk population performs a hard besiege on the prey, as shown in Figure 3. Because the prey has been escaping for a long time, the escape energy is very low. At this time, the Harris Hawk pounces on the prey and updates its position with the formula:

$$X_i(k+1)=X_{prey}(k)-E\left|\Delta X(k)\right|$$

(15)

• (c). Soft besiege with progressive rapid dives

When r < 0.5 and |E| ≥ 0.5, the HHO algorithm assumes that when hawks compete for prey, they can gradually select the best dive path to chase the fleeing prey. During the hunting and escaping phase, Levy Flight (Lf) is adopted to simulate the deceptive actions of the prey and the diverse dive patterns of the hawks. The mathematical representation of this phase is expressed as follows:

$$Y=X_{prey}(k)-E\left|J\cdot X_{prey}(k)-X_i(k)\right|$$

(16)

At the same time, when the prey performs more deceptive movements, the hawks may also perform irregular and rapid dives directly when approaching the prey. As shown in Figure 4, the rules can be expressed as follows:

$$Z=Y+S\times Lf(D)$$

(17)

where D represents the dimension of the problem to be calculated; S is a randomly generated position vector of size 1 × D; Lf represents the Levy flight function, which is expressed as follows:

$$\left\{\begin{array}{c}Lf(x)=0.01\times {\displaystyle \frac{u\times \sigma }{|v{|}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\beta $}\right.}}}\\ \sigma ={\left({\displaystyle \frac{\sin (\frac{\pi \beta }{2})\times \Gamma (1+\beta )}{\beta \times 2^{(\frac{\beta -1}{2})})\times \Gamma (\frac{1+\beta }{2})}}\right)}^{{\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\beta $}\right.}}\end{array}\right.$$

(18)

where u and v are random values obtained randomly in [0, 1]; β is a constant, which is taken as 1.5 in this paper.

Therefore, the mathematical representation of the final position of the hawks update during the soft siege phase is as follows:

$$X_i(k+1)=\left\{\begin{array}{cc}Y,\hfill & if(F(Y)<F(X_i(k)))\hfill \\ Z,\hfill & if(F(Z)<F(X_i(k)))\hfill \end{array}\right.$$

(19)

where F(·) denotes the fitness function.

• (d). Hard besiege with progressive rapid dives

When r < 0.5 and |E| < 0.5, the prey has no energy to escape, and the hawks try to shorten the average distance between themselves and the escaping prey. The mathematical expression for the hard siege state is as follows:

$$X_i(k+1)=\left\{\begin{array}{cc}Y,\hfill & if(F(Y)<F(X_i(k)))\hfill \\ Z,\hfill & if(F(Z)<F(X_i(k)))\hfill \end{array}\right.$$

(20)

where Y and Z are the position vectors obtained by searching using the mathematical forms in Equation (21), as follows:

$$\left\{\begin{array}{c}Y=X_{prey}(k)-E\left|J\cdot X_{prey}(k)-X_m(k)\right|\\ Z=Y+S\times Lf(D)\end{array}\right.$$

(21)

where $X_m(k)$ denotes the average position vector of the population during the kth iteration.

3.2. Chaotic Population Initialization Strategy

Numerous studies have shown that population initialization is critical to algorithm performance [38,39,40]. The standard Harris Hawks Optimizer (HHO) relies on traditional random initialization to generate the initial population. This often leads to uneven population distribution, slow early convergence, and entrapment in local optima. In contrast, chaotic systems, being pseudo-random and ergodic, allow individuals generated through chaotic mapping to cover the search space more uniformly. The resulting enhanced population diversity improves search capabilities and helps avoid premature convergence to local optima.

The skew tent chaotic map (STCM) [41], as shown in Figure 5, is used in this study to improve the performance of the initial population. The skew tent chaotic map is a nonlinear, piecewise-defined function widely used in chaos theory and related applications. It is a variant of the standard tent map, designed to generate chaotic sequences with specific properties, thereby ensuring the diversity of the initial population. The STCM is defined as follows:

$${\chi }_{j+1}=\left\{\begin{array}{l}{\displaystyle \frac{{\chi }_j}{\phi }},0<{\chi }_j<\phi \hfill \\ {\displaystyle \frac{1-{\chi }_j}{1-\phi }},\phi <{\chi }_j<1\hfill \end{array}\right.$$

(22)

$$X_i=Lb+\chi \cdot (Ub-Lb)$$

(23)

where χ_j and φ represent the random number uniformly distributed in the interval [0, 1]; Lb and Ub represent the upper and lower bounds of the problem that can be searched by individuals in the hawk group, respectively. X_i represents the position vector of i-th hawk in the population.

3.3. Exploration Phase Adaptive Strategy

In the exploration phase of the standard HHO, the hawk group employs two strategies to search for possible prey locations, with an equal probability of executing each strategy. However, this equal probability method does not consider the impact of individual performance, which significantly limits the population’s exploration efficiency. Particularly in the q < 0.5 stage, simply considering the intermediate position between the prey and the hawk group, and using the upper limit of the problem space to determine the next stage position of the population, makes it likely that the algorithm cannot guarantee convergence in the later search process. To address this flaw, this study reconstructs the exploration phase by adjusting the strategy for the q < 0.5 stage to account for the average performance of the current hawk’s position and its historical optimal position. Furthermore, it proposes that the hawk balance factor η determines the strategy for individual search, with its magnitude based on the performance of the current hawk within the population. The mathematical expression is as follows:

$$\left\{\begin{array}{c}{A}_1(k)=X_i(k)-r_1\cdot \left|X_{rand}(k)-2r_2\cdot X_i(k)\right|\\ A_2(k)={\displaystyle \frac{(X_i(k)+X_{his}(k))}{2}}\end{array}\right.$$

(24)

$$\left\{\begin{array}{c}{X}_i(k+1)=\eta \cdot A_1+(1-\eta )\cdot A_2\\ \eta ={\displaystyle \frac{F(X_i(k))}{\max [F(X(k))]}}\end{array}\right.$$

(25)

where A₁(k) represents position obtained by the i-th hawk through random perching at k-th iteration; X_his(k) represents historical best position at k-th iteration; A₂(k) represents the position obtained by the i-th hawk through a comprehensive consideration of the current position and historical best position at k-th iteration; max[F(X(k))] represents the maximum fitness within the hawk group during the k-th iteration. η represents the balance factor of the i-th hawk.

3.4. Random Difference Mutation Strategy with Q-Learning

Mutation and crossover are key components in evolutionary algorithms such as differential evolution [42]. Mutation introduces random perturbations into a single solution, promoting exploration and preventing premature convergence by diversifying the population. Crossover, on the other hand, combines features of parent solutions to produce potentially superior offspring, leveraging the best features of each parent solution to improve the overall solution quality. Together, these operators balance exploration and exploitation, enhancing the ability of the algorithm to efficiently search complex solution spaces and find optimal solutions.

The mutation factor is particularly critical in population evolution, as it significantly influences its performance. Q-learning, shown in Figure 6, is a popular reinforcement learning method that mimics human behavior to establish dynamic interactions between the environment and actions, facilitating repeated learning through reward patterns. Since its inception, Q-learning has found extensive applications in various fields, including autonomous driving, gaming, and robotics. In the Q-learning algorithm, the agent learns by interacting with the environment and then selects actions from the Q-table to receive corresponding rewards. The detail process is given in Algorithm 1, and the formula for updating the Q-table values is as follows:

$$Q_{t+1}\left(s,a\right)=Q_t\left(s,a\right)+\alpha [R^a(s,a)+\gamma \max Q_t\left(s^{\prime },a^{\prime }\right)-Q_t\left(s,a\right)]$$

(26)

where Q_t+1(s, a) represents the updated value of the action-value function for the state s and action a at the time step (t + 1); α and γ represent the learning rate and discount factor between [0, 1], respectively; R^a(s, a) represents the reward value for the state s and action a; $\max Q_t\left(s^{\prime },a^{\prime }\right)$ represents the maximum Q-value for all actions in the new state $s^{\prime }$.

The CEHHO algorithm employs Q-learning to adaptively adjust the mutation factor value, aiming to identify the most suitable algorithm parameters to each individual. The probability of selecting action a_j in state s_i is generated by SoftMax technology:

$$\pi \left(\begin{array}{c}{s}_i,a_j\end{array}\right)={\displaystyle \frac{\exp \left[Q_t\left(\begin{array}{c}{s}_i,a_j\end{array}\right)\right]}{{\displaystyle \sum _{j=1}^n\exp \left[Q_t\left(\begin{array}{c}{s}_i,a_j\end{array}\right)\right]}}}$$

(27)

where n denotes the total number of actions taken by the agent; exp[·] denotes the exponential function.

Within this context, two states are defined: (1) The performance of offspring individual surpasses that of the parent, resulting in a reward value of 1. (2) The performance of offspring is inferior to that of the parent, yielding a reward value of 0. Each state corresponds to three possible actions regarding the mutation factor R, (1) r = 0.01; (2) r = 0; (3) r = −0.01; the reward R for each individual is updated as follows:

$$R=R+r$$

(28)

where r represents the mutation adjustment factor obtained through Q-learning. R represents the mutation factor.

Algorithm 1: The adaptive adjustment process of the Q-learning algorithm

1. Initialize the Q-table and randomly select the initial state st. 2. While the search termination condition is not met do: 3.        (a). Select the optimal action at from the Q-table based on the current state st; 4.        (b). Execute the action at and obtain reward r based on the environment; 5.        (c). Calculate the maximum Q-value for the next state st+1; 6.        (d). Update the Q-table using the formula by Equation (26); 7.        (e). Update the current state: st = st+1; 8. Output: State s.

Then, the random difference mutation strategy to improve the level of the HHO to obtain better offspring individuals is expressed as follows:

$$B_{ij}(k)=\left\{\begin{array}{l}{X}_{ij}(k), r_6\le cr or randi(D)=j\hfill \\ X_{rand1}(k)+R\cdot (X_{rand2}(k)-X_{rand3}(k)), else \hfill \end{array}\right.$$

(29)

$$X_i(k+1)=\left\{\begin{array}{l}{X}_i(k), F(X_i(k))\le F(B_i(k))\hfill \\ B_i(k), else \hfill \end{array}\right.$$

(30)

where B_ij(k) represents the mutation position of the i-th hawk in j-th dimension at the k-th iteration; rand1, rand2 and rand3 (rand1 ≠ rand2 ≠ rand3) represent the position indexes randomly selected from the hawk group, respectively; cr represents the crossover factor; randi(D) represents the randomly selected dimension index. r₆ represents the random number in [0, 1].

3.5. The Execution Process of CEHHO

To address the performance deficiencies of the traditional HHO and elevate its exploration and development capabilities, this study integrates three optimization strategies for the first time, proposing the Chaotic-enhanced Harris Hawks Optimizer (CEHHO). Firstly, the skew tent chaotic map (STCM) is introduced during the initialization phase of the HHO to enhance population space traversal, thereby ensuring initial population diversity effectively. Secondly, an adaptive strategy for the exploration phase is introduced to redefine its search strategy, selecting appropriate exploration processes based on individual performance to enhance effective search. Lastly, a random difference mutation strategy is incorporated to enhance the ability of the HHO to generate superior offspring.

Notably, the STCM-based population initialization method does not increase computational complexity of the CEHHO. Similarly to random population initialization, its complexity depends solely on the dimensionality of problem. Similarly, the complexity of the Q-learning adjustment strategy, which is applied individually to each population member, depends only on the population size. This analysis indicates that the introduction of these core strategies does not significantly increase overall computational time of the CEHHO. The specific framework of the CEHHO is illustrated in Figure 7 as follows:

4. Experimental Implementation

To demonstrate the effectiveness of the proposed method CEHHO, a simulation experiment was conducted using four cascade hydropower stations in the lower reaches of the Jinsha River: Wudongde (WDD), Baihetan (BHT), Xiluodu (XLD), and Xiangjiaba (XJB), which form a group of large reservoirs. The Pingshan hydrological station (PS) downstream of XJB was used as the control section for hydrological conditions. The ecological flow process under natural hydrological conditions was obtained by averaging the monthly observed flows at the PS hydrological station over the years 1988 to 2000, as shown in Figure 8.

Additionally, four commonly used methods—Sine Cosine Algorithm (SCA) [43], Repairable Gray Wolf Optimization (RGWO) algorithm [44], Whale Optimization Algorithm (WOA) [45], and Harris Hawks Optimizer (HHO)—were introduced for comparative validation. For all methods, the population size was uniformly set to 30, the maximum number of evaluations was set to 30,000, and each method was independently trialed 20 times for statistical analysis. The detailed parameter settings for each algorithm are shown in

Table 1. The detailed parameter settings of multi algorithms.

Algorithm	Parameters	Value
SCA	Constant factor a	2.0
RGWO	Convergence constant a	Decreased linearly from 2 to 0
	Design parameters ep	5
	Design parameters et	10
WOA	Constant a	2.0
	Random factor l	[−1, 1]
HHO	Initial escape energy E0	Randomly varies within the range [−1, 1]
	Constant β	1.5
CEHHO	Initial escape energy E0	Randomly varies within the range [−1, 1]
	Crossover probably cr	Gaussian distribution with mean 0.6 and variance 0.1
	learning rate α	0.1
	discount factor γ	0.9

below:

4.1. Case Study 1: Analysis in Different Typical Year

To evaluate the effectiveness of the proposed CEHHO algorithm, a statistical analysis of long-term hydrological data was conducted. Water inflow processes corresponding to annual frequencies of 25%, 50%, and 75% were selected as model inputs. The performance of the CEHHO method was compared with other process optimization methods, including SCA, RGWO, WOA, and HHO.

Table 2. Statistical results of multi algorithms in typical years.

Frequency	Indictor	SCA	RGWO	WOA	HHO	CEHHO
25%	Best	639.62	633.75	602.85	618.84	422.77
	Median	754.16	709.38	660.54	823.29	613.87
	Mean	755.74	716.89	682.72	827.45	538.33
	Worst	858.07	794.51	789.00	968.31	622.91
	STD	65.59	52.71	63.50	85.24	90.35
50%	Best	420.91	482.77	308.17	504.00	235.00
	Median	644.57	569.29	471.01	697.42	241.79
	Mean	620.26	565.51	482.48	697.48	244.42
	Worst	750.59	668.71	649.82	816.61	267.91
	STD	85.68	48.03	91.66	96.02	9.52
75%	Best	642.30	546.03	448.96	523.36	415.32
	Median	808.30	738.89	544.82	898.17	416.08
	Mean	785.50	727.47	576.44	832.45	416.93
	Worst	904.95	842.89	753.23	1104.54	420.46
	STD	77.03	71.59	93.76	173.42	1.73

presents the statistical results of the proposed method and the comparison methods, based on the best value, median value, mean value, worst value, and standard deviation (STD).

As shown in Table 2, the proposed CEHHO method outperforms most indicators compared to other control methods. Specifically, for the frequency of 75%, CEHHO achieves optimal values that are 35%, 24%, 7%, and 21% lower than those of SCA, RGWO, WOA, and HHO, respectively. Additionally, the standard deviation is improved by 98%, 98%, 98%, and 99%, respectively.

It is worth noting that, although the STD for the proposed method is higher for the 25% frequency compared to other methods, its overall target value is smaller. This indicates that there may be more minimum points within the distribution range of the better results, suggesting that the proposed strategy effectively finds more accurate solutions. Overall, the results of the proposed method are significantly better than those of other technical solutions, demonstrating its superior performance.

To further examine the effectiveness of the CEHHO, the statistical results of the target values of the five optimizers at 25%, 50%, and 75% frequencies were recorded over 20 repetitions, as shown in Figure 9. The results in Figure 9 indicate that, under all three water inflow frequencies, the proposed method consistently finds better and more stable solutions compared to other mature optimizers. These observations are also consistent with the results presented in Table 2.

Figure 10 shows the statistical ranking of the best value, median, mean, worst value, and STD of different meth. As shown in the inflow scenario shown in Figure 10a, the proposed method performs poorly in terms of standard deviation compared to other methods, but it obtains the best ranking for other metrics. In addition, combined with the data in Figure 9a, it can be clearly seen that the overall distribution of the proposed method is better than that of other methods. In the inflow scenarios shown in Figure 10b,c, it is obvious that the results of the original HHO method are the worst among all methods. In contrast, the proposed CEHHO method outperforms all other methods and obtains the highest ranking. This clearly demonstrates the effectiveness of the proposed strategy.

Figure 11 further illustrates the convergence curves of different methods under three frequencies. As shown in Figure 11, SCA performs the worst at the 25% frequency and performs averagely at the 50% and 75% frequencies. The HHO algorithm exhibits average performance across all three frequencies. In contrast, the proposed CEHHO method demonstrates the best convergence speed and accuracy among the three frequencies. This fully demonstrates the effectiveness and efficiency of the proposed method.

4.2. Case Study 2: Different Guaranteed Output

In this section, different guaranteed output limits are used t to examine their impact on the CEHHO algorithm’s performance. By exploring the algorithm results under various output requirements, the search efficiency and engineering practicability of the proposed method in solving the problem can be demonstrated.

Table 3. Statistical results of multi algorithms in different guaranteed output.

Guaranteed Output (MW)	Indictor	SCA	HHO	RGWO	WOA	CEHHO
100	Best	689.97	708.17	710.47	528.85	414.25
	Median	799.61	929.75	768.45	642.19	436.52
	Mean	798.46	903.29	772.73	664.40	481.15
	Worst	896.20	1002.07	849.67	797.15	620.92
	STD	64.73	91.20	45.96	81.67	82.35
	Time	0.47	0.39	0.57	0.40	0.48
120	Best	616.92	631.54	606.58	607.02	419.31
	Median	860.11	950.48	804.49	676.90	615.61
	Mean	837.74	913.49	804.48	697.16	536.59
	Worst	1013.17	998.80	895.76	806.36	621.09
	STD	100.22	91.06	64.99	61.06	93.73
	Time	0.47	0.35	0.57	0.42	0.49
140	Best	781.29	729.10	686.97	572.43	425.76
	Median	866.50	937.05	869.66	730.34	448.02
	Mean	877.23	930.18	846.25	710.20	492.62
	Worst	977.17	1024.30	914.82	850.37	619.68
	STD	51.98	85.87	57.82	85.50	84.44
	Time	0.46	0.31	0.52	0.39	0.43
160	Best	833.88	585.49	742.38	629.99	448.96
	Median	997.19	925.18	958.55	799.33	492.41
	Mean	995.93	959.40	956.69	784.12	525.33
	Worst	1136.31	1677.82	1083.36	922.40	635.41
	STD	83.83	223.73	86.99	86.56	75.98
	Time	0.48	0.32	0.51	0.35	0.43
180	Best	1084.33	769.78	883.41	648.72	464.82
	Median	1293.38	2179.29	1143.30	829.10	505.58
	Mean	1325.62	1988.80	1143.61	848.52	543.37
	Worst	1657.30	3138.73	1381.95	1067.79	679.75
	STD	154.79	845.83	145.07	107.28	77.69
	Time	0.47	0.32	0.52	0.36	0.43
200	Best	1375.73	1280.25	1165.17	1025.52	494.25
	Median	1873.14	2596.47	1600.74	1356.51	563.32
	Mean	1895.06	3010.82	1622.16	1461.58	594.25
	Worst	2547.47	5290.57	2040.12	3308.89	741.41
	STD	329.92	1558.57	236.56	506.68	75.17
	Time	0.46	0.32	0.51	0.36	0.44

presents the statistical results of the CEHHO and other algorithms for the 50% frequency water inflow process, including the best, worst, median, mean, standard deviation (STD), and execution time (in seconds).

As summarized in Table 3, the proposed method demonstrates excellent results in all simulation scenarios. Specifically, under the guaranteed output limit of 1000 MW, the proposed method improves the optimal value, median, mean, and worst value by 293.92, 493.23, 422.14, and 381.15, respectively, compared to the HHO. Under the guaranteed output limit of 1800 MW, the proposed method improves the optimal value index by 619.51, 304.96, 418.59, and 183.9 compared to SCA, HHO, RGWO, and WOA, respectively, and improves the worst value index by 977.55, 2458.98, 702.2, and 388.04, respectively. These findings fully demonstrate that the proposed method maintains good search accuracy under different guaranteed output requirements and has significant engineering practical value.

Building on the results presented in Table 3, Figure 12 illustrates the average CPU execution times of the five methods under various guaranteed output limits, validating the efficiency of each algorithm. All algorithms were executed in the same hardware and software environment. The execution time for each run was recorded, and the average values were calculated to facilitate a statistical comparison of the execution times of different algorithms. Figure 12 demonstrates that with varying guaranteed output limits, the solution times for all methods remain relatively stable. The average execution times for the SCA, HHO, RGWO, WOA, and CEHHO methods are 0.47 s, 0.34 s, 0.53 s, 0.38 s, and 0.45 s, respectively. The HHO method exhibits the highest efficiency, while the improved CEHHO method shows a moderate execution time. This suggests that the proposed method achieves improved solutions at the cost of an additional runtime of 0.11 s. Compared to other optimization techniques, the proposed CEHHO method effectively reveals an enhanced ecological scheduling scheme for reservoir operations without significantly increasing the algorithm runtime.

To analyze the performance trends of various methods under different guaranteed output constraints, Figure 13 presents the average statistical results for each method, where darker colors represent better performance metrics. As the guaranteed output requirements increase, it is evident that the target values for all methods generally rise, highlighting the growing difficulty in finding optimal solutions across the board. Notably, the increases for the SCA, HHO, RGWO, and WOA methods are particularly pronounced. In contrast, the proposed method maintains target values between 480 and 600 and consistently outperforms the others in every guaranteed output constraint scenario. Specifically, under the condition of a guaranteed output of 2000 MW, the performance of the SCA, HHO, RGWO, and WOA methods is suboptimal, while the proposed method achieves a superior average result. This clearly demonstrates the stability and effectiveness of the proposed approach.

To clearly analyze the distribution of target value results across multiple runs, Figure 14 provides the box plot statistics for different methods. The box plot allows for an intuitive observation of the distribution of target values for each method, including the mean and interquartile range. Specifically, the target value distributions of the SCA and RGWO methods are concentrated, indicating that these methods offer more consistent solutions during the optimization process. The HHO method shows considerable variability, suggesting that its results may differ significantly between runs. The WOA exhibits a relatively poor distribution in the 1000 MW to 1400 MW output limit scenario, while it demonstrates improved stability in the 1600 MW to 2000 MW output limit scenario. The proposed CEHHO method maintains a concentrated solution distribution within an optimal target range across all output scenarios. This analysis provides a crucial basis for evaluating and comparing the optimization performance of different methods, indicating that the proposed method demonstrates superior stability and practicality.

5. Conclusions

This study introduces an advanced version of the Harris Hawks Optimizer (CEHHO), specifically designed to determine the natural flow pattern process following the construction of cascade reservoirs, incorporating chaos-based techniques. The key innovations of this research are as follows:

(1) Chaotic initialization: The skew tent chaotic map enhances population initialization, improving the quality and diversity of the initial population and augmenting the algorithm’s exploration capabilities.

(2) Adaptive exploration strategy: An adaptive strategy is proposed for the exploration phase, improving individual exploration efficiency by introducing a balance factor and selecting exploration strategies based on individual performance.

(3) Integration of random mutation with Q-learning: The integration of random mutation and Q-learning introduces random perturbations to individuals, enhancing the algorithm’s ability to navigate complex solution spaces and generate superior offspring.

(4) Application to practical engineering: The proposed method simulates the natural hydrological situation of four cascade reservoirs in the lower Jinsha River, demonstrating advantages in search accuracy and convergence efficiency compared to several widely used techniques under different water inflow frequencies.

Finally, the proposed method is used to simulate the natural hydrological situation of four cascade reservoirs in the lower reaches of the Jinsha River. Initially, the proposed method is compared against several widely used techniques under three typical water inflow frequencies, demonstrating its advantages in search accuracy and convergence efficiency. Furthermore, to meet the needs of practical engineering applications, a multi-use guaranteed output scenario is established, and various methods are applied for solutions. The experimental results indicate that the proposed CEHHO algorithm significantly outperforms existing popular techniques in search capabilities, demonstrating strong potential for engineering applications.

Despite these encouraging results, the CEHHO algorithm has several limitations. First, this study focused on a relatively modest-sized cascade reservoir system; therefore, the algorithm’s performance and scalability in larger and more complex cascade systems remain to be thoroughly evaluated. Second, combining Q-learning with random mutation inevitably increases computational cost; future work will explore methods that incorporate inherent reinforcement learning heuristics to mitigate this challenge. Finally, while the current implementation runs on a single computational thread, there are plans to incorporate parallel computing techniques to improve scalability and reduce runtimes for large-scale problems.