Research on the Collaborative Optimization Method of Power Prediction and DRL Control

Abstract

This paper proposes a collaborative energy management strategy based on power prediction and deep reinforcement learning (DRL) to address the trade-offs among economic efficiency, durability, and dynamic performance in fuel cell hybrid power systems (FCHPS) under dynamic driving conditions. First, a hybrid prediction model termed LSTM-LSSVM with Cascade Correction (LSTM-LSSVM-CC) is developed. The cascade correction (CC) mechanism adopts a hierarchical structure to capture both low-frequency steady-state trends and high-frequency dynamic fluctuations, which are typically challenging for single models to represent. By integrating an online residual correction mechanism, this model generates accurate future power demand sequences. Second, a Dynamic Spatio-Temporal Fusion (DSTF) method is introduced to construct a high-dimensional DRL state space. This approach integrates predicted data, historical residuals, and real-time system states, enabling the agent to perform anticipatory decision-making. Third, a Dynamic Hierarchical Adaptive Multi-Objective Optimization Framework (DHAMOF) is designed. This framework dynamically adjusts objective weights and constraint boundaries based on real-time operating characteristics, enabling adaptive switching of optimization priorities across diverse scenarios. Furthermore, a closed-loop control architecture comprising “prediction–decision–execution–feedback” is established. By incorporating rolling horizon optimization and a proportional-integral (PI) residual compensation mechanism, the proposed architecture effectively suppresses prediction error accumulation and mitigates communication delays. Simulation results under combined CLTC-P and WLTP driving cycles demonstrate that, compared to conventional fixed-weight strategies, the proposed method achieves an 11.3% reduction in hydrogen consumption, a 30.9% decrease in $SOC$ fluctuation range, and a 55.3% reduction in power tracking error. Moreover, under disturbance scenarios involving prediction errors, sensor noise, and a 200 ms communication delay, the system exhibits superior robustness: the increase in hydrogen consumption is limited to within 8.3 g/100 km, and the power tracking error is reduced by 65.6% relative to uncorrected baselines. This collaborative optimization approach overcomes the limitations of traditional open-loop prediction and fixed-weight control, offering a novel technical pathway for the high-efficiency and stable operation of fuel cell hybrid power systems.

1. Introduction

With the intensifying global energy crisis and environmental pollution issues, new energy vehicles (NEVs) have become the core direction of green transformation in the transportation sector [1]. Among them, fuel cell hybrid power systems (FCHPS), featuring zero emissions, long driving range, and fast hydrogen refueling, are regarded as one of the most promising development pathways for future NEVs [2]. By integrating fuel cells and energy storage devices, such systems can effectively compensate for the shortcomings of single power sources in terms of dynamic response, energy efficiency, and endurance [3,4,5]. However, under complex and dynamic driving conditions—such as urban congestion, highway cruising, and rapid acceleration—the system faces an inherent multi-objective conflict among economic efficiency, durability, and dynamic performance. Pursuing minimum hydrogen consumption for economic optimization may cause the fuel cell to operate long-term in inefficient regions, accelerating membrane electrode assembly (MEA) degradation. Enhancing dynamic response performance requires frequent power redistribution, which may lead to significant State-of-Charge ($SOC$) fluctuations. Conversely, overemphasizing durability protection may compromise power responsiveness and energy utilization efficiency [6]. Traditional energy management strategies—such as rule-based control and dynamic programming (DP)—struggle to achieve dynamic balance among multiple objectives under complex and time-varying conditions. Rule-based control depends heavily on expert experience and lacks adaptability, while DP can obtain globally optimal solutions but suffers from the curse of dimensionality, making it unsuitable for real-time control [7].

Deep Reinforcement Learning (DRL), as an intelligent control method with autonomous decision-making and online learning capabilities, provides a new solution for complex, nonlinear, and strongly coupled system control through continuous interactions between the agent and the environment. In recent years, DRL has been widely applied to the energy management of fuel cell hybrid systems, showing promising adaptability and optimization potential. However, its performance strongly depends on the completeness of the state-space information. Conventional DRL strategies make decisions based solely on the current vehicle states, lacking predictive insight into future driving conditions. As a result, the system exhibits delayed responses to sudden power demand changes, making it difficult to balance dynamic performance and energy efficiency [8,9,10,11,12].

To address this issue, researchers have introduced power prediction techniques to provide foresight for DRL-based decision-making by predicting short-term future power demand. However, most existing prediction models rely on single algorithms such as LSTM or SVM, which exhibit large cumulative errors in long-horizon predictions and poor adaptability to abrupt condition changes [13,14,15,16,17,18]. Moreover, prediction and control are typically coupled in an open-loop structure, allowing prediction errors to propagate directly into the control loop, thereby degrading strategy performance. In addition, the objective weight coefficients in multi-objective optimization are often fixed, limiting the system’s ability to adaptively adjust optimization priorities according to changing driving conditions [19].

To overcome these limitations, this paper proposes a collaborative optimization energy management method that integrates power prediction and deep reinforcement learning (DRL) [20]. The proposed framework adopts an LSTM-LSSVM-CC hybrid prediction model as its core, enhanced by an online residual correction mechanism to improve power demand prediction accuracy. A Dynamic Spatio-Temporal Fusion (DSTF) method is employed to integrate predicted sequences, historical residuals, and real-time operating states into a high-dimensional DRL state space, enabling the agent to make predictive and adaptive control decisions. Furthermore, a Dynamic Hierarchical Adaptive Multi-Objective Optimization Framework (DHAMOF) is developed to dynamically adjust objective weights and constraint conditions based on driving characteristics. Finally, a closed-loop “Prediction–Decision–Execution–Feedback” control architecture is constructed, incorporating rolling optimization and residual compensation mechanisms to suppress the effects of prediction error accumulation and communication delay. Simulation results under CLTC-P and WLTP hybrid driving conditions, as well as multiple disturbance scenarios, verify the effectiveness of the proposed method in improving system economy, durability, and dynamic performance, providing both theoretical support and engineering solutions for the efficient and stable operation of fuel cell hybrid power systems [21].

2. Collaborative Optimization Framework Design

This section proposes a collaborative optimization framework based on power prediction information and Deep Reinforcement Learning (DRL). The framework aims to achieve simultaneous optimization of global energy efficiency and dynamic stability for the fuel cell hybrid power system (FCHPS) through a coupling mechanism that integrates dynamic condition prediction with real-time energy allocation [22].

The framework utilizes the LSTM-LSSVM-CC vehicle condition prediction model (as described in the previous section) as the feedforward input, and the DRL-based energy management strategy as the core controller, forming a closed-loop optimization chain consisting of prediction–decision–execution–feedback [23,24]. The core concept lies in expanding the state space of the DRL agent with predictive information, allowing the control strategy to make forward-looking decisions.

Additionally, a multi-objective optimization strategy is employed to balance energy efficiency, $SOC$ stability, and dynamic response requirements, thereby realizing synergistic enhancement between prediction accuracy and control performance. The overall structure of the collaborative optimization framework is illustrated in Figure 1.

Figure 1. Collaborative optimization framework for prediction and control.

The prediction module (LSTM-LSSVM-CC model) processes historical vehicle speed, acceleration, and battery $SOC$ data to generate power demand forecasts for future time windows (e.g., 1–5 s)$\left\{P_{\mathrm{req},\mathrm{pred}}\left(t + 1\right), \dots ,{ P}_{\mathrm{req},\mathrm{pred}}\left(t + k\right)\right\}$.

The state space integration combines the current state ($SOC$, $P_{\mathrm{req}}$) with the predicted state ($P_{\mathrm{req},\mathrm{pred}}$) to form a multidimensional state vector. This process incorporates fuel consumption, $SOC$ deviation, prediction error penalties, and dynamic response performance (e.g., fuel cell power fluctuations). The system outputs real-time power allocation commands for the fuel cell and energy storage system. The collaborative optimization mechanism employs a sliding window algorithm to dynamically adjust the confidence weights of the DRL strategy while correcting prediction errors in real time. It also adaptively prioritizes optimization objectives based on operating conditions (e.g., CLTC-P congestion scenarios, WLTP highway scenarios). Finally, the actual operational data ($SOC$, fuel consumption) is fed back to the prediction module for online updates.

The prediction module employs the LSTM-LSSVM-CC model with an online residual correction mechanism. It continuously monitors the actual power demand $P_{\mathit{req},\mathit{real}}$ and computes the residual sequence $\mathsf{\Delta }{P}_{\mathit{req}}$ = $P_{\mathit{req},\mathit{real}}$ − $P_{\mathit{req},\mathit{pred}}$. This residual sequence serves as an additional input to drive the LSTM network to dynamically update its weight parameters, thereby reducing the cumulative error in long-term predictions.

The DRL control module employs multidimensional state fusion to integrate predictive data into the DRL state space, effectively addressing the lag in response to abrupt variables characteristic of traditional strategies. Furthermore, it dynamically adjusts the credibility of predicted values based on residual standard deviation, thereby mitigating the interference of abnormal predictions on control operations.

The framework addresses the conflicting objectives of ‘economy, durability, and dynamics’ in fuel cell systems by designing the following collaborative rules, as detailed in Table 1.

Table 1. Collaborative optimization rules.

Type	Condition	Threshold Determination	Operation Strategy	Target Priority
Economy first mode	Predict stable power demand (low volatility)	${\mathsf{\sigma }}_{P_{\mathrm{req},\mathrm{pred}}} < {\mathsf{\sigma }}_{\mathrm{th}}$	Increase fuel consumption weight ${\alpha }_1$ Limiting Fuel Cell Load to Efficient Operating Range	Economy (lowest hydrogen consumption) > Durability > Stability
Dynamic priority mode	Emergency acceleration or sudden slope change detected (sudden power demand)	${\displaystyle \frac{\mathsf{\Delta }{P}_{\mathrm{req},\mathrm{pred}}}{\mathsf{\Delta }t}} > {\gamma }_{\mathrm{th}}$	Relaxing the fluctuation limits of the Shanghai Composite Index ($SOC$) Prioritize supercapacitor response to high-frequency demand Reduce the dynamic response penalty weight ${\alpha }_3$	Dynamism (response speed) > Economy > Durability
Durability compensation mechanism	The battery $SOC$ consistently deviates from the safe range (e.g., $SOC$ < 40% or $SOC$ > 80%)	Continuous time of departure ${ T}_{\mathrm{offset}}$>$T_{\mathrm{th}}$	The penalty weight α2 for $SOC$ deviation increases exponentially. Force DRL to adjust charge–discharge depth to safe range	Durability (battery life) > Economy > Dynamism

This method overcomes the limitations of traditional open-loop prediction or independent control by implementing joint iterative optimization of predictive models and control strategies through dynamic error feedback. It integrates predictive sequences, historical residuals, and real-time states into a high-dimensional state vector, thereby enhancing the deep reinforcement learning (DRL) strategy’s ability to perceive complex operating conditions. By dynamically adjusting optimization objectives’ priorities based on operational characteristics, it avoids performance trade-offs under fixed weighting schemes.

3. Design of DRL Control Strategy Based on Prediction

3.1. Integration of Predicted Data and DRL State Space

To fully exploit the predictive capability of the LSTM-LSSVM-CC model, this section proposes a Dynamic Spatiotemporal Fusion (DSTF) method, which organically integrates the predicted future power demand sequence with the real-time vehicle states into the state space of the DRL algorithm, thereby enabling the agent to make anticipatory and informed control decisions.

The implementation logic and technical details of this fusion process are divided into three main steps.

• State Space Expansion Design

Traditional DRL state spaces rely solely on the current vehicle states (such as $SOC$ and power demand). In contrast, the proposed method introduces predicted power demand sequences and historical residual information to construct a multi-dimensional composite state vector, defined as:

$$s(t)=\left[\begin{array}{ccc}\underset{\mathrm{Current} \mathrm{state}}{\underbrace{\mathrm{SOC}(t),P_{\mathrm{req}}(t)}}& \underset{\mathrm{Prediction} \mathrm{sequence}}{\underbrace{P_{\mathrm{req},\mathrm{pred}}(t+1),\dots ,P_{\mathrm{req},\mathrm{pred}}(t+k)}}& \underset{\mathrm{Historical} \mathrm{residuals}}{\underbrace{\mathsf{\Delta }{P}_{\mathrm{req}}(t-1),\dots ,\mathsf{\Delta }{P}_{\mathrm{req}}(t-m)}}\end{array}\right]$$

(1)

In this framework, the prediction sequence represents the output of LSTM-LSSVM-CC for the next k steps (e.g., $k$ = 5), capturing short-term power demand trends. The historical residual refers to the error between the actual and predicted power over the past m steps ($\mathsf{\Delta }{P}_{\mathrm{req}} = P_{\mathrm{req},\mathrm{real}} - P_{\mathrm{req},\mathrm{pred}}$), reflecting the model’s recent reliability.

Experimental results demonstrate that the DRL strategy achieves optimal balance between energy efficiency and stability when $k$ = 5 (5 s prediction window) and $m$ = 3 (3-step residual backtracking) (Figure 2).

Figure 2. Influence of prediction parameters on system performance.

As illustrated in Figure 2, when k = 5, the system achieves optimal equilibrium between hydrogen consumption efficiency (lowest on the left axis) and $SOC$ stability (lowest on the right axis). Increasing the m value enhances stability (the triangular marker shifts closer to the bottom of the right axis), but results in a slight increase in hydrogen consumption. When k = 7, performance degrades, indicating that an excessively long prediction window causes model overfitting.

2.

Dynamic Confidence Weight Allocation

The reliability of the predicted data varies dynamically with driving conditions; therefore, confidence weights are introduced to adaptively adjust their influence on the state space.

The confidence coefficient is calculated as follows:

$$w\left(t\right) = {\displaystyle \frac{1}{1 + \sqrt{\frac{1}{m}{\sum }_{i=1}^m\mathsf{\Delta }{P}_{\mathrm{req}}^2\left(t - i\right)}}}$$

(2)

When the confidence weight $w$ ∈ [0, 1] is set, the larger the residual fluctuation (indicating unreliable predictions), the smaller $w$ becomes. Then, the state vector is corrected by introducing a weight attenuation factor into the prediction sequence to suppress interference from low-confidence predictions.

$$P_{\mathrm{req},\mathrm{pred},\mathrm{adj}}\left(t +i\right)=w\left(t\right) P_{\mathrm{req},\mathrm{pred}}\left(t +i\right)+\left(1 - w\left(t\right)\right) P_{\mathrm{req}}\left(t\right)(i=1, \dots , k)$$

(3)

As shown in the formula, $P_{\mathrm{req},\mathrm{pred},\mathrm{adj}}\left(t + i\right)$ denotes the equivalent predicted power value after safety boundary correction. when the prediction error is significant (e.g., $w$< 0.3), the predicted value becomes an inertial extension of the current value, preventing erroneous predictions from misleading the DRL strategy.

3.

Data Standardization and Feature Engineering

To eliminate dimensionality differences and enhance DRL training efficiency, the state vector undergoes hierarchical normalization. The current state battery $SOC$ is linearly mapped to [0, 1], with the target range (40–80%) corresponding to [0.4, 0.8]. $P_{\mathrm{req}}$ is standardized using Z-score, based on the historical data mean ${\mathsf{\mu }}_P$ and standard deviation ${\mathsf{\sigma }}_P$.

The predicted value is independently normalized at each step, with the dynamic benchmark being the current power demand, as shown in the formula.

$${\widehat{P}}_{\mathrm{req},\mathrm{pred}}\left(t +i\right)={\displaystyle \frac{P_{\mathrm{req},\mathrm{pred}}\left(t +i\right) - P_{\mathrm{req}}\left(t\right)}{\max \left(P_{\mathrm{req}}\left(t\right),P_{\mathrm{nom}}\right)}}$$

(4)

where $P_{\mathrm{nom}}$ denotes the system’s rated power to prevent the denominator from becoming too small. The historical residuals are standardized using a sliding window, with the window length matching the residual backtracking step size m.

$$\mathsf{\Delta }{\widehat{P}}_{\mathrm{req}}\left(t - i\right)={\displaystyle \frac{\mathsf{\Delta }{P}_{\mathrm{req}}\left(t - i\right) - {\mu }_{\mathsf{\Delta }P}}{{\mathsf{\sigma }}_{\mathsf{\Delta }P}}}$$

(5)

Taking the CLTC-P operating condition as an example, the DRL strategy performance before and after state space expansion is compared, as detailed in Table 2.

Table 2. Performance improvement with DSTF state fusion.

State Space Type	Average Hydrogen Consumption (g/100 km)	$\mathbf{Battery} \mathit{S}\mathit{O}\mathit{C}$ Fluctuation Range (%)	Rapid Acceleration Response Delay (ms)
Traditional state (no prediction)	585.7	25.3	480
Extended State (DSTF Fusion)	538.2	18.9	210

The state-space fusion reduces hydrogen consumption by 8.1%, minimizes $SOC$ fluctuations by 25.3%, and enhances dynamic response speed by 56.3%. This section demonstrates the effectiveness and engineering feasibility of predictive data and DRL state-space fusion through theoretical analysis and simulation validation, laying the foundation for subsequent multi-objective optimization design.

3.2. Multi-Objective Optimization Design

In the field of multi-objective optimization, the goals of economic efficiency, durability, and dynamic performance often exhibit significant conflicts and nonlinear coupling characteristics. To achieve optimal energy management for fuel cell hybrid power systems (FCHPS), this paper proposes a Dynamic Hierarchical Adaptive Multi-Objective Framework (DHAMOF). By integrating prediction information with Deep Reinforcement Learning (DRL) through a collaborative mechanism, this framework constructs hierarchically weighted reward functions and dynamic constraint conditions, enabling the energy management strategy to adaptively balance multi-objective priorities under varying operating conditions.

The core concept of DHAMOF lies in leveraging the predictive guidance provided by future data to achieve dynamic decoupling and rebalancing of optimization objectives, thereby overcoming the limitations of conventional fixed-weight strategies.

In multi-objective optimization problems, Pareto Optimality serves as the fundamental criterion for evaluating solution quality. However, the real-time control requirements of fuel cell hybrid systems demand that the control policy rapidly converge to a feasible near-optimal solution, rather than exhaustively exploring the entire Pareto frontier. To this end, the DHAMOF framework adopts a strategy that combines the weighted-sum method with prediction-driven dynamic constraints, transforming the multi-objective optimization problem into a dynamic single-objective optimization formulation. The mathematical representation is expressed as:

$$\left\{\begin{array}{c}\max R\left(t\right) ={\displaystyle \sum _{i=1}^n}{\alpha }_i\left({\mathbf{P}}_{\mathrm{pred}},{\mathbf{S}}_{\mathrm{real}}\right) r_i\left({\mathbf{S}}_{\mathrm{real}},\mathbf{a}\right)\\ \mathrm{s}.\mathrm{t}. g_j\left({\mathbf{P}}_{\mathrm{pred}},{\mathbf{S}}_{\mathrm{real}}\right) \le 0, j = 1, 2, \dots , m\end{array}\right.$$

(6)

In the formula: ${\alpha }_i$ is the dynamic weight coefficient, which depends on the predicted power sequence ${\mathrm{P}}_{\mathrm{pred}} = \left[{\mathrm{P}}_{\mathrm{req},\mathrm{pred}}\left(\mathrm{t} + 1\right), \dots , {\mathrm{P}}_{\mathrm{req},\mathrm{pred}}\left(\mathrm{t} + \mathrm{k}\right)\right]$ and the real-time state ${\mathbf{S}}_{\mathrm{real}}=\left[\mathit{SOC}\left(t\right),P_{\mathrm{req}}\left(t\right),\dots \right]$ with ${\mathrm{r}}_{\mathrm{i}}$ being the reward function for each sub-objective and ${\mathrm{g}}_{\mathrm{j}}$ representing the constraint based on predicted data.

The advantage of this modeling method is that it not only retains the computational efficiency of the weighted sum method, but also enhances the time-varying nature of the constraints through predictive information, thus better meeting the requirements of actual working conditions.

Based on the core optimization objectives of the fuel cell hybrid system, the following sub-reward functions and their dynamic coupling mechanisms are defined.

(1)

Hydrogen minimization

$$r_{\mathrm{eco}}\left(t\right)=-{\lambda }_1 {\displaystyle \frac{\dot{m_{{\mathrm{H}}_2}}\left(t\right)}{\dot{m_{\max }}}} - {\lambda }_2 {\displaystyle \frac{\left|P_{\mathrm{FC}}\left(t\right)-P_{\mathrm{FC},\mathrm{opt}}\right|}{P_{\mathrm{nom}}}}$$

(7)

In the formula, $\dot{m_{{\mathrm{H}}_2}}\left(t\right)$ denotes the instantaneous hydrogen consumption rate of the fuel cell, while $\dot{m_{\max }}$ represents the maximum allowable hydrogen consumption. $P_{\mathrm{FC},\mathrm{opt}}$ indicates the optimal power point (approximately 70% of rated power) for fuel cell efficiency, driving its operation within the high-efficiency range. When the predicted power demand remains stable (${\mathsf{\sigma }}_{P_{\mathrm{pred}}} < 0.5 \mathrm{kW}$), increasing the weight of ${\lambda }_1$ enhances economic performance.

(2)

Power tracking accuracy

$$r_{\mathrm{dyn}}\left(t\right)=-\gamma {\displaystyle \frac{\left|P_{\mathrm{req}}\left(t\right) - \left(P_{\mathrm{FC}}\left(t\right)+P_{\mathrm{Batt}}\left(t\right)+P_{\mathrm{SC}}\left(t\right)\right)\right|}{P_{\mathrm{nom}}}}$$

(8)

The formula directly penalizes deviations between actual output and required power to ensure real-time power response. In rapid acceleration scenarios ($\mathsf{\Delta }{P}_{\mathrm{pred}}/\mathsf{\Delta }t > 2 \mathrm{kW}/s$), the system prioritizes supercapacitor activation to meet high-frequency demands by adjusting the γ parameter.

(3)

Durability Objective: Battery Life Protection

$$r_{\mathrm{life}}\left(t\right)=-{\eta }_1 e^{\left|\mathrm{SOC}\left(t\right)-0.6\right|} - {\eta }_2 {\displaystyle \frac{\left|I_{\mathrm{Batt}}\left(t\right)\right|}{I_{\max }}}$$

(9)

The exponential term in the formula forces the battery State of Charge ($SOC$) to remain near the midpoint (60%), thereby slowing down battery aging. The linear term suppresses high-current charging and discharging (with $I_{\max }$ as the maximum allowable current). When the $SOC$ consistently deviates from the safe range (e.g., >80% or <40% for more than 5 min), it triggers the time-varying increment of ${\mathsf{\eta }}_1,{\mathsf{\eta }}_2$.

To address the time-varying nature of operating conditions, the DHAMOF framework dynamically adjusts the weight coefficients ${\alpha }_i$ of each sub-objective based on predicted data, as detailed in Table 3.

Table 3. Dynamic weight adjustment strategies.

Trigger Condition	Weight Adjustment Policy	Optimize Objective Priority
Steady-state cruise	${\alpha }_{\mathrm{eco}} \times 1.5,{ \alpha }_{\mathrm{dyn}} \times 0.8, {\alpha }_{\mathrm{life}} \times 0.5$	Economy > Durability > Dynamism
Rapid acceleration/climb	${\alpha }_{\mathrm{dyn}} \times 2.0,{ \alpha }_{\mathrm{eco}} \times 0.6, {\alpha }_{\mathrm{life}} \times 0.3$	Dynamism > Economy > Durability
$SOC$ unusual	${\alpha }_{\mathrm{life}} \times \left(1+0.005 t_{\mathrm{offset}}\right),{ \alpha }_{\mathit{eco}} \times 0.7, {\alpha }_{\mathit{dyn}} \times 0.5$	Durability > Economy > Dynamics

In Table 3, the economic weight is adjusted adaptively according to the forecast volatility as shown in the formula:

$${\alpha }_{\mathrm{eco}}\left(t\right) = {\alpha }_{\mathrm{eco}}^{\mathrm{base}} \left(1 + {\displaystyle \frac{1}{1+{\mathsf{\sigma }}_{P_{\mathrm{pred}}}}}\right)$$

(10)

Here, ${\alpha }_{\mathrm{eco}}^{\mathrm{base}}$ denotes the base value of the economic weight, while ${\mathsf{\sigma }}_{P_{\mathrm{pred}}}$ represents the standard deviation of the predicted power sequence $\left\{P_{\mathit{req},\mathit{pred}}\left(t + 1\right), \dots ,{ P}_{\mathit{req},\mathit{pred}}\left(t + k\right)\right\}$, which characterizes the volatility of future power demand.

The dynamic weight surges during power fluctuations, with the specific variation as shown in the formula:

$${\alpha }_{\mathrm{dyn}}\left(t\right) = {\alpha }_{\mathrm{dyn}}^{\mathrm{base}} \left(1 + \tan \mathrm{h}\left({\displaystyle \frac{\mathsf{\Delta }{P}_{\mathrm{pred}}}{\mathsf{\Delta }t}}\right)\right)$$

(11)

where ${\alpha }_{\mathrm{dyn}}^{\mathrm{base}}$ denotes the base value of the dynamic weight. $\mathsf{\Delta }{P}_{\mathrm{pred}}/\mathsf{\Delta }t$: The power variation rate between adjacent prediction steps ($\mathsf{\Delta }P = P_{\mathrm{req},\mathrm{pred}}\left(t + 1\right) - P_{\mathrm{req},\mathrm{pred}}\left(t\right)$), indicating the magnitude of power demand fluctuations. The $\mathit{tan}\mathrm{h}$ function, a hyperbolic tangent function with an output range of (−1, 1), is employed to mitigate the impact of abrupt changes.

The dynamic adjustment rule for the durability weight ${\alpha }_{\mathrm{life}}$ is based on the following key parameters:

(a)

Battery $SOC$ deviation: |$\mathit{SOC}\left(t\right)$ − 0.6| (absolute value of deviation from the 60% midpoint of $SOC$);

(b)

Charge and discharge current: |$I_{\mathrm{Batt}}\left(t\right)$| (absolute value of the battery’s instantaneous current)

(c)

Trigger time threshold: cumulative time $t_{\mathrm{offset}}$ during which the battery $SOC$ remains outside the safe range (e.g., $SOC$ < 40% for 300 s)

Its dynamic adjustment formula:

$${\alpha }_{\mathrm{life}}\left(t\right) = {\alpha }_{\mathrm{life}}^{\mathrm{base}} \left(1 + {\kappa }_1 \left|\mathit{SOC}\left(t\right) - 0.6\right| + {\kappa }_2 {\displaystyle \frac{\left|I_{\mathrm{Batt}}\left(t\right)\right|}{I_{\max }}}\right) \left(1 + {\displaystyle \frac{t_{\mathrm{offset}}}{T_{\mathrm{th}}}}\right)$$

(12)

where ${\alpha }_{\mathrm{life}}^{\mathrm{base}}$ represents the durability base value weight, κ₁ and κ₂ are the gain coefficients for $SOC$ deviation and current penalty, and $T_{\mathrm{th}}$ denotes the time threshold for triggering compensation (e.g., $T_{\mathrm{th}}$ = 300 s).

(d)

Prediction-driven constraint design

To enhance the strategy’s foresight, dynamic constraints are imposed based on predicted power demand sequences: the fuel cell power limit, with specific settings as shown in the formula:

$$\left\{\begin{array}{c}{P}_{\mathrm{FC},\min } = \underset{i=1,\dots ,k}{\min }{P}_{\mathrm{req},\mathrm{pred}}\left(t + i\right) 0.4\\ P_{\mathrm{FC},\max } = \underset{i=1,\dots ,k}{\max }{P}_{\mathrm{req},\mathrm{pred}}\left(t + i\right) 1.2\end{array}\right.$$

(13)

Ensure the fuel cell output covers future demand with a 20% margin. The charge–discharge current constraints are given by the following formula.

$$I_{\mathrm{Batt}}\left(t\right)\in \left[-\min \left(I_{\max },{\displaystyle \frac{{\sum }_{i=1}^k{P}_{\mathrm{req},\mathrm{pred}}\left(t+i\right)}{V_{\mathrm{Batt}}}}\right), I_{\max }\right]$$

(14)

The current limit is relaxed dynamically according to the forecast of future demand to avoid short-time overload.

Table 4 compares the performance of the DHAMOF framework with fixed-weight strategies under the CLTC-P and WLTP hybrid driving cycle.

Table 4. DHAMOF vs. fixed-weight strategy performance.

Performance Index	Fixed Weight Policy	DHAMOF Frame	Increase Ratio
Average hydrogen consumption (g/100 km)	598.7	531.2	11.3%
$SOC$ wave range (%)	26.8	18.5	30.9%
Power tracking error (kW)	4.7	2.1	55.3%
Battery current limit exceeded times	15	3	80.0%

The dynamic weight adjustment and predictive constraints significantly improve the overall performance of the strategy, especially under high dynamic conditions, where the fuel cell efficiency and battery life protection are optimized simultaneously.

From a control theory perspective, the DHAMOF framework fundamentally constructs an approximate analytical solution space for time-varying multi-objective optimization problems. By incorporating predictive data, the system captures implicit characteristics of future operating conditions within extended state space dimensions, transforming traditional static optimization into a dynamic game process. This mechanism mirrors human drivers’ decision-making: experienced drivers not only respond to current road conditions but also preemptively adjust throttle and braking strategies by anticipating road surface undulations and traffic signal changes, achieving optimal balance between fuel efficiency and driving comfort. Within this framework, the predictive model serves as the agent’s “pre-sensor,” while DRL simulates the brain’s “decision center.” Through adaptive weight adjustment mechanism, both components work in close coordination to achieve global optimization. Notably, although the DHAMOF framework demonstrates significant simulation advantages, its engineering implementation still faces real-time challenges in onboard computing units. Future research could explore edge-cloud collaborative architectures, transferring high-load predictive model computations to the cloud while retaining DRL’s real-time decision-making capabilities in onboard terminals, thereby achieving optimal performance trade-offs under limited resources.

3.3. Real-Time Control and Dynamic Adjustment

To achieve real-time energy management in fuel cell hybrid systems, this paper proposes a predictive-driven dynamic closed-loop control architecture. By deeply integrating DRL strategies with online prediction model updates, it establishes a “perception-decision-correction” real-time control loop. The architecture ensures system stability and efficiency under complex operating conditions through dynamic priority adjustment, rolling optimization, and residual compensation. The following sections elaborate on three key aspects: control flow, dynamic parameter correction, and real-time performance assurance.

Real-time control consists of two core modules (Figure 3): feedforward prediction compensation and feedback dynamic correction, forming a closed-loop optimization system. The feedforward module predicts future power demand using LSTM-LSSVM-CC to generate initial power allocation commands. The feedback module continuously adjusts prediction errors by incorporating real-time operational data (e.g., $SOC$, fuel efficiency) and modifies DRL policy actions accordingly. The high-speed control loop (50 ms cycle) executes DRL actions and power allocation, while the low-speed update loop (1 s cycle) performs real-time updates to prediction model weights and DRL policy parameters.

Figure 3. Real-time control loop architecture.

To mitigate the cumulative effect of prediction errors on DRL strategies, a window-rolling optimization method is proposed. The initial action a(t) is generated from the current prediction sequence {$P_{\mathit{req},\mathit{pred}}(t + 1)$, …, $P_{\mathrm{req},\mathrm{pred}}\left(t + k\right)$}, and the local optimization problem is recalculated at each $\mathsf{\Delta }t$ time step (t = 0.5) using the latest prediction data. The detailed process is illustrated in the formula.

$$a_{\mathrm{new}}=\arg \underset{a}{\max }\left(R\left(t\right)+\gamma {\displaystyle \sum _{i=1}^k}R\left(t+i\right)\right)$$

(15)

Then perform incremental correction on the initial motion:

$$a\left(t\right) \leftarrow a\left(t\right) + \lambda \left(a_{\mathrm{new}} - a\left(t\right)\right)$$

(16)

λ is the smoothing factor, which inhibits the frequent action jump.

To compensate for the prediction residual $\mathsf{\Delta }{P}_{\mathrm{req}} ={ P}_{\mathrm{req},\mathrm{real}} - P_{\mathrm{req},\mathrm{pred}}$, the compensation signal $P_{\mathrm{comp}}\left(t\right) = K_p \mathsf{\Delta }{P}_{\mathrm{req}}\left(t\right) + K_i {\sum }_{j=1}^m\mathsf{\Delta }{P}_{\mathrm{req}}\left(t - j\right)$ is constructed. Here, the proportional gain $K_p$ targets the fast-response instantaneous residual, while the integral gain $K_i$ eliminates the steady-state residual. This compensation signal is directly superimposed onto the DRL output action to form the final power command:

$$P_{\mathrm{FC}, \mathrm{final}}\left(t\right) = P_{\mathrm{FC}, \mathrm{DRL}}\left(t\right) + P_{\mathrm{comp}}\left(t\right)$$

(17)

The DRL strategy handles global optimization, while the compensation module focuses on local error correction to prevent frequent policy adjustments due to prediction errors, thereby enhancing system stability. To avoid compensation overshoot and adapt to varying operating conditions, a dynamic gain mechanism is introduced, as shown in the following equation:

$$\left\{\begin{array}{l}{K}_p\left(t\right) = K_p^{\mathrm{base}} \left(1 - e^{-{\mathsf{\sigma }}_{P_{\mathrm{pred}}}/2}\right)\\ K_i\left(t\right) = K_i^{\mathrm{base}} {\displaystyle \frac{1}{1 + \left|\sum \mathsf{\Delta }{P}_{\mathrm{req}}\right|}}\end{array}\right.$$

(18)

In such cases, when predicting high volatility (${\mathsf{\sigma }}_{P_{\mathrm{pred}}}\uparrow$), reduce $K_p$ to suppress high-frequency noise interference; when the residual accumulation is large ($\left|\sum \mathsf{\Delta }{P}_{\mathrm{req}}\right|\uparrow$), reduce $K_i$ to avoid integration saturation.

4. Algorithm Convergence and Robustness Analysis

4.1. Convergence Analysis

This paper conducts a comprehensive evaluation of the convergence characteristics of the proposed strategy through multidimensional experiments, assessing training stability, multi-scenario adaptability, and robustness boundaries. The analysis reveals the convergence behavior and optimization potential of the Prediction-Embedded DRL (PE-DRL) algorithm within complex dynamic systems.

Under CLTC-P and WLTP hybrid driving conditions, key performance indicators of the PE-DRL algorithm were recorded over 2000 training steps, as illustrated in Figure 4.

Figure 4. Convergence metrics during training.

(a)

Critic network loss function (mean squared TD error):

$$L_{\mathrm{critic}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\left(y_i - Q\left(s_i,a_i|{\theta }^Q\right)\right)}^2$$

(19)

(b)

The dynamic decay of policy entropy is given by the following formula

$$H\left(\pi \right)= -{\displaystyle \sum _a}\pi \left(a|s\right)\log \pi \left(a|s\right)$$

(20)

Convergence progress of PE-DRL versus traditional DDPG under a fixed 2000-step training budget (Table 5):

Table 5. PE-DRL and Traditional DDPG.

Index	DDPG	PE-DRL	Amplitude of Variation
Final Critic loss	0.32	0.15	53.1%
End value of strategy entropy	1.5	0.7	53.3%
Average reward (normalized)	0.68	0.89	30.9%
Power tracking error (kW)	2.8	1.3	53.6%

With the same training steps, PE-DRL demonstrates significantly better convergence depth and policy stability than traditional methods, validating its rapid convergence advantage.

Simulation results demonstrate that PE-DRL achieves rapid and stable convergence within a 2000-step training budget through its predictive embedding and dynamic reward coordination mechanism. The system’s core advantages include reducing invalid action sampling rates to accelerate policy gradient optimization, while its residual compensation and noise suppression modules enhance tolerance to disturbances and prediction errors. The lightweight network architecture combined with priority-based experience replay maximizes learning efficiency within the limited step budget.

4.2. Robustness Analysis

To validate the reliability of the predictive embedding PE-DRL algorithm in complex uncertain environments, this section conducts a robustness evaluation centered on multidimensional disturbance experiments. The assessment covers three dimensions: prediction error tolerance, sensor noise suppression, and communication latency. By comparing it with the DDPG strategy, the study reveals the algorithm’s stability boundaries and failure modes in practical engineering scenarios.

There are three core evaluation indicators in the robustness evaluation index:

(a)

Power tracking error ($e_P = \left|P_{\mathrm{req}} - P_{\mathrm{out}}\right|$): A metric that reflects the system’s responsiveness to sudden dynamic load changes.

(b)

The $SOC$ fluctuation range ($\mathsf{\Delta }\mathit{SOC} = {\mathit{SOC}}_{\mathit{max}} - {\mathit{SOC}}_{\mathit{min}}$) indicates the charge–discharge stability of the energy storage system.

(c)

The increment of hydrogen consumption quantifies the impact of disturbance on economy.

The experimental disturbance scenario design is shown in Table 6.

Table 6. Robustness test scenarios.

Perturbation Type	Perturbation Form	Test Purpose
Failure of forecasting model	Injecting Gaussian noise into the predicted value ($\mathsf{\sigma } = 0.3P_{\mathrm{nom}}$)	Prediction error tolerance
Sensor noise	$SOC$ and power sensor add ±5% random error	Robustness of state estimation
Communication delay	Status signal delay 200 ms	Control Stability in Time-delayed Environment

Performance analysis under multi-perturbation scenarios:

(a)

Failure scenario of forecasting model

The disturbance intensity was set to Gaussian noise with a standard deviation of $\mathsf{\sigma } = 0.3P_{\mathrm{nom}}$ ($P_{\mathrm{nom}} = 1 \mathrm{kW}$) for the predicted power addition; specific performance metrics are shown in Table 7.

Table 7. Performance under prediction model failure.

Norm	DDPG	PE-DRL
Power tracking error (kW)	0.47	0.18
$SOC$ fluctuation, (%)	31.2	19.6
Increment of hydrogen consumption (g/100 km)	+24.6	+8.3

The PE-DRL model dynamically corrects prediction deviations through its residual compensation module ($P_{\mathrm{comp}}$), effectively mitigating the direct interference of outliers on DRL strategies.

• (b) Sensor noise scene

The $SOC$ measurement value is added with ±5% random noise, and the power sensor noise bandwidth is 2 kHz. Specific performance comparison metrics are shown in Table 8.

Table 8. Performance under sensor noise.

Norm	DDPG	PE-DRL	Amplitude of Variation
$SOC$ estimation error (%)	6.8	2.1	69.1%
Fluctuation of fuel cell load (%)	25.4	10.7	57.9%

It is observed that PE-DRL incorporates a sliding average filter layer into the Critic network to mitigate the impact of high-frequency noise on Q-value estimation.

• (c) Communication delay scenario

To simulate real-world vehicular network communication delays, a 200 ms fixed delay is injected into the DRL control loop for critical state signals ($\mathit{SOC}, P_{\mathrm{req}}$). A delay queue is inserted in the state observation channel to ensure that the current values of $\mathit{SOC}$ and $P_{\mathrm{req}}$ are historical state data ($\mathit{SOC}\left(t\right) = \mathit{SOC}(t - 200 \mathrm{ms}), \mathit{Preq}\left(t\right) = \mathit{Preq}(t - 200 \mathrm{ms})$). The test scenarios include frequent start–stop operations under CLTC-P conditions and power spikes under WLTP high-speed conditions, to validate the impact of delays on control strategies.

The robustness of the collaborative optimization framework is evaluated by comparing key metrics of PE-DRL with traditional DDPG algorithms in latency scenarios (results shown in Table 9).

Table 9. Performance under communication delay.

Norm	DDPG (Delay 200 ms)	PE-DRL (Delay 200 ms)
Power tracking error (kW)	3.2	1.1
SOC wave range (%)	34.7	31.3
Increment of hydrogen consumption (g/100 km)	+38.9	+22.5
Fuel cell load jump count	27	9

Table 9 demonstrates that in scenarios with 200 ms vehicle network communication latency, the PE-DRL algorithm outperforms traditional DDPG in terms of robustness and control performance. Specifically, PE-DRL achieves a power tracking error of only 1.1 kW (65.6% lower than DDPG), reduces the $SOC$ fluctuation range to 31.3% (3.4% reduction), and minimizes the hydrogen consumption increment to +22.5 g/100 km (42% lower than DDPG). These results indicate that PE-DRL effectively mitigates control lag caused by latency through its collaborative optimization framework’s delay compensation mechanisms (e.g., historical state prediction and multi-objective dynamic balancing). Under hybrid operating conditions, it combines power response accuracy, energy management stability, and economic advantages, providing enhanced adaptability for energy management strategy deployment in real-world vehicle communication latency environments.

5. Conclusions

This study proposes a novel collaborative optimization framework for fuel cell hybrid electric vehicles, providing a new perspective by restructuring the interaction between prediction and decision-making. Distinct from traditional open-loop or simple serial approaches, our primary innovation lies in a “Prediction–Decision–Execution–Feedback” closed-loop architecture is developed. This framework integrates a Cascade-Corrected Hybrid Power Prediction Model (LSTM-LSSVM-CC) with an online residual correction mechanism to suppress long-horizon error accumulation at the source. We further introduce a Dynamic Spatio-Temporal Fusion state space that incorporates future power sequences with dynamic confidence weighting, endowing the agent with genuine “forward-looking” decision-making capabilities. Complementing this is the Dynamic Hierarchical Adaptive Multi-Objective Optimization Framework, which enables millisecond-level adaptive prioritization of economy, durability, and responsiveness based on real-time operating conditions. Coupled with a dual compensation mechanism of Receding Horizon Optimization and PI residual correction, the proposed strategy demonstrates exceptional robustness against complex disturbances. Simulation results confirm that this approach not only achieves an 11.3% reduction in hydrogen consumption, a 30.9% decrease in $SOC$ fluctuations, and a 55.3% improvement in power tracking accuracy, but also fundamentally resolves the control failure issues caused by prediction error propagation, offering a new theoretical paradigm and technical pathway for intelligent energy management in fuel cell systems.