Hierarchical Energy Management for Fuel Cell Electric Vehicles with Adaptive-Modality Deep Deterministic Policy Gradient

Abstract

Fuel cell electric vehicles (FCEVs) require energy management strategies that can balance hydrogen economy, battery utilization, component protection, and real-time control under varying driving conditions. This paper proposes an Adaptive-Modality Deep Deterministic Policy Gradient and Model Predictive Control hierarchical energy management strategy (AMDDPG–MPC HEMS). In the proposed architecture, the upper-level AMDDPG controller identifies driving-condition patterns and generates adaptive weights for hydrogen consumption, battery power, and state-of-charge regulation, while the lower-level MPC controller performs constrained power allocation between the fuel cell and battery. To improve adaptability, the AMDDPG algorithm incorporates an adaptive modality perception mechanism that extracts driving-condition features and a multi-scale reward mechanism that coordinates short-term energy-saving objectives with long-term component-protection requirements. A dedicated weight-scheduling and switching mechanism is also introduced to ensure smooth transitions between operating conditions. The proposed strategy is evaluated under the World Light Vehicle Test Cycle and Urban Dynamometer Driving Schedule and compared with rule-based, equivalent consumption minimization, and fixed-weight MPC strategies. The results show that the AMDDPG–MPC HEMS achieves the lowest equivalent hydrogen consumption, with reductions of 18.853% and 11.732% relative to the rule-based strategy under the two driving cycles, respectively. It also improves fuel-cell operating efficiency and maintains feasible battery SOC regulation. These results demonstrate the effectiveness and engineering potential of the proposed hierarchical energy management strategy.

1. Introduction

Fuel cell electric vehicles (FCEVs), characterized by zero tailpipe emissions, short hydrogen-refueling time, and long driving range, are regarded as a promising solution for sustainable transportation [1]. The energy management strategy (EMS) is a key element of the FCEV powertrain because it directly affects hydrogen consumption, operational safety, and the service life of powertrain components [2]. However, rapidly varying power demand and uncertain driving conditions make it difficult for conventional EMSs to maintain consistently efficient and stable performance. Therefore, an adaptive EMS capable of coordinating energy efficiency, component protection, and real-time control under diverse driving conditions is required [3,4].

Research on FCEV energy management strategies (EMS) targets efficient, safe power allocation under complex, dynamic conditions. EMS approaches are commonly grouped into rule-based, optimization-based, and learning-based methods [5,6,7]. Rule-based strategies (e.g., logic-threshold control) offer simplicity, low computational cost, and fast real-time response [8,9,10] and were pivotal in early applications; however, their manually tuned parameters lack adaptability to time-varying conditions and component aging, limiting global optimality and often reducing control precision. Optimization-based energy management strategies can be categorized into global and instantaneous optimization approaches, both of which generally deliver superior fuel-economy performance compared with rule-based strategies. Global optimization methods mainly include dynamic programming (DP), Pontryagin’s Minimum Principle (PMP), and various heuristic algorithms such as Genetic Algorithms (GA) and Particle Swarm Optimization (PSO) [11,12,13,14]. These methods determine the optimal control sequence based on complete or predefined driving-cycle information. However, they typically require full-cycle previews or accurate forecasts and entail heavy computational burdens, which makes them sensitive to modeling/forecast errors and difficult to deploy online across highly variable conditions [15]. Instantaneous optimization methods—such as the Equivalent Consumption Minimization Strategy (ECMS) and Model Predictive Control (MPC)—exhibit satisfactory control performance and strong potential for real-time applications [16,17]. ECMS minimizes instantaneous equivalent fuel consumption by introducing an equivalence factor that converts battery energy consumption into equivalent hydrogen consumption [18]. However, its performance strongly depends on the calibration of this factor, and a fixed value cannot readily adapt to varying driving conditions. MPC, on the other hand, generates optimal control sequences by explicitly considering predicted state evolution within a moving time horizon, showing high robustness under various operating scenarios. Learning-based energy management strategies utilize artificial intelligence and machine learning techniques to optimize power allocation [19]. By learning from historical data and predicting system behavior and external conditions, these strategies can automatically adapt control policies to improve efficiency and overall performance. Learning-based approaches are generally divided into supervised-learning-based and reinforcement-learning-based EMS [20]. In supervised learning, each training sample consists of input–output pairs, and the objective is to establish a mapping function from inputs to outputs that enables accurate prediction of unknown states. Supervised-learning-based EMSs establish input–output mappings from labeled training data, but their performance depends strongly on the availability and representativeness of high-quality datasets. Such datasets are difficult to obtain comprehensively for real-world FCEV operating conditions. In contrast, reinforcement learning (RL) enables an agent to learn control policies through trial-and-error interaction with the environment, without requiring explicitly labeled input–output pairs, and seeks to maximize long-term cumulative rewards [21]. The core principle of RL resembles dynamic programming in that both rely on Bellman’s optimality equation and the principle of optimal substructure [22,23,24]. However, unlike DP—which assumes complete environmental knowledge—RL acquires information through interaction and is therefore often referred to as adaptive dynamic programming. In summary, no single paradigm can simultaneously satisfy the fourfold requirements of real-time performance, feasibility, cross-scenario adaptability, and long-term durability [25,26]. MPC provides a constraint-aware framework for predictive power allocation and has strong potential for real-time implementation. However, its control performance depends strongly on the weighting coefficients assigned to hydrogen consumption, battery utilization, SOC regulation, and control smoothness. Fixed weights are generally unable to maintain an appropriate balance among these objectives under rapidly changing driving conditions. Therefore, adaptive weight scheduling is necessary to improve the cross-condition performance of MPC-based FCEV energy management.

To address the optimization of MPC for FCEV energy management, various strategies have been proposed, including feedforward control, feedback control, and weight optimization. Among these, weight optimization can be effectively approached using deep reinforcement learning (DRL) strategies. DRL in this context encompasses several categories: value-based algorithms, policy-gradient algorithms, and Actor–Critic-based algorithms. Value-based algorithms (e.g., DQN and its variants) derive decision policies by learning the state–action value function. These methods are mature and easy to implement in discrete control scenarios [27]. However, since the continuous power-output space of FCEVs must be discretized, such methods often lose control precision, making it difficult to meet fine-grained energy-management requirements. Policy-gradient algorithms (e.g., REINFORCE, A2C) directly optimize the policy function to maximize cumulative returns and can naturally handle continuous action spaces, but they suffer from high variance, unstable convergence, and a tendency to fall into local optima [28]. Actor–Critic-based algorithms (e.g., DDPG, TD3, SAC) use an Actor network to generate actions and a Critic network to evaluate their value, combining the stability of value-function methods with the continuous-control capability of policy-gradient methods [29,30,31,32,33]. These approaches have become the mainstream research direction [34,35]. Unlike rule-based strategies, which depend heavily on empirical calibration, and optimization-based strategies, which rely on precise models, DRL methods reformulate the energy-management problem as a Markov Decision Process (MDP). By leveraging deep neural networks to approximate the policy or value function, DRL autonomously learns power-distribution patterns from large-scale driving data during offline training—without requiring explicit derivation of the system dynamics [36]. This property effectively mitigates the model-dependency and prediction-reliance issues of MPC, providing intrinsic adaptability to handle component aging and dynamic disturbances [37]. Specifically, the Deep Deterministic Policy Gradient (DDPG) algorithm, as a representative Actor–Critic method, is well suited to continuous decision-making problems and can therefore be used to generate continuously varying optimization weights for FCEV energy management [38]. MPC- and DRL-based controllers can be integrated using either centralized or hierarchical architectures [39]. A centralized architecture jointly processes all control decisions, but its computational burden and implementation complexity increase rapidly with the number of states, actions, and component constraints. A hierarchical architecture separates strategic decision-making from real-time execution, thereby improving modularity, scalability, and controller interpretability. This structure is particularly suitable for the present study, because the upper-level DRL agent can focus on adaptive weight generation while the lower-level MPC controller handles constrained power allocation.

In this context, this paper proposes an Adaptive-Modality Deep Deterministic Policy Gradient and Model Predictive Control hierarchical energy management strategy (AMDDPG–MPC HEMS) for FCEVs. The proposed strategy employs a two-level architecture in which the upper-level AMDDPG controller generates adaptive multi-objective weights, while the lower-level MPC controller performs constrained power allocation between the fuel cell and battery. The AMDDPG algorithm incorporates an adaptive modality perception mechanism to identify driving-condition patterns and a multi-scale optimization mechanism to coordinate short-term energy economy with long-term component-protection objectives. In addition, a dedicated weight-scheduling and switching mechanism is introduced to ensure smooth weight transitions under varying operating conditions. The effectiveness of the proposed strategy is evaluated under WLTC and UDDS driving cycles through comparisons with rule-based, ECMS, and fixed-weight MPC strategies.

The main contributions of this paper are as follows:

• A two-level AMDDPG–MPC hierarchical architecture is developed to decouple adaptive weight generation from constrained power allocation. The upper-level AMDDPG controller generates the optimization weights, while the lower-level MPC controller allocates power between the fuel cell and battery under physical constraints. This structure improves the modularity and scalability of the energy management system and facilitates real-time implementation.
• A dynamic weight-scheduling and switching mechanism is introduced to update the hydrogen-consumption, battery-power-limitation, and SOC-regulation weights according to the current driving condition and component states. The mechanism enables smooth transitions among different weight combinations and improves the adaptability of the MPC controller compared with fixed-weight strategies.
• An AMDDPG algorithm incorporating adaptive modality perception and multi-scale optimization is proposed. The adaptive modality perception mechanism provides driving-condition information for real-time weight generation, while the multi-scale optimization mechanism balances short-term energy-saving performance and long-term component-protection requirements.

The remainder of this paper is organized as follows. Section 2 presents the FCEV powertrain model and its main components. Section 3 describes the proposed AMDDPG–MPC HEMS, including the hierarchical architecture, MPC formulation, and adaptive weight-generation algorithm. Section 4 evaluates the proposed strategy under representative driving cycles and compares its performance with benchmark EMSs. Section 5 summarizes the main conclusions and outlines future research directions.

2. Modeling

This study considers an FCEV with a hybrid powertrain layout, as shown in Figure 1. The vehicle is driven by a front-axle motor whose torque is transmitted to the front wheels through a differential, while two independent in-wheel motors provide direct propulsion at the rear wheels. The required electrical energy is supplied by a combined energy system consisting of a hydrogen storage tank, a fuel cell system, and a battery pack. The specific parameters of the vehicle are detailed in Table 1.

Figure 1. The schematic of the FCEV configuration.

Table 1. Basic parameters of the studied FCEV.

Vehicle Body	Mass	1860 [kg]
	Tire Rolling Radius	0.35 [m]
Front Motor	Motor type	Permanent-magnet synchronous motor
	Speed	0~14,000 [rpm]
	Torque	−137~137 [Nm]
	Maximum efficiency	93%
Rear Motor	Motor type	Permanent-magnet synchronous in-wheel motor
	Speed	0~10,000 [rpm]
	Torque	−97~97 [Nm]
	Maximum efficiency	92%
Battery	nominal voltage	365 [V]
	capacity	40 [Ah]
	Discharging efficiency	95%
Fuel Cell	Idle Power	2 [kW]
	Maximum Output Power	50 [kW]
	Maximum system efficiency	41%

2.1. Vehicle Dynamic

The traction force generated by the powertrain is used to overcome rolling resistance, grade resistance, aerodynamic drag, and inertial resistance. Using standard SI units, the required longitudinal force at the wheels is expressed as

$$F_{\mathrm{req}}=mgf\cos \alpha +mg\sin \alpha +{\displaystyle \frac{1}{2}}\rho C_{D}A{v}^2+m{\displaystyle \frac{\mathrm{d}v}{\mathrm{d}t}}$$

(1)

According to Equation (2), the vehicle power demand can be shown as

$${\widehat{m}}_{FC}={\displaystyle \int P_{FC}\cdot R_{H_2}}(P_{FC})$$

(2)

where m is the vehicle mass in kg, $g$ is the gravitational acceleration in m/s², $f$ is the rolling-resistance coefficient, and $\alpha$ is the road-slope angle in rad; it is set to zero in the WLTC and UDDS simulations presented in Section 4. $\rho$ is the air density in kg/m, $C_D$ is the aerodynamic drag coefficient, $A$ is the frontal area in m², $v$ is the vehicle speed in m/s, $F_{\mathrm{req}}$ is the required traction force in N, and $P_{\mathrm{req}}$ is the vehicle power demand in W. The power demand formulation used in this study focuses on longitudinal vehicle motion and the corresponding power required from the powertrain, since these terms dominate the energy flow in the considered EMS problem. This simplified representation captures the main effects of traction, acceleration, rolling resistance, aerodynamic drag, and road slope and is therefore suitable for energy management design while avoiding unnecessary model complexity.

2.2. Fuel Cell

Since this study mainly evaluates the control performance of the proposed energy management strategy, environmental effects on the fuel-cell system, such as temperature and humidity variations, are not considered in the model. Figure 2 presents the relationships among fuel-cell output power, system efficiency, and hydrogen-consumption rate. The fuel-cell efficiency curve was adapted from the empirical fuel-cell model reported in Ref. [40]. Based on this curve, the hydrogen-consumption rate RH2(PFC) was obtained and used to calculate the cumulative hydrogen consumption according to Equation (4).

Figure 2. Fuel-cell system efficiency and hydrogen-consumption rate. Redrawn based on Ref. [40].

Theoretically, the hydrogen consumption of the fuel cell system can be calculated as

$$m_{FC}={\displaystyle \frac{1}{LH{V}_{H_2}}}{\displaystyle \int \frac{P_{FC}}{{\eta }_{FC}(P_{FC})}}$$

(3)

where $P_{FC}$ [w] is the fuel cell power, $m_{FC}$ [J] is the hydrogen consumption of the fuel cell system, ${LHV}_{H_2}$ is the hydrogen lower heating value, taken here as 120 MJ/kg, and ${\eta }_{FC}$ is the dimensionless fuel cell system efficiency.

According to the polarization curve in Figure 2, the hydrogen consumption of the fuel cell system can be expressed as

$${\widehat{m}}_{FC}={\displaystyle \int P_{FC}\cdot R_{H_2}}(P_{FC})$$

(4)

where ${\widehat{m}}_{FC}$ [g] is the hydrogen consumption; $R_{H_2}$ [g/s] is the hydrogen consumption rate.

2.3. Battery

Analogous to the fuel-cell modeling approach, a simplified R-C equivalent-circuit model is adopted to describe the main electrical characteristics of the battery while reducing computational complexity. Transient electrochemical and thermal dynamics are not explicitly modeled; therefore, battery temperature is treated as a constant parameter rather than a dynamic state. In all simulations, the battery temperature is fixed at 25 °C. The terminal voltage of the battery is expressed as

$$U_{batt}=E_{batt}-I_{batt}{R}_{batt}$$

(5)

where $U_{batt}$ [V] is the load voltage, $E_{batt}$ [V] is the open-circuit voltage, $R_{batt}$ [Ω] is the internal resistance of the battery, and $I_{batt}$ [A] is the battery current.

The state of charge (SOC) of the battery is shown as

$$SOC=\left\{\begin{array}{l}{SOC}_{init}+{\displaystyle \frac{1}{{\eta }_{batt}{C}_{batt}}}{\displaystyle {\int }_0^t{I}_{batt}dt \mathrm{for} I_{batt}>0 \left(\mathrm{Charging}\right)}\\ {SOC}_{init}-{\displaystyle \frac{{\eta }_{batt}}{C_{batt}}}{\displaystyle {\int }_0^t(-I_{batt})dt \mathrm{for} I_{batt}<0 \left(\mathrm{Discharging}\right)}\end{array}\right.$$

(6)

where $SO{C}_{init}$ is the initial value of battery SOC, dimensionless, and $C_{batt}$ is the battery capacity in Ah. $I_{batt}>0$ means the battery is in the charging phase.

The battery efficiency ${\eta }_{batt}$ is defined as the ratio of battery power to total power, and it can be expressed as

$${\eta }_{batt}=\left\{\begin{array}{l}{\displaystyle \frac{P_{batt}}{P_{batt}+P_{batt\_loss}}}={\displaystyle \frac{U_{batt}{I}_{batt}}{U_{batt}{I}_{batt}+{I^2}_{batt}{R}_{batt}}},P_{batt}<0\\ {\displaystyle \frac{P_{batt}+P_{batt\_loss}}{P_{batt}}}={\displaystyle \frac{U_{batt}{I}_{batt}+{I^2}_{batt}{R}_{batt}}{U_{batt}{I}_{batt}}},P_{batt}>0\end{array}\right.$$

(7)

where $P_{batt}$ [W] is the battery power; $P_{batt\_loss}$ [W] is the battery power loss due to internal resistance. $P_{batt}>0$ means the battery is in the charging phase. The battery internal resistance is obtained from the SOC–temperature resistance maps. Because thermal dynamics are not included, the temperature input is fixed at $T_{\mathrm{ref}}=25$ °C, and the resistance varies only with SOC during each simulation:

$$R_{\mathrm{batt}}=\left\{\begin{array}{ll}{R}_{\mathrm{chg}}(SOC,T_{\mathrm{ref}}),\hfill & I_{\mathrm{batt}}>0,\hfill \\ R_{\mathrm{dis}}(SOC,T_{\mathrm{ref}}),\hfill & I_{\mathrm{batt}}<0.\hfill \end{array}\right.$$

(8)

The resistance characteristic of the battery during charging and discharging is shown in Figure 3.

Figure 3. Presents the complete SOC–temperature resistance maps, whereas only the 25 °C slices of these maps are used in the simulations.

2.4. Motor

The studied FCEV employs a front-axle permanent-magnet synchronous motor (PMSM) and two rear permanent-magnet synchronous in-wheel motors. PMSMs offer high efficiency, high power density, fast torque response, and good regenerative-braking capability, making them suitable for electric-vehicle propulsion. The rear in-wheel motors provide direct drive and flexible wheel-level torque distribution while reducing mechanical transmission losses. To reduce computational complexity, the motor thermal and transient electromagnetic dynamics are neglected, and the motor efficiencies are obtained from torque–speed lookup maps. The electrical power of the front and rear motors is calculated as follows:

$$P_{elec\_m}=\left\{\begin{array}{c}{\displaystyle \frac{T_m{\omega }_m}{{\eta }_m}},T_m>0\\ T_m{\omega }_m{\eta }_{gen},T_m\le 0\end{array}\right.$$

(9)

where $P_{elec\_m}$ denotes the motor electrical power, $T_m$ is the motor torque, ${\omega }_m$ is the motor rotational speed, ${\eta }_m$ represents the motor efficiency in traction mode, and ${\eta }_{gen}$ represents the efficiency in regenerative mode. When $T_m>0$, the motor operates in traction mode; otherwise, it works in regenerative braking mode.

In this study, motor efficiency is obtained from a two-dimensional look-up table indexed by torque and speed. The efficiency relationship is expressed as

$${\eta }_m=\eta \left(T_m,{\omega }_m\right)$$

(10)

The steady-state efficiency maps of the front PMSM and rear in-wheel PMSMs are shown in Figure 4.

Figure 4. Motor efficiency map.

3. Hierarchical Energy Management Strategy

To endow the energy management strategy (EMS) of fuel cell electric vehicles (FCEVs) with fast processing capability, robustness, and engineering application potential, this study proposes a hierarchical energy management strategy (AMDDPG–MPC HEMS) integrating Adaptive-Modality Deep Deterministic Policy Gradient (AMDDPG) and Model Predictive Control (MPC). The overall workflow of the proposed strategy can be summarized as driving condition feature extraction and power allocation optimization. A novel fast computation algorithm is adopted in the EMS to realize energy management based on predefined logic for driving condition identification. Figure 5 presents the implementation framework of the AMDDPG–MPC HEMS. This hierarchical control architecture is designed to enhance adaptability to diverse driving conditions and improve control performance. The upper level integrates driving mode perception and weight generation logic, processes driver demand and component states, and then generates adaptive optimization weights according to the current driving condition. The lower level serves as an advanced executive control layer, processes the observed driver demand and component states, and determines the power allocation in the powertrain through the integrated MPC method. Figure 6 illustrates the logic flow of weight generation and optimization.

Figure 5. Schematic diagram of the AMDDPG–MPC HEMS.

Figure 6. Logic flow of weight generation and optimization.

To improve real-time performance, the MPC implementation incorporates offline training strategies and online inference calls, significantly reducing online computational requirements. To further enhance the control effect of the MPC-based energy management strategy, a multi-dimensional weight generation process is developed in this study, which adaptively adjusts according to the state of charge (SOC), vehicle power demand, and prediction horizon length to cope with different driving scenarios.

3.1. Hierarchical Control Architecture

Given variable driving conditions, powertrain component interactions strongly influence FCEVs’ energy efficiency and service life. To maximize powertrain energy-saving potential and optimal energy management, a hierarchical control architecture is critical. This architecture decomposes the control strategy, simplifies the problem, and enables precomputation of strategic decisions, alleviating real-time MPC’s computational load. Consequently, it accelerates and optimizes power allocation, while boosting MPC adaptability via real-time updates of target weights and constraints. It allows MPC to focus on specific operational goals, yielding precise control and enhanced vehicle performance.

3.1.1. Upper-Level Controller

To optimize the adaptability and control performance of the AMDDPG–MPC HEMS under dynamic driving conditions, the upper-level controller defines specific operating modes of FCEVs (steady cruising, acceleration, and congestion modes) and presets engineering threshold parameters offline to assist in mode identification and boundary constraints (these thresholds ensure stability without participating in training). It categorizes modes based on energy source activation states and driving condition characteristics: in steady cruising mode, the fuel cell meets power demand to minimize hydrogen consumption; in acceleration mode, the fuel cell and battery coordinate to respond to transient loads; and in congestion mode, the battery acts as a buffer to reduce fuel cell start–stop losses. Simultaneously, the upper level outputs a dynamic weight vector $w_t={[w_{1,t},w_{2,t},w_{3,t}]}^T$ (hydrogen consumption weight $w_{1,t}$, battery power limitation weight $w_{2,t}$, and SOC balance weight $w_{3,t}$) in real time, transmitting it to the lower level via the CAN bus every 100 ms.

Figure 5 illustrates the control algorithms assigned to different operating modes within the hierarchical framework, whereas the adaptive adjustment of optimization weights is further detailed through the weight-generation logic. Since mode transitions in practical driving are continuous and coupled with powertrain states, the upper-level controller must provide a clear rule for updating the MPC weights. In the proposed AMDDPG–MPC HEMS, this process is carried out by jointly considering driver power demand and key component states. Specifically, vehicle power demand and speed are used to identify the current driving mode in real time, while SOC, battery power, and fuel-cell power provide additional constraints for weight adjustment. Based on these variables, the upper layer generates matched optimization weights for the lower-level MPC, as shown in Figure 6, thereby improving energy allocation efficiency and supporting component durability.

In Figure 6, $w_{1,t}$, $w_{2,t}$ and $w_{3,t}$ denote the hydrogen consumption weight threshold, battery power limitation weight threshold, and SOC balance weight threshold, respectively, with values of 0.4, 0.3, and 0.3 as shown in Table 2. These values are derived based on multiple factors, including ensuring the balance between component life and performance, maintaining sufficient reserves under dynamic driving conditions, and achieving efficient energy recovery in various driving scenarios. $v_{th}$ [km/h] is the maximum constrained vehicle speed for mode switching, $P_{dem,th}$ [kW] is the power demand threshold, $P_{bat,th}$ [kW] is the battery power threshold, and $P_{fc,th}$ [kW] is the fuel cell output threshold, with values of 80, 60, 40, and 50, respectively, which are related to the specific energy requirements and operating efficiency under different driving conditions.

Table 2. Upper-level controller threshold parameter.

Parameter Symbol	Description	Value	Unit/Remarks
$w_{1,th}$	Hydrogen consumption weight threshold	0.4	Used for cruising mode switching boundary
$w_{2,th}$	Battery power limitation weight threshold	0.3	Used for acceleration mode response threshold
$w_{3,th}$	SOC balance weight threshold	0.3	Used for congestion mode buffer priority
${\lambda }_{adapt}$	Adaptive adjustment coefficient	0.1	Mode encoding fusion adjustment factor
$v_{th}$	Vehicle speed threshold for mode switching	80	Maximum constrained vehicle speed
$p_{dem,th}$	Power demand threshold	60	Demand trigger point
$p_{bat,th}$	Battery power threshold	40	Charging/discharging boundary
$p_{fc,th}$	Fuel cell output threshold	50	Maximum output limitation

3.1.2. Lower-Level Controller

At the lower level, the MPC determines powertrain power allocation, taking upper-level weights $w_t$, current state $x_t={[SO{C}_t,P_{fc,t}]}^T$, and prediction horizon $N_p=10$ as inputs. It outputs the fuel cell power change rate $\Delta P_{fc}$, further calculating battery power $P_{bat,t+1}=P_{dem,t+1}-{\eta }_{fc}{P}_{fc,t+1}$ and motor power allocation (residual power to front/rear motors) with a 100 ms rolling execution frequency. In specific modes, steady cruising mode relies on the fuel cell to meet main power demand for hydrogen consumption optimization; acceleration mode uses MPC to coordinate battery and fuel cell outputs (detailed in subsequent chapters); and congestion mode operates the fuel cell at its high-efficiency point, where its output serves both buffering and response needs.

Power allocation is determined by optimizing a multi-objective cost function. The optimization problem is formulated as follows:

$$J={\displaystyle {\displaystyle \sum }_{i=1}^{N_p}}\left(w_{1,t}\Delta H_2(k+i)+w_{2,t}|P_b(k+i)|+w_{3,t}|SOC(k+i)-SO{C}_{\mathrm{ref}}|+w_{4,t}{(\Delta P_{fc}(k+i))}^2\right)$$

(11)

where $w_t={[w_{1,t},w_{2,t},w_{3,t}]}^T$ is the dynamic weight vector generated by the upper level, $w_{4,t}$ is an intrinsic control-smoothing weight native to the MPC layer, $\Delta H_2(k+i)$ is the hydrogen consumption rate, $P_b(k+i)$ is the battery power, $SOC(k+i)$ is the battery state of charge, and $\rho$ is the control smoothing coefficient, which can be expressed as

$$\Delta H_2={\displaystyle \frac{P_{fc,k}}{LHV\cdot {\eta }_{fc}}}$$

(12)

$$P_{b,k}=P_{dem,k}-{\eta }_{fc}{P}_{fc,k}$$

(13)

where $P_{fc,k}$ denotes the fuel-cell power variation between two consecutive sampling instants. Equation (13) represents the high-level source-side power balance used in the MPC optimization, in which the fuel cell provides the commanded power and the battery supplies the residual power demand. The lower-level MPC does not independently optimize the power outputs of the front, rear-left, and rear-right motors. Instead, the required traction power is distributed among the three motors according to the predefined motor power-allocation rule, and the corresponding electrical power is subsequently calculated using the individual motor-efficiency maps introduced in Equation (19). Therefore, Equation (13) serves as a computationally efficient abstraction within the MPC cost function, whereas Equation (19) is used in the detailed powertrain model to account for the efficiency differences among the three motors.

The MPC controller is implemented in a receding-horizon manner to ensure smooth and feasible control actions. At each sampling instant, the optimal fuel-cell power variation is calculated, and only the first control action is applied before the prediction horizon is shifted forward. This procedure enables the fuel cell and battery to coordinate under rapidly varying power demand while limiting abrupt fuel-cell power changes.

In Section 4, identical driving-cycle inputs, initial conditions, component constraints, and operating-mode definitions are applied to all benchmark EMSs. This setting ensures consistent simulation conditions and enables a fair comparison of their energy-management performance.

3.2. Energy Management Based on Implicit MPC

In the proposed hierarchical energy management strategy for FCEVs, the lower-level controller adopts implicit MPC for real-time optimal power allocation. Unlike explicit MPC with offline precomputed affine control laws, implicit MPC solves optimization online at each sampling step, using efficient QP solvers to address powertrain nonlinearities and time-varying parameters. It adapts to dynamic weights from the upper-level AMDDPG module, ensuring robust energy distribution across operating conditions. The formulation resolves fuel cell–battery–motor power coupling via a cost function balancing hydrogen consumption, battery degradation, and FC stress, under component physical constraints.

3.2.1. Basic Implicit MPC

Model Predictive Control (MPC) is a receding-horizon optimization framework that computes an optimal sequence of control actions over a prediction horizon by solving a finite-time optimal control problem at each sampling step, while only applying the first control input before shifting the horizon forward. For linear time-invariant (LTI) systems, the core process involves (1) modeling the system dynamics via a discrete-time state-space representation; (2) defining a quadratic cost function to penalize deviations from a reference trajectory and control effort; (3) enforcing input, state, and output constraints; and (4) reformulating the problem as a QP for efficient online solution.

Consider a discrete-time LTI system described by

$$x(t+1)=Ax(t)+Bu(t)$$

(14)

where $x(t)\in R^n$ is the state vector, $u(t)\in R^m$ is the control input vector, and $A$, $B$ are the system matrices. The MPC optimization problem over a prediction horizon $N_p$ and control horizon $N_c$ (with $N_c\le N_p$) is formulated as

$$\underset{U}{\min }{\displaystyle {\displaystyle \sum }_{i=1}^{N_p}}\left(\parallel y(k+i|k)-r(k+i){\parallel }_Q^2+{\displaystyle {\displaystyle \sum }_{j=0}^{N_c-1}}\parallel \Delta \mathrm{u}(k+j|k){\parallel }_R^2\right)$$

(15)

subject to the system dynamics, constraints on states $x\in \mathcal{X}$, inputs $u\in \mathcal{U}$ and outputs $y\in \mathcal{Y}$. Here, $U={[\Delta \mathrm{u}{(k\mid k)}^T,\dots ,\Delta \mathrm{u}{(k+N_c-1\mid k)}^T]}^T$ stacks the control increments, $y(k+i\mid k)$ denotes the predicted output at step $i$ based on current measurements at $k$, $r(k+i)$ is the reference, and $Q$, $R$ are weighting matrices.

By stacking predictions and substituting the state-space model, the problem is recast as a QP:

$$\underset{U}{\min }{\displaystyle \frac{1}{2}}{U}^{T}HU+f^{T}U, GU\le w$$

(16)

where $H$ is the Hessian, $f$ is the linear term, and $GU\le w$ encapsulates the constraints affine in $U$. The optimal $U^{\ast }$ is obtained via QP solvers and only the first element $\Delta \mathrm{u}(k\mid k)$ is applied, yielding the control law $u(k)=u(k-1)+\Delta \mathrm{u}(k|k)$.

This implicit formulation, solved online, contrasts with explicit MPC by avoiding offline region partitioning, thus reducing preprocessing complexity at the expense of mild computational overhead, which is negligible for embedded solvers in automotive applications.

3.2.2. Application of MPC in EMS

In the hierarchical energy management strategy for fuel cell electric vehicles (FCEVs), the Model Predictive Control (MPC) component enables efficient real-time power allocation through online optimization. At each sampling instant, MPC solves a constrained optimization problem leveraging quadratic programming (QP) solvers, which accommodates the dynamic weights from the upper-level Adaptive-Modality Deep Deterministic Policy Gradient (AMDDPG) module while ensuring computational efficiency. The states are selected as the battery state of charge (SOC) and fuel cell output power $P_{fc}$, with the control input defined as the fuel cell power rate $\Delta P_{fc}$, facilitating gradual power adjustments to minimize component degradation. This implementation is grounded in the FCEV powertrain model established in Section 2, balancing real-time adaptability and computational feasibility.

The state-space equations are derived to model energy allocation dynamics. The discrete SOC update, based on battery electrochemistry, is given by

$${\mathrm{SOC}}^{\prime }=-{\displaystyle \frac{E_{\mathrm{batt}}-\sqrt{E_{\mathrm{batt}}^2-4R_{\mathrm{batt}}{P}_{\mathrm{batt}}}}{2R_{\mathrm{batt}}{C}_{\mathrm{batt}}}}$$

(17)

where $E_{\mathrm{batt}}$ is the battery open-circuit voltage, $R_{\mathrm{batt}}$ is the internal resistance, $C_{\mathrm{batt}}$ is the capacitance, and $P_{\mathrm{batt}}=P_b$ is the battery power.

The electrical-mechanical power coupling is

$$\left\{\begin{array}{l}{P}_b+P_{fc}={\displaystyle \frac{P_{mF}}{{\eta }_{mF}}}+{\displaystyle \frac{P_{mRL}}{{\eta }_{mRL}}}+{\displaystyle \frac{P_{mRR}}{{\eta }_{mRR}}}\\ P_{mF}+P_{mRL}+P_{mRR}=P_{\mathrm{req}}\\ P_{\mathrm{elec},\mathrm{req}}=P_{fc}+P_b\end{array}\right.$$

(18)

where $P_{mF}$, $P_{mRL}$, and $P_{mRR}$ are the front, rear-left, and rear-right motor powers, respectively; ${\eta }_{mF}$, ${\eta }_{mRL}$, and ${\eta }_{mRR}$ are their efficiencies; and $P_{\mathrm{req}}$ is the vehicle power demand.

From Equation (18), the battery power is expressed as

$$P_b={\displaystyle \frac{P_{\mathrm{req}}}{{\eta }_{mF}}}+\left({\displaystyle \frac{1}{{\eta }_{mRL}}}-{\displaystyle \frac{1}{{\eta }_{mF}}}\right)P_{mRL}+\left({\displaystyle \frac{1}{{\eta }_{mRR}}}-{\displaystyle \frac{1}{{\eta }_{mF}}}\right)P_{mRR}-P_{fc}$$

(19)

Substituting Equation (19) into Equation (17) yields the coupled SOC dynamics:

$${\mathrm{SOC}}^{\prime }=-{\displaystyle \frac{E_{\mathrm{batt}}-\sqrt{E_{\mathrm{batt}}^2-4R_{\mathrm{batt}}\left[\frac{P_{\mathrm{req}}}{{\eta }_{mF}}+\left(\frac{1}{{\eta }_{mRL}}-\frac{1}{{\eta }_{mF}}\right)P_{mRL}+\left(\frac{1}{{\eta }_{mRR}}-\frac{1}{{\eta }_{mF}}\right)P_{mRR}-P_{fc}\right]}}{2R_{\mathrm{batt}}{C}_{\mathrm{batt}}}}$$

(20)

Discretizing these with sampling time $T_s$ using zero-order hold, and incorporating fuel cell first-order dynamics,

$${\dot{P}}_{fc}=-{\displaystyle \frac{1}{{\tau }_{fc}}}(P_{fc}-P_{fc,ref})$$

(21)

with ${\tau }_{fc}$ as the time constant results in the linear time-varying (LTV) state-space model:

$$x(k+1)=Ax(k)+B\Delta P_{fc}(k)$$

(22)

where $x={[\mathrm{SOC},P_{fc}]}^T$

$$A=\left[\begin{array}{cc}1& -{\displaystyle \frac{T_s{\eta }_m}{V_b{Q}_b}}\\ 0& 1-{\displaystyle \frac{T_s}{{\tau }_{fc}}}\end{array}\right], B=\left[\begin{array}{c}0\\ {\displaystyle \frac{T_s}{{\tau }_{fc}}}\end{array}\right]$$

(23)

This linear time-invariant (LTI) approximation, linearized around nominal operating points, incorporates measurable disturbances like $P_{dem}$ and ${\eta }_m$.

The finite-horizon optimal control problem is formulated with a quadratic cost:

$$J={\displaystyle {\displaystyle \sum }_{i=1}^{N_p}}\left(w_{1,t}\Delta H_2(k+i)+w_{2,t}|P_b(k+i)|+w_{3,t}|SOC(k+i)-SO{C}_{\mathrm{ref}}|+w_{4,t}{(\Delta P_{fc}(k+i))}^2\right)$$

(24)

where weights $w_t={[w_{1,t},w_{2,t},w_{3,t}]}^T$ are parameterized offline based on AMDDPG mappings ($w_1,w_2,w_3\ge 0$ with $w_1+w_2+w_3=1$) and $w_4$ is an intrinsic control-smoothing weight native to the MPC layer ($w_4>0$); it enforces physically feasible and smooth control actions. $\Delta H_2(k+i)={\displaystyle \frac{P_{fc}(k+i)T_s}{{\eta }_{fc}LHV}}$ quantifies hydrogen consumption at time step $k+i$, $P_b(k+i)$ penalizes the magnitude of battery power to mitigate degradation from extreme charging/discharging, $SOC(k+i)$ is the battery State of Charge, and ${SOC}_{\mathrm{ref}}=0.5$ is the reference SOC selected to balance battery longevity and buffer capacity; $\Delta P_{fc}(k+i)=P_{fc }(k+i)-P_{fc }(k+i-1)$ represents the fuel cell power variation between consecutive time steps.

To safeguard component integrity and vehicle performance, the optimization incorporates box and linear constraints on states, controls, and derived variables, ensuring feasible operation within physical limits. These constraints, affine in the control sequence $U$, are expressed as

$$\left\{\begin{array}{l}SO{C}_{\min }\le SOC(k+i)\le SO{C}_{\max }\\ P_{fc,\min }\le P_{fc}(k+i)\le P_{fc,\max }\\ \Delta P_{fc,\min }\le \Delta P_{fc}(k+j)\le \Delta P_{fc,\max }\\ P_{b,\min }\le P_b(k+i)\le P_{b,\max }\\ P_{dem,\min }\le P_{dem}(k+i)\le P_{dem,\max }\\ {\eta }_{m,\min }\le {\eta }_m(k+i)\le {\eta }_{m,\max }\end{array}\right., i=1,\dots ,N_p;j=0,\dots ,N_c-1$$

(25)

where $SO{C}_{\min /\max }$ prevents overcharge/discharge; $P_{fc,\min /\max }$ bounds fuel cell output; $\Delta P_{fc,\min /\max }$ limits slew rates for durability; $P_{b,\min /\max }$ reflects battery ratings; and $P_{dem,\min /\max }$ enforces demand and efficiency bounds. Voltage and resistance constraints are embedded via $P_b=V_b{I}_b-R_b{I}_b^2$, linearized around nominals.

Stacking predictions over horizons $N_p$ and $N_c$ reformulates $J$ as an mp-QP:

$$\underset{U}{\min }{\displaystyle \frac{1}{2}}{U}^{T}HU+f^{T}U, GU\le w$$

(26)

where $U={[\Delta P_{fc}{(k\mid k)}^T,\dots ,\Delta P_{fc}{(k+N_c-1|k)}^T]}^T$ stacks the control increments; $H,f,G,w$ are derived from $A$, $B$, the weights and constraints. The resulting optimization is solved online using a QP solver. The optimal $U^{\ast }$ yields $\Delta P_{fc}(k|k)$ for the $P_{fc}$ update, and the residual battery power is $P_b(k)=P_{dem}(k)-{\eta }_m(k)P_{fc}(k)$.

This MPC integration with AMDDPG enables adaptive, predictive control, surpassing static methods by anticipating demands and handling uncertainties, thus enhancing FCEV efficiency, durability, and robustness to extremes.

3.3. Design and Implementation of the AMDDPG Algorithm

As the core weight generator in the hierarchical architecture, the Adaptive-Modality Deep Deterministic Policy Gradient (AMDDPG) learns the coupling between driving condition features and component states offline and outputs online real-time adaptive MPC multi-objective weights ($w_1$—hydrogen consumption; $w_2$—battery power limitation; $w_3$—SOC balance). This section details its framework, offline training, and online application, highlighting innovations in adaptive mode perception and multi-scale optimization versus traditional DDPG.

3.3.1. Basic Formulas and Framework of the Algorithm

The AMDDPG is extended based on the traditional Deep Deterministic Policy Gradient (DDPG) framework, incorporating an adaptive mode perception module and a multi-scale optimization mechanism, and is formalized in the paradigm of the Markov Decision Process (MDP). The core of this framework lies in transforming the weight generation problem into policy optimization in the continuous action space, ensuring robust decision-making response to the dynamic driving conditions of fuel cell electric vehicles (FCEVs). The core formulas and network structure are detailed as follows.

The energy management weight generation problem is modeled as an MDP five-tuple $(S,A,P,R,\gamma )$. The state space $S$ is designed to capture current driving conditions and component states, defined as

$$S=[v,P_{dem} ,SOC,{\dot{P}}_{dem} ,D_{fc },D_{bat} ]$$

(27)

where v denotes vehicle speed (km/h), $P_{dem}$ is the driver’s power demand (kW), $SOC$ is the battery state of charge (0–1), ${\dot{P}}_{dem}$ is the power demand change rate (kW/s), $D_{fc }$ is the fuel cell degradation degree (0–1, with 0 indicating a new state and 1 indicating failure), and $D_{bat}$ is the battery degradation degree (0–1). The action space $A$ corresponds to the output MPC weight vector:

$$w={[w_{1t} ,w_{2t} ,w_{3t} ]}^T$$

(28)

where $w_1$ prioritizes hydrogen consumption reduction, $w_2$ suppresses battery power fluctuations to mitigate degradation, and $w_3$ balances $SOC$ at ${SOC}_{\mathrm{ref}}=0.5$ for optimal buffering and longevity. The state transition probability $P(s_t+1|s_t ,a_t )$ is governed by FCEV powertrain dynamics, while the reward function $R(s_t ,a_t )$ quantifies the immediate benefit of weight decisions, formulated as

$$R(s_t ,a_t )=-(w_1 \cdot \Delta H_2 +w_2 \cdot |P_b|+w_3 \cdot |SOC-SO{C}_{ref}|)+{\lambda }_t\cdot \left(1-\max (D_{fc,t},D_{bat,t})\right)$$

(29)

In this, the hydrogen consumption rate $\Delta H_2$ is calculated as

$$\Delta H_2 =P_{fc} /(LHV\cdot {\eta }_{fc })$$

(30)

where $LHV=120 \mathrm{M}\mathrm{J}/\mathrm{k}\mathrm{g}$ is hydrogen’s lower heating value and ${\eta }_{fc}$ is fuel cell efficiency; $|P_b|$ is the absolute battery power; $|SOC-SO{C}_{ref}|$ is the SOC deviation; and ${\lambda }_t\in [0,1]$ is a dynamically adjusted coefficient based on driving modes. The discount factor $\gamma \in (0,1)$ balances short-term and long-term rewards.

To improve the clarity of the proposed control formulation, the physical meanings, units or ranges, and selection or acquisition basis of the main variables and parameters involved in the MPC and AMDDPG controllers are summarized in Table 3.

Table 3. Physical meanings, units, and selection basis of the main variables and parameters.

Symbol	Physical Meaning	Unit/Range	Selection or Acquisition Basis
$v$	Vehicle speed	km/h	Obtained from the vehicle-state measurement
$P_{\mathrm{dem}}$	Driver-demand power	kW	Calculated from the longitudinal vehicle dynamics
${\dot{P}}_{\mathrm{dem}}$	Variation rate of driver-demand power	kW/s	Calculated from consecutive power-demand samples
$SOC$	Battery state of charge	0–1	Estimated using the battery model
${SOC}_{\mathrm{ref}}$	Reference battery SOC	Dimensionless; 0.5	Selected near the middle of the allowable SOC range to retain traction and regenerative-braking capacity
$P_{\mathrm{fc}}$	Fuel-cell output power	kW	Constrained by the fuel-cell operating range in Table 1
$P_{\mathrm{bat}}$	Battery power	kW	Determined from the power balance and limited by battery ratings
$\Delta P_{\mathrm{fc}}$	Fuel-cell power variation between consecutive control steps	kW/control step	Limited to suppress rapid fuel-cell load changes
$P_{\mathrm{bat},\min }, P_{\mathrm{bat},\max }$	Battery charging and discharging power limits	kW	Determined from battery voltage, capacity, allowable current, and SOC limits
$w_1$	Hydrogen-consumption weight	0–1	Generated online by the trained AMDDPG Actor network
$w_2$	Battery-power-limitation weight	0–1	Generated online by the trained AMDDPG Actor network
$w_3$	SOC-regulation weight	0–1	Generated online by the trained AMDDPG Actor network
$w_4$	Fuel-cell power-smoothing weight	Dimensionless	Selected through offline calibration under the fuel-cell slew-rate constraint
$D_{\mathrm{fc}}$	Normalized fuel-cell degradation indicator	0–1	Obtained from the estimated fuel-cell health state
$D_{\mathrm{bat}}$	Normalized battery degradation indicator	0–1	Obtained from the estimated battery health state
$N_{\mathrm{p}}$	MPC prediction horizon	10 steps	Selected by balancing control performance and online computational burden
$T_{\mathrm{s}}$	Controller update period	0.1 s	Selected according to the real-time hierarchical control cycle
$\lambda$	Mode-dependent reward adjustment coefficient	0–1	Determined according to the identified driving mode
$\gamma$	Reward discount factor	0–1	Selected during offline training to balance current and future rewards
$\beta$	Multi-scale loss balance coefficient	Dimensionless	Calibrated offline to balance short-term and long-term losses

The physical constraints are selected according to the component specifications and allowable operating ranges, whereas the learning-related parameters are determined through offline calibration by balancing energy economy, component protection, training stability, and computational efficiency.

The Actor–Critic network structure comprises two subnetworks. The Actor network takes state $s_t$ as input and outputs a deterministic weight decision $a_t =\mu (s_t |{\theta }^{\mu })$, with ${\theta }^{\mu }$ as network parameters. It adopts a three-layer fully connected (FC) structure (input layer → hidden layer 128 → hidden layer 64 → output layer 3), with a Softmax activation function at the output to ensure weight normalization. The Critic network takes $s_t$ and $a_t$ as inputs and outputs the action value $Q(s_t ,a_t |{\theta }^Q)$ to evaluate decision quality, utilizing a three-layer FC structure (state–action concatenation → hidden layer 128 → hidden layer 64 → output layer 1) optimized via mean squared error (MSE) loss. To enhance training stability, target Actor $\mu \prime (s|{\theta }^{\mu \prime })$ and Critic $Q\prime (s,a|{\theta }^{Q\prime })$ networks are introduced, with parameters synchronized via a soft update mechanism:

$${\theta }^{\mu \prime }\leftarrow \tau {\theta }^{\mu }+(1-\tau ){\theta }^{\mu \prime },{\theta }^{Q\prime }\leftarrow \tau {\theta }^Q+(1-\tau ){\theta }^{Q\prime }$$

(31)

where $\tau$ is the soft update coefficient. The replay buffer stores samples $(s_t ,a_t ,r_t ,s_{t+1} )$, and random sampling is performed during training to avoid sample correlation.

3.3.2. Offline Training

The offline training phase enables AMDDPG to learn the optimal “state → weight” mapping, with its “Adaptive-Modality“ (AM) essence manifested in adaptive mode perception and multi-scale optimization mechanisms, as elaborated below.

Adaptive Mode Perception Mechanism

To achieve precise weight adaptation to dynamic driving conditions, an adaptive mode perception module based on fuzzy clustering is designed, classifying driving scenarios into five distinct modes (Table 4) and integrating mode-specific features into the network input to guide context-aware weight decisions.

Table 4. Classification of driving modes and corresponding weight priorities.

Driving Mode	Classification Criteria	Core Objective of Weight Decision
High-speed Cruising	$v>80 \mathrm{k}\mathrm{m}/\mathrm{h}, {\sigma }_v <0.1$	Prioritize hydrogen consumption reduction
Medium-speed Steady	$\begin{array}{l}50\le v\le 80 \mathrm{k}\mathrm{m}/\mathrm{h}\\ 0.1\le {\sigma }_{v }<0.3\end{array}$	Balance efficiency and responsiveness
Low-speed Congestion	$\begin{array}{l}v<50 \mathrm{k}\mathrm{m}/\mathrm{h}\\ {\sigma }_v \ge 0.3\end{array}$	Suppress battery power fluctuation to reduce fuel cell start-stop loss
Rapid Acceleration	${\dot{P}}_{dem} >5 \mathrm{k}\mathrm{W}/\mathrm{s}$	Coordinate fuel cell and battery for transient load response
Rapid Deceleration	${\dot{P}}_{dem} <-5 \mathrm{k}\mathrm{W}/\mathrm{s}$	Prioritize SOC balance for energy recovery

The mechanism operates through two key steps: first, driving condition features are extracted by calculating vehicle speed volatility ${\sigma }_v =std(v)/mean(v)$ and average power demand ${\overline{P}}_{dem}$ over a 5 s sliding window, capturing the dynamic nature of scenarios. Second, a mode embedding network is integrated before the Actor network’s input layer, converting clustering results into one-hot vectors $m_t\in {\{0,1\}}^5$ that are concatenated with state features $s_t$. This fusion enables the network to focus on mode-specific core needs and dynamically adjusts the coefficient $\lambda$ in the reward function via

$${\lambda }_t={{\displaystyle \sum }}_{i=1}^5{m}_{t,i}\cdot {\lambda }_{base,i}$$

(32)

where ${\lambda }_{base,i}$ is a preset base value for each mode, ensuring the reward function’s sensitivity to driving contexts. The “adaptive” optimization here lies in real-time scenario identification and targeted weight tuning guided by mode encoding, allowing the network to prioritize objectives that align with current driving conditions.

Multi-Scale Optimization Mechanism

To address the inherent trade-off between short-term driving dynamics and long-term component aging, a dual-scale optimization framework is proposed, integrating both immediate and extended operational horizons into the training process.

The short-term scale aligns with the MPC prediction horizon $N_p=10$, focusing on the Critic network’s evaluation of weight impacts on instantaneous hydrogen consumption $\Delta H_2$ and power fluctuations $\left|P_b\right|$. This ensures rapid responsiveness to abrupt changes in power demand ${\dot{P}}_{dem}$, stabilizing transient performance. Conversely, the long-term scale incorporates a “long-term reward pool” within the replay buffer, storing cumulative rewards over 100 consecutive steps ${\displaystyle {\sum }_{t=0}^{99}{\gamma }^t{R}_t}$. Long-term samples are sampled every 100 training steps to emphasize suppression of component degradation $D_{fc} ,D_{bat}$, guiding the network to prioritize life-cycle optimization.

These scales are fused via a multi-scale loss function that balances their contributions:

$$J=J_{short}+\beta J_{long}$$

(33)

where $J_{short}$ denotes the standard temporal difference (TD) error for short-term performance, $J_{long}$ represents the long-term reward error, and $\beta$ is a balance coefficient. This “multi-scale” optimization ensures the learned weights not only adapt to immediate driving demands but also mitigate long-term component wear, achieving robust performance across both transient and extended operational periods.

3.3.3. Online Application

In the online phase, trained network parameters are frozen to ensure efficient inference, seamlessly integrating with the 100 ms vehicle control cycle and communicating via the Controller Area Network (CAN) bus with end-to-end latency < 10 ms. Real-time state $s_t=[v,P_{\mathrm{dem}},\mathrm{SOC},{\dot{P}}_{\mathrm{dem}},D_{\mathrm{fc}},D_{\mathrm{bat}}]$ is acquired every 100 ms from on-board sensors, with $D_{\mathrm{fc}}$ and $D_{\mathrm{bat}}$ estimated via the On-Board Diagnostics (OBD) system. Based on ${\sigma }_v$ and ${\overline{P}}_{\mathrm{dem}}$ within the current 5 s window, the driving mode is identified using a pre-trained fuzzy clustering model (solidified during offline training), generating mode encoding $m_t$. State features $s_t$ and mode encoding $m_t$ are input into the Actor network (with only forward computation enabled) to output normalized weights $w={[w_1,w_2,w_3]}^T$, which are transmitted to the MPC controller via CAN. For robustness, if the MPC constraint violation rate exceeds 5% over five consecutive cycles (500 ms), weight fine-tuning is triggered—temporarily increasing $w_3$ (or $w_2$) until constraints are satisfied, then restoring normal output. This emergency mechanism is retained as a supervisory safety layer for extreme or previously unseen operating conditions. During the WLTC and UDDS simulations reported in Section 4, no constraint violation persisted for five consecutive control cycles; therefore, the emergency weight-tuning mechanism was not activated. This design leverages AMDDPG’s adaptive mode perception (to handle diverse scenarios) and multi-scale optimization (to balance short- and long-term goals), enabling dynamic weight generation across full driving conditions and enhancing hierarchical EMS performance.

4. Comparison of Simulation Results

The AMDDPG–MPC HEMS for the FCEV is developed based on the vehicle model in Section 2 and the proposed control algorithm in Section 3. This section evaluates the economic (energy-saving) performance of this AMDDPG-MPC strategy. For this purpose, the proposed strategy is tested under two representative driving cycles: the World Light Vehicle Test Cycle (WLTC) and the Urban Dynamometer Driving Schedule (UDDS). These cycles cover both mixed driving conditions and urban stop-and-go scenarios, providing representative evaluation scenarios for assessing the adaptability and energy-management performance of different EMSs. The WLTC SOC trajectory used in the comparison is shown in Figure 7.

Figure 7. The changes in SOC in WLTC.

The overall control performance of the AMDDPG–MPC HEMS is evaluated by comparing it with several benchmark energy management strategies and the corresponding reference results, as described below:

Rule-based EMS (RB): The rule-based EMS (RB) is utilized as a benchmark strategy in this study [41]. Its operation is governed by predefined logic thresholds that determine the vehicle operating mode. In this study, the powertrain is divided into three modes: electric, hybrid, and charging. In electric mode, the required traction power is supplied only by the battery. In hybrid mode, the fuel cell and battery work together to meet the power demand. In charging mode, the fuel cell provides the propulsion power while producing additional energy to recharge the battery.

ECMS: A prominent instantaneous optimization-based EMS extensively explored in the literature [42] is also included. The central element of ECMS is the equivalent factor, which serves to unify the measurement of disparate energy sources by converting battery energy consumption into its hydrogen equivalent. For this study, a constant equivalent factor of $\lambda$ = 0.55 is adopted. This value is derived from the average efficiencies of the fuel cell and the battery. Employing a fixed $\lambda$ reduces real-time computational complexity while maintaining a balance between the energy sources.

Model Predictive Control (MPC) is also benchmarked, a control algorithm renowned for its inherent advantages of rolling optimization and feedback correction, which has led to its widespread adoption in practical engineering problems [43]. In this application, the MPC objective function, as formulated in Equation (28), is structured to manage the optimal trade-off between hydrogen consumption and battery usage. The respective weighting factors for this balance are set to $\alpha$ = 0.7 and $\beta$ = 0.3. The selected parameters used for the benchmark EMSs are summarized in Table 5.

All comparative simulations are conducted with a fixed time step of 0.01 s. The WLTC and UDDS simulations are performed on a level road, with the road slope angle set to α = 0, corresponding to a road grade of 0%. To ensure consistent initial conditions among the tested EMSs, the vehicle starts in electric mode and the initial battery SOC is set to 0.85. The simulations are carried out on a computer equipped with an Intel i5-12400 CPU and 32 GB of memory. No emergency weight-tuning event was triggered during either the WLTC or UDDS simulations, because no state or input constraint violation persisted for five consecutive control cycles.

4.1. Assessment of Energy Consumption by Different EMSs in WLTC Driving Cycles

The vehicle simulation results for the four EMSs under four WLTCs are illustrated in this section. The WLTC provides a comprehensive, real-world test environment, covering diverse segments (urban, suburban, highway) and behaviors (acceleration, deceleration, etc.). This ensures a thorough evaluation of the strategies’ adaptability and robustness across different scenarios. Table 6 summarizes the key performance metrics: final SOC, total equivalent hydrogen consumption (EHC), and EHC optimization percentage. The SOC trajectory during the simulation is shown in Figure 7, and the cumulative EHC is presented in Figure 8. To eliminate the influence of different terminal SOC values, all strategies are evaluated using an SOC-corrected equivalent hydrogen consumption metric. After each driving cycle, the electrical energy required to restore the battery from the terminal SOC to the common initial value of 0.85 is calculated as

$$E_{\mathrm{rec}}={\displaystyle \frac{E_{\mathrm{bat},\mathrm{nom}}(SO{C}_{\mathrm{ini}}-SO{C}_{\mathrm{end}})}{{\eta }_{\mathrm{chg}}}}$$

(34)

where $E_{\mathrm{bat},\mathrm{nom}}$ is the nominal battery energy, $SO{C}_{\mathrm{ini}}=0.85$, $SO{C}_{\mathrm{end}}$ is the terminal SOC, and ${\eta }_{\mathrm{chg}}$ is the overall charging efficiency, including the assumed charging losses. The SOC-corrected equivalent hydrogen consumption is then calculated as

$$m_{{\mathrm{H}}_2,\mathrm{eq}}=m_{{\mathrm{H}}_2,\mathrm{cycle}}+{\lambda }_{\mathrm{eq}}{E}_{\mathrm{rec}}$$

(35)

where $m_{{\mathrm{H}}_2,\mathrm{cycle}}$ is the actual hydrogen consumed during the driving cycle and ${\lambda }_{\mathrm{eq}}$ is the electrical-energy-to-hydrogen equivalence coefficient. In this study, ${\eta }_{\mathrm{chg}}=0.90$ and ${\lambda }_{\mathrm{eq}}=75 \mathrm{g}/\mathrm{kWh}$ are applied consistently to all EMSs. Therefore, the total EHC values reported in the subsequent performance comparison tables already include the energy required to restore the terminal SOC to 0.85.

Figure 8. The changes in total EHC in WLTC.

The RB strategy maintains a relatively smooth SOC trajectory during the driving cycle, indicating its ability to preserve the battery charge within a narrow range and provide a stable energy reserve. Nevertheless, this behavior is achieved with limited optimization capability, since the rule-based controller depends on preset thresholds and cannot adjust its decisions to changing driving conditions. As a result, RB produces the highest total equivalent hydrogen consumption among the compared methods. Compared with RB, ECMS leads to more pronounced SOC variations because it continuously adjusts the power split between the fuel cell and the battery to minimize instantaneous equivalent fuel consumption. Although this mechanism improves hydrogen economy, it increases battery charge–discharge activity, which may be unfavorable for battery durability. MPC achieves a further reduction in total EHC relative to RB and ECMS. By incorporating predicted state evolution into the optimization process, MPC can select more effective control actions and improve fuel economy over the driving cycle.

By generating adaptive MPC weights in real time based on perceived driving modes, the AMDDPG-MPC achieves the lowest total EHC 155.957 g among all analyzed EMSs. It leverages a trained deep reinforcement learning agent to continuously output optimal weights (${\lambda }_H$, ${\lambda }_{Pbatt}$, ${\lambda }_{SOC}$) for the underlying MPC controller. This agent uses its adaptive modal awareness to adjust weights for transient phases and its multi-scale optimization to balance short-term economy with long-term component health. The results indicate that AMDDPG-MPC can adapt effectively to the changing power demand in the WLTC and maintain favorable control performance over the whole test. Its terminal SOC reaches 0.541, showing that the strategy makes fuller use of the battery energy while coordinating the fuel cell to operate more efficiently. This behavior contributes to lower hydrogen consumption and demonstrates the practical potential of the proposed method for improving fuel economy without causing uncontrolled SOC deviation.

For a fair energy-consumption comparison, the final SOC is then restored to the initial value of 0.85 by stationary charging, and the additional charging energy is converted into equivalent hydrogen consumption.

4.2. Assessment of Energy Consumption by Different EMSs in UDDS Driving Cycles

The performance of the four energy management strategies is further examined under the UDDS cycle. Unlike WLTC, which contains a wider mixture of driving segments, UDDS places greater emphasis on urban operating conditions, including repeated acceleration, braking, and idle periods. Therefore, using UDDS together with WLTC helps evaluate whether the proposed strategy can maintain stable energy-management performance in both mixed and city-oriented driving scenarios. Table 7 reports the terminal SOC, total equivalent hydrogen consumption (EHC), and energy-saving improvement of each EMS under the UDDS condition. The corresponding SOC evolution and cumulative EHC profiles are presented in Figure 9 and Figure 10, respectively.

Table 5. The selected parameters.

Parameters	Value	Description
$\lambda$	0.55	Equivalent factor
$\alpha$	0.7	Weight for fuel cell energy in the objective function
$\beta$	0.3	Weight for battery energy in the objective function
$E_{batt}$	Varies based on demand	Energy consumed from the battery
$E_{FC}$	Varies based on load	Energy consumed from the fuel cell

Table 5. The selected parameters.

Parameters	Value	Description
$\lambda$	0.55	Equivalent factor
$\alpha$	0.7	Weight for fuel cell energy in the objective function
$\beta$	0.3	Weight for battery energy in the objective function
$E_{batt}$	Varies based on demand	Energy consumed from the battery
$E_{FC}$	Varies based on load	Energy consumed from the fuel cell

Table 6. Energy consumption performance by different EMSs in WLTC driving cycles.

EMSs	End SOC	Total Equivalent Hydrogen Consumption (g)	Energy-Saving Optimality (%)
RB	0.695	192.191	-
ECMS	0.687	178.687	7.026
MPC	0.638	162.833	15.275
AMDDPG-MPC	0.541	155.957	18.853

Table 6. Energy consumption performance by different EMSs in WLTC driving cycles.

EMSs	End SOC	Total Equivalent Hydrogen Consumption (g)	Energy-Saving Optimality (%)
RB	0.695	192.191	-
ECMS	0.687	178.687	7.026
MPC	0.638	162.833	15.275
AMDDPG-MPC	0.541	155.957	18.853

Table 7. Energy consumption performance by different EMSs in UDDS driving cycles.

EMSs	End SOC	Total Equivalent Hydrogen Consumption (g)	Energy-Saving Optimality (%)
RB	0.709	155.922	-
ECMS	0.698	144.269	7.474
MPC	0.683	139.770	10.359
AMDDPG-MPC	0.597	137.630	11.732

Figure 9. The changes in SOC in UDDS.

Figure 10. The changes in total EHC in UDDS.

Following the WLTC analysis, the UDDS cycle is used to further assess the behavior of AMDDPG-MPC in urban traffic. As shown in Figure 9, the SOC trajectory fluctuates under the repeated start–stop events of UDDS, but the proposed strategy still regulates the battery charge within an acceptable range. Compared with MPC, AMDDPG-MPC maintains a higher SOC level, which helps preserve sufficient energy reserve for city driving. The cumulative EHC curve in Figure 10 also shows that AMDDPG-MPC achieves a low energy-consumption level under the UDDS condition. According to Table 7, the proposed method obtains an energy-saving improvement of 11.732%. Although this improvement is smaller than that under WLTC, it remains evident compared with the benchmark EMSs. This benefit mainly comes from the adaptive modal awareness mechanism of AMDDPG, which identifies urban operating patterns such as low-speed congestion and rapid acceleration and then provides suitable weight commands for the MPC controller.

The results under WLTC and UDDS indicate that AMDDPG-MPC can adapt to different driving characteristics. In WLTC, the strategy improves overall fuel economy under mixed-speed conditions, while in UDDS it responds effectively to frequent power-demand changes caused by acceleration, braking, and idling. By recognizing the current driving pattern and updating the MPC weights accordingly, the AMDDPG layer enables the lower-level controller to balance hydrogen consumption reduction and SOC regulation. Therefore, the proposed EMS shows stable control behavior across both mixed and urban scenarios, supporting its applicability to practical FCEV operation.

Figure 11 and Figure 12 compare the fuel-cell operating-point distributions obtained by the different EMSs. In the WLTC case, AMDDPG-MPC places a larger share of operating points in the high-efficiency region of the fuel-cell system. The corresponding pie chart shows that only 7.4% of its operating points are located in the lowest-efficiency interval, which is lower than those of the other compared strategies. This result indicates that the proposed hierarchical strategy can guide the fuel cell away from inefficient operating regions. Since hydrogen consumption and fuel-cell degradation are considered during training, the AMDDPG agent tends to generate weights that encourage more economical and durability-aware operation.

Figure 11. Fuel cell operating points by four EMSs in WLTC.

Figure 12. Fuel cell operating points by four EMSs in UDDS.

For AMDDPG-MPC, the fuel-cell operating points are concentrated more reasonably within the efficient power range. Through adaptive adjustment of the hydrogen-consumption weight, the upper-level AMDDPG controller helps MPC respond to changing power demand while keeping the fuel cell near favorable efficiency regions. Compared with the benchmark methods, the proposed strategy reduces low-efficiency operation, thereby decreasing energy loss and improving fuel-cell utilization. This behavior not only contributes to hydrogen saving but also helps relieve unnecessary stress on the fuel-cell system, which is consistent with the long-term component-protection objective introduced in the multi-scale optimization design.

Under the UDDS cycle, AMDDPG-MPC still keeps a considerable share of fuel-cell operation within the high-efficiency region. As indicated by the pie chart, 32.64% of the operating points fall in this efficient range, showing that the proposed strategy can maintain favorable fuel-cell utilization even under urban conditions with repeated stops and low-speed operation. Such scenarios involve frequent acceleration and braking events, which impose rapid power-demand variations and therefore place higher requirements on EMS adaptability. AMDDPG-MPC’s ability to sustain high efficiency under these conditions demonstrates its robustness, which stems from its adaptive modal awareness mechanism identifying and adapting to these challenging urban modes. AMDDPG-MPC minimizes the proportion of low-efficiency operating points in the UDDS cycle. By shortening the operating duration in low-efficiency regions, AMDDPG-MPC improves the overall quality of energy distribution and makes the control response more stable. This advantage is especially important for urban driving, where rapid variations in vehicle speed and power demand make fixed-weight strategies less effective. The reduced proportion of low-efficiency fuel-cell operation further shows that the real-time weight-generation mechanism enables AMDDPG-MPC to better cope with stop-and-go traffic conditions.

As shown in Figure 13, the battery and fuel cell jointly meet the vehicle power demand during the driving cycle. The fuel cell output remains comparatively smooth, which helps avoid frequent load variations and thereby reduces load-cycling stress associated with membrane degradation and catalyst aging. Meanwhile, the battery is mainly used to absorb transient power demand and high-frequency fluctuations. Although the battery power changes more actively, AMDDPG-MPC keeps the SOC within a reasonable range, preventing excessive discharge and overcharge. In this way, the proposed strategy not only aims to reduce hydrogen consumption but also considers component protection through coordinated power allocation, helping improve the durability of both the fuel cell and the battery.

Figure 13. Fuel cell and battery power of AMDDPG-MPC in WLTC.

In summary, AMDDPG-MPC stands out for its remarkable adaptability and robustness across different driving conditions, as demonstrated by the fuel cell operating points in both WLTC and UDDS simulations. The strategy’s ability to maintain high efficiency under varying power demands and driving scenarios underscores its versatility. This adaptability ensures that the vehicle can achieve optimal performance whether it is cruising on a highway or navigating through city traffic, as the hierarchical AMDDPG layer is designed to identify these distinct driving modes and provide tailored objectives for the MPC controller.

5. Conclusions

In this paper, a hierarchical AMDDPG–MPC energy management strategy is proposed for an FCEV, strengthening hydrogen-economy potential and improving the operating efficiency and durability awareness of the powertrain. Firstly, the adoption of a hierarchical control architecture enhances cross-scenario adaptability by decoupling upper-level dynamic weight generation from lower-level power-split optimization. Secondly, an implicit MPC with offline design and online inference reduces real-time computational burden while retaining constraint handling and receding-horizon optimality. Thirdly, the proposed AMDDPG-based multifaceted weight-generation process—featuring adaptive mode perception and a multi-scale reward—improves the anti-interference ability of MPC and further enhances adaptability to time-varying driving conditions. Finally, compared with rule-based, ECMS, and fixed-weight MPC baselines, the proposed strategy achieves notable reductions in equivalent hydrogen consumption on standard cycles (about 18.9% on WLTC and 11.7% on UDDS), while maintaining stable SOC regulation and concentrating fuel-cell operation in high-efficiency regions. The simulation results verify that the hierarchical AMDDPG–MPC can effectively release economic potential with real-time feasibility.

However, to guarantee fair comparison, the required power in this study is determined from given a priori driving cycles, and the EHC evaluation employs a fixed equivalence factor with simplified component aging/thermal models. Future work will focus on integrating short-horizon power/speed prediction, embedding degradation-aware models, enabling online adaptation to forecast errors, and conducting hardware-in-the-loop validations.