Optimization of Energy Management Strategy for Hybrid Power System of a Liquid Cargo Ship

Abstract

To enhance the potential application of ships in energy conservation and emission reduction, a parallel hybrid power system simulation model was developed using Matlab/Simulink based on the operational scenario of a 3300 m³ inland LPG tanker. A rule-based control strategy was employed to simulate and analyze the impact of different power modes on the energy efficiency and emissions of the ship’s power system. An optimization method for an energy management strategy based on the ship’s operating cycle was proposed. The results show that using an LNG engine as the main engine in the hybrid system significantly reduces fuel consumption and pollutant emissions throughout the ship’s operating cycle. When the hybridization ratio is 0.2, the system achieves relatively optimal overall energy consumption and emission levels. Building on this, an Equivalent Consumption Minimization Strategy (ECMS) was introduced, and multi-objective optimization was carried out using an improved particle swarm optimization algorithm. Additionally, reinforcement learning was applied to optimize the energy management strategy, resulting in further reductions in fuel consumption over the operating cycle.

1. Introduction

After the “carbon peaking” and “carbon neutrality” targets were proposed, high-efficiency energy-saving and emission-reduction technologies for ships have increasingly become a focal research topic in the maritime industry [1,2]. Hybridization refers to the integration of multiple power sources, typically combining internal combustion engines (ICE) with electric motors, to create a more energy-efficient and environmentally friendly propulsion system. Unlike conventional mechanical propulsion, hybrid marine propulsion integrates electric motors and internal combustion engines as power sources and, together with an energy management strategy, enables multi-mode switching to meet propulsion demands under diverse operating scenarios. By optimizing the engine operating region and mitigating the poor efficiency associated with low-load operation, hybridization can effectively reduce both pollutant emissions and fuel consumption [3,4].

Several hybrid architectures have been developed for marine applications, with the most common being series [5], parallel [6], and series–parallel configurations. In a series hybrid system, the internal combustion engine (ICE) acts as a generator, supplying power to the electric motor. In a parallel hybrid, both the ICE and electric motor directly drive the propeller, either separately or in combination. A series–parallel system combines the benefits of both, allowing more flexible operation under various load conditions. Examples of hybrid power systems in ships include the use of diesel–electric propulsion in ferries and LNG-powered hybrid systems in container ships.

Current research on marine hybrid power systems has primarily concentrated on energy management strategies and component matching/parameter optimization. Pan Haibang and Fan Liyun et al. [7] investigated series and parallel hybrid architectures, respectively, performed key-component parameter matching, and then evaluated overall system performance under assumed operating conditions using switching-based control strategies. Their simulation results indicated that, compared with conventional mechanical propulsion, hybrid power systems can achieve notable improvements in both economic performance and emissions. Byongug et al. [8] adopted a multi-criteria decision-making framework in which cost, environmental impact, and risk were monetized to compare the total costs of hybrid propulsion, diesel–electric systems, and diesel–mechanical systems across different operating scenarios; they concluded that hybrid systems can reduce pollutant emissions while enhancing the safety and reliability of propulsion systems. Xia Jingting et al. [9] applied a diesel–electric hybrid system to an inland waterway service vessel and managed the system using a fuzzy-logic control strategy, demonstrating that the proposed approach can significantly reduce pollutant emissions.

Nevertheless, the aforementioned studies largely rely on rule-based energy management or analyses conducted under static and idealized conditions, and therefore, do not fully capture the advantages of real-time, dynamic optimization under complex, real-world operating profiles. In particular, research on advanced intelligent methods, such as deep reinforcement learning, remains limited. Motivated by this gap, this paper investigates a 3300 m³ LPG inland liquid cargo ship operating on the Yangtze River. Real navigation data are used to construct representative simulation profiles, and the effects of different powertrain configurations on system efficiency and emissions are systematically examined. Building on conventional rule-based strategies and the Equivalent Consumption Minimization Strategy (ECMS), a multi-objective optimization approach based on an improved particle swarm optimization algorithm (APSO) is developed. Furthermore, a reinforcement learning method based on the Deep Deterministic Policy Gradient (DDPG) algorithm is introduced to enable real-time optimization of energy allocation. The proposed framework aims to further exploit the energy-saving and emission-reduction potential of hybrid systems and to provide a more efficient energy management solution that is better suited to complex navigation scenarios.

2. Power System Modeling

2.1. Prototype Vessel Description

The prototype vessel considered in this study is a 3300 m³ LPG liquid cargo ship manufactured by Nantong CIMC Sinopacific Offshore & Engineering Co., Ltd. (Nantong, China), operating on the Yangtze River along the route from Chongqing to Nanjing. The vessel has a length of 88.0 m, a beam of 16.0 m, a depth of 5.6 m, and a design draft of 4.0 m. It is equipped with a twin-engine, twin-propeller propulsion arrangement, with a design speed of 13 knots, a design displacement of 1900 t, and a rated total propulsion power of 2216 kW. Following retrofitting, the propulsion system comprises two parallel hybrid propulsion units, each configured as a twin-engine, single-shaft, single-propeller system. The primary components include the internal combustion engine, energy storage system, shore power interface, a DC network incorporating busbars, electric motors, power conversion equipment, a clutch, a gearbox, and propellers. The configuration of the two-sided system is illustrated in Figure 1, and the detailed specifications of the prototype vessel are summarized in

Table 1. Specifications of the prototype vessel.

Item	Specification
Vessel type	3300 m3 LPG liquid cargo ship
Navigation area	Class A, J2 route segment
Route	Chongqing–Nanjing (main route: Yueyang–Nanjing)
Propulsion configuration	Twin-engine twin-propeller
Design speed	13 knots (matched at 90% engine load)
Actual speed (upstream)	≤8 km/h
Actual speed (downstream)	≤16 km/h
Design displacement (at design draft)	1900 t
Maximum loaded displacement	2200 t
Gearbox ratio	4.47:1

.

The engine and the energy storage system are the two onboard power sources. The energy storage system drives the electric motor to operate in coordination with the engine to generate and combine propulsion power, which is then transmitted to the propeller through the power-combining gearbox. The key components of the parallel hybrid propulsion system and their matched parameters are summarized in

Table 2. Matching results of key component parameters of the system.

Key Component	Main Parameter	Value
Engine	Engine type	LNG engine
	Rated power/kW	886.4
	Rated speed/RPM	750
Electric motor	Motor type	PMSM
	Rated power/kW	221.6
	Rated speed/RPM	1500
Energy storage system	ESS type	Lithium iron phosphate battery
	Rated capacity/A.h	577.1
Gearbox	Engine-side gear ratio	4.47
	Motor-side gear ratio	8.94
Propeller	Series	B-series
	Design speed/RPM	167.8
	Diameter/m	3
	Disk area ratio	0.55
	Pitch ratio	0.86

.

2.2. Modular Modeling

2.2.1. Engine Modeling

At present, engine modeling is primarily carried out using two approaches: (i) physics-based simulation models formulated from engine structural parameters and thermodynamic equations, and (ii) map-based simulation models constructed via interpolation of experimental data [10,11]. Owing to the complexity of engine structures and the large number of parameters involved, developing a purely mathematical-function model would require establishing numerous high-order equations, measuring multiple structural parameters, and introducing various simplifying assumptions, which makes it difficult to satisfy real-time requirements in engineering applications. To improve computational efficiency and shorten the simulation time, this study adopts a two-dimensional interpolated lookup-table method to build the engine simulation model. The model consists of a governor module and a universal performance-map module. Universal characteristic curves are obtained from steady-state engine tests, and the throttle opening is adjusted according to operating demands to enable real-time outputs of fuel consumption rate, NOx emissions, CO emissions, and other key parameters. The governing equations are given as follows:

$$T_{\mathrm{e}}=R_{\mathrm{thr}}\ast T_{\mathrm{e},\max }$$

(1)

$$\left[g_{\mathrm{e}},g_{\mathrm{NO}x},g_{\mathrm{CO}}\right]=f\left(n_{\mathrm{e}},T_{\mathrm{e}}\right)$$

(2)

where $n_e$ denotes the instantaneous engine speed (rpm); $T_e$ is the instantaneous engine torque (N·m); $R_{thr}$ represents the engine throttle opening; $g_e$ is the engine natural-gas specific consumption (g/kWh); $g_{NOx}$ is the engine $N{O}_x$ specific emission rate (g/kWh); and $g_{co}$ is the engine CO specific emission rate (g/kWh).

2.2.2. Motor Modeling

When developing the motor model, the built-in Permanent Magnet Synchronous Motor (PMSM) module from the Simulink library is adopted. This module is widely used for modeling electric machines in hybrid power systems, and its dynamics are described by the standard PMSM governing equations in the $dq$-reference frame. Specifically, the stator voltage equations can be expressed as follows:

$$V_d=R_s{i}_q+L_d{\displaystyle \frac{d{i}_d}{dt}}-w_e{L}_q{i}_q$$

(3)

$$v_q=R_s{i}_q+L_q{\displaystyle \frac{d{i}_q}{dt}}+{\omega }_e(L_d{i}_d+{\psi }_f)$$

(4)

and the electromagnetic torque is given by

$$T_e={\displaystyle \frac{2}{3}}p[{\psi }_f{i}_q+(L_d-L_q)i_d{i}_q]$$

(5)

where $v_d$ and $v_q$ are the $dq$-axis voltages; $i_d$ and $i_q$ are the $dq$-axis currents; $R_s$ is the stator resistance; $L_d$ and $L_q$ are the $dq$-axis inductances; ${\omega }_e$ is the electrical angular speed; ${\psi }_f$ is the permanent magnet flux linkage; and $p$ denotes the number of pole pairs.

Based on the motor parameters required for the prototype vessel, the back-electromotive-force (back-EMF) constant $K_E$ and the torque constant $K_T$ are determined using the following formulations:

$$K_{\mathrm{E}}={\displaystyle \frac{N\varphi }{60a}}$$

(6)

$$K_{\mathrm{T}}={\displaystyle \frac{N\varphi }{2\mathsf{\pi }}}$$

(7)

where $N$ is the number of turns of the armature winding; $\varphi$ denotes the air-gap magnetic flux (Wb); and $a$ is the number of parallel branches in the motor winding.

2.2.3. Energy Storage System Modeling

The operating-mode decision of the hybrid power system is directly influenced by the state of charge (SOC) of the energy storage system. Accordingly, the energy storage system model developed in this study incorporates SOC estimation and terminal-voltage calculation functions [12], as expressed by the following equations:

$$\mathrm{SOC}={\mathrm{SOC}}_{\mathrm{init}}-{\displaystyle \int \frac{I_{\mathrm{out}}}{{\mathrm{Capacity}}_{\max }}}dt$$

(8)

$$U_{\mathrm{out}}=\mathrm{Voc}-K{\displaystyle \int I_{\mathrm{out}}dt}-R_0{I}_{\mathrm{out}}-R_{\mathrm{p}}{I}_{\mathrm{p}}$$

(9)

where SOC denotes the real-time remaining charge of the energy storage system; $SO{C}_{init}$ is the initial remaining charge; $I_{out}$ is the charge/discharge current of the energy storage system (A), where a negative value indicates charging and a positive value indicates discharging; $Capacit{y}_{max}$ is the maximum charge capacity under the current C-rate and thermal state; $U_{out}$ and $V_{oc}$ are the single-string terminal voltage and open-circuit voltage (V), respectively; $K$ is a calibration coefficient $\left(\mathrm{V}/(\mathrm{A}\cdot \mathrm{s})\right)$; $R_0$ and $R_p$ are the series internal resistance and polarization resistance (Ω) of a single string, respectively; and $I_p$ is the polarization current of the cell (A).

2.2.4. Gearbox Modeling

The gearbox model incorporates clutch engagement/disengagement and speed-ratio (gear) conversion functions. The clutch function determines whether the clutch is engaged or disengaged according to the operating states of the motor and the engine. The speed-ratio conversion function transmits power based on the gear ratios at the engine side and motor side of the powertrain, as formulated as follows:

$$T_{\mathrm{p}}={\eta }_{\mathrm{c}}\left(T_{\mathrm{m}}{\displaystyle \frac{n_{\mathrm{m}}}{n_{\mathrm{p}}}}+T_{\mathrm{e}}{\displaystyle \frac{n_{\mathrm{e}}}{n_{\mathrm{p}}}}\right)$$

(10)

where $n_m$, $n_e$, and $n_p$ denote the instantaneous rotational speeds of the motor, engine, and propeller, respectively (rpm); and ${\eta }_c$ is the transmission efficiency of the gearbox.

2.2.5. Propeller Modeling

According to propeller operating principles, the propeller thrust and torque can be calculated using the following equations:

$$T=K_{\mathrm{Q}}\rho n_{\mathrm{p}}^2{D}^4$$

(11)

$$Q=K_{\mathrm{T}}\rho n_{\mathrm{p}}^2{D}^5$$

(12)

where $T$ is the theoretical propeller thrust (N); $Q$ is the theoretical propeller torque (N·m); $K_T$ and $K_Q$ are the propeller thrust and torque coefficients, respectively; $\rho$ is the water density in the propeller operating region (${\mathrm{k}\mathrm{g}/\mathrm{m}}^3$); and $D$ is the propeller diameter (m).

2.2.6. Hull Modeling

As the carrier of the propulsion system, the ship exhibits different acceleration responses under varying external loads. According to Newton’s second law, a vessel’s longitudinal dynamics can be expressed as follows:

$$T-f=m{\displaystyle \frac{d{V}_{\mathrm{S}}}{dt}}$$

(13)

where f denotes the hull resistance (N), m is the total ship displacement (kg), and $V_S$ represents the ship speed (m/s). Notably, hull resistance is dependent on the navigation environment and operating condition of the vessel [13]. In this study, hull resistance is determined using the following formulation:

$$f={\displaystyle \frac{P_{\mathrm{D}}{\eta }_{\mathrm{h}}{\eta }_0{\eta }_{\mathrm{r}}}{V_{\mathrm{S}}}}$$

(14)

where $P_D$ is the propeller delivered power in open-water conditions (W); ${\eta }_h$ is the hull efficiency; ${\eta }_0$ is the open-water efficiency; and ${\eta }_r$ is the relative rotative efficiency.

3. Equivalent Fuel Consumption Minimization Strategy Optimized via Improved PSO and DDPG

3.1. Equivalent Consumption Minimization Strategy (ECMS)

Pontryagin’s Minimum Principle (PMP) provides a theoretical foundation for solving optimal control problems and is applicable to continuous or discrete systems with constrained control variables [14]. Inspired by PMP, the energy split problem of a parallel hybrid marine power system can be formulated as an optimal control problem, leading to the development of the Equivalent Consumption Minimization Strategy (ECMS) [15,16]. The core idea of ECMS is to convert the variation in battery energy into an equivalent fuel consumption, and to minimize—at each time step—the sum of the engine’s actual fuel consumption and the battery’s equivalent fuel consumption. This is achieved by appropriately allocating the output torques of the engine and electric motor such that the instantaneous equivalent fuel consumption is minimized [17].

$$J_{\min }={\displaystyle \sum _t^{N-1}\left\{m_{\mathrm{fuel}}+s(t)\left[\lambda \frac{P_{\mathrm{batt}}\left(t\right)}{{\eta }_{\mathrm{dis}}\left(t\right)Q_{\mathrm{low}}}+\left(1-\lambda \right)\frac{{\eta }_{\mathrm{char}}\left(t\right)P_{\mathrm{batt}}\left(t\right)}{Q_{\mathrm{low}}}\right]\right\}}$$

(15)

where $J$, $m_{fuel}$, and $m_{batt}$ denote the total equivalent fuel consumption, the engine fuel consumption, and the battery-equivalent fuel consumption, respectively (g/s). $P_{batt}$ is the electrochemical power of the energy storage system (kW), $Q_{low}$ is the lower heating value (LHV) of the fuel (J/kg), and $s$ is the equivalence factor. The equivalence factor is determined using the average-efficiency method [18], as follows:

$$\left\{\begin{array}{cc}{s}_{\mathrm{chg}}\left(t\right)={\displaystyle \frac{{\eta }_{\mathrm{m}}{\eta }_{\mathrm{dis}}}{{\eta }_{\mathrm{e}}}}& T_{\mathrm{m}}<0\\ s_{\mathrm{dis}}\left(t\right)={\displaystyle \frac{1}{{\eta }_{\mathrm{e}}{\eta }_{\mathrm{m}}{\eta }_{\mathrm{chg}}}}& T_{\mathrm{m}}\ge 0\end{array}\right.$$

(16)

where $s_{chg}\left(t\right)$ and $s_{dis}\left(t\right)$ are the charging and discharging equivalence factors, respectively; ${\eta }_e$ is the average operating efficiency of the engine; ${\eta }_m$ is the average efficiency of the electric motor; ${\eta }_{dis}$ is the average discharging efficiency of the energy storage system; and ${\eta }_{chg}$ is the average charging efficiency of the energy storage system. Based on the calculations, the charging equivalence factor is $s_{chg}\left(t\right)=3.39$, and the discharging equivalence factor is $s_{dis}\left(t\right)=1.94$.

3.2. Optimization of the Equivalence Factors via Adaptive Particle Swarm Optimization (APSO)

Because the selection of the equivalence factor is critical to the performance of energy management in marine hybrid power systems, the values obtained using the average-efficiency method may not be globally optimal. Therefore, it is necessary to employ intelligent algorithms to conduct multi-objective optimization. To mitigate issues such as convergence to local optima, premature convergence, and stagnation, an adaptive particle swarm optimization algorithm with adaptive inertia weighting (Adaptive Particle Swarm Optimization, APSO) is introduced [19]. In this study, a nonlinear, dynamically varying inertia-weight formulation is adopted to update the inertia weight during the optimization process.

$$w=\left\{\begin{array}{cc}{w}_{\min }-{\displaystyle \frac{\left(w_{\max }-w_{\min }\right)\left(f-f_{\min }\right)}{f_{avg}-f_{\min }}}& ,f\le f_{avg}\hfill \\ w_{\max }& ,f>f_{avg}\hfill \end{array}\right.$$

(17)

where $w_{max}$ and $w_{min}$ denote the maximum and minimum values of the inertia weight $w$, respectively; $f$ is the current objective-function value of an individual particle; and $f_{avg}$ and $f_{min}$ are the mean and minimum objective values of the entire particle population at the current iteration.

When APSO is applied to optimize the ECMS-based energy management strategy for the parallel hybrid marine power system, the decision variables are primarily the charging and discharging equivalence factors, which are defined as follows:

$$X=\left[s_{\mathrm{chg}}\left(t\right),s_{\mathrm{dis}}\left(t\right)\right]$$

(18)

The pollutant emissions $\left(M_{NOx,base},M_{CO,base}\right)$ and operating cost $\left(cos{t}_{base}\right)$ obtained under the rule-based energy management strategy are adopted as baseline reference values. The emission-related optimization objectives are defined separately for different pollutants, namely ${\mathrm{N}\mathrm{O}}_x$ and CO, and are calculated as follows:

$$f_{\mathrm{cost}}={\displaystyle \frac{{\mathrm{cost}}_{\mathrm{LNG}}+{\mathrm{cost}}_{\mathrm{elc}}}{{\mathrm{cost}}_{\mathrm{base}}}}$$

(19)

$$f_{{\mathrm{NO}}_x}={\displaystyle \frac{M_{{\mathrm{NO}}_x}}{M_{{\mathrm{NO}}_x,\mathrm{base}}}}$$

(20)

$$f_{\mathrm{CO}}={\displaystyle \frac{M_{\mathrm{CO}}}{M_{\mathrm{CO},\mathrm{base}}}}$$

(21)

Following the linear weighted-sum approach commonly used in multi-objective optimization, this subsection aggregates multiple optimization objectives into a single composite objective function within the particle swarm optimization framework by applying weighted summation, as expressed below:

$$f={\lambda }_{\mathrm{cost}}{f}_{\mathrm{cost}}+{\lambda }_{{\mathrm{NO}}_x}{f}_{{\mathrm{NO}}_x}+{\lambda }_{\mathrm{CO}}{f}_{\mathrm{CO}}$$

(22)

where ${\lambda }_{cost}$, ${\lambda }_{NOx}$, and ${\lambda }_{CO}$ are the weighting coefficients for operating cost, ${\mathrm{N}\mathrm{O}}_x$ emissions, and CO emissions, respectively, and are set to 0.8, 0.1, and 0.1 in this study. The values for these parameters were determined based on expert judgment, with a higher weight given to operating cost to make the optimization objectives more aligned with economic efficiency. The emission-related objectives were assigned lower weights in order to emphasize cost reduction while still considering environmental impacts. These values were finalized after preliminary tuning tests to ensure a reasonable trade-off between economic performance and emission reduction under the investigated operating profiles.

For the upstream and downstream navigation profiles of the liquid cargo ship, APSO is employed to obtain, at each iteration, the system fitness value and the corresponding decision variables. Under both the upstream and downstream operating conditions, the population fitness converges within 100 iterations. The global optimal fitness value is 0.95996 for the upstream condition and 0.9651 for the downstream condition.

The objective fitness values corresponding to the population’s local optima and global optimum under upstream and downstream conditions are illustrated in Figure 2. Through multi-objective optimization, when the fitness function reaches its global optimum, the associated decision variables simultaneously correspond to the globally optimal design point. Specifically, under the upstream condition, the optimal charging equivalence factor is $s_{chg}\left(t\right)=2.6487$ and the optimal discharging equivalence factor is $s_{dis}\left(t\right)=1.8764$. Under the downstream condition, the optimal charging equivalence factor is $s_{chg}\left(t\right)=2.3368$ and the optimal discharging equivalence factor is $s_{dis}\left(t\right)=1.5606$.

3.3. Reinforcement Learning-Based Optimization of the ECMS

In the equivalent fuel consumption minimization problem for hybrid ships, the ship simulation model and the navigation profile are treated as the environment. The environmental state vector typically includes the battery state of charge (SOC), demanded torque, time, and other relevant variables. The energy management strategy is modeled as an agent: after observing the current environment state, the agent selects an action through a learned mapping, and the environment returns corresponding feedback. Guided by the expected return estimated by the target network, the agent evaluates and improves its control policy across different states, continuously adapting to the environment and refining the policy to maximize long-term cumulative rewards. Ultimately, an optimal control policy is obtained and applied to the environment to generate the state vector at the next time step.

Within a deep reinforcement learning framework, the agent serves as the core decision-making module that perceives system states, makes control decisions, and iteratively learns to optimize its policy. Through repeated interactions with the environment, the agent selects actions based on the current state; the environment then provides a reward signal and the next state, which are both used to update the policy so as to maximize the long-horizon cumulative return. For energy management in hybrid power systems, the agent evaluates the ship’s real-time operating conditions (e.g., battery SOC and power/torque demand) and outputs the optimal torque split between the engine and electric motor, thereby enabling efficient energy utilization and emission control [20]. With continued learning, the agent can adapt to different navigation conditions and progressively improve control performance. The fundamental concept of reinforcement learning is illustrated in Figure 3.

3.4. DDPG Reinforcement Learning Algorithm

3.4.1. DDPG Algorithm

DDPG (Deep Deterministic Policy Gradient) is a reinforcement learning algorithm based on policy gradient. Its core idea is to directly optimize the policy in the continuous action space, rather than indirectly optimizing the policy through Q-value as in Q-learning [21]. DDPG consists of two main components: the Actor network and the Critic network. In DDPG, the Actor network outputs a continuous action value, while the Critic network draws on the idea of Q-learning to calculate the Q-value of the action and updates the parameters of the Actor network based on this Q-value. To improve the stability of training, DDPG adopts techniques such as target networks and experience replay buffers. By using a deterministic policy, DDPG enables the agent to determine specific action values according to the current state in the continuous action space, and optimizes the policy through the policy gradient method, thereby maximizing the cumulative reward. The framework of the energy management algorithm based on DDPG is shown in Figure 4. The Actor network is composed of two hidden layers and one output layer, with the ReLU function as the activation function for the hidden layers and the tanh activation function for the output layer. The structure of the Critic network is similar to that of the Actor network, which is also composed of two hidden layers and one output layer, with the ReLU function as the activation function for the hidden layers and the tanh function as the activation function for the output layer.

3.4.2. State Space Definition

In this study, the hybrid power system is treated as the environment that interacts with the energy management agent. The state information fed back to the agent includes the battery pack state of charge (SOC), the required (demanded) torque $ReqTor$, and time $t$. Accordingly, the state space of the hybrid system model is defined as follows:

$$state=(\mathrm{Re}qtor,SOC,t)$$

(23)

3.4.3. Action Space Definition

For a reinforcement learning-based energy management strategy, the key objective is to allocate torque between the engine and the electric motor. Upon receiving the environment state, the agent outputs an action in the action space $A$, namely the torque-split coefficient $a$. It should be noted that the output torque of the selected marine motor is constrained by symmetric saturation limits, with an upper bound of $T_{\mathrm{m},\mathrm{m}\mathrm{a}x}=+1410.8 \mathrm{N}·\mathrm{m}$ and a lower bound of $T_{\mathrm{m},\mathrm{m}\mathrm{i}\mathrm{n}}=-1410.8 \mathrm{N}·\mathrm{m}$. These bounds represent the maximum motoring and generating torques, respectively, rather than a constant torque value. Therefore, during the simulation trials, the motor torque is not held constant; instead, it varies dynamically according to the torque demand and the agent’s action, while being clipped within $\left[T_{\mathrm{m},\mathrm{m}\mathrm{i}\mathrm{n}},T_{\mathrm{m},\mathrm{m}\mathrm{a}\mathrm{x}}\right]$. Accordingly, the action space is defined as follows:

$$A=\left\{\alpha \right\}$$

(24)

The allocated torques of the engine and the electric motor are computed as follows:

$$N_e=a*\mathrm{Re}qTor$$

(25)

$$N_m=(1-a)*\mathrm{Re}qPow$$

(26)

where $N_e$ denotes the allocated engine torque, $N_m$ denotes the allocated motor torque, and $ReqTor$ is the demanded torque of the hybrid power system.

3.4.4. Reward Function Definition

Fuel consumption is a key indicator for evaluating the energy efficiency of a hybrid power system; therefore, it is incorporated into the reward function. In addition, to maintain the battery SOC within a desirable range, a quadratic penalty term is introduced to quantify the deviation between the actual SOC and a reference SOC value. Because both factors adversely affect system performance, the corresponding coefficients in the reward function are defined as penalty weights (i.e., assigned negative values). The specific penalty weights are denoted by $a$ and $b$, and the reward function is defined as follows:

$$reward=-a \ast fuel-b \ast {(SOC-SOC\_target)}^2$$

(27)

3.4.5. Pseudocode

Based on the above definitions, a multi-ECMS hybrid energy management strategy using the DDPG algorithm is proposed. The corresponding pseudocode is presented in Figure 5.

3.5. Model Validation and Comparison

To verify the feasibility and effectiveness of the proposed DDPG-based energy management strategy, a full-ship simulation model of the hybrid power system was developed in the MATLAB R2020a/Simulink environment. Meanwhile, the reinforcement learning algorithm was implemented in Python 3.7 using the TensorFlow framework, and co-simulated with the MATLAB/Simulink model to realize strategy training and performance evaluation. The operating profile used for strategy validation is shown in Figure 6.

3.5.1. Hyperparameter Configuration and Convergence Analysis

In DDPG, the online (current) networks and the target networks share identical architectures, with the input and output layers corresponding to the state variables and action variables, respectively. The selection of hyperparameters has a direct impact on training convergence and the resulting performance of the energy management strategy. Based on comparative tuning, the learning rate is set to $\alpha =0.001$, the discount factor to $\gamma =0.9$, and the exploration parameter to $\epsilon =0.01$. The number of training episodes is set to 250. Under the demanded-torque operating profile, the reward-function coefficients a and b are set to 2 and 1500, respectively.

The average return per episode is used to characterize the training progress of the agent, as shown in Figure 7, where one episode corresponds to a complete training run over the demanded-torque profile. The initial average return is approximately $-\mathrm{22,500}$ and increases monotonically as training proceeds, converging after about 50 episodes. This behavior indicates that the deep reinforcement learning-based approach is well-suited for energy management of the hybrid power system.

3.5.2. Performance Analysis and Comparison of Energy Management Strategies

Under the input operating profile shown in Figure 6, the SOC trajectories are presented in Figure 8. Both the rule-based strategy using equivalence factors determined by the average-efficiency method and the rule-based strategy using equivalence factors tuned via the PSO algorithm can effectively maintain SOC within reasonable upper and lower bounds. When the demanded torque is relatively low, the system tends to charge the lithium battery, driving the SOC toward its upper limit. In contrast, when the demanded torque is high, the system either operates the generator/engine alone to sustain SOC or employs coordinated engine–motor operation, which results in a decrease in SOC.

By comparison, after training, the DDPG-based energy management strategy is able to output a torque-split decision at each control step that maximizes the reward function. The SOC results indicate that this strategy exhibits a stronger preference for electric propulsion, thereby reducing frequent engine start–stop events under low-power conditions and achieving improved fuel economy. Meanwhile, while prioritizing fuel efficiency, the DDPG-based strategy also maintains SOC within an appropriate charge–discharge operating window.

To further examine how the three energy management strategies shape the operating regions of the engine and motor, the corresponding MAP distributions are presented in Figure 9. The engine MAP in Figure 9a is a fuel-consumption map, where BSFC (brake-specific fuel consumption) denotes the fuel mass flow required per unit power (or torque) output. As shown in Figure 9a, all three strategies effectively prevent the engine from operating in high-BSFC regions. Moreover, under the PSO-optimized strategy and the DDPG-based strategy, a larger proportion of operating points are concentrated in the high-efficiency region of the engine map, indicating that both PSO and DDPG optimization can further improve engine operating conditions. As shown in the motor efficiency MAP in Figure 9b, under the rule-based strategy, most motor operating points are distributed below 90% efficiency. With PSO-based optimization, the motor efficiency is mainly maintained within 84–94%. In contrast, under the DDPG-based strategy, motor efficiency is generally kept above 90%, with the majority of points located in regions exceeding 96% efficiency. These results suggest that both PSO-optimized and DDPG-based strategies reduce motor energy losses compared with the rule-based approach, while the DDPG-based energy management strategy achieves the lowest motor losses during electric-machine.

Considering the large differences in magnitude among the evaluated indicators, a normalization procedure is applied to enable a clear horizontal comparison of the overall performance of the ship’s hybrid power system under the three energy management strategies. In this study, the operating cost, total ${\mathrm{N}\mathrm{O}}_x$ emissions, total CO emissions, and total ${\mathrm{C}\mathrm{O}}_2$ emissions obtained under the rule-based strategy are used as baseline values (normalized to 100%). As shown in Figure 10, compared with the rule-based energy management strategy, the PSO-optimized strategy reduces the operating cost by 3.77% and decreases the total ${\mathrm{N}\mathrm{O}}_x$, CO, and ${\mathrm{C}\mathrm{O}}_2$ emissions by 4.55%, 6.66%, and 3.49%, respectively, over the entire voyage cycle. Furthermore, when the DDPG-based strategy is applied, the operating cost is reduced by 31.06%, while the total ${\mathrm{N}\mathrm{O}}_x$, CO, and ${\mathrm{C}\mathrm{O}}_2$ emissions decrease by 19.10%, 38.31%, and 15.43%, respectively. These results indicate that both optimization-based strategies significantly improve the energy-saving and emission-reduction performance of the hybrid power system, with the DDPG-based strategy providing the most substantial overall benefits.

4. Discussion

The results demonstrate that the hybridization ratio plays a critical role in balancing the economic and environmental performance of the parallel hybrid propulsion system. A hybridization ratio that is too low limits the contribution of the electric motor, whereas a high hybridization ratio increases battery dependence and may lead to more intensive charge–discharge behaviors under real navigation conditions. Therefore, a moderate hybridization ratio provides a more balanced operating profile for both the engine and the energy storage system.

In addition, optimized strategies show clear advantages over the baseline rule-based strategy. APSO improves ECMS performance by tuning key equivalence factors, which helps shift engine operation toward higher-efficiency regions and enhances motor utilization. Furthermore, the DDPG-based strategy provides superior adaptability due to its ability to learn an effective torque-split policy under varying load demands, thereby improving overall energy utilization and reducing inefficient operating modes.

Despite the promising results, this study still has several limitations. The engine and motor models are mainly based on steady-state performance maps, and transient effects during rapid mode transitions are not fully captured. Moreover, the proposed strategies have been validated through simulation and co-simulation; experimental verification using hardware-in-the-loop platforms or test benches will be conducted in future research to further evaluate real-time performance and robustness. In addition, the emission assessment in this study mainly focuses on ${\mathrm{N}\mathrm{O}}_x$, CO, and ${\mathrm{C}\mathrm{O}}_2$, while other pollutants such as ${\mathrm{S}\mathrm{O}}_x$ and particulate matter (PM) are not quantitatively evaluated due to the lack of validated sub-models and measurement data. Future work will further incorporate ${\mathrm{S}\mathrm{O}}_x$ and PM emission modeling (or measurement-based estimation) into the evaluation framework to provide a more comprehensive environmental assessment of LNG-fueled hybrid propulsion systems under realistic operating conditions.

5. Conclusions

This study focused on a 3300 m³ LPG liquid cargo ship and established a parallel hybrid propulsion system simulation model on the MATLAB/Simulink R2021a and Python 3.7 platforms to comprehensively evaluate the influence of different energy management strategies on the overall energy efficiency and emission characteristics of the propulsion system. To investigate the effect of the hybridization ratio on system performance, an LNG engine was selected as the main power source, and a series of power-matching schemes were designed under hybridization ratios of 0.1, 0.2, 0.3, and 0.4. Based on these configurations, the upstream and downstream navigation conditions of the 3300 m³ LPG carrier were simulated in MATLAB/Simulink using representative operational profiles. By jointly considering economic performance, pollutant emission characteristics, and energy storage system utilization, the results indicate that the hybrid propulsion system achieves the best overall trade-off when the hybridization ratio is approximately 0.2.

The simulation results further demonstrate that adopting an LNG engine as the primary power source can significantly reduce fuel consumption and pollutant emissions, and that the overall system efficiency reaches its optimum at a hybridization ratio of 0.2. Compared with the rule-based control strategy, the equivalent consumption minimization strategy (ECMS) optimized using an improved particle swarm optimization method (APSO) exhibits superior performance in terms of both fuel economy and emission reduction. Moreover, by introducing deep reinforcement learning through the deep deterministic policy gradient (DDPG) algorithm, the energy distribution between the engine and the energy storage system becomes more rational and adaptive to varying operating conditions. As a result, the overall fuel consumption is reduced by 31.06%, while total ${\mathrm{N}\mathrm{O}}_x$, CO, and ${\mathrm{C}\mathrm{O}}_2$ emissions are decreased by 19.1%, 38.31%, and 15.43%, respectively, over the entire voyage cycle. These findings validate the significant advantages of intelligent, algorithm-driven energy management strategies in enhancing the energy-saving and emission-reduction capability of marine hybrid propulsion systems, while also improving their adaptability to complex and variable navigation conditions.