A Global Optimization Framework for Energy Efficiency of Wing–Diesel Hybrid Ships Under Distinct Sail-Statuses Based on Improved Deep Q-Network and D*Lite Algorithm

Abstract

Wing–diesel hybrid ships are a practical approach to sustainable maritime transport that harnesses wind energy to supplement diesel propulsion and reduce carbon emissions. The core optimization problem addressed in this study is the global energy efficiency optimization of path planning and propulsion system cooperative control for wing–diesel hybrid ships under two typical sail operation statuses (sail-deployed and sail-stowed) with dynamic changes in complex maritime meteorological and hydrological conditions. To address this issue, this paper proposes a global energy efficiency optimization framework based on an improved Deep Q-Network (DQN) and D*Lite algorithm. Firstly, the D*Lite algorithm is reconstructed with an incremental replanning mechanism and risk-aware cost function to generate real-time safe path constraints. Secondly, the DQN is improved by adopting a dueling network, noisy exploration and prioritized experience replay, and a differentiated reward function dynamically weighted by sail statuses is designed for it. Finally, a fuel consumption prediction model based on the gradient boosting algorithm is integrated into the reward function to realize an accurate energy efficiency assessment. Empirical results confirm that the framework achieves remarkable carbon reduction effects: the optimized routes reduce the total fuel consumption by 5.02%, cut carbon dioxide emissions by 140.66 tons, and improve the energy efficiency operational index by 7.50%. This framework provides an effective technical solution for the dynamic energy efficiency optimization of wing–diesel hybrid ships under different sail operation statuses.

1. Introduction

1.1. Background

The issue of greenhouse gas emissions from maritime ships has garnered significant attention across the maritime industry [1]. Recently, the acceleration of the globalization process and trade growth has propelled the growth of the shipping industry [2], which exerts a significant influence on the global economy [3]. With the increase in cross-border trade [4], the demand for ships continues to rise, especially for large bulk carriers used in the transportation of commodities and goods. Secondly, sustainable development and safe navigation have become important focal points for the shipping industry. The international community’s concerns about environmental pollution are growing, prompting the International Maritime Organization (IMO) and numerous countries to adopt progressively stricter emission reduction targets and measures [5]. The shipping industry is striving to reduce carbon emissions and other pollutants. Faced with mounting pressure to reduce emissions, the shipping industry is increasingly turning to renewable energy-assisted propulsion technologies, with wind energy technology emerging as a pivotal strategy for sustainable development [6]. This shift is driving the adoption of more environmentally friendly ship designs and encouraging the integration of sustainable energy sources, as highlighted in the study by [7]. Among these innovations, wing–diesel hybrid ships constitute a particularly promising technological pathway within the spectrum of sustainable maritime transportation. These ships leverage wind energy by converting it directly into propulsive force, which significantly reduces reliance on conventional fuels and offers a practical means to address critical emission challenges faced by the shipping industry.

1.2. Literature Review

In ensuring high energy efficiency and achieving sustainable maritime operations, the considerations in global energy efficiency optimization are quite complex, involving multiple factors. For ships equipped with wind-assisted technologies, the global energy efficiency optimization strategy focuses on identifying optimal wind zones for energy utilization and the prudent avoidance of high-risk regions, thus leveraging wind energy to achieve significant fuel savings. It is also important to choose suitable approaches for sea area division and global planning methods to reduce fuel consumption and pollutant emissions. Therefore, these issues have attracted the attention of numerous scholars, and their respective influences are considered below:

1.2.1. Research on Weather Routing Approaches

A core challenge in global energy efficiency optimization is coping with complex and variable meteorological and oceanographic conditions, an area to which numerous studies are dedicated.

Meteorological data gathered from areas adjacent to intended shipping routes are subjected to K-means clustering analysis for the identification of predominant weather patterns [8]. The advantages of employing weather-based ship routing optimization are examined, and the assessment of shipping pollutants is integrated with their reduction using weather routing optimization, evaluating the emissions of ships for routes with minimum distance and optimized routes [9]. A novel dynamic weather routing framework is introduced, utilizing the Dijkstra algorithm for shortest paths, aimed at determining the optimal route to enhance ship performance in varying sea conditions [10].

However, most of these methods are designed for conventionally powered ships, where the optimization logic is primarily based on avoiding adverse weather to save fuel. For wing–diesel hybrid ships, the interaction with the environment undergoes a fundamental shift: wind transitions from a resistance that needs to be avoided into a propulsive force that can be utilized. Consequently, traditional energy efficiency optimization methods cannot be directly applied to this new scenario, which requires the active harnessing of wind energy. This inherent mismatch reveals the first critical research gap: traditional weather routing logic is incompatible with the wind-utilization characteristics of wing–diesel hybrid ships.

1.2.2. Research on Wind-Assisted Propulsive Force in Optimizing Energy Efficiency

The effectiveness of wind-assisted propulsive force in optimizing energy efficiency during ship operations has also attracted increasing attention from scholars.

An advanced weather routing system is put forward for ships utilizing the A* algorithm to ascertain the most efficient ship route and operations involving rotors with wind assistance [11]. The framework employs multiple data sources to generate a more accurate estimation of fuel consumption in varying sea conditions. A numerical model is introduced to conduct a theoretical evaluation of how the key parameters of wind speed and the planform area of the kite determine the propulsive force generated by a kite sail [12]. A comprehensive evaluation of the impact of resulting air pollution is conducted from hybrid propulsion with wind assistance under two distinct strategies for emission reduction, aiming to illustrate the practicality of the proposed framework [13]. In research on improving seaborne trade efficiency, a novel software tool is created to determine the fuel-optimal shipping route and evaluate the capabilities of ship propulsion systems with wind assistance, alongside conventional diesel engine propulsion [14]. In one study, potential energy savings are emphasized by constructing more slender bulk carriers paired with propulsion assisted by wind. In comparison to alternative transportation modes, the authors evaluated seaborne transport and dry cargo vessels for their energy efficiency [15]. In one study, the optimization of navigation routes was analyzed for wind-assisted ships, and different goals of optimization were defined [16]. Furthermore, the authors constructed optimization algorithms to achieve lower fuel consumption within a constrained voyage time under fixed main engine power conditions.

Nevertheless, these studies are often limited to localized optimization and fail to construct a global dynamic framework that collaboratively optimizes route safety, wind energy utilization, total fuel consumption, and voyage duration throughout the entire voyage. This lack of global, multi-objective collaborative optimization constitutes the second key research gap.

1.2.3. Research on Global Optimization Algorithms

Addressing the aforementioned problem requires a method capable of global optimization, making the selection of the specific algorithm itself particularly crucial. Currently, two main categories of mainstream algorithms have been applied in general path planning, including graph search algorithms and intelligent optimization algorithms.

(1)

These approaches are designed to create a path-search graph and to identify an energy efficiency optimal method [17]. A fuel-optimal path design method is developed based on the D*Lite algorithm, leveraging Automatic Identification System data and Simple Recurrent Units, in [18]. The issue of low-carbon global planning is formulated as an optimization issue, proposing an approach based on the hybrid A* algorithm to jointly determine fuel-optimal paths, in [19]. A detailed assessment of inherent fuel efficiency characteristics is provided to ice-strengthened vessel designs to enhance arctic ship navigation fuel efficiency for economic and environmental gains in [20].

While research on graph search algorithms benefits from their ability to find optimal solutions, they suffer from low computational efficiency. Particularly in dynamic environments, any change necessitates global replanning, making it difficult for these methods to meet real-time requirements.

(2)

Research efforts have also been directed towards applying intelligent algorithms to the problem of ship energy efficiency optimization. A comprehensive framework is developed for automated machine learning approaches to optimize global energy efficiency using AIS data, incorporating the sequential steps of route planning algorithms and pattern extraction, in [21]. An objective function is formulated using a multi-scenario collaborative optimization framework, which transforms the global energy efficiency planning issues of a VLCC into a problem with multiple optimization criteria, in [22]. An analysis technique is introduced using big data, designed to proactively mitigate the risk of inaccuracy in fuel consumption prediction, in [23]. A Deep Q-Network serves as the approach foundation for implementing a reward–penalty mechanism that considers both designated navigable zones and restricted areas for the energy efficiency of the entire voyage between ports in [24]. According to these studies, intelligent algorithms are characterized by their strong adaptability and ability to acquire strategies through learning.

More importantly, existing global optimization algorithms rarely consider the fundamental impact of different wing-sail operational statuses on the hybrid power system and optimization objectives. The switching of sail-deployed and sail-stowed states directly determines the power coordination mode and optimization priorities, yet such state awareness and adaptive adjustment are absent in current algorithms. This deficiency leads to the third research gap.

1.3. Gaps and Contribution

According to the comprehensive review of research on weather routing approaches, wind-assisted propulsive force for energy efficiency optimization, and global optimization algorithms in Section 1.2, the identified limitations of existing studies directly lead to three key research gaps in the global dynamic energy efficiency optimization for wing–diesel hybrid ships, as elaborated below: (1) Traditional global energy efficiency dynamic optimization methods (i.e., weather routing approaches) are primarily based on the logic of avoiding adverse weather to save fuel, which is not suitable for wing–diesel hybrid ships. For the latter, wind transitions from an obstacle that needs to be avoided to a propulsive force that must be actively harnessed. (2) Existing research on wind energy utilization often focuses on localized optimization, lacking a global dynamic framework that collaboratively optimizes route safety, wind energy utilization, total fuel consumption, and voyage duration throughout an entire journey. (3) Current global optimization algorithms generally fail to fully consider the fundamental impact of different operational states of the wing-sail on the ship’s power system and optimization objectives. Under varying wind conditions, the wing-sail’s state determines the coordination of the ship’s hybrid power modes. This necessitates optimization strategies with state-aware capability and adaptive weight adjustment based on state differences, a critical dimension that has not been adequately incorporated into existing path planning frameworks.

In response to the limitations of existing research, this paper proposes an adaptive energy efficiency dynamic optimization framework that integrates the D*Lite algorithm with a Deep Q-Network (DQN). The selection of these two algorithms and their combination is fully justified by the problem’s characteristics:

(1)

The global path planning of wing–diesel hybrid ships involves dynamic obstacles, real-time meteorological changes, and risk areas, which requires an algorithm with incremental replanning and high computational efficiency. The D*Lite algorithm is a reverse incremental dynamic search method naturally suitable for dynamic environments, making it the optimal choice for global safe path generation.

(2)

The energy efficiency decision-making under different sail statuses is a high-dimensional sequential decision problem with strong nonlinearity, which cannot be well handled by traditional optimization or rule-based methods. Our improved DQN based on deep reinforcement learning can adaptively learn optimal strategies through environmental interaction, matching the requirements of sail-status-aware dynamic control.

(3)

The combination of D*Lite and DQN is complementary and synergistic: D*Lite provides stable global safety constraints, while DQN performs local adaptive energy efficiency optimization within the safe corridor. This hybrid structure overcomes the weaknesses of single algorithms—such as the insufficient decision-making adaptability of graph search methods and a lack of global safety guarantees in single DQNs.

The main contributions of this paper are as follows:

(1) The proposal of a differentiated energy efficiency optimization framework based on wing-sail state awareness. By deeply analyzing the energy transfer mechanism of the ship’s hybrid power configuration, this work distinguishes optimization objectives under different scenarios, such as deployed or stowed wing-sails. It establishes a collaborative energy efficiency optimization framework capable of dynamically adapting to energy mode transitions, thereby resolving the mismatch between traditional optimization logic and the characteristics of wing–diesel hybrid ships. (2) The algorithm combination is determined by the complementary technical features of each method and the core navigation demands of real-time safety, dynamic decision-making and accurate energy efficiency assessment: the reconstructed D*Lite algorithm, with an incremental replanning mechanism and risk-aware cost function, generates real-time safe path constraints, making it ideal for path planning in dynamic marine environments; the improved DQN, with a dueling network, noisy exploration, prioritized experience replay and a sail status-weighted differentiated reward function, excels at capturing the marine environment–ship operation nonlinear relationship and enabling adaptive dynamic decision-making for different sail statuses. (3) The gradient boosting-based fuel consumption prediction model is embedded into the DQN’s reward function to achieve accurate energy efficiency assessment, as the algorithm can precisely capture the multi-factor–fuel consumption nonlinear correlation and provide a reliable quantitative basis for energy efficiency optimization.

This paper’s structure is organized as follows: Section 2 introduces the materials and methodologies. Section 3 presents the experimental results. Section 4 discusses the optimization performance and compares the proposed framework with existing studies. Section 5 summarizes the main conclusions of this study.

2. Materials and Methods

2.1. Materials

2.1.1. Ship and Wing-Sail Parameters

The research object is a wing–diesel hybrid ship operating on the route from Singapore to Cape Town. This voyage is set as the unified reference evaluation scenario of this study, covering typical open waters, restricted waters, and complex meteorological and hydrological environments in the Indian Ocean, which can fully verify the adaptability and optimization effect of the proposed framework. Schematic representations of the target ship and wing-sails are illustrated in Figure 1, while the ship’s core operational parameters and wing-sail design parameters are determined based on practical engineering applications and relevant research [25], including ship speed range, main engine power, wing-sail height (deployed: ≥20 m; stowed: <20 m), thrust coefficient range, and the side projection area of the wing-sail. The wing-sail system is designed to provide additional propulsive force under favorable wind conditions, and its operational status is switched according to preset environmental thresholds and navigation scenarios.

2.1.2. Data Sources

Three types of multi-source data form the foundation of the proposed optimization framework, providing support for path planning, intelligent decision-making, and fuel consumption prediction:

(1)

Ship operational data

Collected from the target wing–diesel hybrid ship during its actual voyage from Singapore (8:20 a.m., 22 October 2022) to Cape Town (4:30 p.m., 9 November 2022), including ship speed, course, main engine RPM, draft, trim, and fuel consumption data, with a sampling interval of 10 min.

(2)

Meteorological and hydrological data

Obtained from the European Centre for Medium-Range Weather Forecasts (ECMWF), including wind speed, wind direction, significant wave height, wave direction, and water depth data, with a spatial resolution of 1° × 1° and a temporal resolution of 6 h. Within this study, the distribution characteristics of navigation conditions are illustrated in Figure 2, Figure 3 and Figure 4. A notable correlation exists between wind speed and wave height, quantifiable through the Pearson correlation coefficient method [26]. The findings delineate distinct sequential patterns within meteorological data, while the parameters demonstrate discernible fluctuations across diverse temporal scopes and geographical regions.

(3)

Piracy risk data

Piracy risk data are derived from the International Maritime Bureau (IMB) piracy armed robbery incident reports, which are used to identify piracy-prone areas in the target sea area.

2.1.3. Target Sea Area Demarcation

Utilizing hydrological data, an electronic chart is created relying on methods for coordinate transformation and scale conversion [27]. The data for the environment employed in this study maintains an accuracy of $1°\times 1°$, with a grid size of $1°\times 1°$. Following rasterization preprocessing, the electronic chart identifies grid cells presenting navigation impediments, which are then designated as restricted grids. Employing the binary approach, the navigable sea region is partitioned, assigning navigable grids a value of land, designating restricted grids with a value of 0. Furthermore, regions with wave heights over 6 m or more than Grade 8 wind are categorized as restricted grids [28]. To determine navigational suitability, the water depth across the grid is measured against the ship’s details. Grid points failing to meet these requirements are labeled as non-navigable points: $Navi(x,y)=0$.

The relevant data on pirate armed robbery incidents were obtained from the IMB’s reports on piracy. The results of processing the relevant data occurring in the target sea area using the K-means algorithm are depicted in Figure 5. Figure 5 demonstrates that when the cluster centers stabilize, the distribution of pirate robbery activities can be divided into three clusters. The corresponding positions on the grid map will then be supplemented onto the original grid map in an incremental mapping manner.

Likewise, through the integration of ship performance data with meteorological features, the navigational feasibility of candidate grid points at time $t$ is evaluated. Grid points failing to conform to the specified criteria are designated as non-navigable points: $Navi(x,y)=0$. This method significantly streamlines the search area and enhances efficiency, with the blue area denoting the region of pirate activity. In addition, the environmental state is updated every hour, including meteorological data and obstacle positions. The navigability status (0 or 1) of grid nodes is refreshed in real time to simulate the dynamic changes in a real maritime environment. The preparation of grid points using this methodology can be formulated as follows:

$$Navi\left(x,y,t\right)=\left\{\begin{array}{c}0,Non\text{-}navigation\\ 1,Navigation\end{array}\right.$$

(1)

2.2. Data Preprocessing

To ensure the consistency and validity of data for model training and algorithm optimization, the collected multi-source data is subjected to a series of preprocessing steps, including abnormal data cleaning, spatiotemporal interpolation, and relative wind parameter calculation:

(1)

Abnormal data cleaning

Outliers and noise in ship operational data and meteorological data are removed using the 3σ criterion to eliminate the impact of sensor errors and data transmission anomalies.

(2)

Spatiotemporal interpolation

Since the sampling interval of ECMWF meteorological data (6 h) is inconsistent with that of ship operational data (10 min), cubic spline interpolation is used to interpolate the meteorological data along the planned route, synchronizing the temporal resolution of all data to 10 min [29], as depicted in Figure 6. This ensures that each ship’s operational data point is matched with corresponding high-precision meteorological data.

Utilizing the approach of cubic spline extrapolation, the weather data frequency was adjusted to match the actual ship data intervals, facilitating the development of the efficiency database. A data segment obtained through interpolation calculations is illustrated in

Table 1. Partially interpolated data.

Sample Time	Latitude (°)	Longitude (°)	Sailing Speed (Knot)	Course	…	Characteristic Wave Height (m)	Wave Direction (°)	Relative Wave Direction (°)
22 October 2022 8:20	1.32	104.16	13.35	160.50	…	0.153	332.99	172.49
22 October 2022 8:30	1.31	104.16	13.23	142.35	…	0.154	331.25	188.89
22 October 2022 8:40	1.30	104.16	13.13	140.30	…	0.156	334.96	194.66
…	…	…	…	…	…	…	…	…
9 November 2022 16:10	−34.55	18.06	13.70	334.82	…	2.697	224.08	−110.74
9 November 2022 16:20	−34.51	18.04	13.80	337.98	…	2.711	223.92	−114.06
9 November 2022 16:30	−34.48	18.02	13.85	337.05	…	2.726	223.76	−113.28

.

(3)

Relative wind parameter calculation

ECMWF provides true wind data, while the wing-sail’s aerodynamic performance depends on relative wind (the vector combination of true wind and ship motion). The relative wind speed and relative wind direction are calculated using vector analysis based on the ship’s speed, course, true wind speed, and true wind direction, which are key input parameters for the wing-sail thrust calculation and fuel consumption prediction model. The corresponding coordinate system and the process of vector composition for the wing-sail model are illustrated in Figure 7, and the dataset on wing-sail efficiency after calculation is displayed in

Table 2. Subset of relative wind data.

Sample Time	Latitude (°)	Longitude (°)	Sailing Speed (Knot)	Course	Wind Velocity (m/s)	Wind Direction (°)	Relative Wind Velocity (m/s)	Relative Wind Direction (°)
22 October 2022 8:20	1.32	104.16	13.35	160.50	3.95	279.06	8.37	122.20
22 October 2022 8:30	1.31	104.16	13.23	142.35	3.96	279.55	8.96	108.68
22 October 2022 8:40	1.30	104.16	13.13	140.30	3.94	280.54	8.31	101.00
…	…	…	…	…	…	…	…	…
9 November 2022 16:10	−34.55	18.06	13.70	334.82	6.44	203.54	11.42	83.98
9 November 2022 16:20	−34.51	18.04	13.80	337.98	6.45	201.70	13.54	60.84
9 November 2022 16:30	−34.48	18.02	13.85	337.05	6.46	199.92	12.40	58.11

.

2.3. Fuel Consumption Prediction Model Based on Gradient Boosting Algorithm

2.3.1. Wind-Assisted Diesel Power System

In the wind-assisted diesel power system, the primary driving force for navigation is generated through a dual-source configuration, comprising both sail-based aerodynamic propulsion and the conventional main propulsion engine. Figure 8 depicts the design configuration of the hybrid power system, which combines the wind assistance with the diesel engine. If the sailing conditions are ideal for unfurling the sail, it is raised to increase thrust force [30]. During this period, the wind-assisted system and diesel engine work together to counteract resistance while the ship is underway, consequently reducing the power output needed from the wing–diesel hybrid ship’s diesel engine [31].

The dynamic maritime environment induces fluctuations in ship operational states, which subsequently modify the operational behavior of the aerodynamic surfaces, the main engine, and propellers. These interdependent variations ultimately impact the thermodynamic efficiency of the combustion-based power generation units, thereby influencing the holistic energy utilization effectiveness of the ship. The systemic energy flow dynamics within this propulsion architecture are quantitatively illustrated in Figure 9.

2.3.2. Model Principle

Focusing on wing–diesel hybrid ships, this study employs a gradient boosting algorithm for fuel consumption prediction, rather than methods like Random Forest suited for single-factor (e.g., speed) analysis in conventional vessels [32]. This approach is necessitated by the need to model the coupled effects of multiple dynamic factors, such as wing-sail state transitions and changing ocean environments. The gradient boosting algorithm is an ensemble learning method that improves predictive performance by iteratively training multiple weak learners (decision trees) to correct prediction errors [33]. It can accurately capture the nonlinear interdependencies between fuel consumption and its key driving variables, and its residual learning mechanism enables continuous optimization of predictive results through sequential training. The final ensemble model is formed by a weighted summation of the predictive outputs of all weak learners, with higher weights assigned to learners with lower prediction errors. The complete computational workflow for model development and hyperparameter optimization is illustrated in Figure 10.

Notably, this gradient boosting-based fuel consumption prediction model serves as a core surrogate model for the entire global energy efficiency optimization framework proposed in this study, and its prediction accuracy is a foundational guarantee for the effective operation of the framework. The model is the direct quantitative basis for the construction of the DQN algorithm’s reward function, especially the fuel consumption reward component that accounts for the core optimization objective of energy efficiency. The rationality of the DQN’s reward signal, which guides the algorithm’s adaptive learning and action selection, is directly determined by the accuracy of the fuel consumption predicted by this model; in turn, the convergence speed and the optimality of the final learned strategy of the DQN algorithm are closely related to the rationality of the reward signal, ultimately exerting a decisive influence on the global energy efficiency optimization effect of the entire framework for wing–diesel hybrid ships.

2.3.3. Model Training

The sequential phases of model training are as follows:

(1)

Training data preprocessing

Abnormal data cleaning and dataset standardization are performed on the training set to eliminate noise and outliers that may interfere with model training. The experimental data used in this study is derived from the operational parameters of the target wing–diesel hybrid ship [34], with the key input features of the model listed in

Table 3. Features in dataset.

Features	Unit
Ship speed	$\mathrm{k}\mathrm{n}$
Significant height of wind waves	$\mathrm{m}$
Wave direction	$°$
Relative wave direction	$°$
Relative wind direction	$°$
Relative wind speed	$\mathrm{m}/\mathrm{s}$

.

(2)

Hyperparameter setting of the gradient boosting algorithm

The critical hyperparameters of the gradient boosting algorithm are calibrated and fixed, as shown in

Table 4. Parameters setting for the gradient boosting algorithm.

$\mathbf{n\_}\mathbf{E}\mathbf{s}\mathbf{t}\mathbf{i}\mathbf{m}\mathbf{a}\mathbf{t}\mathbf{o}\mathbf{r}\mathbf{s}$	$\mathbf{L}\mathbf{e}\mathbf{a}\mathbf{r}\mathbf{n}\mathbf{i}\mathbf{n}\mathbf{g\_}\mathbf{R}\mathbf{a}\mathbf{t}\mathbf{e}$	$\mathbf{M}\mathbf{a}\mathbf{x\_}\mathbf{D}\mathbf{e}\mathbf{p}\mathbf{t}\mathbf{h}$	$\mathbf{S}\mathbf{u}\mathbf{b}\mathbf{s}\mathbf{a}\mathbf{m}\mathbf{p}\mathbf{l}\mathbf{e}$	$\mathbf{M}\mathbf{i}\mathbf{n\_}\mathbf{S}\mathbf{a}\mathbf{m}\mathbf{p}\mathbf{l}\mathbf{e}\mathbf{s\_}\mathbf{L}\mathbf{e}\mathbf{a}\mathbf{f}$
100	0.03	8	0.85	5

, to balance the model’s fitting ability and generalization performance and avoid overfitting or underfitting.

(3)

Initialization of training set weight distribution

The initial weight distribution of the training samples is set uniformly to ensure that each sample has an equal contribution to the training of the first weak learner [35].

(4)

Iterative training of weak learners

In each training iteration, a new weak learner is trained on the current weighted training set. After training, the weight distribution of the training samples is dynamically adjusted: the weights of correctly classified samples are reduced to reduce their influence on subsequent training, while the weights of misclassified samples are increased to focus the model’s learning on these error-prone cases. The adjusted weight distribution is then used to train the next weak learner in the boosting sequence.

(5)

Ensemble of strong learners

The final strong learner is constructed by a weighted summation of the predictive results of all trained weak learners. In the ensemble process, a weighting coefficient is assigned to each weak learner based on its prediction accuracy: learners with lower classification errors receive higher weighting coefficients and thus exert a greater influence on the final prediction, while those with higher error rates are assigned lower weights to minimize their adverse impact on the ensemble model.

2.3.4. Model Performance Verification

The model’s predictive performance is evaluated against quantitative criteria by measuring statistical alignment between forecasted results and observed values. This study utilizes Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) as principal evaluation criteria, with their computational formulations mathematically defined in Equations (2) and (3).

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-f(x_i\left)\right)}^2}$$

(2)

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|y_i-f\left(x_i\right)\right|$$

(3)

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^n}(y_i-\widehat{y_i}{)}^2}{{\displaystyle \sum _{i=1}^n}(y_i-\overline{y}{)}^2}$$

(4)

where $f\left(x\right)$ represents the actual engine fuel consumption; $y$_i represents the model-predicted value; and $n$ is the sample number.

2.4. Optimization Algorithm Framework

The overall adaptive energy efficiency optimization framework proposed in this study is illustrated in Figure 11. This framework is explicitly defined as a hybrid global–local collaborative optimization framework: the improved D*Lite algorithm is responsible for real-time global safe path planning with risk awareness and incremental replanning, and the improved DQN algorithm performs local adaptive energy efficiency decision-making within the safe path corridor, both of which work together to achieve dynamic optimization under different sail operation statuses. This framework integrates ship operational data, comprehensive weather forecasts, and improved hybrid algorithm optimization techniques, forming a joint optimization method combining an improved Deep Q-Network (DQN) and the D*Lite algorithm for the global energy efficiency optimization of wing–diesel hybrid ships under uncertain meteorological conditions.

In summary, this study formulates the global energy efficiency optimization problem for wing–diesel hybrid ships as a multi-constraint joint optimization problem. During the optimization process, safety, navigation, and performance-related constraints are strictly enforced, including the maximum heading change amplitude, speed limits, obstacle avoidance, water depth restrictions, and tolerable maximum wind and wave conditions. When the sail is deployed, the optimization objective focuses on minimizing total fuel consumption; when the sail is stowed, the optimization objective dynamically increases the weight assigned to navigation safety. This adaptive weighting mechanism, based on sail operational statuses, achieves multi-scenario joint energy efficiency optimization while ensuring that the ship reaches its destination within the specified time. Ultimately, the fuel-optimal route generated by the proposed framework can be dynamically adjusted according to real-time weather conditions and operational constraints, thereby achieving the co-optimization of ship energy efficiency and navigation safety.

2.4.1. Wing-Sail Operational Status Definition

Table 5. Selection criteria for different sail status strategies.

Sai Status	Environmental Threshold	Shipping Scenario	Duration Requirement
Sail-deployed status	$V_{wind}\le {\displaystyle \frac{23.5 \mathrm{m}}{s}},$ $H_{wave}\le 6 \mathrm{m},$ $\beta \in [20°,340°]$	Open waters	$T_{duration}\ge 5 \mathrm{m}\mathrm{i}\mathrm{n}$
Sail-stowed status	$V_{wind}>{\displaystyle \frac{23.5 \mathrm{m}}{s}},$ $H_{wave}>6 \mathrm{m}$	Departure/Berthing, Restricted waters, Piracy-prone areas	Instant trigger

demonstrates the two typical operational statuses of wing-sails, with switch criteria based on environmental thresholds, navigation scenarios, and duration requirements [36]. The thrust coefficient of the wing-sail in the deployed status is consistently higher than that in the stowed status under the same wind direction angle due to the larger wind-facing area and better aerodynamic performance of the fully deployed wing-sail, as shown in Figure 12. This characteristic can be attributed to the increased sail height providing a larger wind-facing area and superior aerodynamic performance.

(1)

Sail-deployed status

The sail-deployed status was triggered in open waters when wind speed ≤ 23.5 m/s, wave height ≤ 6 m, and relative wind direction angle β ∈ [20°, 340°], with a minimum duration of 5 min. The core objective is to maximize wind energy utilization and minimize fuel consumption.

(2)

Sail-stowed status

The sail-stowed status was instantly triggered in departure/berthing waters, restricted waters, piracy-prone areas, or when wind speed > 23.5 m/s or wave height > 6 m. The core objective is to prioritize navigation safety and maneuverability, with fuel economy as a secondary objective.

2.4.2. Improved D*Lite Algorithm for Global Safe Path Planning

The traditional D* Lite algorithm is a reverse incremental dynamic path search algorithm with a variable starting point. Its core principle is to find the shortest distance from the destination to each grid by minimizing the $rhs\left(s\right)$ value [37]. To adapt to the dynamic marine environment and the characteristics of wing–diesel hybrid ships, the D*Lite algorithm is improved with an incremental replanning mechanism and a risk-aware cost function:

(1)

Cost function optimization

When meteorological and hydrological environmental changes during the ship’s voyage along the pre-planned path render previously navigable areas impassable or high-risk, thus affecting the subsequent route, the algorithm takes the ship’s current position as the new starting point and proceeds to update the heuristic values and estimated costs. The value of $rhs\left(s\right)$ is given by the following expression:

$$rhs\left(p\right)=\left\{\begin{array}{c}0, if p=p_{start}\\ {min}_{p^{\prime }\in Suss\left(p\right)}c\left(p^{\prime },p\right)+g\left(p^{\prime }\right),if not\end{array}\right.$$

(5)

where $p$ and $p^{\prime }$ are the current point and the parent grid point respectively, $Suss\left(p\right)$ denotes the child grid points of $p$, $c\left(p^{\prime },p\right)$ demonstrates the cost distance between the grid point and the parent grid point, and $g\left(p^{\prime }\right)$ is the actual cost from the currently expanded grid node to the target point.

To facilitate the efficient calculation of the node evaluation function in the improved D* Lite algorithm for ship pathfinding, the Manhattan distance serves as the primary measure for inter-node path costs. Secondly, to obtain a more accurate path cost function and reduce the number of expanded nodes with identical evaluation function, $E$, the algorithm is refined. Additionally, the influence of wing-sails on the grid cost function during the ship navigation is considered. Based on the above, the updated expression for $c\left(p,p^{\prime }\right)$ is as follows:

$$\left(p,p^{\prime }\right)={\tau }_1·\left[\sqrt{2}\mathit{min}\left(\left|x_p-x_{p^{\prime }}\right|,\left|y_p-y_{p^{\prime }}\right|\right)+\left|\left|x_p-x_{p^{\prime }}\right|-\left|y_p-y_{p^{\prime }}\right|\right|\right]+{\tau }_2·(1-{\displaystyle \frac{F_{wind}(x_p,y_p)}{F_{max}}})$$

(6)

$$F_{wind}={\displaystyle \frac{1}{2}}\rho C_T(\alpha )v_{wind}^2$$

(7)

where $x_p$ and $x_{p^{\prime }}$ are values on the x-axis for the current grid point and the parent grid point respectively, $y_p$ and $y_{p^{\prime }}$ are values on the y-axis for the current grid point and the parent grid point respectively, $F_{wind}$ is the aerodynamic thrust, $\rho$ is air density, and $C_T$ is the thrust coefficient.

As described above, the cost of moving to adjacent cells is 1.5 for horizontal/vertical steps and $\sqrt{3}$ for diagonal steps. This definition better aligns with the actual path cost characteristics of wing–diesel hybrid ships. A comparative analysis of the computed paths using the improved search method is shown in Figure 13. In Figure 13b, the node expansion number (highlighted in light blue) significantly decreases in the improved D* Lite algorithm, thereby enhancing both computational efficiency and route planning precision.

(2)

Incremental replanning mechanism

The algorithm updates the navigability state, $Navi(x,y,t)$, of the grid map in real time (every 10 min) based on dynamic obstacle data (e.g., other ships’ positions from AIS). If the path cost change exceeds a preset threshold, the algorithm triggers incremental replanning with the ship’s current position as the new starting point, avoiding global replanning and improving computational efficiency.

(3)

Node expansion optimization

The Manhattan distance is used to calculate the inter-node path cost, reducing the number of expanded nodes with the same evaluation function, $E$, and improving route planning precision. $E$ is defined as a two-dimensional vector as follows:

$$E=\left[\begin{array}{c}{E}_1\\ E_2\end{array}\right]=\left[\begin{array}{c}\mathit{min}\left(g\left(p\right),rhs\left(p\right)\right)+h\left(p,p_{initial}\right)\\ \mathit{min}\left(g\left(p\right),rhs\left(p\right)\right)\end{array}\right]$$

(8)

$$h\left(p,p_{initial}\right)=\left\{\begin{array}{c}0,if p=p_{start}\\ c\left(p,p^{\prime }\right)+h\left(p^{\prime },p_{goal}\right),if not\end{array}\right.$$

(9)

where $h\left(p,p_{initial}\right)$ is the heuristic function representing the cost between the starting point and the current node, and $E$ determines the order of node expansion that firstly compares $E_1$ or compares $E_2$ if $E_1$ values are equal.

2.4.3. Improved DQN Algorithm for Dynamic Adaptive Decision-Making

The traditional DQN algorithm integrates Q-learning with deep learning, using a neural network to approximate the action-value function and select optimal actions based on the current state [38]. To enhance its adaptability to the marine environment and sail status changes, the DQN is improved by integrating a dueling network, noisy exploration, and prioritized experience replay, and a sail status-aware differentiated reward function is designed.

(1)

State and action space definition

The state space in this study encompasses all possible positions of the ship, as shown in Figure 14. The environmental states are represented using grid map points, defined by their two-dimensional coordinates. The specific expression is below:

$$s=\left[x,y,\psi ,v,v_{wind},{\theta }_{wind},swh,\alpha \right]$$

(10)

where $\psi$ represents ship course, $v$ denotes ship speed, ${\theta }_{wind}$ indicates wind direction, and $\alpha$ is the attack angle of the wing-sail.

Navigation in this study was quantized, and the composition of the specific action space is expressed as follows:

$$a=\left[\begin{array}{c}\begin{array}{c}\Delta \alpha \\ \Delta RPM\end{array}\\ \Delta \psi \end{array}\right]$$

(11)

where $\Delta \mathrm{\alpha }$ represents the attack angle adjustment of the wing-sail, $\Delta \mathrm{\alpha }\in \left\{-30°,0,30°\right\}$; $\Delta RPM$ denotes the diesel engine RPM change, $\Delta RPM\in \left\{-5\%,\mathrm{0,5}\%\right\}$; and $\left|\Delta \psi \right|\le \Delta {\psi }_{max}$ (maximum heading change amplitude) to ensure maneuver smoothness. The main engine RPM must satisfy ${RPM}_{min}\le RPM+\Delta RPM\le {RPM}_{max}$ to avoid unsafe operation.

(2)

DQN Improvements

Dueling network: The action-value function, $Q(s,a)$, is decoupled into a state value function, V(s), and an advantage function, $A\left(s,a\right)$, improving the accuracy of state and action evaluation [39]. A diagram of the dueling network architecture is presented in Figure 15, and its functional expression is presented as follows:

$$Q\left(s,a,w\right)=V\left(s\right)+\left[A\left(s,a,w\right)-{\displaystyle \frac{1}{\left|A\right|}}\sum _{a^{\prime }}A\left(s,a^{\prime },w\right)\right]$$

(12)

where $\left|A\right|$ represents the number of available actions.

Noisy exploration: Parameterized noise is added to the network weights to replace the ε-greedy strategy, enabling smoother policy exploration and avoiding random blind exploration. The modified weight expression is

$$w={\mu }_w+{\sigma }_w⨀ϵ,ϵ\sim \aleph \left(\mathrm{0,1}\right)$$

(13)

where ${\mu }_w$ and ${\sigma }_w$ are the arithmetic mean and statistical dispersion of the noisy network, respectively, and $ϵ$ is the random noise parameter.

Prioritized experience replay: The Temporal Difference (TD) error is used as an importance indicator for experience samples, with higher sampling priority assigned to samples with larger TD errors to optimize training efficiency. The specific expression is as follows:

$$P\left(i\right)\propto {\left|{\delta }_i\right|}^{\xi },{\delta }_i=r_i+\gamma \underset{a^{\prime }}{\mathit{max}}{Q}_{target}\left(s_{i+1},a^{\prime }\right)-Q(s_i,a_i)$$

(14)

where $P\left(i\right)$ represents the sampling probability, and $\xi$ is the hyperparameter.

(3)

Sail Status-Aware Differentiated Reward Function

A multi-component reward function is designed, integrating fuel consumption reward, progress reward, and safety reward, with dynamic weight adjustment based on wing-sail operational statuses. The total reward, $r_t$, is defined as

$$r_t=\left\{\begin{array}{c}{w}_{fuel}·r_{fuel}+w_{safe}·r_{safety\_depoyed}+w_{progress}·r_{progress}, H_{sail}\ge 20m\\ w_{fuel}·r_{fuel}+w_{safe}·r_{safety\_stowed}+w_{progress}·r_{progress}, H_{sail}<20m\end{array}\right.$$

(15)

where $w_{fuel}$, $w_{safe}$, and $w_{progress}$ are the weights of fuel consumption reward, safety reward, and progress reward, respectively. The weight of $w_{fuel}$ is set to the maximum in the sail-deployed status, while the weight of $w_{safe}$ is set to the maximum in the sail-stowed status.

• Fuel reward function, $r_{fuel}$

This is designed to minimize total fuel consumption and is quantified based on the fuel consumption prediction model and wing-sail thrust contribution:

$$r_{fuel}=1-{\displaystyle \frac{F_{total}}{F_{ref}}}=1-{\displaystyle \raisebox{1ex}{$(\sum _{k=1}^N\left[\frac{P_{diesel}\left(k\right)·SFOC·\Delta t_k}{{10}^6}-\Delta F_{wing}(k)\right])$}\!\left/ \!\raisebox{-1ex}{$F_{ref}$}\right.}$$

(16)

$$\Delta F_{wing}(k)={\displaystyle \frac{1}{2}}{\rho }_a·C_T(\alpha ,\beta )·S_w·v_a^2(k)·{\displaystyle \frac{\Delta t_k}{{10}^6}}$$

(17)

where $F_{ref}$ serves as a reference fuel consumption value for reward normalization, and $N$ indicates the number of path segments. The net fuel consumption aggregates the contribution across all segments: the main engine’s fuel usage, $P_{diesel}\left(k\right)·SFOC·\Delta t_k$, minus the savings, $\Delta F_{wing}\left(k\right)$, is attributable to the wing-sail per segment. The quantity, $\Delta F_{wing}\left(k\right)$, captures the aerodynamic contribution of the wing-sail and is calculated using the air density, ρ_a; the thrust coefficient, $C_T(\alpha ,\beta )$; the wing-sail’s side projection area, $S_w$; and the segment’s relative wind speed, $v_a^2\left(k\right)$.

• Progress reward, $r_{progress}$

This is designed to incentivize efficient navigation to the destination and is quantified based on the ratio of the straight-line reference distance to the actual route length:

$$R_{progress}={\displaystyle \frac{D_{ref}}{D_{total}}}$$

(18)

where the total route length $D_{total}$ is calculated as the sum of the Euclidean distances separating each waypoint along the planned path. For reference, $D_{ref}$ typically represents the straight-line distance from the journey’s starting point to its final endpoint.

• Safety reward function, $r_{safety}$

This is designed to penalize deviations from the global safe path generated by the D*Lite algorithm, with differentiated designs for the two sail statuses:

Sail-deployed status: A tolerant linear penalty is applied, allowing for reasonable deviations within a safety corridor to explore wind-rich areas:

$$r_{safety\_depoyed}=-{\lambda }_{deployed·}\mathit{max}\left(0,{\displaystyle \frac{{dist}_{deviation}-d_{safe}}{d_{safe}}}\right), if H_{sail}\ge 20m$$

(19)

where ${dist}_{deviation}$ is the shortest distance from the ship’s current position to the D*Lite safe path, $d_{safe}=k·R_{turning}$ (k = 2) is the safety corridor width, $R_{turning}$ is the ship’s handling radius, and ${\lambda }_{deployed·}$ is a small penalty coefficient.

Sail-stowed status: A quadratic penalty is applied to strictly constrain path deviations and ensure navigation safety:

$$r_{safety\_stowed}=-{\lambda }_{stowed·}·{\left({\displaystyle \frac{{dist}_{deviation}}{d_{safe}}}\right)}^2, if H_{sail}<20m$$

(20)

where ${\lambda }_{stowed·}\gg {\lambda }_{deployed·}$ is a large penalty coefficient to impose severe penalties for any deviation.

2.4.4. Integration of D*Lite and DQN Algorithm

The improved D*Lite and DQN algorithms are integrated in a hybrid framework to achieve collaborative optimization of global safety and local energy efficiency:

(1)

Initialization

The D*Lite algorithm generates a global safe path, which is used as prior data to initialize the Q-network weights of the DQN algorithm:

$$Q_{initial}\left(s,a\right)=\eta ·P_{reward}(s,a)$$

(21)

where $\eta$ represents the network weight initialization coefficient, and $P_{reward}(s,a)$ denotes the initial safe path.

(2)

Real-time collaborative optimization

The D*Lite algorithm maintains the global safe path constraint and triggers incremental replanning when the marine environment changes; the DQN algorithm performs fine-tuned local path optimization within the safety corridor generated by the D*Lite algorithm, dynamically adjusting actions (wing-sail attack angle, main engine RPM, course) based on the current sail status and environmental conditions to optimize energy efficiency.

(3)

Experience update and network training

The experience tuple $(s_t,a_t,r_t,s_{t+1})$ generated by the ship’s interaction with the environment is stored in the replay buffer; the DQN network is trained using batch sampling based on prioritized experience replay, and the network weights are updated using the loss function:

$$L_{loss}=E\left[(r+\gamma \underset{a^{\prime }}{\mathit{max}}{(Q\left(s^{\prime },a^{\prime },w\right)-Q(s,a,w\left)\right))}^2\right]$$

(22)

where $s$ and $a$ represent the current time step’s chosen state and action respectively, $s^{\prime }$ and $a^{\prime }$ represent the state and action picked at the next time interval respectively, $r$ is the environmental reward, $\gamma$ is the discount factor, and w represents the network weight parameters.

A detailed flowchart of this process is shown in Figure 16.

2.4.5. Algorithm Flow Design

The improved DQN with the D*Lite algorithm adheres to the prescribed sequence of steps, as delineated in Figure 17.

(1)

A comprehensive set of multi-source data is leveraged, including ship performance metrics, wing-sail parameters, diesel engine specifications, and dynamic meteorological and hydrological conditions, which are loaded and subjected to preprocessing. The fuel consumption prediction model is subsequently initialized based on this curated data, serving as the cornerstone for evaluation in the ensuing optimization framework.

(2)

We construct a risk-factor grid map and introduce a risk function. We initialize the priority queue U as an empty set. We set the $rhs\left(s\right)$ required heuristic value and actual cost from the current node to the target node, $g\left(s\right)$, of all grids to infinity. Specifically, we set the $rhs$ value of the goal node, $s_{goal}$, to 0, calculate the key value of $s_{goal}$ based on the evaluation function ($E$), and insert $s_{goal}$ into the priority queue $U$.

(3)

For the expansion nodes of the current node, we calculate the $k$ values derived from the key value of nodes in the priority queue and insert these expansion nodes into the priority queue $U$. We pop the node with the smallest $k$ value from $U$ and designate it as the new current node. We repeat this node expansion and selection process until the start node $s_{start}$ is reached. On this basis, we compute the pre-planned path to initially generate a feasible navigation path.

(4)

After moving to the next waypoint according to the pre-planned path generated by the D*Lite algorithm, we check whether the environment has changed. The environment change is determined by synchronizing external data sources to refresh the navigability state, $Navi(x,y,t)$, of the grid map, to show that grids occupied by new obstacles or assessed as high-risk indicate an environment change.

If the environment has changed, we update the specific parameters of the affected nodes and their adjacent nodes, return to Step (3) to re-calculate the path, and convert the segments where the new path deviates from the previous path into experience samples.

If the environment remains unchanged, continue calculating the path to the next waypoint until the goal node $s_{goal}$ is reached.

(5)

Based on the initial safe path generated in Steps (2) to (4), we initialize the weights of the dueling DQN rather than a traditional Q-table. Specifically, we substitute the initial safe path’s reward, $P_{reward}(s,a)$, to determine the initial Q-values of the dueling network.

(6)

We set the initial iteration count: $I_n=0$. We compare whether the iteration count, $I_n$, exceeds the maximum allowed iterations per cycle, $I_{max}$:

If $I_n>I_{max}$, proceed to Step (8);

If $I_n\le I_{max}$, proceed to Step (7).

(7)

We select an appropriate action using the noisy exploration strategy. The selected action must comply with the ship’s physical maneuvering constraints, and we calculate the comprehensive reward value, $r_t,$ using the state-dependent reward function.

We update the agent’s (wing–diesel hybrid ship) position, refresh the Q-values of the dueling network, and increment the iteration count, $I_n$, by 1.

(8)

When $I_n>I_{max}$, we output the current Q-values of the dueling network and check for convergence. Convergence is determined by whether the fluctuation of Q-values across consecutive iterations is below a preset threshold:

If converged, we terminate the computation and proceed to Step (10);

If not converged, we proceed to Step (9).

(9)

We update the experience pool using prioritized experience replay:

We incorporate the transition tuple ($s,a,r,s^{\prime }$) into the experience pool and assign sampling priorities based on the TD error.

We perform batch sampling from the experience pool, train the dueling DQN using the loss function, and update the network weights.

We reset the iteration count, $I_n=0$, and return to Step (5).

(10)

The algorithm concludes by defining the optimal navigation policy. This policy is constructed such that for any state, $s$, it selects the action, $a$, that maximizes the Q-value function. The resulting policy can, therefore, generate optimal paths that dynamically balance the contributions of wing-sail thrust and diesel engine power while ensuring navigation safety.

3. Results

According to Figure 18, ship operational data and marine weather data form the foundation of the entire adaptive energy efficiency optimization framework. They provide the basis for D*Lite to plan safe paths, supply the perceptual and decision-making basis for the DQN agent, and offer the learning and prediction samples for the gradient boosting-based fuel consumption model.

For the core hyperparameters of the hybrid algorithm, including the discount factor, γ (0.85–0.99), and learning rate, α (0.001–0.01), of the improved DQN, as well as the risk weight coefficient, τ₂ (0.1–0.5), of the improved D*Lite, we adjusted the parameter values within a reasonable range and analyzed the changes in algorithm convergence and optimization performance.

(1)

DQN hyperparameter analysis: When the discount factor, $\gamma$, varied within 0.90–0.99, the fluctuation of the algorithm convergence steps was less than 8%, and the fuel consumption optimization rate remained stable at 4.8–5.02%. When the learning rate, $\alpha$, was adjusted within 0.003–0.01, the algorithm maintained good convergence; while the convergence speed slowed down when $\alpha < 0.003$, there was no significant decline in the optimization effect. These results confirm that the DQN hyperparameters have strong robustness within a reasonable range.

(2)

D*Lite hyperparameter analysis: When the risk weight coefficient, ${\tau }_2$, varied within 0.2–0.5, the planning efficiency of the global safe path did not decrease significantly, and the fluctuation of the fuel consumption optimization rate was only ±0.3%. This verifies the rationality of the design of the risk-aware cost function, and the adjustment of hyperparameters will not have a significant impact on the core optimization effect of the algorithm.

3.1. Fuel Consumption Prediction Model Performance

The gradient boosting-based fuel consumption prediction model is trained and validated using the preprocessed ship operational and meteorological data. The model’s predictive performance was evaluated by three key quantitative indicators—Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and coefficient of determination (R²)—with the results shown in

Table 6. Model performance evaluation.

Model	RMSE	MAE	R2
Gradient Boosting	21.0	$7.09$	0.987

. The model achieved a low prediction error with RMSE = 21.0, MAE = 7.09, and a high fitting degree with R² = 0.987, indicating that the model can explain 98.7% of the actual variation in ship fuel consumption and has high prediction accuracy.

The scatter plot of predicted vs. actual fuel consumption values in Figure 19 shows a strong linear correlation between the two, with the fitting line equation of y = 0.985x + 12.36 (x for actual value, y for predicted value). The predicted values closely fit the actual values, confirming the model’s reliability for fuel consumption assessment and reward function calculation in the DQN algorithm and providing an accurate quantitative basis for the subsequent energy efficiency optimization of the wing–diesel hybrid ship.

3.2. Route Optimization Results

The proposed computational framework leverages navigation data to adaptively optimize ship energy efficiency while enhancing navigational safety. To objectively evaluate the optimization effect, three clear comparison baselines are uniformly set in the same reference scenario:

(1)

Actual ship route: The real operational AIS trajectory and fuel consumption data of the target ship on the corresponding route (actual voyage data, with a sampling interval of 10 min), as the practical engineering baseline.

(2)

Traditional D*Lite algorithm optimized route: The algorithm is set with the same cost function and environmental constraints as the improved version, only removing the incremental replanning mechanism and risk-aware weight design, as the single global path planning baseline.

(3)

Traditional DQN algorithm optimized route: The algorithm retains the basic network structure and training parameters, removing the dueling network, noisy exploration, prioritized experience replay and sail status-aware reward function, as the single local decision-making baseline.

A comparison of the planned route trajectories is visualized in Figure 20. In Figure 20, the actual ship route is marked in red, the trajectory optimized by D*Lite is shown in gray, the trajectory optimized by the DQN is shown in pink, and the trajectory generated by the improved DQN with the D*Lite algorithm is highlighted in green against the marine environmental conditions.

Analysis of the navigation route comparison data in

Table 7. Route comparison.

Shipping Route	Voyage Distance (n Mile)	Sailing Time (h)	Sailing Fuel Consumption (t)
Actual ship route	5464.56	440.26	944.98
Optimized route by D*Lite	5503.4	450.33	921.47
Optimized route by DQN	5560.37	457.71	905.21
Optimized route by D*Lite with DQN	5612.91	467.29	899.81

and the trajectory distribution in Figure 20 reveals that the optimization framework combining the improved DQN and D*Lite algorithm yields the optimal performance in fuel consumption optimization, and its optimization effect reaches the optimal lower bound among all comparative algorithms. Specifically, compared with the actual ship route, the hybrid algorithm framework achieves a fuel consumption reduction of 45.17 tons with a relative decrease of 5.02%, which is the largest reduction amplitude among all optimized routes and sets the optimal lower bound for fuel consumption in this research’s algorithm comparison system. In contrast, the single DQN optimized route, as a comparative benchmark algorithm, only realizes a fuel consumption reduction of 39.77 tons (a 4.21% decrease), which is higher than the optimal lower bound achieved by the hybrid algorithm, verifying the superior fuel-saving performance of the improved hybrid framework. In terms of voyage distance and sailing time, all optimized routes show a slight increase compared with the actual ship route, and the hybrid algorithm route has a moderate growth range among them: the voyage distance of the hybrid optimized route increases by 148.35 n mile (2.71%) and the sailing time increases by 27.03 h (6.14%); the single DQN optimized route has a smaller increase (1.75% for distance, 3.96% for time), and the single D*Lite optimized route is between the two in terms of distance and time growth.

Comparative analysis of pre-optimization and post-optimization EEOI was conducted to validate fuel efficiency enhancement efficacy. As calculated through Equation (23), the EEOI metric inversely correlates with environmental performance: higher values denote greater CO2 emissions per transport unit in wind-assisted marine operations. The experimental findings reveal a 7.50% improvement in EEOI.

$$EEOI={\displaystyle \frac{\sum F_{Ck}\times C_{Fk}}{M_{cargo}\times D_{total}}}$$

(23)

where $k$ represents the fuel type; $C_{Fk}$ indicates the CO₂ emission factor; $F_{Ck}$ signifies fuel consumption; $M_{cargo}$ is load capacity; and $D_{total}$ is cumulative sailing distance.

3.3. Sail Status-Aware Optimization Performance

To verify the effectiveness of the proposed state-aware differentiated energy efficiency optimization framework for distinct sail statuses, this section divides the entire voyage into two stages based on the sail operation status trigger criteria in Table 1 and analyzes the optimization performance of the proposed framework under different statuses.

The division of stages strictly adheres to the environmental parameter thresholds and ship scenarios defined. The first stage corresponds to the sail-stowed status (departure/berthing waters, restricted channels, and partial high-wave areas), while the second stage corresponds to the sail-deployed status (open waters with favorable wind-wave conditions).

Figure 21 presents the data distribution chart illustrating the impact of sail operational statuses on fuel consumption in the overall optimized route by the improved DQN and D*Lite algorithm.

As illustrated in Figure 21, characterized by a cluster of low-fuel-consumption data points, the sail-deployed state clearly outperforms the sail-stowed state. The comparative analysis of the two statuses further validates the core advantage of the proposed D*Lite-DQN hybrid framework, which can optimize global energy efficiency with dynamic adaptation to sail status transitions.

4. Discussion

4.1. Analysis of Optimization Performance

The proposed D*Lite-DQN hybrid framework achieves a 5.02% fuel consumption reduction, 140.66 tons of CO₂ emissions reduction, and a 7.50% EEOI improvement compared with the actual ship route, which is attributed to three key design features of the framework:

(1)

Improved D*Lite algorithm with incremental replanning

The algorithm generates real-time global safe path constraints and adapts to dynamic marine environment changes via incremental replanning, avoiding high-risk areas and ensuring navigation safety without excessive computational overhead. The risk-aware cost function reduces the cost of wind-rich grids, guiding the DQN algorithm to select routes with favorable wind conditions.

(2)

Improved DQN with sail status awareness

The integration of a dueling network, noisy exploration, and prioritized experience replay enhances the DQN’s ability to capture the nonlinear ship-environment interaction and adapt to dynamic sail status changes. The differentiated reward function with dynamic weight adjustment aligns the optimization objectives with the physical characteristics of the wing–diesel hybrid ship, maximizing wind energy utilization in the sail-deployed status and prioritizing safety in the sail-stowed status.

(3)

Gradient boosting-based fuel consumption prediction model

The high prediction accuracy of the gradient boosting-based fuel consumption prediction model is a critical factor for the framework to achieve excellent optimization performance, and the model’s accuracy directly affects the DQN algorithm’s learning process and decision-making results through the reward function mechanism. Specifically, the model’s high-precision prediction of fuel consumption ensures that the fuel consumption reward component in the DQN’s differentiated reward function can accurately reflect the actual energy savings effect brought by wind energy utilization and main engine power optimization in different sail operation statuses and marine environmental conditions. This accurate reward feedback enables the DQN algorithm to correctly identify the correlation between ship operation actions and actual energy efficiency gains, guiding the algorithm to learn the optimal energy efficiency decision-making strategy along the correct direction.

In addition, the model’s ability to accurately capture the nonlinear correlation between multiple marine environmental factors, ship operational parameters and fuel consumption also provides a reliable quantitative basis for the dynamic adjustment of the reward function weights based on sail statuses, ensuring the scientific and rationality of the entire optimization objective setting.

4.2. Comparison with Existing Studies

The proposed framework outperforms existing wind-assisted ship energy efficiency optimization methods in three key aspects and further exhibits significant advantages over mainstream simple/lightweight optimization approaches in solving the global energy efficiency optimization problem of wing–diesel hybrid ships, as elaborated below:

(1)

Global collaborative optimization

Unlike existing studies that focus on local optimization, our framework achieves global collaborative optimization of route safety, wind energy utilization, total fuel consumption, and voyage duration by integrating the D*Lite algorithm (global path planning) and the DQN algorithm (local dynamic decision-making). For simpler optimization approaches, such as traditional linear programming and basic Q-learning, it is impossible to realize the joint optimization of multiple global and local objectives due to the limitation of the algorithm’s expression ability.

(2)

Sail status-aware differentiation

Existing optimization algorithms mostly ignore the impact of wing-sail operational states on the ship’s power system, while the proposed framework designs a differentiated reward function and optimization objectives based on sail statuses, fully considering the dynamic interaction between the wing-sail and diesel engine. Simple approaches lack the ability to capture the state-dependent nonlinear coupling relationship between wing-sail operation and ship propulsion and can only adopt a unified optimization strategy for different sail statuses, which leads to a serious mismatch between optimization objectives and actual ship operation characteristics.

(3)

High computational efficiency and real-time performance

The incremental replanning mechanism of the improved D*Lite algorithm avoids global replanning, and the improved DQN algorithm with noisy exploration and prioritized experience replay accelerates convergence speed, which both contribute to the framework’s real-time performance, making it suitable for dynamic marine environments. Although some simple single algorithms have high computational efficiency, they cannot balance the requirements of real-time performance, safety and optimization effect in dynamic marine environments.

5. Conclusions

This study proposes an adaptive global energy efficiency optimization framework for wing–diesel hybrid ships based on an improved D*Lite algorithm and an improved DQN algorithm, addressing the core challenge of collaborative optimization of path planning and propulsion system control under different sail operation statuses and dynamic marine environments. The framework integrates multi-source data (ship operational data, meteorological data, and piracy risk data), a gradient boosting-based fuel consumption prediction model, and a sail status-aware differentiated reward function to achieve a global balance between navigation safety and energy efficiency. The key conclusions of this study are as follows:

(1)

Superior optimization performance: The proposed framework achieves a 5.02% reduction in total fuel consumption, a 140.66-ton reduction in CO₂ emissions, and a 7.50% improvement in EEOI compared with the actual ship route, confirming its significant energy efficiency and carbon reduction effects. The slight increase in voyage distance and time is a reasonable trade-off for safety and wind energy utilization.

(2)

Effective sail status-aware adaptation: The framework dynamically adjusts optimization objectives, reward function weights, and action selection strategies based on wing-sail operational statuses (sail-deployed and sail-stowed). In the sail-deployed status, it maximizes wind energy utilization to minimize fuel consumption; in the sail-stowed status, it prioritizes navigation safety and maneuverability while balancing fuel economy. This design fully aligns with the physical characteristics of wing–diesel hybrid ships and the dynamic changes of the marine environment.

(3)

Reliable fuel consumption prediction: The gradient boosting-based fuel consumption prediction model achieves high prediction accuracy (RMSE = 20.9994, MAE = 7.0864), accurately capturing the nonlinear correlation between ship operational parameters, environmental factors, and fuel consumption. The embedding of the model into the DQN’s reward function provides a reliable quantitative basis for energy efficiency assessment and optimization.

(4)

Practical technical value: The proposed framework provides an effective technical solution for the dynamic energy efficiency optimization of wing–diesel hybrid ships, and its real-time performance and adaptability make it suitable for practical maritime operations. It also provides a reference for the design of wind-assisted ship systems and the formulation of maritime emission reduction policies, promoting the sustainable development of the shipping industry.

In the future, the framework will be further optimized by expanding the research scope, integrating economic cost indicators, and developing a more detailed wing-sail aerodynamic model. It is expected to provide a more comprehensive and intelligent optimization tool for the energy efficiency management of wing–diesel hybrid ships and the deep integration of artificial intelligence and renewable energy utilization in the maritime industry.