A Machine Learning Framework for Condition-Based Maintenance of Marine Diesel Engines: A Case Study

Abstract

The development of artificial intelligence-based tools is having a big impact on industry. In this context, the maintenance operations of important assets and industrial resources are changing, both from a theoretical and a practical perspective. Namely, conventional maintenance reacts to faults and breakdowns as they occur or schedules the necessary inspections of systems and their parts at fixed times by using statistics on component failures, but this can be improved by a predictive maintenance based on the real component’s health status, which is inspected by appropriate sensors. In this way, maintenance time and costs are saved. Improvements can be achieved even in the marine industry, in which complex ship propulsion systems are produced for operation in many different scenarios. In more detail, data-driven models, through machine learning (ML) algorithms, generate the expected values of monitored variables for comparison with real measurements on the asset, for a diagnosis based on the difference between expectations and observations. The first step towards realization of predictive maintenance is choosing the ML algorithm. This selection is often not the consequence of an in-depth analysis of the different algorithms available in the literature. For that reason, here the authors propose a framework to support an initial implementation stage of predictive maintenance based on a benchmarking of the most suitable ML algorithms. The comparison is tested to predict failures of the oil circuit in a diesel marine engine as a case study. The algorithms are compared by considering not only the mean squared error between the algorithm predictions and the data, but also the response time, which is a crucial variable for maintenance. The results clearly indicate the framework well supports predictive maintenance and the prediction error and running time are appropriate variables to choose the most suitable ML algorithm for prediction. Moreover, the proposed framework can be used to test different algorithms, on the basis of more performance indexes, and to apply predictive maintenance to other engine components.

1. Introduction

1.1. Background

In recent years, new technologies have radically changed industry such that the concept of Industry 4.0 has been coined, meaning the pervasive introduction of information technologies that create networks in which machines communicate with each other [1]. Many sectors have been affected by this deep change, including the maritime sector. In fact, the term Shipping 4.0 has been introduced next to Industry 4.0 to refer to “smart ships”, namely, fully interconnected and potentially autonomous vessels [2]. To make this scenario possible, it is important to design suitable devices able to communicate information to the outside world, that is, all the infrastructure correlated to the maritime world, not only the ships. For that reason, the term “smart ship” has been expanded to “smart ports” and “smart transportation” [3]. However, the development of autonomous and smart plants has opened new challenges. An example is given in the maritime sector by the necessity to update the dock-to-dock operations [4]. In this context, a virtual counterpart of the physical asset, namely, a so-called digital twin (DT), is often used for different purposes, i.e., maintenance or design. In fact, implementing the physical laws in cyberspace gives the possibility to create both a copy of the whole ship [5] or only a single component, such as the main engine [6]. An important observation is that Industry 4.0 and Shipping 4.0 do not forget that humans are the core of the change in paradigm. In fact, technicians can be involved in new activities if equipped with new smart devices, such as augmented reality instruments [7].

1.2. The Shift to Proactiveness in Maintenance

This revolution has influenced many fields and applications, among which maintenance has radically changed. Maintenance is defined as “any activity carried out on an asset in order to ensure that the asset continues to perform its intended functions, or to repair the equipment” [8].

Many types of maintenance strategies have been proposed in the literature. Some are comparable, some have similarities, some are clearly distinct. Moreover, different authors may define the same strategy differently. In general, maintenance is classified as reactive (also called passive or corrective) or (pro)active (which is sometimes indicated as preventive) [9,10,11,12,13].

Reactive maintenance (RM) is applied once a complete failure occurs, after breakdown, and requires fixing equipment problems and repairing faults as they arise; this is also described in software and automation systems [14].

In more detail, industry can define it as breakdown maintenance if an asset is not working anymore and needs to be repaired; as run-to-failure maintenance if an asset is used until it breaks down or requires replacement or repair; as corrective maintenance if it provides a temporary solution; or as emergency maintenance if a sudden failure occurs in a critical asset which needs to be repaired as soon as possible, which makes this type of maintenance expensive. The approach of RM is simple, with no costs for planning and scheduling regular maintenance tasks. However, especially in the maritime industry, long-term costs are high because of the emergency repairs and related expensive rush charges for parts and because of the downtime, which is unpredictable. Moreover, RM decreases the equipment lifespan, can introduce safety risks, and determines a poorer asset utilization with reduced performance. So, RM is considered for non-essential equipment, when the replacement cost is low, and when the equipment is close to its end of life.

Proactive maintenance prevents failures. The terms proactive and preventive are often used as synonyms, but there are differences. Proactive maintenance aims to anticipate failures or faults and stop them before they occur. It consists in periodic rigorous inspections and condition monitoring, in collecting data from sensors, and in replacements at scheduled intervals, regardless of whether a failure has occurred. Then, it considers data to determine when maintenance should occur. Basically, proactive actions try to remove the root causes of failure (wrong or degraded materials, unexpected operations, human errors, etc.) to extend the equipment life and let it function at its best without unexpected breakdown. Proactive maintenance can save costs by avoiding unnecessary maintenance (e.g., repairs or replacements when nothing is wrong), so it can be convenient with respect to preventive maintenance, which is defined below. However, it requires monitoring technology and data analysis. Finally, if the equipment is proactively kept safe, then the risk of accidents is reduced as well as the consequent cost of repairs or replacements.

Preventive maintenance (PvM) schedules audits, inspections, and replacements to guarantee the continuous operation of the asset, with the purpose to prevent failures before their occurrence [15,16]. The aim is to reduce the chances of equipment failure and unplanned downtime and to increase safety. Regular and routine inspections are scheduled with periodicity on a time basis or on the basis of the equipment’s iterative utilization, according to a pre-established plan of tasks, while the equipment is still working to prevent unexpected breakdowns. This maintenance approach has advantages (reduced downtime, extended equipment life, saving costs due to emergency situations, improved safety, increased efficiency, and energy saving) but also presents various drawbacks and problems (initial investment costs, unnecessary maintenance with waste of time and resources, dedicated personnel, pause of equipment operation, complex scheduling of maintenance tasks). In summary, it is somewhere between RM and predictive maintenance, which is the latest trend based on emergent digital technologies (Internet of Things, cloud computing, data analytics).

One particular form of prevention is what some researchers and practitioners call planned maintenance or scheduled maintenance [17,18,19,20,21], which some authors consider the most common onboard a ship [19].

It requires planning, documentation, and scheduling. The concept of planned maintenance is much used in industry, and is classically conceived to reduce the unexpected downtime by making all resources available as much as possible, according to a pre-defined strategy. Namely, regularly scheduled maintenance identifies and solves the issues that could become major problems. This is based on statistical observations such that, based on similar applications, maintenance is planned before the failure is likely to happen. Since the scheduled maintenance can be planned in advance, the need to react to breakdowns is quite uncommon. Moreover, the maintenance operations are faster and safer, because technicians know the procedures, the best practices, the parts to fix, etc. In this way, industry may optimize the running conditions and extend the lifespan of the equipment. However, the main challenge of planned maintenance is the costs associated with labor, potential downtime, parts, and the budgetary constraints.

Moreover, it can lead to audits and operations that are not really required, increasing idle time and wasting money. Finally, note that the terms planned and preventive maintenance are often used interchangeably, but refer to two different things. PvM is the foundation for a planned maintenance strategy. Planned maintenance allows proactive management of facilities by encompassing both RM and PvM such that reliance on RM is reduced. It is often required for critical systems (e.g., regular inspection of a fire safety system) and many times is a condition to attaining permissions to operate.

Until recently, maintenance operations, especially in shipping, were mainly performed by RM and PvM [22,23,24,25].

Although many industrial manufacturers decide to use RM because maintenance is perceived as a negative necessary cost and investments in PvM are not seen as profitable, RM is today considered outdated. Namely, it was already found that an emergency repair costs three times and takes ten times longer than a scheduled maintenance task [26], and many industries are developing digital tools, automated systems, and prevention strategies to be competitive [27,28,29]. At the same time, PvM is widely used in the specific field of maritime applications, as recognized by experts in the field who discourage use of broken equipment on ships, also for safety reasons, as well as repairing broken machinery [27]. In detail, maintenance on ships or on shore is performed in three ways: (1) Emergency maintenance, only to repair equipment breakdowns, usually not at sea; (2) scheduled maintenance, especially on board ships, with periodic inspections and replacements, but highly expensive; (3) preventive maintenance, with various tasks and analysis of equipment conditions. Moreover, experts are in favor of early detection of failures and defects, which demands predictive maintenance. They also highlight that this new tool can be cost-effective with respect to preventive maintenance and can help in deciding (predicting) the next maintenance required for machines in service.

1.3. Leveraging Data to Predict Maintenance

In this context, a new kind of maintenance has been introduced, predictive maintenance (PdM) [30,31,32], which is perceived as a groundbreaking approach for maritime operations, in which RM and PvM are no longer sufficient [25,28]. In this case, the aim is to predict failures by using sensors that are installed onboard. Some authors prefer using the prognostic and health management (PHM) definition [23]. The medical term “prognostic” highlights that it is not only necessary to provide information on the health status of machines but also to predict incoming failures. Some authors also call PdM condition-based maintenance (CBM) [32,33]. In shipping, one of the main problems of CBM is to guarantee continuous data storage and communication between the ship and onshore operators. To address this issue, a Hierarchical Data Format version 5 was proposed to store a large amount of data [34]. Herein, the authors will use the acronym PdM to refer to both PHM and CBM. The advantage of the PdM approach is to give information about when components and machines actually need maintenance. In this way, the audit timetable is not fixed in advance but depends on the current, real machinery health status. This approach reduces the idle time and wasted money for unnecessary inspection. Moreover, an intelligent and continuous parameter monitoring allows us to decrease sudden failures, thereby increasing the safety for ships, transported goods, and passengers.

To be more specific, PdM can be performed in two different ways, depending on how data are collected:

(a)

Offline monitoring: Data are acquired when the machinery is not in service or at regular time intervals using measurement systems that are not integrated with the equipment. Examples are vibration analysis [35], oil analysis [36], and ultrasound monitoring [37]. Many times, offline predictive maintenance demands industry professionals to perform data collection manually. In [35], a comprehensive analysis of the importance of vibration analysis for diagnostic purposes of an internal combustion engine is outlined. In [36], the authors find the best oil analysis interval scheme for a marine engine to investigate the degradation trends from off-line oil samples. In [37], once the engine is not in service, wave spectrum analysis through ultrasonic signals is used as a condition-based technique to analyze the health status of the engine. The main advantage of this technique is to provide very precise and helpful information on the engine’s health status; nevertheless, the engine needs to be out of service, thus limiting the applicability of the approach, because failures can happen during working hours, e.g., when ships are sailing.

(b)

Online monitoring: Data are collected during machinery operations [38]. Companies of the Industry 4.0 movement tend to combine IoT sensors installed in the machines with artificial intelligence, and collect data automatically, 24 h a day, every day. Examples are given in [39,40], where the main approach consists in training the predictive algorithm on previously acquired data, and then, applying it during the operation of the monitored device. The main advantage of this method is that it provides information on the engine’s health status continuously. In this way, it is easier to catch failures earlier. However, issues related to interruption of data acquisition and noise may have to be addressed.

It is important to be aware of the differences between the two methods and to use valid reasoning to select the best for monitoring industrial assets. A well-motivated decision determines good asset maintenance and has an impact on costs and profitability. In offline monitoring, the collection of data requires a significant amount of time and expert technicians as well as a measurement plan, and sometimes assets are difficult to access, posing possible risks for the technicians. The method is simple but requires high expertise and care.

In online monitoring, equipment sensors allow data to be collected in real time, maybe stored on the cloud, with no time or effort from humans. The whole data collection process works automatically and remotely, including data processing and analysis by a platform based on artificial intelligence tools. The output is sent to managers who may have information and insights on the equipment’s health status. So, equipment monitoring is possible by remote networked digital devices. Another reason to prefer online monitoring is because providing engine status information continuously is the best way to catch sudden failures in time, avoiding catastrophic accidents (see the Baltimore bridge collapse after cargo-ship power loss in March 2024).

This paper contributes to online monitoring methods by a framework based on predictive algorithms. This framework consists in an easy-to-implement procedure supporting an initial-stage CBM. Namely, to perform online monitoring in an effective way, the first necessary step is to choose the most suitable predictive algorithm. As highlighted in the following sections, the motivation for the choice of the predictive algorithm is often omitted or underestimated or is not the consequence of a comparative analysis. So, to the best of the authors’ knowledge, the novelty of the present study is a framework for an initial and quick benchmarking among three different machine learning (ML) algorithms that can be easily developed and implemented. The analysis compares the three ML algorithms based on their time response and predictive error, which, in the authors’ opinion, are the most important variables to assess the algorithmic performance. Namely, the response time is crucial in maritime applications because giving failure information on time is necessary to guarantee the safety of the machinery, passengers, and crew. On the other hand, the predictive error must be low to have a reliable prediction.

The paper is divided as follows. Section 2 gives an overview of the literature on the predictive maintenance approach based on efficient and profitable use of data. Section 3 shows the proposed framework to select the best algorithm for CBM. Section 4 synthetically describes the simulation model of the engine, in particular of the significant oil filter circuit, used as a case study for applying the framework strategy. Section 5 shows and briefly discusses the simulation results. Finally, Section 6 draws the conclusions.

2. Predictive Maintenance

2.1. Predictive Maintenance by Data-Driven Models

Once the monitoring strategy is chosen, the next step is to select the failure detection and prediction model. The available models are classified based on the diagnostic strategy [41].

(a)

Data-driven models: These typically apply machine learning (ML) or deep learning (DL) algorithms. An example is given in [42], where the authors compared the prediction of the ship speed loss due to fouling by two completely different methods: the ISO 19030 standard [43,44] procedure and a data-driven digital twin. The results clearly showed the data-driven model had better prediction capability than the ISO 19030 standard. In general, the main advantage of data-driven models is a better determination of the time between overhauls (i.e., the time between two maintenance actions).

(b)

Physics-based models: In this case, the outputs from the real asset are compared to those given by a physical model that can be developed to represent the system under maintenance. An example is given in [45], where the authors proposed an adaptive extended Kalman filter for estimating the fault magnitude in an autonomous surface vehicle. In this case, a digital twin was developed to implement all the mathematical modeling equations which represent the autonomous vessel’s physics. The main advantage of this approach is independence from the amount of collected data, differently from the data-driven model, but a substantial limiting drawback is the mathematical approximation, which can reduce the method’s reliability.

(c)

Knowledge-based models: These try to mimic an expert’s reasoning; the main advantage is that complex physical models are not needed. An example is given in [36], where the authors analyzed the deterioration of a marine engine lubrication system using a dynamic Bayesian network (dBN). The model was realized by using both the real data provided by the shipping company and the experts’ background knowledge. The latter was used to assess the cost of the risk model of the dBN (condition monitoring cost, maintenance expenses, repair expenses and failure expenses).

Although all the aforementioned models are valid for diagnosis, the authors focus on online monitoring by data-driven models.

Data-driven models are linked to the current technology development, especially in the maritime sector regarding propulsive systems. In fact, the objectives that must be reached by the 2030 Agenda and Paris Agreement 2050 determine a high interest for investigating new fuels for marine propulsion, such as hydrogen fuel cells [46] and ammonia engines [47]. For that reason, physical models or experts’ knowledge are often not available. Instead, data-driven models do not require the implementation of physical laws, which gives high flexibility to the detection and prediction algorithms, such that they can be applied to different engines without deep changes to the algorithmic structure.

2.2. A Bird’s-Eye View of Prediction Models

Data-driven models have been widely investigated in the literature and applied to different diagnosis problems. They can be used as classifiers and as predictors. In both cases, the aim is to answer three fundamental maintenance questions:

(1)

Will there be a failure or fault?

(2)

Where will the failure or fault be?

(3)

When will the failure or fault occur?

Answering all the questions is necessary to apply the PdM in a good way. In fact, it is possible to know which component to maintain and when the failure or fault is predicted to occur. Focusing the attention on the maritime sector, anticipating a failure occurrence may allow the ship owner to manage the maintenance and order the spare parts. The difference between classifiers and predictors is the way in which they are applied.

A classifier is trained to compare new data with data from the different engine behaviors it was trained on. Then, it assigns a score to each comparison, and finally, outputs the behavior with the highest score that matches the data best. Examples of PdM by classification are given in [32,48,49].

On the contrary, predictors are trained to predict the expected values of engine variables. They are then compared with the outputs from the real asset. An example is given in [50], where an artificial neural network (ANN) evaluates the expected values of parameters that are identified as failure indicators. It is remarked that finding failure indicators is decisive in diagnostic and prognostic models. In fact, reducing the computational cost is important to realize an intelligent diagnostic tool that could be installed onboard. For that reason, as concluded by [51], modeling and simulation can be used to overcome the poor availability of faulty data. Synthetic data produced by simulation make it possible to compare healthy with faulty behaviors, hence to find the most suitable failure indicators. Other neural networks were implemented in [52,53]. Moreover, random forests can be applied for maintenance purposes [49,54], and finally, some authors propose the application of so-called ensemble neural networks [55,56].

In this innovative context, the main research gap is that frequently it is not well explained how the algorithms used for diagnostic purposes are chosen, which may affect the predictive maintenance success and cost, as also discussed in Section 1.3. To provide a more robust motivation and prevent unreliable predictions, some authors propose a benchmarking between algorithms, as shown in [22,31], but neither of the mentioned two analysis techniques are definitively the best choice for predictive maintenance. For that reason, the aim of this work is to propose an easy-to-use framework to select the best algorithm for an initial stage of CBM implementation in marine industrial settings. A procedure is developed to easily find the best algorithm for each component of the system that is subject to faults and is fundamental for a marine engine’s operation. In all cases, the procedure marks the best algorithm. The novelty of the current study is to realize an effective comparison by the response time and the prediction error, and by a formula evaluating a performance index that weights the two previous time and error indexes. In the authors’ opinion, these two indexes are the most suitable ones to evaluate the performance of a predictive algorithm. It is remarked that, differently from [31], the authors also consider the time variable, which is crucial for maintenance purposes.

Finally, once all the components have been analyzed, the algorithm which is marked the most number of times is chosen as the predictive algorithm to be used in the online monitoring system.

3. Proposed Framework

In this section, we describe the proposed strategy for data-driven predictive maintenance. Industrial machinery has different numbers and types of sensors installed on it. The aim of the sensors is to monitor the health status of the main components continuously. These components are those more sensitive to failure or those necessary for the good operation of the entire machinery. In the case of a marine diesel engine, we take the oil circuit as an example, since filtering the impurities is important. Namely, both a non-optimal filtering operation and excessive clogging can cause an unhealthy engine status. The consequence of this is the reduction of the engine life and, in catastrophic situations, it can lead to dangerous conditions for the ship crew and passengers. So, it is extremely important for the engine status to monitor the oil filter, and the associated pressure drop must be measured by sensors to detect clogging. Similar considerations can be made for other engine components.

Assuming that m is the number of sensors in the machinery, let us indicate as $[x_1,x_2,\dots ,x_m]$ all the monitored variables. Then, $[u_1,u_2,\dots ,u_r]$ are the variables chosen as inputs to the considered ML algorithms, which are trained to provide the predictions $[{\widehat{x}}_1,{\widehat{x}}_2,\dots ,{\widehat{x}}_m]$ by processing the employed inputs. The predictions give the expected values in the case of a healthy engine. They are next compared to the real values from the asset, thus allowing the technicians to take decisions regarding the maintenance plan.

The next step is to standardize the $x_j$ variables, for $j=1,\dots ,m$, by the Z-score method to yield $x_{z_j}$, with $j=1,\dots ,m$, as shown below:

$$x_{z_j}=(x_j-{\mu }_j)/{\sigma }_j$$

(1)

where ${\mu }_j$ is the mean and ${\sigma }_j$ is the standard deviation.

Now, we may apply k different ML algorithms to the m variables and compute the mean squared error ($MSE$), say $ms{e}_{i,j}$ for $i=1,\dots ,k$ and $j=1,\dots ,m$, between the predictions and the real experimental values.

The $MSE$ is given by

$$MSE={\displaystyle \frac{1}{m}}\sum _{j=1}^m{(x_{z_j}-{\widehat{x}}_{z_j})}^2$$

(2)

where $x_{z_j}$ is the real standardized value and ${\widehat{x}}_{z_j}$ the predicted standardized value. For each algorithm, the necessary time $t_{pr}$ for obtaining the predicted value is also measured. In this way, a performance index $J=f(MSE,t_{pr})$ is evaluated: the higher J is, the better the algorithm’s performance. Then, for each variable $x_{z_j}$, the algorithm number $i_j$, $1\le i_j\le k$, with the highest J, i.e., $J_{i_j,j}$, is chosen. Finally, algorithm number $i^{*}$, which presents the highest J for the largest number of times, is selected for the overall prediction. The procedure is summarized in Figure 1.

Figure 1 shows the following: the inputs are the standardized measurements from sensors; J is computed for each algorithm prediction and for each sensor; the outputs provide the lowest J for each sensor. The rationale is to select the algorithm which provides the highest J for the highest number of sensors. This optimal algorithm is used subsequently for predictive maintenance with all sensors.

The following algorithms were considered for the benchmark after a careful analysis of the literature:

(1)

An artificial neural network (ANN);

(2)

An ensemble neural network (ENN), which provides an arithmetical mean of the outputs from the single neural networks;

(3)

An ENN providing the weighted mean of outputs from single neural networks;

(4)

A random forest (RF).

3.1. Artificial Neural Network and Ensemble Neural Network

An ANN is an ML algorithm that consists of different interconnected hidden layers, where neurons process the data to produce an output (Figure 2).

The inputs to the ANN are represented by numbers, so in the current application the sensor measurements can be used as inputs to the algorithm. After that, the values are multiplied by the weights in the hidden layers, which are initialized randomly. The multiplication results are added together to produce the neuron output. Finally, once the information arrives at the final layer, the ANN outputs are produced. In the present study, the outputs are the expected sensor measurements associated with the inputs. This basic kind of information propagation is called forward propagation. During the training phase, there is also a second kind of propagation, backward propagation (not shown in Figure 2). In this case, the error is propagated backwards (from the output to the input) to update the weight values.

The output can be a regression or a classification. More details can be found in [57]. The algorithm is described by (3)–(7):

$$y=f\left(\sum _{i=1}^n{w}_i{x}_i+b\right)$$

(3)

$$f\left(x\right)={\displaystyle \frac{2}{1+e^{-2x}}}-1$$

(4)

$$h_j=f\left(\sum _{i=1}^n{w}_{ij}{x}_i+b_j\right)$$

(5)

$$y_k=f\left(\sum _{j=1}^m{w}_{jk}{h}_j+b_k\right)$$

(6)

$$\Delta w_{jk}=\eta {\delta }_k{h}_j$$

(7)

where (3) allows us to compute the output of the neural unit, $x_i$ is an input, $w_i$ is a weight, b is the bias, and f is the activation function. In more detail, in (5), $w_{ij}$ is the weight associated with the i-th neuron input $x_i$ and the j-th neuron output $h_j$; $b_j$ is the bias of the j-th neuron. In (6), $b_k$ is the bias of the k-th output; $w_{jk}$ is the weight associated with the j-th hidden layer input and the k-th output. The $tanh$ function in (4) is the activation function used in the present study and introduces nonlinearity in the system. The computations in (5) and (6) represent the feedforward operations for the hidden layer and the output layer, respectively. They describe the propagation of data to compute the output. Here, $h_j$ is the neuron output of the j-th hidden layer; $y_k$ is the k-th output. Equation (7) describes the backpropagation, namely, the error computation to update the weights: $\eta$ is the learning rate and ${\delta }_k$ the error between the network output and real data.

An ENN represents an extension of the previous neural architecture. The ENN output is obtained by combining the outputs of several ANNs. The authors propose two different ENNs: the first one generates an output by arithmetically averaging the single neural network outputs; the second one by computing the output as the weighted mean of the single neural network outputs. The weights are assigned on the basis of the error made by each ANN. The ANN with the smallest error has the biggest weight.

3.2. Random Forest

We develop an RF with the same regression purpose. An RF consists of a combination of decision trees [49].

A single decision tree can be used as a predictive algorithm for classification or regression. In its structure, the first node (first rectangular box in Figure 3) is called the root and takes the first action or decision; in the same way, each subsequent internal node (rectangular box) represents another action or decision; an arc between two nodes represents the consequent decision flow; finally, a leaf is the node where the final decision is taken. Each decision is taken by considering a probability distribution. The final result can be a classification (classification tree) or a regression (regression tree).

The RF is introduced to improve the performance of a single decision tree. To realize it, the first step is to obtain bootstrap aggregating (bagging) datasets from an initial dataset: they are obtained by random sampling with replacement. This technique consists of taking elements from the original dataset and using them for training one of the decision trees which makes the forest. These elements are then put back in the original dataset. In this way, they can be used for different trees. For each sampling and replacement operation, a decision tree is trained. Then, during the learning phase, a second bagging is performed to find the features (i.e., specific characteristics or variables) that allow the node split: the aim is to obtain the features that can allow a very reliable prediction to obtain the correct output. The final output stems from averaging the outputs of the trees or selecting the output from the highest number of trees.

For example, Figure 3 shows an RF composed of three decision trees: the yellow circle is the RF input; the yellow rectangle is the root node of the decision trees. Since the input is the same for all trees, the first node has the same color (yellow). Then, based on the decision taken by each tree, the internal nodes and the leaves have different colors. Two out of the three displayed trees produce the same output, the red one, which is chosen as the global output with the same color (red circle).

The base of the decision process of the single decision tree is the Gini index, given by (8) below, namely, the evaluation of the probability that a sample belongs to a class.

$$G\left(t\right)=1-\sum _{i=1}^C{p}_i^2$$

(8)

where C is the total number of classes and $p_i$ is the fraction of elements belonging to class i in a specific node. Moreover, the entropy, namely, the “measurement” of the disorder in the data, is evaluated by

$$H\left(t\right)=-\sum _{i=1}^C{p}_{i}log\left(p_i\right)$$

(9)

After every node split, the entropy decreases to have a more homogeneous dataset.

Finally, for a regression problem, with B the total number of trees that compose the forest, the output is evaluated by

$$\widehat{y}={\displaystyle \frac{1}{B}}\sum _{b=1}^B{\widehat{y}}_b$$

(10)

where ${\widehat{y}}_b$ is the output of a decision tree in the forest.

4. Case Study

An engine model was developed using the software GT-Suite (v2024) ^® [58]. The GT-Suite Integrated Multi-Physics Systems simulation software is a package allowing us to build a virtual prototype system. In this way, we may describe physical phenomena with great precision and details and accurately simulate the system dynamics.

The GT-Suite^® software implements the conservation laws of mass, energy, and momentum by (11)–(13) to predict the engine performance [51]:

$${\displaystyle \frac{\partial \rho }{\partial t}}+{\displaystyle \frac{\partial v}{\partial x}}=0$$

(11)

$${\displaystyle \frac{\partial \rho e}{\partial t}}+{\displaystyle \frac{\partial \rho vh}{\partial x}}=0$$

(12)

$${\displaystyle \frac{\partial \rho v}{\partial t}}+v{\displaystyle \frac{\partial \rho x}{\partial x}}+{\displaystyle \frac{\partial p}{\partial x}}=0$$

(13)

where $\rho$ is the fluid density, v the fluid velocity, p the fluid pressure, x the longitudinal dimension of the flow, h the enthalpy, and e the internal energy. The previous mathematical modeling equations are taken into account in each component of the engine, such as the turbine and compressor, air-cooling system, cylinders, and valves. The validity of the results from this software is testified by the amount of technical publications available [59]. The developed model regards a typical large-sized diesel engine with three different supercharged systems and sixteen cylinders.

We chose the oil filter as the first element to which to apply the ML algorithms. Figure 4 shows the simplified circuit with the employed sensors: PT is a pressure sensor and TT a temperature sensor.

The dashed signal in Figure 4 changes the volumetric flow at the pump inlet. Then, on the basis of the achieved pressure, measured by $P{T}_1$, the pump increases or decreases its rotational speed, affecting the pressure drop $P{T}_{in}-P{T}_{ou}$. The brake-specific fuel consumption may increase because the pump is engaged on the main engine, so a higher energy is required to compensate a higher pressure drop. To conclude, higher filter clogging determines a higher consumption and worse engine operation.

Since the pressure drop is a quadratic function of the volumetric flow rate, both nominal and excessive clogging are simulated by multiplying the function by a coefficient $a_f\in [1,5]$, as shown in Figure 5 and explained in [51].

The simulation by GT-Suite^® highlighted that the maximum pressure drop for $a_f=4.0$ is equal to 1.94 bar. Since the sensor warning starts for a pressure drop equal to 2.0 bar, the last $a_f$ value for which the clogging is considered nominal is $a_f=3.7$, for which the maximum pressure drop is 1.7 bar. In this way, we keep a certain distance from the condition of excessive clogging.

5. Simulation Results

The developed ANN has the characteristics reported in

Table 1. Specifications of the artificial neural network.

Inputs	$P{T}_1$/$T{T}_1$ and pump rotational speed
Outputs	Oil filter pressure drop
No. of hidden layers	2
No. of neurons	24 (1st hidden layer) and 12 (2nd hidden layer)
Training algorithm	Levenberg–Marquardt

.

The results shown in Figure 6 and Figure 7 are obtained by using the pressure given by $P{T}_1$ (see Figure 4) and the pump rotational speed as inputs. In the figures, the horizontal axis shows the values of the pressures $P{T}_{in}$ and $P{T}_{ou}$ predicted by the ANN algorithm; the vertical axis shows the values of $P{T}_{in}$ and $P{T}_{ou}$ that were obtained by the simulation.

It can be seen that the trained ANN is able to predict the expected value of the pressure. Equivalent results were obtained for other combinations of inputs (e.g., using the temperature given by $T{T}_1$).

Since the outputs are the pressures by $P{T}_{in}$ and $P{T}_{ou}$, two different random forests were trained because a single one gives one output only. The parameters were tuned in MATLAB R2023b ^® by using the maximum number of splits in the tree (MaxNumSplit) and the minimum number of observations required to create a node (MinLeafSize). The different combinations of parameters are compared based on the $MSE$, and then, the best one is chosen.

Table 2. Specifications of the random forest for $P{T}_{ou}$.

Inputs	$P{T}_1$ and pump rotational speed
Output	$P{T}_{ou}$
Number of trees	91
MinLeafSize	1
MaxNumSplit	92

reports the specifications for the RF making the $P{T}_{ou}$ prediction, and Figure 8 shows the results.

The simulation was performed on a computer with an Intel(R) Core(TM) i7-1065G7 CPU processor and 8.00 GB of RAM. Although the results shown in Figure 6, Figure 7 and Figure 8 may seem similar, tuning the ANN parameters required less time and effort than the RF. Moreover, one ANN can provide two different outputs, unlike an RF. For this reason, implementing two different RFs could require a higher computational cost. Finally, the $MSE$ is lower with the ANN. See

Table 3. Comparison between ANN and RF.

Performance	ANN	RF
$MSE$	$6.2\times {10}^{-5}$	$3.03\times {10}^{-3}$ ($P{T}_{in}$) and $7.2\times {10}^{-5}$ ($P{T}_{ou}$)
Computation time	12 s	105 s

for a comparison.

After comparing the ANN and RF, two different ENNs were developed. A first one was obtained by training three different ANNs and the final output was the arithmetical mean of the three ANNs’ outputs. A second ENN was obtained by training three different ANNs and the final output was calculated as the weighted mean. The weights were assigned based on the $MSE$ of each ANN during the training, namely, the weight 1.0 was assigned to the ANN with the lowest $MSE$ and 0.70 was assigned to the other ANNs.

Table 4. Specifications of the ensemble neural networks.

Inputs	$P{T}_1$ and pump rotational speed
Outputs	Oil filter pressure drop
No. of Neural Networks	3
Output Logic	Arithmetic mean and weighted mean
Weight Values	1.0, 0.7, 0.7

shows the characteristics of the ENNs.

The results showed that:

• A well-trained ANN is a good predictor thanks to its simplicity; in fact, it is the best algorithm for the response time but it shows a higher error than an ENN;
• An RF has the highest error and highest computation time;
• A not well-trained ENN gives better results than a not well-trained ANN;
• An ENN shows a smaller error for a higher number of observations than ANN, even with a lower number of training epochs (i.e., comparing an ENN trained on 25 epochs and an ANN trained on 50 epochs, the first has a smaller error for 65% of the observations).

Table 5. Results from the ensemble neural networks.

Performance	Arith. Mean ENN	Weight. Mean ENN
$MSE$	$2.32\times {10}^{-6}$	$2.93\times {10}^{-6}$
Computation time	46 s	24 s

shows the performance by the two ENNs and Figure 9 shows the simulation results from the weighted ENN.

The main conclusion arising from the comparison between the ANN and ENN is that the higher computational cost of the ENN is motivated by easy tuning and higher accuracy. This is because multiple ANNs, that is, an ENN, allow us to overcome the weakness of the individual regressors. For this reason, ENNs are preferred to monitor components or evaluate physical quantities that are crucial for the engine, when faults can impair the operation.

Finally, for each ML algorithm, an aggregate and simple performance index J can combine the values of the $MSE$ and computation time T using (14) below:

$$J={\displaystyle \frac{MS{E}_{best}/MSE+T_{best}/T}{2}}$$

(14)

where $MS{E}_{best}$ and $T_{best}$ are the best values obtained by the three algorithms. The following results are obtained (higher values of J indicate better performance): $J=0.52$ (ANN), $J=0.07$ (RF), $J=0.63$ (ENN). This gives a compact, easy-to-interpret indication of the best ML algorithm for PdM (or CBM) of the oil filter.

The shown simulation results highlighted that the ENN is the best algorithm for monitoring the oil filter and predicting its sensor measurements for online maintenance. The proposed framework for benchmarking can be applied to all the most important parts of the monitored machinery. In this way, by comparison, it is possible to choose the best algorithm overall. The aim is to increase the safety of the marine engine by ensuring that the predictive algorithm is chosen accurately and after a comparison with other performing ones. As previously remarked, the time and the error were chosen as key indicators for the benchmarking due to their importance. The first indicator is for ensuring that operators receive information about the engine health status in a timely manner; the second is for reliable predictions. In future and different applications, new type of algorithms and new key performance indicators can be included in the framework.

6. Conclusions

Maintenance of marine diesel engines, as with many other industrial processes, is being affected by the revolution brought about by artificial intelligence. So, a thorough change in its fundamental principles and its application is necessary. This paper follows the path highlighted by ML algorithms but contributes to the early-stage engine design, when the choice of the diagnostic algorithm is decisive for an effective maintenance procedure. To the best of the authors’ knowledge, the motivation behind the preference of one predictive algorithm over another is missing in the literature. To overcome this limitation, the authors proposed a framework to follow in the initial stage of CBM implementation: different predictive algorithms were considered and compared based on the time response and the prediction error. The best one was evaluated based on a cost index J weighting the response time and the $MSE$ as key performance indicators. Although the oil filter was chosen as case study due to its importance, the framework can be easily extended and applied to other plant components.

To summarize, this study is helpful to increase the reliability and safety of engine operation. The best predictive algorithm is selected by a comparative analysis among different algorithms. This result is crucial to prevent a plant working in faulty conditions and can avoid a high environmental impact: the engine will not operate under faulty conditions, thus reducing the polluting emissions by 3% [60].