Machine Learning Framework for Fault Detection and Diagnosis in Rotating Machinery

Abstract

Rotating machinery are essential elements in industrial systems and strongly present aboard vessels and maritime platforms, whose unexpected failure can lead to significant economic and operational losses, both for the maritime industry and for industry in general. Condition Monitoring (CM), through the analysis of specific parameters, aims to assess equipment health and enable the early detection of deviations from normal operating conditions. Among existing techniques, vibration analysis stands out for its effectiveness. However, when applied to naval environments, it requires human resources and equipment that are not always prepared or available. Aligned with the principles of Industry 4.0, maintenance has been integrating technologies that enhance data collection and analysis, becoming more autonomous and intelligent. The integration of Machine Learning (ML) into CM offers an alternative to conventional approaches, enabling systems to learn real operating behavior and recognize fault patterns with high accuracy and reduced human intervention. Addressing a real industrial challenge, this paper proposes an automatic framework for fault detection and diagnosis using ML models. As a case study, vibration data from rotating machinery were analyzed, encompassing common faults such as unbalance, misalignment, and the combination of both. The obtained results highlight the potential of the proposed framework for CM in maritime environments, modernizing it with new trends and making it more autonomous, efficient, and less dependent on specialized knowledge.

1. Introduction

Condition monitoring emerged in the 1970s [1] as a fundamental discipline for the early detection of faults through the analysis of signals extracted from machinery, enabling the planning and execution of maintenance actions before an actual failure occurs. Among the various available techniques, vibration analysis is one of the most widely used in industry, particularly for fault detection in rotating machinery [2], due to its ability to effectively describe the dynamic behavior of such systems.

Industry 4.0 has introduced new challenges focused on increasing productivity, safety, profitability, and competitiveness, while simultaneously reducing machine downtime. Consequently, maintenance practices have evolved accordingly, integrating data acquisition and analysis techniques that have led to more autonomous and intelligent maintenance systems, enabling the optimization of activities and more informed and precise decision-making processes. Maintenance is no longer an option for large companies, becoming an integral part of their strategy to remain competitive in the market, with condition monitoring playing a decisive role in the operational equipment’s efficiency.

However, the direct applicability of these models in real environments remains limited, particularly in maritime environments, where CM is still carried out by specialized technicians using dedicated monitoring equipment according to a specific time schedule, resources that are often unavailable or inaccessible when needed.

The conventional CM process presents several economic, technical and practical disadvantages for the industry and for the business process in general. Technicians are required to be physically present at the machine site to collect data and analyze a large volume of information to make an accurate diagnosis. Furthermore, not all machines are monitored using the same technique, making it unrealistic to expect a single individual to have expert knowledge of all CM methods. As a result, companies often need to hire multiple specialists, invest in technician training, or rely on outsourced services.

The integration of intelligent systems with emerging Industry 4.0 technologies, such as the Internet of Things (IoT) and Artificial Intelligence (AI) [3], offers a faster, more efficient, and more reliable approach to fault diagnosis. This integration is expected to drive innovation in CM, leading to a new generation of modern, autonomous, and efficient systems that are less dependent on specialized human expertise. Such systems can help address recurring challenges such as the shortage of specialized technical knowledge on board, limited material resources, and low equipment availability, thereby increasing operational availability and enhancing the safety of maritime operations.

This paper introduces a framework for automatic fault detection and diagnosis capable of integrating multiple machine learning models, heterogeneous datasets, and diverse fault types. The experimental evaluation shows that most of the proposed models present satisfactory generalization performance when applied to different datasets, retain reasonable diagnostic capability in the presence of previously unseen fault conditions, and demonstrate robustness to noise within a limited Signal-to-Noise Ratio (SNR) range. Unlike the conventional CM process, which is essentially based on the acquisition of signals from machines and their comparison with reference standards, such as ISO 10816 [4] for vibration analysis, historical machine data, or fault patterns predominantly identified through traditional methods such as statistical parameter comparison and frequency domain analysis, the proposed framework enables the automation of this entire process. This is achieved by training a ML model using samples collected directly from the machine under well defined fault conditions, thereby eliminating human intervention in signal acquisition, processing, and analysis. The framework was validated using vibration signals acquired from rotating machinery, aiming to support the automation of fault diagnosis in maritime rotating equipment, where reliability and continuous operation are critical. It can be seamlessly deployed in both real-time data acquisition environments and offline analysis pipelines, demonstrating strong adaptability and practical relevance.

This paper is structured as follows: Section 2 presents an overview of rotating machinery, the most common faults associated with it, and related works involving the application of ML algorithms for fault detection and diagnosis. Section 3 outlines the proposed framework. Section 4 presents the experimental evaluation. Section 5 presents the design of each developed ML model, while Section 6 discusses the results obtained from applying the proposed framework. Finally, Section 7 offers conclusions.

2. Rotating Machinery

2.1. Rotating Machinery in the Maritime Context

Rotating machinery lies at the center of industrial processes. It is almost impossible to find a manufacturing or industrial system that does not incorporate at least one rotating machine, whose health and availability have a direct impact on production schedules, product quality, and operational costs [2].

Rotating machinery plays a fundamental role in the maritime sector as well, being responsible for converting energy into mechanical motion for various onboard applications on maritime platforms, regardless of their size, operating speed, configuration, or structural complexity. Common examples include centrifugal pumps, electric motors, gearboxes, turbines, and propulsion shafts that ensure the efficient operation of cooling, ventilation, fluid transport, and propulsion systems on these platforms.

The reliability and performance of such machines are crucial for the safety and operational continuity of maritime platforms. For this reason, the study, monitoring, and maintenance of maritime rotating machinery play a central role in naval engineering, contributing to reducing operational costs and enhancing the sustainability of maritime operations.

2.2. Faults in Rotating Machinery

Rotating machinery installed on maritime platforms and vessels is often exposed to demanding conditions [5] that promote the occurrence of mechanical and electrical failures. These include harsh environmental factors such as vibration, humidity, salinity, dust, and significant temperature variations. Among the most common faults observed in rotating machinery, as illustrated in Figure 1, unbalance stands out as the most prevalent mechanical fault [2,6,7,8], affecting the performance of these equipment.

Unbalance occurs due to an uneven distribution of mass within a rotating component relative to its axis of rotation [9,10], which moves the center of mass and, consequently, the center of rotation from the geometric center, creating a centrifugal force $F_c$ that increases with the square of the machine rotation $\omega$. This relationship is defined by Expression (1), where M is the mass of the rotor and the eccentricity e is the distance between the geometric center and the center of mass of the rotor [6,10].

$$F_c=Me{\omega }^2$$

(1)

As a result, unbalance becomes increasingly noticeable as the machine’s rotational speed rises. The uneven mass distribution responsible for unbalance can arise from several causes, including manufacturing defects, accumulation of dirt or debris, or physical degradation such as erosion and corrosion. This is the reason why unbalance is so common, as it can easily develop during regular operation, particularly in harsh environments.

The centrifugal force $F_c$ generated by unbalance leads to a noticeable increase in the machine’s vibration levels, making vibration analysis the most widely adopted technique for its detection. Since the unbalance mass rotates at the same speed as the rotating component, the primary symptom of unbalance is an increase in vibration amplitude at a frequency equal to the rotational speed (1 × RPM) of the machine. Moreover, the vibration amplitude is proportional to the degree of unbalance [6,7].

The presence of unbalance introduces dynamic forces and additional stress concentrations in machine components, thereby increasing the likelihood of other failures appearing. Examples include bearing defects, internal rubbing that may lead to seal degradation or cracking, structural looseness in components such as machine supports and couplings, and even contact between rotating blades and stationary housings, which can result in catastrophic damage.

On the other hand, the own design, operating conditions and environment of rotating machinery, typically operating mechanically coupled to other equipment such as fans, pumps, compressors, or shafts through couplings, can also introduce new faults into the system. These may include bearing defects [11] and misalignment [6].

Bearings are fundamental components found in nearly all rotating equipments. Their main function is to support the relative motion between stationary and rotating elements, ensuring smooth and stable operation. Because they are in continuous contact and operate throughout the machine’s entire running time, bearings are subjected to constant stress, making them highly susceptible to wear and failure. Depending on their design and construction [12], bearing faults can occur in the inner race, outer race, rolling elements, or other components.

Misalignment occurs when the centerline of rotation of two or more coupled components are not in line with each other [2].

The occurrence of such faults can lead to the shutdown of critical systems, compromising the safety and operational reliability of maritime platforms. Consequently, the early detection and accurate diagnosis of faults in rotating machinery are essential to ensure the safety, operational continuity, and minimize the costs associated with corrective maintenance and unplanned downtime.

2.3. Fault Diagnosis in Rotating Machinery Using Machine Learning

Recognizing the importance of rotating machinery and being aware of the impact of its unexpected downtime on the operational sustainability of companies, several researchers have explored the potential of using intelligent algorithms for machinery automatic fault detection and diagnosis.

The majority of these studies address highly representative faults in industrial environments, directly related to the design and operating mode of the machines themselves, which typically operate coupled to other equipment and perform power generation tasks. Among the most frequently studied faults are unbalance, misalignment, and bearing defects.

In [13], for example, hourly vibration and temperature records were used to develop models based on Random Forest (RF), Support Vector Machines (SVMs), and K-Nearest Neighbors (KNNs) for fault detection in pumps responsible for fluid transfer in an oil and gas processing plant, with the RF algorithm achieving a performance of 98.5% in classifying the different machine states. Similarly, in [14], the potential of these approaches was demonstrated through the development of models based on Convolutional Neural Networks (CNNs), RF, and SVM for gears fault detection, achieving accuracies of 97.11%, 86.33%, and 88.78%, respectively.

In [15], a real-time monitoring system, based on RF models, was developed for fault detection and diagnosis in water pumps from a water treatment and distribution station. This model was initially trained with vibration signal samples transformed into the frequency domain using the Fast Fourier Transform (FFT), and later applied in a real-world context to classify the pump condition as healthy, misaligned, or with an unknown fault. By feeding the model with vibration signals acquired from four points on the pumps, corresponding to the bearing housings, an accuracy of 97% was achieved in detecting misalignment, demonstrating the model’s effectiveness and its potential for application in real industrial environments.

Also following a frequency domain approach, in [16], an SVM model was developed, achieving a performance of 98.8% in misalignment diagnosis. This study highlighted the potential of applying dimensionality reduction techniques, such as Principal Component Analysis (PCA), to improve the performance of such models, which often become less effective due to high dimensionality issues inherent to the nature of the input signals.

In contrast, in [17], an SVM model was developed for the diagnosis of unbalance, misalignment, and bearing faults through the extraction of statistical parameters such as Root Mean Square (RMS), peak, shape factor, impulse factor, and kurtosis, after amplification and filtering of the vibration signals using the Continuous Wavelet Transform (CWT) for noise reduction. The SVM model developed with a polynomial kernel achieved an overall performance of 95.56%, slightly higher than the same architecture using a linear kernel, which achieved 93.67%.

Highlighting the importance of applying similar approaches in maritime environments, reference [18] explored the high customizability of neural network architectures to develop a system based on an Improved Deep Residual Shrinkage Network (IDRSN) for diagnosing unbalance and misalignment in rotating machinery operating under high noise levels. In this study, an experimental test bench simulating a ship’s propulsion shaft driven by an electric motor was used.

The developed IDRSN is based on a Residual Deep Neural Network (ResNet) architecture that incorporates a soft thresholding function to suppress vibration signal features related to noise. These signals are filtered through this function, combined with wavelet thresholding and Variational Mode Decomposition (VMD) during preprocessing, generating Intrinsic Mode Functions (IMFs) that are subsequently fed into the IDRSN.

From the experimental tests carried out on the test bench, an overall performance of 85.3% was achieved when the model was fed with vibration signals converted into two-dimensional images. The diagnosis of misalignment samples reached 100%, a result attributed to their distinct characteristics in the frequency domain, whereas 25% of the unbalance samples were incorrectly classified as normal.

Overall, CNN models have demonstrated superior performance when compared to classical ML algorithms such as RF and SVM. For example, in [19], the same dataset was used to develop both a CNN and an SVM model, with the CNN achieving a performance of 100%, surpassing the 98.29% previously obtained by the SVM.

The studies in [20,21] also demonstrate the versatility of CNN, with the authors employing pre-trained architectures such as GoogLeNet and AlexNet, respectively, for fault diagnosis using transfer learning approaches. In [20], spectrograms generated from vibration signals indicative of bearing faults were used as input to the pre-trained model, which achieved an overall performance of 99% in classifying anomalies such as inner race, outer race, or rolling element faults.

3. Proposed Framework for Condition Monitoring

In this section, a framework based on the application of ML algorithms is proposed for the automatic fault detection and diagnosis in the condition monitoring of mechanical maritime equipment.

As illustrated in Figure 2, the proposed framework can be segmented into six main components:

• The first component corresponds to the framework input, which can receive either previously acquired signals (offline) or real-time signals (online) from the sensors. The input data may include vibration signals, electrical current, temperature, or other measured physical quantities, depending on the condition monitoring technique employed and the type of machine or its application.
• The second component corresponds to the preprocessing phase, where the input signal is processed to match the format of the samples used during the development of the applied ML model. For example, if the model was trained using feature extraction based on statistical functions, the same extraction procedure must be applied to the raw input signal. Alternatively, if the model was developed using signals in the frequency domain, the input signal must first be converted from the time domain to the frequency domain.
• The third component represents the application of the Machine Learning model, which represents the core of the proposed framework. This model, previously trained with a labeled dataset mapping known machine fault patterns to fault classes, performs the classification task.
• The fourth component corresponds to the model’s output. The output consists of mutually exclusive classes, where the model assigns a score to the input sample across all possible classes, typically expressed as a probability distribution. The first class, Normal, indicates the absence of fault, confirming that the machine is operating under normal conditions. The remaining classes, from Fault 1 to Fault n, correspond to the different fault types or, alternatively, to varying severity levels of a particular fault if such gradation was included during model training. For example, Fault 1 and Fault 2 may both indicate unbalance, with Fault 2 corresponding to a higher severity level than Fault 1. The total of n + 1 output classes enables the proposed framework to perform automatic multi-fault diagnosis.
• In order to extend the robustness of the proposed framework, a fifth component was introduced after the model output. This threshold mechanism retrieves the highest probability value P assigned by the model to its predicted classes and compares it to a threshold value t. The value of t is selected by the user within the range [0, 1], according to the desired confidence level to the system. If a more rigid and accurate system is required, a higher value of t should be chosen, ensuring that a sample is classified as belonging to a known class only when its classification probability P exceeds t. Conversely, lower values of t reduce the confidence level associated with the sample classification process.
• The sixth component corresponds to the framework output. If P is greater than or equal to t, the system classifies the input signal as belonging to one of the known classes. Otherwise, the system classifies the input signal as an unknown fault, not represented within the model’s training data.

The flexibility to operate with both offline and online data enhances the practical applicability of the proposed framework across diverse industrial contexts and predictive maintenance scenarios. Moreover, its sequential architecture ensures adaptability and scalability, allowing for future extensions through the integration of additional modules between components, such as outlier detection, noise reduction, or specialized signal processing procedures.

The introduction of a threshold mechanism allows the framework not only to classify known fault types but also to identify abnormal conditions outside the fault classes present in the training dataset, thus enhancing its applicability in real-world scenarios. Input samples corresponding to a novel or unseen fault condition can be incorporated into a continuous learning process, in which the model is retrained to include the new fault class once it has been properly labeled. However, this labeling process requires the intervention of a specialized maintenance technician.

4. Experimental Evaluation

The proposed framework is designed to incorporate any type of intelligent model capable of receiving a signal as input and providing a class membership probability as output. Nevertheless, the overall effectiveness of the framework is inherently dependent on the predictive performance of the model employed.

Accordingly, to evaluate the feasibility, sensitivity, and robustness of the proposed approach, a set of models was developed using the following algorithms:

• Random Forest [2], selected for its robustness and frequent use in multiclass classification tasks;
• Support Vector Machines [2,16,22], chosen for their ability to define complex decision boundaries and effectively manage high-dimensional data;
• Convolutional Neural Networks [23,24], adopted due to their superior performance and capability to automatically extract complex patterns from temporal and spectral data.

In the development of these models, Dataset I was used, containing vibration signals representing only the unbalance fault with different severity levels, and Dataset II, which, in addition to unbalance, introduces the misalignment fault and the combination of both faults. The use of these datasets not only enabled the development and validation of the proposed models but also allowed their optimization and analysis of their generalization capability through a proper sensitivity and robustness assessment.

In this paper, only vibration signals were analyzed, since vibration analysis is the most widely used condition monitoring technique for fault detection in rotating machinery within the industry [2], and is also the most extensively studied method in the academic community, particularly in the context of ML applications.

The performance of the developed models and their ability to make correct predictions on unseen data were evaluated using the accuracy metric (Expression (2)), which represents the proportion of true positives (TP) and true negatives (TN) among all classified samples. This choice was mainly due to the fact that the datasets used in this work are balanced, and accuracy provides a general measure of how often a model makes correct predictions.

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(2)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(3)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(4)

$$F1\text{-}Score=2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}}$$

(5)

Accuracy was complemented by the F1-Score value (Expression (5)), which provides a balanced evaluation of the model’s performance without requiring the calculation of additional metrics such as precision (Expression (3)), which emphasizes the presence of false positives (FP), and recall (Expression (4)), which emphasizes the presence of false negatives (FN).

The F1-Score summarizes, in a single value, the balance between precision and recall, which is particularly important in the context of maintenance, where FP can lead to unnecessary maintenance actions due to the diagnosis of faults that do not actually exist, and FN can lead to the progression of severe faults due to the absence of maintenance actions when they are in fact required.

Additionally, the confusion matrix was used mainly to identify error patterns in the classification of different samples, while model’s training time was considered as a measure of computational cost. From the confusion matrices, it is possible to derive additional metrics such as FN, FP, TN, and TP rates, as well as recall and precision. Together, these indicators provide both a quantitative and qualitative understanding of the predictive capability of the developed models, as well as their computational efficiency.

4.1. Experimental Setting

The experiments conducted aimed to develop intelligent models capable of classifying a vibration signal from a rotating machine according to its correct operational condition, thus formulating a multiclass classification problem and enabling these models to be considered for integration into the proposed framework.

Based on this objective, a structured experimental sequence was followed, encompassing different data preprocessing strategies tailored to each type of algorithm used.

• Random Forest and Support Vector Machines:
  • Training with raw time domain signals, without any preprocessing;
  • Training with statistical features extracted from the signals in the time domain;
  • Training with vibration signals transformed from the time domain to the frequency domain.
• Convolutional Neural Networks:
  • Training a one-dimensional CNN using the raw time domain vibration signals;
  • Training a one-dimensional CNN with vibration signals transformed from the time domain to the frequency domain;
  • Training a two-dimensional CNN using representative images of raw vibration signals;
  • Training a two-dimensional CNN with images corresponding to the spectrograms of the acquired vibration signals.

The raw vibration signal in the time domain was used to evaluate the capacity of each model to identify faults patterns directly from unprocessed data. Given the high dimensionality of vibration signals, with one amplitude value per time unit, a feature extraction approach was implemented, where a set of statistical parameters were extracted from the signals. This step aimed to reduce data dimensionality while emphasizing the most relevant signal characteristics.

In line with traditional and widely adopted practices in vibration-based condition monitoring [25], particularly for unbalance detection, the frequency domain representation of the signal was also used to evaluate model performance in that domain.

Considering that in real-world scenarios, such as industrial and maritime environments, vibration signals are generally contaminated by noise from various sources, typically modeled as white noise [26], the highest performing models associated with each algorithm were selected for additional experiments. These experiments followed a three-phase protocol in which white noise was introduced at different intensity levels, with SNR values of −10, −5, −3, 0, 3, 10, 15, and 20 dB. In the first phase, noise was applied to the training and validation datasets, while model performance was evaluated using a clean testing dataset. In the second phase, the models were trained and validated using contaminated datasets, while testing was performed on a clean testing dataset. In the third phase, all datasets were contaminated, and the models were trained, validated, and tested under noisy conditions. The objective was to assess the robustness of the developed models under different noisy conditions and, consequently, to evaluate their potential practical applicability within the proposed framework.

To identify each developed model and associate it with the corresponding experiment, a naming convention was adopted based on the structure “Algorithm + Applied Preprocessing Technique (when applicable)”. The “Algorithm” parameter can take the values RF, SVM, CNN 1D or CNN 2D, corresponding to the algorithm used. The “Applied Preprocessing Technique” parameter can be designated as “FeatExt”, when statistical features are extracted from the signal in the time domain, or as FFT, when the signal is transformed to the frequency domain through the application of the Fast Fourier Transform. For example, SVM + FeatExt corresponds to an SVM model trained with statistical features extracted from the vibration signal in the time domain.

The preprocessing and feature extraction phases are detailed in Section 4.3 and Section 4.4, respectively.

4.2. Data Description

After an extensive search for publicly available and open access datasets suitable for fault analysis in rotating machinery, Dataset I, described in Section 4.2.1, was selected for the development and tuning dataset of the proposed models. This dataset was chosen because it is widely referenced in the literature and serves as a benchmark in this research field, despite addressing only one type of the most common fault in rotating machinery: the unbalance.

To further assess the performance of the studied models, as well as their sensitivity, robustness, and generalization capability, Dataset II, described in Section 4.2.2, was also selected. This dataset has a broader scope, encompassing faults such as unbalance, misalignment, and their combination, thereby allowing the evaluation of the models’ ability to handle multi-fault and combined fault scenarios.

Although both datasets were recorded in a laboratory environment, they consist of vibration signals acquired directly from three-phase induction electric motors operating under different fault conditions. This type of machine is highly representative of maritime platforms, as it is one of the most widely used machines in maritime systems, including fluid handling systems, sewage systems, firefighting systems, fans, air conditioning systems, and various electric pumps. The fact that these datasets rely on vibration analysis as the recorded data further highlights their similarity to the maritime environment, since vibration analysis is one of the most widely employed condition monitoring techniques on maritime platforms for diagnosing the faults considered.

4.2.1. Dataset I

Dataset I was specifically developed in [27] for research on intelligent approaches applied to the detection of unbalance in rotating machinery based on vibration data. This dataset was obtained through the simulation of different levels of unbalance on a shaft equipped with an unbalance mass support driven by an electric motor. Vibration signals were recorded from three sensors installed on the bearing housing of the shaft, along the horizontal and vertical directions, and on the motor casing, as shown in Figure 3. The unbalance severity level was adjusted by varying both the mass and its positioning radius on the unbalance support.

The vibration signals were acquired at a sampling rate of 4096 Hz for four unbalance conditions, corresponding to different severity levels, and one normal condition, across eight measurement sessions. Each session recorded 135 min of shaft rotation, with a speed variation ranging from 360 to 2330 RPM over time. After segmentation, this dataset resulted in a total of 40,000 samples evenly distributed among the different classes normal, unbalance I (3.3 g/14 mm), unbalance II (3.3 g/18.5 mm), unbalance III (3.3 g/23 mm), and unbalance IV (6.6 g/23 mm), where an increase in the index corresponds to an increase in unbalance severity level.

4.2.2. Dataset II

Dataset II, publicly available in [29], contains vibration signals acquired from the experimental test bench for rotating machinery shown in Figure 4. The setup consists of an electric motor coupled to a shaft, which has an inertia disk mounted at its center, operating under multiple conditions and speeds ranging from 720 to 3600 RPM.

This dataset stands out for including combined fault conditions with different severity levels, such as normal condition, unbalance, horizontal misalignment, vertical misalignment, and three combinations of these faults. This allows for a more realistic study of automatic fault detection and diagnosis, closely resembling industrial scenarios where multiple faults coexist.

The unbalance was simulated by attaching known masses to the inertia disk at a fixed radius. Horizontal and vertical misalignment were simulated by laterally displacing the motor base and by inserting slot shims of known thickness under the base, respectively. The dataset consists of 2162 vibration signals, each with a duration of 5 s, acquired at a sampling rate of 50,000 Hz.

In the present work, only signals corresponding to normal, unbalance, horizontal misalignment, and combined unbalance and horizontal misalignment conditions were considered. After segmentation, this dataset resulted in a total of 1330 samples, distributed among the different classes with 240 samples for the normal class, 220 samples for the classes unbalance I (6 g), unbalance II (20 g), and unbalance III (35 g), 240 samples for the misalignment (2 mm) class, and 190 samples for the combined unbalance II and misalignment condition.

4.3. Preprocessament

In unbalance fault detection through vibration analysis, vibration levels tend to be higher in the radial direction [7]. Consequently, according to best practices in vibration analysis, this direction is generally sufficient for identifying an unbalance condition in a rotating machine. However, most datasets containing vibration signals are provided in more than one axis. For this reason, for each dataset, the signals measured along all axes were compared in order to determine which axis exhibited the highest amplitude expressiveness and, naturally, to select the signals from that axis as input for model training, as illustrated in Figure 5.

For Dataset I, Figure 5 shows that the vibration signal acquired from sensor 3, mounted on the motor, is the least expressive, exhibiting the lowest amplitude and peak-to-peak values. Sensor 1, mounted horizontally on the bearing block, presents slightly higher amplitudes than sensor 3. In contrast, sensor 2, also mounted on the bearing block, exhibits the highest peak-to-peak and RMS values. For this reason, data from sensor 2 were selected for model development, as they provide a more expressive signal with stronger patterns, thereby facilitating model learning and potentially improving performance.

As illustrated in Figure 6, the initial phase of these signals was removed to ensure that the models were trained exclusively with data corresponding to the stationary phase of the measurements, thereby eliminating the possibility of noise from the initial transient phase interfering with the training process. Subsequently, each signal was segmented into one second windows, properly labeled according to the associated fault, to serve as input for the training of the selected supervised learning models.

Finally, for the RF and SVM models, the datasets were partitioned into 80% training and 20% for testing. In the case of the CNN, the datasets were split into 70% for training, 10% for validation, and 20% for testing.

In the RF and SVM models, the raw vibration signals were standardized by removing the mean and subsequently normalizing them to unit variance, in order to reduce the sensitivity of these algorithms to the scale of the input data. To evaluate the sensitivity of these models to vibration signal preprocessing, additional experiments were conducted in which only amplitude normalization to the range [0, 1] was applied, as well as experiments without any form of preprocessing.

Since the vibration signals in the datasets used correspond to the measurement of real physical parameters acquired through a signal acquisition and processing system, it was assumed that these signals are consistent with the machine’s operating condition, and therefore, no outliers were considered during processing. Similarly, no missing values were identified across the different vibration signals.

4.4. Feature Extraction

Following an approach similar to that found in other studies within the same research domain [28,30,31], several statistical parameters commonly reported in the literature were extracted from the vibration signals in the time domain, with the goal of forming a representative feature set while simultaneously reducing the dimensionality of the original signals. The calculated parameters are the maximum and minimum values, peak amplitude, peak-to-peak amplitude, mean amplitude, standard deviation, crest factor, RMS, kurtosis, and skewness.

For the models developed in the frequency domain, the FFT was applied to convert the raw vibration signal from the time domain to the frequency domain, as performed in previous studies [16] and as is commonly adopted in the CM field.

5. Design and Development of Machine Learning Models

The RF and SVM models were developed locally using Python libraries such as Pandas, Numpy, Scikit-learn, TensorFlow, Keras, and Matplotlib on a standard laptop equipped with an Intel i5-10210U CPU @ 1.60 GHz and 8 GB RAM.

In contrast, CNN models, which required additional computational power, were developed using the Graphics Processing Unit (GPU) resources freely available on Kaggle1.

5.1. Random Forest

The architecture of the RF models was developed methodically and iteratively. The process began with the optimization of the number of trees to be used, testing values within a range of 20 to 300, with an increment of 5, while keeping the Gini criterion [32] as the splitting criterion and the maximum depth of each tree as unlimited. It was observed that most models reached maximum accuracy with a different number of trees. However, it was found that 240 trees produced accuracy values very close to the maximum accuracy of each model, with differences only on the order of thousandths. Therefore, 240 trees were adopted for subsequent experiments.

To prevent underfitting or overfitting, tree depth was first evaluated under no depth restriction. Since, the maximum depth is a critical hyperparameter for controlling the trade-off between bias and variance. Very deep trees tend to overfit, whereas limiting tree depth acts as a regularization mechanism. In this case, the decision trees nodes were expanded until all leaves were pure or until all leaves contain less than the minimum number of samples required to split an internal node, which was defined as 2 by default. Evaluating the minimum, maximum, and average depth of each model, it was observed that the average depth of the models ranged between 30 and 36, with minimum and maximum values around 26 and 50, respectively. Considering these results and computational limitations, a prepruning strategy was applied by fixing the maximum depth at 40, slightly above the observed average.

After defining the number of trees and their respective depth, both Gini and Entropy [32] decision criteria were compared. Both produced similar accuracy, but Entropy yielded a marginal improvement in F1-score at the cost of higher training time. Given the balance between performance and efficiency, the Gini criterion was chosen, thereby enhancing the overall efficiency of these models.

This process ensured an appropriate selection of hyperparameters in the developed architecture, allowing, in addition to the applied preprocessing, control over model complexity, ultimately improving generalization and test performance.

5.2. Support Vector Machines

Given the high sensitivity of SVMs to parameterization, a grid search mechanism was implemented instead of an iterative approach, aiming to balance generalization capability with computational efficiency. The search was based on the hyperparameters listed in

Table 1. Hyperparameters tested during the grid search in the development of the SVM model.

Kernel	Regularization Coefficient	Gamma	Degree
Linear	{0.1, 1, 10, 100}	—	—
RBF	{0.1, 1, 10, 100}	{0.001, 0.01, 0.1, Scale}	—
Polynomial	{0.1, 1, 10}	{0.01, 0.1}	{2, 3, 4}

, using Dataset I as the development dataset. Accuracy was used as the evaluation criterion due to the balanced nature of the datasets, leading to the configurations presented in

Table 2. SVM models grid search hyperparameters founded for Datasets I.

Model	Dataset I
	Kernel	Reg. Coef.	Gamma
SVM	RBF	10	Scale
SVM + FeatExt	RBF	100	0.1
SVM + FFT	RBF	100	0.001

.

However, when applying the same hyperparameters to Dataset II, acquired at a higher sampling rate and characterized by distinct amplitude and noise properties, a loss of generalization capability was observed during the validation process, particularly for representations in the frequency domain. This revealed the sensitivity of the algorithm to parameterization.

Therefore, a new grid search process was carried out for Dataset II, resulting in a hyperparameter set specifically tuned for this dataset and distinct from those obtained for Dataset I.

Nonetheless, based on the performance achieved and the information presented in Table 2 and

Table 3. SVM models grid search hyperparameters founded for Datasets II.

Model	Dataset II
	Kernel	Reg. Coef.	Gamma	Degree
SVM	Polynomial	1	0.01	2
SVM + FeatExt	RBF	100	0.1	–
SVM + FFT	Linear	0.1	–	–

, a flexible yet sufficiently generic configuration was defined to serve both datasets. Consequently, the Radial Basis Function (RBF) kernel was selected due to its ability to produce smooth decision boundaries and adapt to nonlinear data. The gamma parameter was set to Scale, allowing the RBF kernel to automatically adjust the influence of near and distant data points according to the variance of the input features, which is directly dependent on the preprocessing applied. To penalize misclassifications more strongly, the regularization coefficient C was fixed at 100, introducing some additional rigidity to the model.

Considering that the SVM algorithm is inherently designed for binary classification problems, a one-versus-one approach was adopted to adapt this classifier to the present multiclass classification problem. In this approach, binary classifiers are constructed between all pairs of classes, and the final class is determined by majority voting.

5.3. Convolutional Neural Networks

The proposed architecture for the development of CNN models, illustrated in Figure 7, begins with a convolutional layer, one-dimensional for the CNN 1D—Time and CNN 1D—Frequency models, and two-dimensional for the CNN 2D—Time and CNN 2D—Frequency models. This first convolutional layer employees 32 filters with a kernel size of 3, aiming to extract relevant local features from the signals while preserving their temporal structure. The convolution operation is followed by a MaxPooling layer with a reduction factor of 2, reducing dimensionality and contributing to invariance to small variations in the signals. Batch normalization is then applied to stabilize and normalize the activations of the previous layer.

This pattern is repeated across two additional convolutional layers, where the number of filters is increased to 64 and 128, respectively. The pooling and normalization operations associated with each convolutional layer reinforce hierarchical feature extraction and enhance training stability. Rectified Linear Unit (RELU) was defined as the activation function for all convolutional layers, since it is widely recommended due to its ability to introduce non-linearity without saturation, thereby accelerating weight convergence during training.

After the convolutional layers, the output is flattened, enabling connection to a fully connected neural network consisting of two dense layers. The first dense layer has 128 neurons and also uses RELU activation function, while the output layer contains as many neurons as the number of faulty classes. The Softmax activation function was adopted in the output layer as it provides a probability distribution over the classes, with the probabilities summing to one. This makes Softmax the most appropriate choice for multiclass classification problems involving mutually exclusive classes, as also adopted by numerous other researchers in similar studies.

To prevent overfitting, a dropout rate of 20% was applied during the training of the fully connected network, thereby forcing the architecture to improve its generalization capacity.

The architecture was compiled using the Adam optimizer, which implements the Adaptive Moment Estimation algorithm. Adam was selected as it is one of the most commonly used optimizers, capable of automatically adapting the learning rate, ensuring fast convergence, and performing well across a wide range of scenarios. This guarantees that the network weights are updated appropriately during training. The cost function employed was sparse categorical cross-entropy, which is suitable for multiclass classification problems where class labels are encoded as integers [33]. The categorical cross-entropy is defined by expression (6), where n is the number of training samples, m is the number of classes, $y_{ij}$ represents the true label of the ith class (1 if the sample belongs to the class, else 0), and $p_{ij}$ is the predicted probability of the ith class belonging to the jth class.

This formulation avoids the need for one-hot encoding of the labels, simplifying data preprocessing and reducing computational cost during training. Accuracy was defined as the primary metric for evaluating model performance across epochs, due to its intuitive interpretation and direct relationship to the correct classification objective.

$$categoricalcross\text{-}entropy=-{\displaystyle \frac{1}{n}}\sum _{i=1}^n\sum _{j=1}^m{y}_{ij}\times log\left(p_{ij}\right)$$

(6)

All models were trained in batch mode with a default batch size of 32, with sample shuffling enabled. To prevent overfitting and avoid performance degradation, an early stopping mechanism was employed with a patience parameter of 50 for a maximum of 250 epochs.

Different preprocessing techniques were applied to adapt the data to the proposed CNN architecture and to the specific requirements of this algorithm. For the CNN 1D—Time model, feature scaling was applied by standardizing the signals, since CNN are highly sensitive to the scale of their input data. For the CNN 1D—Frequency model, preprocessing was limited to the direct transformation of raw signals into the frequency domain using the FFT.

For the two-dimensional CNN models, it was necessary to convert vibration signals into image representations suitable for CNN input. Each sample in the training dataset was transformed into an image of 128 × 128 pixel representing the machine’s vibration signature over one second. For the frequency domain analysis, spectrograms were generated directly from the time domain representation for each sample, such as performed in [34]. To improve the performance of the CNN 2D—Frequency model, spectrograms were generated with signal amplitude represented on a logarithmic scale and the resulting pixel values normalized between 0 and 1, as illustrated in Figure 8.

6. Results and Discussion

6.1. Known Faults

When trained directly with vibration signals in the time domain, the RF and SVM algorithms exhibited low performance, with accuracy around 30%, confirming their limited ability to identify patterns in raw vibration signals, such as shown in

Table 4. Experimental results—Dataset I.

Models	Dataset I
	Acc.	F1-Score
		Normal	Unb. I	Unb. II	Unb. III	Unb. IV
RF	0.30	0.32	0.30	0.16	0.21	0.40
RF + Feat. Ext	0.78	0.81	0.85	0.72	0.73	0.78
RF + FFT	0.98	0.99	0.97	0.95	0.98	0.99
SVM	0.31	0.34	0.35	0.20	0.22	0.40
SVM + FeatExt	0.43	0.39	0.41	0.43	0.39	0.51
SVM + FFT	0.98	0.98	0.98	0.99	0.98	1
CNN 1D—Time	0.94	0.95	0.93	0.90	0.92	0.99
CNN 1D—Frequency	0.99	1	1	1	1	1
CNN 2D—Time	0.82	0.82	0.78	0.74	0.81	0.97
CNN 2D—Frequency	0.99	1	1	0.99	1	1

. To explore the sensitivity of these algorithms to the scale of the input data, both models were retrained without applying any preprocessing and only with normalization of the input data. It was observed that the performance of the RF algorithm remained unchanged, with only a 5% reduction in training time when standardization was applied, indicating that this algorithm shows low sensitivity to data scaling. Similarly, the SVM model showed only a 3% improvement in accuracy when normalization was used instead of standardization, maintaining its performance when no preprocessing was applied.

The introduction of statistical feature extraction significantly improved the performance of the RF algorithm, with the RF + FeatExt model increasing accuracy to 78%, a value considered reasonable for a multiclass classification problem. This model also showed consistent F1-Score values across all classes, demonstrating a good balance between FP and FN.

However, Figure 9 shows that, for the particular case of unbalance fault, not all extracted features assume the same importance. Among the ten features analyzed, the mean exhibited the highest importance, followed by kurtosis, standard deviation, root mean square, skewness, crest factor and peak-to-peak values. Ranked within the lowest 20% of importance, indicating that they are largely irrelevant in this specific context, are the minimum, maximum, and peak values.

In contrast, the SVM + FeatExt model remained below 50% accuracy and required twice the training time of the RF + FeatExt model. From these results, it can be concluded that the RF algorithm is sensitive to the structure of the input data and that the application of proper preprocessing contributes to enhancing its robustness.

However, when vibration signals were transformed into the frequency domain, both algorithms showed significant performance improvements. The RF + FFT and SVM + FFT models achieved accuracy values close to 98%, along with high F1-Score values, demonstrating the relevance of frequency domain representations in fault diagnosis and confirming this technique as the most suitable for this type of analysis. In this case, the SVM + FFT model exhibited higher overall performance compared to the RF + FFT model, as both achieved very similar accuracy values, but the SVM + FFT model required less computational effort, with a reduction in training time of approximately 31%.

On the other hand, the CNN models achieved high accuracy and F1-Score results, even when trained with raw signals in the time domain.

The CNN 1D—Time model, although trained directly with raw time domain signals, also achieved a strong accuracy of 93%. However, this result came at the cost of the longest training time. This indicates that the proposed CNN architecture is capable of handling raw vibration signals directly from the sensor, which simplifies the data processing pipeline required for integration into the proposed framework. However, this approach can be computationally expensive.

The lowest performance was observed with the CNN 2D—Time model, which reached an accuracy of 82% and showed degraded F1-score values for unbalance classes close to normal operation, such as Unb. I and Unb. II.

Comparing both time domain approaches, the CNN 1D—Time model proved to be more effective. A possible explanation lies in the nature of the generated images. For instance, Figure 8 shows a confusing representation of the time domain signature, appearing as a predominantly dark image, making it difficult to extract the waveform shape and discriminative features. This issue becomes more pronounced with signals acquired at very high sampling frequencies, such as in Dataset II, where the resulting images contain large dark areas that obscure waveform features and peak analysis. The performance of the CNN 2D—Time model on Dataset II validates this observation. Thus, special attention must be given to the way input images are generated when applying two-dimensional models, particularly for time domain vibration signatures, in order to enhance their discriminative power.

Overall, the CNN models based on frequency domain signals delivered superior performance, achieving accuracy of 99% and F1-score values close to 100% across all unbalance scenarios. In particular, the CNN 1D—Frequency model achieved outstanding performance with an accuracy of 99%, maintaining consistency across all unbalance classes, while requiring a relatively short training time. These findings suggest that transforming vibration signals into the frequency domain enhances the extraction of robust discriminative patterns, thereby improving the model’s generalization ability. This conclusion is further supported by the extension of the models to Datasets II, where the performance of frequency domain models remained high.

The CNN proved to be sensitive to the size of the dataset, specially in the time domain, with their performance decreasing slightly when using Dataset II, specially for the CNN 1D—Time model. This suggests the need for large and balanced datasets to ensure effective generalization. Conversely, the SVM + FeatExt model improved its performance on dataset II (

Table 5. Experimental results—Dataset II.

Models	Dataset II
	Acc.	F1-Score
		Normal	Unb. I	Unb. II	Unb. III	Misal.	Unb. II + Misal.
RF	0.38	0.45	0.38	0.30	0.49	0.37	0.31
RF + FeatExt	0.84	0.80	0.72	0.83	0.93	0.85	0.92
RF + FFT	0.90	0.96	0.90	0.87	0.77	0.90	0.95
SVM	0.37	0.35	0.38	0.29	0.53	0.36	0.31
SVM + FeatExt	0.63	0.73	0.51	0.49	0.67	0.61	0.75
SVM + FFT	0.95	0.94	0.92	0.94	0.98	0.99	0.95
CNN 1D—Time	0.37	0.46	0.25	0.25	0.40	0.37	0.32
CNN 1D—Frequency	1	1	1	1	1	1	1
CNN 2D—Time	0.64	0.69	0.45	0.62	0.70	0.62	0.73
CNN 2D—Frequency	0.99	1	0.99	1	0.99	1	1

), suggesting that smaller datasets may facilitate its ability to identify an effective separating hyperplane between different classes.

The cross-analysis of the results obtained for both datasets showed that the remaining models maintained similar performance when applied to new fault conditions or combined faults, confirming their robustness and generalization capability.

Figure 10 compares the accuracy value of all developed models for both datasets I and II. The figure shows that both RF and SVM models perform poorly when applied directly to raw time domain vibration signals. In contrast, the RF + FFT and SVM + FFT models demonstrate that frequency domain approaches achieve substantially higher accuracy, indicating superior predictive performance. As shown in

Table 6. Developed models training time on dataset I.

Model	Training Time (Seconds)
RF	796
RF + FeatExt	32
RF + FFT	552
SVM	14,226
SVM + FeatExt	74
SVM + FFT	384
CNN 1D—Time	1662
CNN 1D—Frequency	584
CNN 2D—Time	694
CNN 2D—Frequency	277

, the primary limitation of these models lies in their computational cost rather than their predictive capability.

Training time is particularly important when a continuous learning process, based on the identification of unknown faults, is incorporated into the proposed framework, as it represents the computational cost associated with each model training cycle.

The SVM model exhibits the highest training time, such as shown in Table 6, which is consistent with initial expectations, since the SVM algorithm must determine a decision boundary that correctly separates data with different patterns. In the time domain, each input sample has a high dimensionality, proportional to the signal sampling frequency. A similar effect is observed in the RF algorithm, as the number of nodes is also related to the dimensionality of the input data.

The fastest models are RF + FeatExt and SVM + FeatExt. However, despite its low training time, the SVM + FeatExt model exhibits poor predictive performance and is therefore not suitable for integration into the proposed framework. Although frequency domain models present slightly higher training times, they offer a favorable trade-off between accuracy and training time.

Since the CNN models were developed using the GPU resources freely available on Kaggle, rather than the local resources employed for the development of the remaining models, they cannot be directly compared. However, using the accuracy values obtained together with the training times from Kaggle, the CNN 2D—Frequency model achieved the best overall performance, outperforming the CNN 2D—Time model by exhibiting lower training time and higher accuracy. Although the CNN 2D—Frequency model achieved an accuracy value very close to that of the CNN 1D—Frequency model, its training time was 53% lower, indicating that the two-dimensional approach is more efficient than the one-dimensional counterpart. The CNN 1D—Time model achieved an accuracy of 94%, which is notably high for a raw time-domain signal approach. However, its significantly longer training time may represent a limitation.

Overall, transforming the signals into the frequency domain proved to be the most effective processing technique for developing robust and low sensitivity models, as shown by the confusion matrices in Figure 11, Figure 12, Figure 13, Figure 14, Figure 15 and Figure 16.

The confusion matrix of the RF + FeatExt model (Figure 11) reveals a more pronounced dispersion of classification errors among the different classes, as evidenced by the tonal variations across the matrix. Analyzing this matrix, it can be observed that most misclassifications occur between classes that are significantly different. For instance, 139 samples labeled as Normal were incorrectly classified as Unb. II and 145 as Unb. III. Similarly, 167 samples were misclassified as Unb. I when they actually corresponded to samples labeled as Unb. IV. These results suggest that, although the RF + FeatExt model demonstrates reasonably acceptable performance for a multiclass classification model, it exhibits difficulty in distinguishing between distant fault severities, occasionally misclassifying samples from opposite severities with a particularly tendency to confuse normal operational states with fault conditions of intermediate or high severity, limiting its reliability for precise fault diagnosis.

The confusion matrix of the SVM + FeatExt model (Figure 13) confirms its poor performance by showing a heterogeneous pattern largely related to confusions between classes with very different severity levels. For example, 319 samples belonging to Unb. I class were misclassified as Unb. IV, being this class the one with the highest number of correct predictions.

Contrarily, this behavior was not observed for the frequency domain models, as the RF + FFT, SVM + FFT, CNN 1D—Frequency and CNN 2D—Frequency models exhibited more homogeneous confusion matrices, characterized by a clearly defined diagonal, indicating mostly correct classifications. The observed classification errors occurred predominantly between adjacent classes, namely those representing similar levels of unbalance fault severity.

According to vibration analysis theory, this behavior can be explained by the fact that, in the frequency domain, the main distinguishing factor between fault severities of the same type is represented by a slight increase in amplitude at the faulty frequency. These small variations in amplitude values can make it challenging for the models to distinguish between classes with closely related severities.

For example, the confusion matrix of the RF + FFT model (Figure 12) shows that 93 samples labeled as Unb. II were classified as Unb. I, and a smaller number of confusions occurred between Unb. II and Unb. III classes. When analyzing the diagonal elements, the Normal class shows the highest number of correctly classified samples, corresponding to a high F1-score. This result indicates that the RF + FFT model performs particularly well in distinguishing between normal and faulty operating conditions.

The strong colorful diagonal in Figure 14, surrounded by a clear pattern, once again highlights the high performance of the SVM + FFT model.

In general, both the CNN 1D—Frequency and CNN 2D—Frequency models demonstrate high precision across all classes, without significant error patterns. Showing the CNN 2D—Frequency model the lowest number of misclassifications, which is consistent with the accuracy and F1-Score values shown in Table 4.

Analysis of the confusion matrices shows that, out of 6372 faulty samples in the test set of dataset I, 311, 16, 854, and 28 samples were incorrectly classified as Normal by the RF + FeatExt, RF + FFT, SVM + FeatExt, and SVM + FFT models, respectively, corresponding to misclassification rates of 4.8%, 0.2%, 13.4%, and 0.4%. In contrast, the CNN 1D—Frequency and CNN 2D—Frequency models misclassified only 1 and 7 faulty samples as Normal, yielding substantially lower error rates of 0.01% and 0.1%. This result is particularly relevant in the domain of intelligent condition monitoring and maintenance safety, as the use of models that are unable to accurately detect and diagnose fault conditions in a timely manner may lead to catastrophic failures with severe material damage and risk to human life.

The Receiver Operating Characteristic (ROC) curves in Figure 17 and Figure 18 confirm that the SVM + FFT and CNN 2D—Frequency models exhibit behavior very close to that of an ideal classifier, with the different curves positioned near the upper-left corner of the graph and their corresponding Area Under the Curve (AUC) values very close to, or equal to, 1. These are characteristic features of high performance classifiers, demonstrating a high TP rate and a low FP rate. The same pattern observed for the SVM + FFT model is also verified for the RF + FFT model, just as the CNN 1D—Frequency model exhibits a pattern identical to that of the CNN 2D—Frequency model.

Based on these results, the RF + FFT, SVM + FFT, and CNN 2D—Frequency models were selected for experiments involving the controlled addition of white noise. It was observed that the performance of the RF + FFT model decreased by 8% when noise with an SNR value of 10 dB was added exclusively to the training set. However, its performance dropped significantly to 36% when the same amount of noise was added to the test set after the model had been trained on a clean training set. A similar performance was observed when both the training and test sets contained noise, with the model achieving an accuracy of only 30%.

This behavior can be explained by the fact that the RF algorithm is composed of multiple decision trees whose nodes are defined based on rigid threshold values used to classify samples.

In the case of the RF + FFT model, these nodes rely on features such as frequency components and their corresponding amplitude values. When white noise is added to the vibration data, random fluctuations are introduced, altering the spectral amplitude values that characterize a given fault type. This results in instability in the decision thresholds of the tree nodes, node splits driven by random variations, and consequently, trees that are highly sensitive to noise.

As a result, model accuracy drops to 36% when the model is trained on clean data and tested using noisy data. This occurs because, during training, the model learns a set of parameters that it is subsequently unable to correctly identify in the test set, which contains features that fall outside the domain learned during training. Consequently, the classification process becomes largely arbitrary.

When both the training and testing datasets contain noisy data, the true fault patterns are significantly degraded, as the randomness introduced by noise causes high variability in the spectral amplitude values of the samples. This variability hinders the consistent repetition of fault patterns required for effective model learning, leading to excessive node fragmentation within each decision tree and, consequently, to a substantial deterioration in model performance.

These results indicate that the RF + FFT model is not suitable for development in a controlled laboratory environment with subsequent implementation in real world conditions, unless more advanced processing techniques are applied to remove noise from the input data.

In contrast, the SVM + FFT and CNN 2D—Frequency models maintained their high performance when subjected to the same noisy datasets. The SVM + FFT model exhibited only about 1% difference in accuracy between experiments, demonstrating that the SVM algorithm is capable of identifying characteristic features associated with different unbalance levels even in noisy environments with a SNR of 10 dB, provided that the analysis is performed in the frequency domain.

Similarly, the CNN 2D—Frequency model showed an accuracy degradation of only 7% when noise with an SNR of 10 dB was added exclusively to the training and validation sets or only to the test set, with accuracy remaining above 93% in both scenarios. When noise was added simultaneously to the training, validation, and test sets, the model accuracy remained close to 100%, showing performance similar to that obtained in experiments free of noise.

However, in real-world maritime and industrial environments, noise levels are highly variable and unpredictable. For this reason, Figure 19 presents the accuracy of both models for SNR values of −10, −5, −3, 0, 3, 10, 15 and 20 dB, considering models trained on clean datasets and evaluated using noisy test set. Under these conditions, both models maintained stable performance for SNR values above 10 dB. As the SNR decreased, that is, as the noise level progressively increased relative to the original signal, model accuracy began to decline. Even so, the CNN 2D—Frequency model achieved a relatively acceptable accuracy of approximately 71% at an SNR of 3, compared to 62% for the SVM + FFT model. As the SNR values further decreased into negative ranges, the accuracy of both models dropped sharply to values below 50%, corresponding to performance comparable to that of a random classifier. At an SNR of −10 dB, the SVM + FFT and CNN 2D—Frequency models achieved accuracy values of 22% and 30%, respectively.

These results indicate that, up to a certain noise level, the SVM + FFT and CNN 2D—Frequency models can be considered robust and therefore suitable for deployment within the proposed framework. In this context, the models may be trained using data acquired in a controlled laboratory environment and subsequently applied using data collected under real operational conditions. Nevertheless, it should be noted that monitoring and controlling the noise level in the operational environment is essential in order to ensure that it does not approach a threshold that compromises the reliable operation of the proposed models.

Despite the degradation in accuracy as noise levels increase, the observed behavior of the models is consistent with initial expectations. Since fault symptoms in the frequency domain are primarily associated with increased amplitudes at characteristic frequencies, it is expected that, as noise progressively overlaps with the original signal, these spectral components become masked. This effect increasingly hinders their clear identification and, consequently, limits the models ability to accurately learn and distinguish fault related features during the training phase.

In contrast to the RF algorithm, which inherently learns local decision rules, the SVM algorithm performs classification by defining a hyperplane that serves as a global decision boundary, aiming to maximize the margin between samples of different classes. This characteristic allows the SVM to maintain stable performance even when noise alters the spectral amplitude of the samples, as the relative positions of the samples are only slightly affected, causing minor adjustments to the optimal hyperplane. In the case of vibration signals, faults are manifested as frequency bands, harmonics, and structured patterns. This structure contributes to preserving the relative positions of samples even in the presence of random noise. Furthermore, the use of kernels, such as the RBF kernel employed in this work, enhances the model’s ability to learn smooth decision boundaries, making it less rigid compared to the threshold-based node splits in the RF algorithm.

On the other hand, the CNN 2D—Frequency model takes input images representing the spectrogram of each sample. The spectrogram highlights different amplitude values at specific frequencies on a clearly distinguishable scale. CNNs are capable of learning very specific local patterns, and when noise is added, these patterns are not completely destroyed but only attenuated, unless the noise level is so high that it effectively masks the original signal. This is illustrated in Figure 19, where the CNN 2D—Frequency model begins to exhibit a random behavior only for SNR values below −3 dB. The ability of this algorithm to learn highly specific local patterns makes it particularly well suited for frequency domain data represented through spectrograms. As a result, the CNN 2D—Frequency model is less sensitive to white noise and exhibits greater robustness and stability across a wider range of SNR values when compared to the SVM + FFT model, as shown in Figure 19.

Among the developed models, the CNN 2D—Frequency model exhibits the most satisfactory overall performance, achieving high accuracy, relatively low training time, satisfactory F1-score values, and robustness when applied to Dataset II, even though its performance was slightly lower than that of the SVM + FFT model for a SNR value of 10. However, it should be noted that the development of CNN-based models requires access to more demanding computational resources. In scenarios where such resources are not available, and considering only the standalone classification performance, the SVM + FFT model offers a good balance between accuracy and computational effort, with the additional advantage of robust performance in noisy environments within a specific SNR range.

6.2. Unknown Faults

Nevertheless, when considering the integration of the classification model into the proposed framework, its compatibility with the framework’s capability to identify and handle unknown fault conditions must also be taken into account. The practical implementation of the proposed framework using the SVM + FFT and CNN 2D—Frequency models, trained exclusively with unbalance related samples from Dataset I and subsequently evaluated using 3800 new samples associated with bearing faults, revealed that both models tend to exhibit overconfidence in their classifications. As a result, they showed very low rates of classifying these samples as unknown faults. Specifically, the SVM + FFT model yielded rejection rates of 0% for threshold values t = 0.7, 0.8, 0.9 and 0.95, while the CNN 2D—Frequency model achieved rejection rates of 1.71%, 2.82%, 4.55%, and 6.47%, respectively, increasing to 10.29% when a threshold value t of 0.99 was applied.

This behavior is mainly attributed to the intrinsic characteristics of the employed algorithms. CNNs are known to be inherently overconfident, often assigning high class membership probabilities even when predictions are incorrect. This behavior is partly associated with the activation function used. The softmax function lacks the ability to identify samples as being outside the training distribution, instead forcing each input to be classified into the least incorrect known class by matching partial patterns, without allowing for rejection of out of distribution samples.

The loss function used during training also contributes to this overconfidence, as loss functions that strongly penalize low-probability assignments to the correct class, such as the sparse categorical cross-entropy function employed in this work, tend to encourage overly confident predictions.

A similar phenomenon is observed for SVM models, which inherently assign a known class label to any input sample, even if it does not belong to the training distribution. This behavior reflects a common characteristic of many classifiers, known as the closed-set assumption, whereby models are restricted to known classes and lack an explicit rejection mechanism for unknown samples.

In contrast, the Random Forest algorithm demonstrated low sensitivity to overconfidence. The RF + FFT model achieved rejection rates of 100% for threshold values t of 0.7, 0.8, 0.9 and 0.95. Similarly, although the RF + FeatExt model achieved lower accuracy, it exhibited rejection rates of 71.68% for t = 0.7 and 100% for t of 0.8, 0.9 and 0.95, respectively.

7. Conclusions

This paper proposes a novel framework based on the use of ML models for the automatic detection and diagnosis of faults in rotating machinery. To validate the potential of this framework, representative data of unbalance, the most common fault in rotating machinery, was used for the development of ML models based on the RF, SVM, and CNN algorithms, evaluated according to different processing techniques and performance metrics. These models were subsequently validated through a sensitivity and robustness analysis by extending their application to faults such as misalignment and the combination of unbalance and misalignment.

The experimental results demonstrated that CNN models achieved superior performance compared to others ML models such as RF and SVM, particularly when applied to signals in the time domain. However, when analyzing vibration signals in the frequency domain, all models showed accuracy and F1-Score values above 98%. In the case of the RF algorithm, the application of statistical feature extraction resulted in accuracy values around 78%, highlighting the importance of appropriate preprocessing practices in the development of intelligent fault detection and diagnosis models.

Despite the promising results achieved with CNNs, this algorithm proved to be challenging, particularly with regard to defining an architecture suitable for the available dataset and handling specific noise conditions. Similar limitations were observed for the SVM algorithm, whose performance is highly dependent on the adequate selection of hyperparameters, especially in scenarios prone to overfitting and in the presence of significant levels of noise in the input data.

The experiments conducted with controlled noise addition to the original datasets revealed that, although deep learning models generally exhibit higher accuracy, their performance also degrades under certain noise conditions. Considering that in industrial and maritime environments the noise level is often unpredictable, this behavior represents a relevant limitation of the proposed models. Such limitation may be mitigated through the incorporation of a signal preprocessing phase focused on noise reduction, performed immediately after signal acquisition. This step aims to preserve the authenticity of the measured signal and enhance the effectiveness of subsequent feature extraction processes. Within the context of vibration analysis, several denoising techniques can be considered, including conventional filtering approaches, such as band-pass filters for the removal of undesired frequency components, as well as more advanced methods like Empirical Mode Decomposition (EMD), Variational Mode Decomposition (VMD), Discrete Wavelet Transform (DWT), applicable in both time and frequency domains, the Hilbert–Huang Transform (HHT) applied in the time–frequency domain, Fourier-based filtering techniques, and Fast Independent Component Analysis (FastICA).

The conducted experiments allow concluding that the performance of the developed models is strongly influenced by the type of data used, the presence or absence of noise, and the computational resources available during the training process.

Overall, the obtained results and the flexibility of the proposed framework, which allows the use of both previously acquired and real-time data, combined with emerging Industry 4.0 technologies such as smart sensors, cloud computing, and the Internet of Things (IoT), demonstrates a strong potential to transform the traditional paradigm of CM through the application of intelligent approaches, thereby enhancing efficiency in maintenance management within maritime environments, where resources are often limited.

The implementation of autonomous fault detection and diagnosis systems can significantly contribute to mitigating recurring issues such as the shortage of specialized technical knowledge on board, limited material resources, and low equipment availability, thereby increasing operational availability and enhancing the safety of maritime operations.

Considering that the results obtained are highly satisfactory, it is considered that the developed models should be applied to different types of faults in order to broaden the application domain of the proposed framework, as well as to integrate noise mitigation techniques that enable their implementation in maritime industrial scenarios with a higher probability of success. These developments will allow monitoring systems to progressively become more accurate, precise, and robust, enhancing their generalization capability and consolidating their role as an intelligent tool for fault detection and diagnosis.

A comparison between the developed models and alternative architectures, such as Long Short-Term Memory (LSTM) and transformer-based models, should also be conducted in order to broaden the comparison among different approaches in this field and to assess the potential of recent baseline models. Although these architectures are typically more computationally demanding, they may represent a suitable alternative to the developed models, particularly for offline applications.