Classification of Highly Imbalanced Supervisory Control and Data Acquisition Data for Fault Detection of Wind Turbine Generators

Abstract

It is common knowledge that wind energy is a crucial, strategic component of the mix needed to create a green economy. In this regard, optimizing the operations and maintenance (O&M) of wind turbines (WTs) is key, as it will serve to reduce the levelized cost of electricity (LCOE) of wind energy. Since most modern WTs are equipped with a Supervisory Control and Data Acquisition (SCADA) system for remote monitoring and control, condition-based maintenance using SCADA data is considered a promising solution, although certain drawbacks still exist. Typically, large amounts of normal-operating SCADA data are generated against small amounts of fault-related data. In this study, we use high-frequency SCADA data from an operating WT with a significant imbalance between normal and fault classes. We implement several resampling techniques to address this challenge and generate synthetic generator fault data. In addition, several machine learning (ML) algorithms are proposed for processing the resampled data and WT generator fault classification. Experimental results show that ADASYN + Random Forest obtained the best performance, providing promising results toward wind farm O&M optimization.

1. Introduction

Wind energy has experienced sustained growth over the last decade, as shown in a recent report by the Global Wind Energy Council (GWEC), which informed that 78 WTs were installed in 2022. This has facilitated a total of 906 WTs of installed wind power capacity around the world [1]. Moreover, the rapid development of Artificial Intelligence (AI) techniques and their tools has positioned the fault prediction of wind turbines (WTs) as a field of great interest in the scientific community [2].

WTs are equipped with a SCADA (Supervisory Control and Data Acquisition) system for performance monitoring, supervision, and remote control. A SCADA system typically uses 10 min intervals to monitor more than 200 signals generated by a WT [3] and creates historical datasets which, after being processed by appropriate data analysis techniques, are a valuable source of information on the WT’s status. However, datasets in the real world are unevenly available, especially in industrial machinery fault detection, which means that the amount of normal data (majority class) is much larger than that of fault-related data (minority classes).

In this context, it is important to mention that WTs are routinely in a normal operating state, i.e., a large amount of normal SCADA data and little abnormal SCADA data are generated. The normal operating state of the WT comprises many stages, such as the maximum power point tracking stage, the constant rotational speed stage, the rated power output stage, etc. [4].

Most of the models used for fault classification, and the metrics employed to evaluate such models, assume that the distribution of classes is equal. However, imbalanced distributions exist across the board between abnormal and normal classes, leading to inaccurate failure diagnoses and predictions, because these models tend to be biased toward the most widespread class.

This paper analyzes high-frequency and highly imbalanced SCADA data and alarms on an in-service WT to predict failure in the induction generator. The data used for the study are sampled every 30 s, thus being high-frequency. We highlight this aspect since the sampling frequency of SCADA data used in most studies on WT fault prediction is 10 min [5].

In this work, the class imbalance problem, commonly disregarded, is first tackled. To this end, modern resampling techniques are used to remove or generate synthetic data randomly. Novel machine learning (ML) algorithms are then used for data processing and fault classification. Finally, in order to avoid indicators that focus exclusively on one of the classes, we use specific performance metrics that evaluate the overall performance of the classification algorithms.

The main contributions of this article are summarized as follows:

• We carried out an exhaustive analysis of the SCADA operating data and alarms. In the case of operational data, their high sampling frequency allowed us to better observe the behavior patterns in the variables of interest.
• We used different resampling techniques to analyze our dataset, which is characterized by a severe imbalance between classes (normal and faults), with an approximate imbalance rate of 10,000:1. To reduce the classification bias of the data toward the majority class, the data were resampled in the training set with different oversampling methods.
• Using different metrics, we compared the performance of four binary classification algorithms. We tried to use evaluation metrics that measure the overall performance of the classification algorithms without solely focusing on one particular class.

Following this introduction, the paper is structured as follows: Section 2 describes the state of the art and motivation of the present work. Section 3 presents the dataset used for the analysis and the different methods implemented. The results are described in Section 4, including the performance metrics. Finally, the conclusions drawn from the analysis are summarized in Section 5, and the acknowledgments are included.

2. State of the Art and Motivation

A dataset is defined as imbalanced when the classification categories are not equally represented [6], which generally occurs in industrial rotating machinery for failure data versus normally operating data.

A recent work, presented in [7], performs a state-of-the-art review on intelligent machine fault diagnosis using small and imbalanced data. The research results are divided into three categories: data augmentation-based, feature learning-based, and classifier design-based. The authors conclude that the data augmentation-based strategy using the Synthetic Minority Oversampling Technique (SMOTE) can effectively enlarge the volume of fault data; the feature learning-based strategy identifies faults accurately by extracting features from small and imbalanced data, whereas the classifier design-based strategy achieves high accuracy in fault diagnosis by building classifiers suitable for small and imbalanced data. Additionally, Ref. [8] proposes a deep learning (DL) model with data pre-processing and hyper-parameter tuning to achieve early anomaly detection in WTs. The imbalanced classes in the records are addressed by SMOTE, with the experimental results showing that the proposed model can identify possible failures 72 h before they occur.

In [9], a Minority Clustering SMOTE method (MC-SMOTE) is proposed, which involves clustering minority class samples to improve classification performance. Here, minority class samples are clustered and then combined with SMOTE. Subsequently, an experiment is performed on real SCADA data on WT blade icing, the results of which demonstrate the superiority of MC-SMOTE over SMOTE. Additionally, in [10], an imbalance learning algorithm based on the SMOTE (Easy-SMT) technique is proposed and augments the faulty classes. The feasibility and effectiveness of the proposed method in a real WT icing fault forecasting challenge are determined. The experiments show that Easy-SMT achieves better performance in binary and multi-fault classifications.

Similarly, in [11], the MiniBatch K-means clustering algorithm is combined with SMOTE (MBK-SMOTE), allowing for the distribution of samples to be balanced. The authors then use the Random Forest (RF) algorithm to predict icing on WT blades, concluding that MBK-SMOTE significantly improves prediction accuracy.

Another method to diagnose WT blade icing faults is presented in [12]. To overcome the shortcomings of data imbalance, a Safe Circle SMOTE (SC-SMOTE) is proposed and a fault diagnosis method based on an improved k-Nearest Neighbor (kNN) classification is adopted. The experimental results show the effectiveness of the method.

Furthermore, in [13], different analysis techniques are applied to imbalanced SCADA data to improve temperature-related faults of the gearbox of a WT. Principal Component Analysis (PCA) is used for data modeling and reduction, and an oversampling technique is used for imbalanced data, which is able to increase the information per sample. The combination of these techniques leads to a better performance of the classification algorithms.

To overcome the problem of data imbalance between the majority and minority classes, and the problem that models tend to be biased toward the majority class, a new intelligent fault diagnosis methodology based on Deep Neural Networks (DNNs) is proposed in [4]. Here, the problem of imbalance between classes is tackled by learning a deep representation that can preserve within-class information and between-class information. The effectiveness and generalization of the proposed method are validated on SCADA data from two WTs.

Meanwhile, a new Spatio-Temporal Multiscale Neural Network (STMNN) is presented in [14], to extract fault features from imbalanced SCADA data and execute the multi-class fault diagnosis of WTs. To address the SCADA data imbalance problem and improve the fault diagnosis performance, the model adopts Focal Loss (FL) as the loss function; the experiments show that the STMNN method achieves the best performance.

In [15], a novel approach based on the Synthetic and Dependent Wild Bootstrapped Oversampling Technique (SDWBOTE) is proposed for fault detection and localization of WTs with imbalanced data. The authors design an improved oversampling algorithm to generate the balanced dataset. They then introduce Convolutional Neural Networks (CNNs), with experimental results of seven cases using the datasets collected from two real wind farms in China to validate the effectiveness and robustness of the proposed approach.

A further work is presented in [16], which aims to predict fault-related alarms in WTs. SCADA data are analyzed, and SMOTE is used to balance the classes. Then, Support Vector Classifiers (SVCs) and Decision Trees (DTs) are compared, and the obtained performance results incline DT to be selected for the prediction model. Additionally, an investigation on WT gearbox failure diagnosis is presented in [17], the aim of which is to identify the most suitable classification technique that least depends on the imbalance level of the dataset. The authors conclude that DT is the most suitable prediction model for this task.

Moreover, in [18], a fault diagnosis algorithm combining SMOTE with adaptive Transfer Learning (TL) is proposed. A case study demonstrates that the proposed algorithm can effectively train models with highly imbalanced data and identify the types of faults and the time windows in which they occur. In [19], models are developed using Artificial Neural Networks (ANNs) to characterize the behavior and predict failures of the WT, gearbox, and generator. The proposed method is tested on real WTs in Italy to verify its effectiveness.

An approach for detecting faults in WT converters using CNN and SCADA data is presented in [20], finding that the accuracy of the proposed model is up to 98% compared to other models.

In [21], TL algorithms on SCADA data are used for WT fault diagnosis. TL algorithms are used to solve the data imbalance problem and two failure modes are compared with the proposed algorithm, which is shown to perform better in dealing with data imbalance. A study on WT failure data is presented in [22], demonstrating through case studies that the proposed system can effectively detect failure events in the generator, the transformer, and the hydraulic system. Another experimental study shows that, by using SCADA data, SMOTE, and ML tools, WT failures can be predicted with 80% accuracy 18 days in advance [23].

While the above shows that SMOTE has been successfully used in a number of studies for imbalanced datasets in the wind industry, other oversampling techniques are widely used in the AI community and, in some cases, may be more effective than SMOTE. This is the case of Adaptive Synthetic Sampling (ADASYN) and Ranked Minority Oversampling of Minorities in Boosting (RAMOBoost). However, as stated in [24], these have not yet been used for the condition monitoring (CM) of WTs. Other studies that motivated this work were [25,26,27,28,29,30,31,32].

In summary, there are a considerable number of scientific studies that combine techniques for treating imbalanced SCADA data with the application of ML models in specific problems of WT fault detection. Several factors, such as ice accumulation on the blades, gearbox heating, and converter component malfunctions, among others, can cause these failures. However, studies using highly imbalanced and high-frequency SCADA data for WT generator fault detection are limited. We aim to fill this research gap, providing new perspectives and innovative solutions.

Table 1. Summary of relevant studies on resampling techniques and ML tools for WT fault detection.

Ref. and Year	TDID 1	Learning Algorithms	Tools/ Methods	Main Components	Results
[4] 2019	Triplet loss function	classification	DNN, kNN	blade	90.85% (precision), 63.10% (recall), 74.47% (F1-Score)
[7] 2021	SMOTE	classification	GANs 2, DNN, SVM 3	WT	not specified
[8] 2021	SMOTE	classification	DNN, SVM 3, RF, kNN	WT	>90% (precision)
[9] 2020	MC-SMOTE	classification, clustering	SVM 3, CART 4, Bayesian classifiers	blade	91.70% (precision), 92.10% (recall), 91.90% (F1-Score)
[10] 2018	SMOTE	classification	RF, GBDT 5, XGBoost 6	WT	86.82% (precision), 96.50% (recall), 91.49% (F1-Score)
[11] 2017	MBK-SMOTE	classification, clustering	K-means, RF	blade	54.00% F1-Score), 74.85% (AUC)
[12] 2020	SC-SMOTE	classification	kNN	blade	78.90% (acc)
[13] 2021	Random oversampling, data reshaping	classification	kNN, SVM 3, RUSBoost 7	gearbox	99.82% (recall), 95.48% (F1-Score), 95.36% (acc)
[14] 2021	FL function	classification	STMNN, LSTM 8, CNN	WT	94.70% (F1-Score), 90.10% (G-mean)
[15] 2021	SDWBOTE	classification	CNN	WT	97.51% (F1-Score)
[16] 2021	SMOTE	classification	SVC, DT	WT	80.00% (precision), 83.00% (recall), 81.00% (F1-Score), 87.00% (acc)
[17] 2018	SMOTE, undersampling, cost-sensitive learning	classification	MLP 9, Naive Bayes, kNN, DT, Bagging, Rotation Forest	gearbox	85.00% (MCC)
[18] 2023	SMOTE	classification	CNN, ResNet 10	gearbox, generator bearing, hydraulic system	>75.00% (recall), >51.43% (F1-Score)
[20] 2021	Upsampling operations	classification	CNN	converter	98.41% (precision), 97.66% (recall), 98.04% (acc)
[21] 2021	Re-sampling	classification	kNN, RF, TrAdaBoost 11, Inception V3	blade, pitch system	>73.20% (recall), >89.40% (acc)
[23] 2017	SMOTE	classification	ANN, SVM 3, KNN, Naive Bayes	generator	96.34% (recall), 94.80% (acc)
[25] 2022	SMOTE, FL function, re-sampling	classification	GANs 2, JAFTN 12, WDCNN 13	gearbox	99.70% (precision), 99.60% (F1-Score), 99.60% (acc)
[26] 2020	SMOTE	classification, clustering	LSTM 8, XGBoost 6	WT	97.00% (acc)
[27] 2022	SMOTE, cost-sensitive learning	classification	LGBM 14, GMM 15	blade	97.80% (F1-Score)
[28] 2020	SMOTE, cost-sensitive learning	classification	Logistic regression, RF, XGBoost 6, LSTM 8	gearbox	52.00% (precision), 86.00% (recall), 57.00% (F1-Score)
[29] 2022	SMOTE	classification	LSTM 8, CNN, MLP 9	hydraulic system, generator, converter	>80.20% (F1-Score), >90.60% (acc)
[30] 2023	SMOTE+ENN	classification	DT, AdaBoost 16, KNN, RF	WT	99.60% (precision), 99.20% (recall), 99.60% (F1-Score)
[31] 2023	SMOTE, Adasyn	classification	MCL 17, STMNN, STFNN 18	WT	94.70% (F1-Score)
[32] 2022	SMOTE, Adasyn, SMOTE+ENN	classification	DT, KNN, SVM 3, MLP 9	blade	94.20% (F1-Score), 94.20% (G-mean), 93.80% (MCC)

¹ Techniques for Dealing with Imbalanced Data (TDID), ² Generative Adversarial Networks (GANs), ³ Support Vector Machine (SVM), ⁴ Classification and Regression Trees (CARTs), ⁵ Gradient Boosting Decision Tree (GBDT), ⁶ eXtreme Gradient Boosting (XGBoost), ⁷ Random Under-Sampling Boosting (RUSBoost), ⁸ Long Short Term Memory (LSTM), ⁹ Multi-Layer Perceptron (MLP), ¹⁰ Residual Neural Network (ResNet), ¹¹ Transfer Adaptive Boosting (TrAdaBoost), ¹² Joint Attention Feature Transfer Network (JAFTN), ¹³ Deep Convolutional Neural Networks with Wide First-layer Kernels (WDCNN), ¹⁴ Light Gradient Boosting Machine classifier (LGBM), ¹⁵ Gaussian Mixture Model (GMM), ¹⁶ Adaptive Boosting (AdaBoost), ¹⁷ Matching Contrastive Learning (MCL), ¹⁸ Spatio-Temporal Fusion Neural Network (STFNN).

summarizes the most relevant articles analyzed in this work. The objective is to succinctly show the different data resampling techniques used in WT fault detection research. Likewise, the AI algorithms used, the WT component or subsystem on which the study is focused, and the results achieved are shown.

In particular, the results column offers the reader the possibility of comparing the performance of the algorithms used in fault detection based on the metrics of precision, recall, F1-Score, Area Under Curve (AUC), accuracy (acc), geometric mean (G-mean), and Matthews Correlation Coefficient (MCC).

3. Materials and Methods

This section describes the dataset used in the present work and the methodology implemented. The methodology includes algorithms to tackle the class imbalance problem and, subsequently, the ML algorithms for fault detection. The complete process is summarized in Figure 1.

3.1. Data Description

This paper analyzes high-frequency SCADA data from an operating onshore wind farm located in the province of Albacete in Spain for a period of nearly a year, representing a total of 782,233 records. Due to a confidentiality agreement between the wind farm owners and the authors, the wind farm name is anonymized in this study.

The SCADA data specifically correspond to a pitch-controlled WT Gamesa G90-2MW, equipped with a doubly fed induction generator (DFIG). The SCADA system of the WT under study collects 20 variables every 30 s.

In order to reduce the full dataset, a correlation matrix was used to determine the most suitable variables for the study. The matrix shows the color-coded Pearson’s correlation coefficient scale for each pair of variables. In this study, we have chosen this correlation method for its ability to calculate the correlation between quantitative variables and to identify linear relationships between variables in the SCADA dataset.

As can be seen in Figure 2, there is a low correlation between the different temperatures and the different power-related variables. Regarding the speed-related variables, these present a medium-high correlation with the power-related ones, as opposed to the former with the temperature-related variables, which again show a low correlation.

After analyzing this matrix, the following variables were selected for the study: wind speed, slip ring temperature, stator temperature, ambient temperature, rotor power, stator power, and rotor speed. These variables were selected based on the target variable. In this study, stator power is the target variable as it is closely related to the generator. In addition, the authors made a final selection of the resulting variables based on their domain knowledge.

Besides the variables shown, the SCADA system also collects alarms and warnings, which indicate abnormal behavior in the operation of WT components or system-operating states. According to [33], alarms can be classified into general alarms, operational alarms, environmental alarms, and communication alarms.

During the period of study, the SCADA system recorded generator failures that resulted in WT shutdown. In addition, a few days prior to these failure events, running alarms associated with failure, which did not involve WT shutdown, were recorded. These operating alarm records were considered in this study, and their details are presented in Figure 3.

3.2. Methods to Balance the Class Distribution

As previously explained, imbalanced training datasets can negatively affect the performance of most ML algorithms, with it thus being necessary to balance the class distribution prior to implementing fault detection using ML.

In this study, the WT failure data represent the minority class. The majority class (no anomaly or class 0) represents 99.99067% of the dataset, while the minority class (anomaly or class 1) represents 0.00933% of the dataset, with an imbalance ratio of close to 10,000:1.

Data resampling is an effective strategy for balancing the class distribution. The two main data resampling methods for the training dataset are oversampling and undersampling. Undersampling is the method by which the number of instances or samples of the majority class are removed. Among the most commonly used undersampling methods are random undersampling, near miss undersampling, and Tomek Links undersampling [34].

The oversampling technique aims to increase the number of instances or samples of the minority class until the training dataset is balanced. The most commonly used oversampling methods are random oversampling [13] and SMOTE, the latter having several variations, such as Borderline-SMOTE [35].

Another oversampling technique is ADASYN, which has been widely applied in the healthcare domain, but has not been used with WT SCADA data. In the present work, ADASYN and SMOTE (including two variations) are proposed.

The ADASYN procedure is briefly described as follows [36].

• First, the number of synthetic data examples that need to be generated for the minority class is determined:

  $$G=(m_l-m_s)·\beta$$

  (1)

  where $m_s$ represents the number of minority class examples, $m_l$ represents the number of majority class examples, and $\beta$ is used to specify the balance level after the generation of the synthetic data, being between [0,1].
• Afterward, for each example, the minority class $\left(x_i\right)$, finds k-nearest neighbors based on the Euclidean distance in n dimensional space, and calculates the ratio $r_i$ defined as

  $$r_i=\frac{{\Delta }_i}{k};i=1,\dots ,m_s$$

  (2)

  where ${\Delta }_i$ represents the number of examples in the k-nearest neighbors of $\left(x_i\right)$, associated with the predominant class.
• Then, normalize the ratio $r_i$ according to

  $${\widehat{r}}_i=\frac{r_i}{{\sum }_{i=1}^{m_s}{r}_i}$$

  (3)

  where ${\widehat{r}}_i$ represents the density distribution, and ${\sum }_i{\widehat{r}}_i=1$.
• Finally, the number of synthetic data instances that must be developed is calculated for each $x_i$:

  $$g_i={\widehat{r}}_i·G$$

  (4)

The other technique used in the present work, SMOTE, generates synthetic samples in the feature space formed by samples of the minority class and the KNN. According to [7], the procedure used by the SMOTE algorithm is as follows:

• The nearest neighbor sample for each minority class sample is found.
• Random samples to interpolate are chosen among the nearest neighbor samples.
• The original minority class samples and their neighbor samples are linearly interpolated. The synthetic sample $\left({\chi }_{new}\right)$ is generated by

  $${\chi }_{new}=\chi +rand(0,1)·({\chi }_i-\chi ),(i=1,2,\dots ,N)$$

  (5)

  where $\chi$ represents the original samples used to obtain the new samples, $rand(0,1)$ is a random number from 0 to 1, which guarantees that ${\chi }_{new}$ is on the line joining the original data $\chi$ and one of its nearest neighbors, and ${\chi }_i$ represents a randomly selected sample among the minority class samples.

In addition, two variations of SMOTE are implemented in the present work, namely SMOTE + Tomek and SMOTE + ENN. Both variations are regarded as hybrid methods, since they combine oversampling and undersampling techniques, providing the best of both techniques for improved results.

3.3. ML Methods for Fault Detection

The present work uses four binary classification algorithms to predict WT failures. These were trained using the default parameters and adjusting the hyperparameters. A brief description of the algorithms used is presented below.

• Random Forest (RF). Proposed by Leo Breiman in 2001 [37], this is a supervised learning algorithm used for classification or regression problems, which works by building many decision trees, where each tree yields a class prediction, and the class with the most votes becomes the model prediction [38]. The general RF classifier pseudocode used in this study is shown in Algorithm 1.
• Decision Trees (DTs). Non-parametric algorithms obtained from a set of learning cases, labeled by class and attribute values. This prediction algorithm is suitable for regression and classification problems, although it can be highly unstable, and overtraining should be avoided [39]. Algorithm 2 shows the pseudocode of the DT classifier.
• Multilayer Perceptron (MLP). ANNs are applied to a sequence of time series forecasting problems. These have an input layer, one or more hidden layers, and an output layer [40]. The advantages of MLP include the ability to withstand elevated levels of noise in the input data and to learn independently of the linear and nonlinear relationships existing in the variables under study.
  For the binary classification problem, the MLP can be described as

  $$y\left(x\right)=\phi ·(\sum _{i=1}^n{W}_i{X}_i+b)$$

  (6)

  where $W_i$ denotes the weights associated with the neuron, $X_i$ are its inputs, b is the bias, and $\phi$ is the transfer or activation function.

Algorithm 1: Random Forest Classifier

Algorithm 2: Decision Tree Classifier

• The MLP structure used in this study consists of four hidden layers in addition to the input and output layers. The activation functions of the hidden layers are rectified linear unit (ReLU) and Sigmoid for the output layer. In addition, in the first hidden layer, a dropout rate equal to 0.5 was applied to avoid the model overfitting problems. The MLP pseudocode is shown in Algorithm 3.
• Boosting Decision Tree (BDT). In this algorithm, each tree depends on the previous trees (boosting). Therefore, each tree has information about the errors made by the previous one, thus helping to refine the result [10]. The pseudocode for the BDT classifier employed in this study is presented in Algorithm 4.

Algorithm 3: Multilayer Perceptron Classifier

Algorithm 4: Boosting with Decision Trees Classifier

3.4. Metrics Used to Quantify the Prediction Accuracy

Various performance metrics are used to quantify prediction errors of oversampling methods, on the one hand, and of ML methods, on the other hand. The metrics selected for use in the present study are accuracy (acc), recall, precision, AUC, specificity, F1-Score, geometric mean (G-mean), Negative Predictive Value (NPV), weighted accuracy (W_acc), and Matthews Correlation Coefficient (MCC). Of these, acc, recall, specificity, AUC, G-mean, NPV, and W_acc are specifically useful for classification models with imbalanced data since they focus specifically on the performance of a class [13,23].

Conversely, acc can be a very unreliable metric when the class distribution is highly biased [4,11], and is thus not used in this study to evaluate the ML methods for fault detection.

All of the above-mentioned metrics are briefly described as follows, where TP, TN, FP, and FN refer to true positives, true negatives, false positives, and false negatives, respectively.

• Acc: The total percentage of correctly classified elements is given by the total number of correct predictions divided by the total number of m predictions (Equation (7)).

  $$acc=\frac{TP+TN}{m}$$

  (7)
• Recall: Known as True Positive Rate, this is the percentage of positive instances correctly classified, i.e., the ability of the model to predict the class correctly, as shown in Equation (8). Recall is useful for failure prediction as it effectively measures the minority class prediction coverage [13].

  $$recall=\frac{TP}{TP+FN}$$

  (8)
• Precision: As indicated in Equation (9), precision measures the number of positive class predictions that belong to the positive class. In other words, precision refers to the number of positive items that the model properly detected out of all the potentially positive elements [9].

  $$precision=\frac{TP}{TP+FP}$$

  (9)
• AUC: This metric represents the area under the ROC curve. The ROC (Receiver Operating Characteristic) curve is a graph that shows the performance of a classification model. The graph represents the distribution of the True Positive Rate on the y-axis versus the False Positive Rate on the x-axis.

  $$AUC=\frac{recall+specificity}{2}$$

  (10)
• Specificity: This is a metric that determines the harmonic mean between precision and the recall [13]. Specifically, Equation (11) defines the classification specificity.

  $$specificity=\frac{TN}{FP+TN}$$

  (11)
• F1-Score: This is a metric for quantifying model performance. It is useful for imbalanced data because it attempts to find the balance between precision and recall [36]. The F1-Score measure is calculated as follows:

  $$F1-Score=2·\frac{precision·recall}{precision+recall}$$

  (12)
• G-mean: This evaluates the performance of the majority and minority classes compared to each other. Although negative cases are successfully classified, a low G-mean suggests poor performance in classifying positive cases [9]. G-mean can be calculated as follows:

  $$G-mean=\sqrt{recall+specificity}$$

  (13)
• NPV: This is the ratio of properly categorized negative class labels to the total number of predicted negative labels [13]. The formula for this metric is given by

  $$NPV=\frac{TN}{FN+TN}$$

  (14)
• W_acc: This is the average of the recall and specificity (Equation (15)), and measures the average accuracy of both minority and majority classes. If a classifier performs equally well on both classes, this number drops to the traditional accuracy. In contrast, if the high value of traditional accuracy is due to the classifier exploiting the majority class distribution, the balanced accuracy will be lower than the accuracy [21].

  $$W\_acc=0.5·\left(recall+specificity\right)$$

  (15)
• MCC: This includes all the elements of the confusion matrix (TP, TN, FP, and FN), as shown in Equation (16). It is the metric least influenced by imbalanced data. It ranges from −1 to +1; a value of +1 indicates a perfect prediction, and −1 indicates the worst possible prediction [17].

$$MCC=\frac{(TP·TN)-(FP·FN)}{\sqrt{(TP+FP)(TP+FN)(TN+FN)(TN+FN)}}$$

(16)

4. Results

This section presents the results after performing the methodology described in Figure 1, with the ultimate goal of improving the fault detection algorithms by improving the imbalance problem.

4.1. Data Resampling

As mentioned in Section 3.2, this study has an imbalance between the normal and fault classes. To delve deeper into the problem of imbalance, a further correlation analysis was performed to determine the relationship between the variables under study for both the normal and anomaly classes of samples.

In this regard, as depicted in Figure 4, a high positive correlation was found between stator power and rotor power (for both class 0 and class 1 samples), which denotes a close relationship between the generator’s electrical and mechanical behavior. Deviations in this correlation during WT operation could be an early indication of possible generator failure.

Moreover, similar correlations were observed between stator power and wind speed and between ambient temperature and slip ring temperature. The latter association can be explained by the direct effect of ambient temperature on the increase in slip ring temperature. On the other hand, the correlations with the other variables were lower due to the dispersion of the samples analyzed.

The differences between the synthetic samples generated with the resampling techniques are visually imperceptible when considering the complete dataset. Therefore, a sample of 10,000 data was selected for better visualization of the resampling on the power curve. Figure 5 shows the application of the four resampling techniques, and a magnified segment of the power curve is used to show how each technique generates different synthetic samples.

The class imbalance problem was then tackled. A DT classifier was used to evaluate the different oversampling techniques. This decision was taken due to the computational benefits of this algorithm [41], which is crucial due to the considerable volume of data being processed in this study. The experiments were carried out with the hyperparameters shown in

Table 2. Hyperparameters of the DT classifier.

Hyperparameter	Value	Description
max_depth	5	Maximum Tree Depth
min_samples_split	2	Minimum number of samples to split node
min_samples_leaf	1	Minimum number of samples in a leaf
max_features	None	Maximum number of characteristics
criterion	gini	Measure of the quality of a node split

.

The results of applying the oversampling techniques to the complete dataset were analyzed using the specific metrics selected, explained in Section 3.4, and shown in Figure 6. As can be seen, the ADASYN technique performed best, yielding the following metrics: acc = 92.80%, recall = 60.27%, specificity = 92.80%, AUC = 76.85%, G-mean = 74.79%, NPV = 99.99%, and W_acc = 76.58%.

4.2. Performance of Classification Algorithms

After having implemented (and analyzed) the techniques to overcome the class imbalance problem, the fault detection algorithms described in Section 3.3 were implemented, with these being MLP, RF, DT and BDT. To this end, the dataset was first divided into a training set (70% of the data) and a test set (remaining 30% of the data), and various analyses were performed by gradually adjusting the hyperparameters in the specific algorithms.

As in the previous step, the metrics proposed in Section 3.4 were used to quantify the results. Since the aim of this study was to predict generator failures, (i.e., faults – class 1), class 1 is considered the more important one in the analysis. In this sense, recall is regarded as a key metric since one of the objectives in fault classification problems is to minimize the number of false negatives. Erroneous classification of observations as normal operation (class 0) instead of fault (class 1) may cause unplanned downtime and, thus, additional costs in the O&M of WTs.

Table 3. Performance evaluation values for the classifiers used in this study.

MLP
hidden_layer sizes	alpha	Precision	Recall	F1-Score	Specificity	AUC	G-mean	NVP	W_acc	MCC
100	0.0001	0.000	0.150	0.001	0.766	0.610	0.481	0.989	0.558	0.016
100	0.001	0.000	0.200	0.001	0.762	0.571	0.639	0.989	0.581	0.021
100	0.01	0.001	0.625	0.001	0.895	0.771	0.769	0.999	0.775	0.024
(50, 50)	0.0001	0.000	0.105	0.001	0.787	0.597	0.322	0.989	0.518	0.008
(50, 50)	0.001	0.000	0.100	0.001	0.788	0.595	0.314	0.989	0.544	0.020
BDT (lr: 50, max_depth: 0.01)
n_estimators	min_samples split	Precision	Recall	F1-Score	Specificity	AUC	G-mean	NVP	W_acc	MCC
0	3	0.000	0.300	0.000	0.005	0.784	0.589	0.924	0.723	0.013
1	3	0.000	0.500	0.000	0.005	0.784	0.589	0.953	0.723	0.002
2	3	0.000	0.600	0.001	0.005	0.844	0.739	0.994	0.750	0.014
3	5	0.000	0.500	0.000	0.003	0.830	0.576	0.907	0.744	0.001
4	5	0.000	0.400	0.000	0.003	0.830	0.576	0.907	0.740	0.001
DT (criterion: gini)
max_depth	min_samples split	Precision	Recall	F1-Score	Specificity	AUC	G-mean	NVP	W_acc	MCC
None	2	0.082	0.500	0.131	0.993	0.546	0.555	0.999	0.546	0.227
None	2	0.082	0.300	0.131	0.993	0.546	0.315	0.999	0.546	0.227
5	2	0.094	0.600	0.162	0.993	0.822	0.805	0.999	0.826	0.247
5	5	0.022	0.300	0.111	0.918	0.595	0.525	0.999	0.609	0.219
5	5	0.022	0.300	0.111	0.918	0.595	0.525	0.999	0.609	0.219
RF (n_stimators: 50)
max_depth	min_samples split	Precision	Recall	F1-Score	Specificity	AUC	G-mean	NVP	W_acc	MCC
None	2	0.149	0.400	0.426	0.937	0.648	0.722	0.999	0.523	0.420
None	5	0.219	0.400	0.425	0.996	0.668	0.722	0.999	0.523	0.419
None	10	0.019	0.500	0.425	0.996	0.661	0.622	0.999	0.523	0.419
10	2	0.183	0.500	0.489	0.934	0.589	0.743	0.999	0.657	0.413
10	5	0.391	0.643	0.487	0.999	0.821	0.802	0.999	0.821	0.502

shows the classifiers’ performance when using the resampled dataset using the Adasyn technique, mentioned above as the most effective in terms of resampling. Experimental tests were carried out by varying the values of the hyperparameters. The best performance obtained by each classifier is shown in bold.

The metrics obtained after implementing the ML algorithms are shown in Figure 7. As can be seen, RF is the best performing algorithm, with the following scores: precision = 39.16%, recall = 64.38%, F1-Score = 48.70%, specificity = 99.99%, AUC = 82.18%, G-mean = 80.23%, NPV = 99.99%, W_acc = 82.18%, and MCC = 50.21%. It is also noticeable that, for some metrics, DT reaches values close to those obtained for RF. As previously explained, acc was not used to evaluate the proposed models since it can be very unreliable when the class distribution is highly biased, as is the case here.

The performance of the ML techniques selected for fault detection were further analyzed before and after implementing the oversampling techniques, through confusion-matrix plots and ROC curves.

On the one hand, the results of the confusion-matrix plots prove that, without applying resampling techniques, the classifiers overfit the majority class, as can be seen in Figure 8a. On the other hand, the results of the evaluation of the classifiers trained with the resampled datasets, SMOTE (Figure 8b) and ADASYN (Figure 8c), highlight the performance of RF in classifying faults, especially the minority class.

Finally, the performance of these models was analyzed using ROC curves. As can be observed in Figure 9, the results obtained improve after applying SMOTE and slightly more after applying ADASYN. This is especially true for RF and DT techniques, with RF providing better results than DT.

5. Conclusions

This paper presents a solution for the prediction of faults in the WT generator, which is one of the main components of a WT. The solution is based on high-frequency and highly imbalanced normal-operating SCADA data and fault-related data. The proposed solution includes testing four resampling techniques to compensate for biases that occur in imbalanced classification problems.

The resampling techniques implemented in the present work compare commonly used methods for the CM of WTs, such as SMOTE and its variations, against newer techniques, such as ADASYN, which is used in other fields but has not previously been implemented in the wind energy sector. These oversampling techniques were analyzed using different performance metrics as well as confusion-matrix plots and ROC curves, where ADASYN obtained the highest scores on most evaluation metrics. Furthermore, four classification algorithms for WT generator fault detection were tested before and after implementing the proposed resampling techniques, with RF (after ADASYN) obtaining the best results on different evaluation metrics and showing better results than DT in the confusion-matrix plots and ROC curves.

Therefore, the methodology proposed in this study, developed using high-resolution SCADA data from real WTs, is able to overcome the imbalance problem normally found in operating wind farms and, thus, improve fault detection which will serve to optimize the O&M of WTs.

Future research directions include exploring advanced oversampling methods such as Gaussian Copula Oversampling, Distance-based Arranging Oversampling, and hybrid techniques like SMOTE-NC + RUS (SMOTE for Nominal and Continuous features + Random Under Sampling) to address the imbalance between SCADA system operational data and WT fault data. We will also investigate the use of DL algorithms, such as LSTM, due to their ability to identify complex data patterns without additional resampling techniques. This will allow us to evaluate the benefits and limitations of different models, thus improving the detection of WT faults.