A Cluster-Based Filtering Approach to SCADA Data Preprocessing for Wind Turbine Condition Monitoring and Fault Detection

Abstract

The high cost of wind turbine maintenance has intensified the need for reliable fault detection and condition monitoring methods. While Supervisory Control and Data Acquisition (SCADA) systems provide valuable operational data, the raw signals often contain noise, outliers, and missing or redundant entries, which can compromise analysis accuracy. This study presents a novel cluster-based outlier removal approach for SCADA data preprocessing, featuring a unique flexibility to include or exclude negative power values—a factor rarely investigated but potentially critical for fault detection performance. The method applies the K-Means++ unsupervised clustering algorithm to group data points along the wind speed–power curve. The number of clusters is determined heuristically using the elbow method, while outliers are identified through Mahalanobis distance with thresholds derived from Chebyshev’s inequality theorem. The approach was validated using SCADA data from a wind farm in Portugal and further assessed with a CUSUM test-based structural change detection method to study how preprocessing choices—outlier thresholds (5% vs. 1%) and inclusion/exclusion of negative power values—affect early fault identification. Results demonstrate reliable fault detection up to 14 days before failure, retaining over 99% of the original dataset. This work provides key insights into preprocessing impacts on model reliability and offers an open-source Python implementation for reproducibility.

1. Introduction

In 2024, global wind power capacity continued to grow, with 117 GW of new installations—an 11% year-over-year increase—bringing the total capacity to 1136 GW [1]. Of this growth, 109 GW came from onshore wind projects, while offshore installations accounted for the remaining 8 GW. China is the world’s leader commissioner of wind power making almost 70% of the overall 2024 wind power growth. However, the growth of wind power installations slowed in 2024 due to macroeconomic pressures, including rising raw material and component costs as well as supply chain disruptions, which increased maintenance expenses and delayed turbine production [2]. Therefore, it is important to mitigate the unexpected downtimes of wind turbines to avoid unexpected costs and properly plan for maintenance [1]. Renewable energy sources account for more than 40% of global electricity production in 2024. While hydropower plants and nuclear power plants contributed 23.3% of the total generated electric power, wind and solar power plants produced 15% with wind power contribution at 8.1%. Other renewable energy sources like bioenergy and geothermal energy delivered 2.6% of global electricity in 2024. Solar and wind energy contributions to electricity generation are steadily increasing, while the shares from nuclear and hydropower are gradually declining. This trend reflects growing interest and investment in wind- and solar-based renewable energy sources [1].

According to a recent study [3], about one-fifth of the wind energy generation cost comes from operation and maintenance efforts over the turbine’s lifetime. Equipment failures lead to turbine downtime, directly impacting both revenue and electricity prices. To mitigate these effects, condition monitoring and early fault detection systems have been developed, aiming to identify issues proactively and reduce unplanned outages, as reported in the literature [4,5,6]. Generator, gearbox, and blade pitch systems are among the most failure-prone components in wind turbines, often causing unplanned downtime and expensive repairs [7]. Due to their high replacement costs, significant research efforts have been directed toward developing early failure detection methods specifically for these critical modules [8,9]. Condition monitoring and fault detection systems methodologies are divided into sensor-based, model-based, and SCADA-based data-driven approaches [10,11,12]. Sensor-based methods rely on signals from sensors that acquire operational data from a wind turbine and then process the data into meaningful information about equipment maintenance status. These methods encompass a wide range of techniques, including vibration analysis, acoustics, strain and torque measurements, oil property monitoring, image recognition, and electrical signal analysis [13]. However, implementing these approaches often incurs additional costs due to the need for installing auxiliary sensors. Additional equipment will increase operating costs due to the maintenance of sensor infrastructure and associated software. This is why SCADA-based approaches are widely preferred [14]. A Supervisory Control and Data Acquisition (SCADA) system collects and aggregates sensor and actuator data from wind turbines, enabling operators to make informed, data-driven decisions. SCADA allows for fault detection using fixed thresholds or by analyzing signal trends over time. Most wind turbines come equipped with proprietary SCADA systems, eliminating the need for additional sensors or wiring. Data is typically recorded at fixed intervals, often at a 1 Hz frequency, providing high-resolution monitoring and sensitivity to signal changes.

Condition monitoring and fault diagnosis techniques typically rely on classical statistical methods, machine learning approaches, or a combination of both [11,12]. Classical statistical techniques often use control charts and statistical process control to detect system failures or anomalies. The study in [15] introduced a statistical method for monitoring wind turbine generators by analyzing the deviation of an individual turbine’s behavior from the average performance of other turbines on the same wind farm. Four key operational parameters were assessed: electrical energy output, tower vibration, nacelle yaw, and gearbox temperature. Control charts were then employed to identify abnormal behavior in the monitored turbine. Approaches based on the autoregressive models are also widely used for wind turbine data analysis and forecasting. Multi-class autoregressive moving average models were employed to address the seasonality issue of wind turbine operating conditions and proved to achieve better model parameters estimation [16]. In recent years, many advanced statistical methods have been explored for wind turbine condition monitoring. Examples include probabilistic power curve estimation [17], multivariate statistical hypothesis testing [18], change-point detection techniques [19,20,21], cumulative sum (CUSUM)-based methods [22,23], Wilcoxon rank sum test-based method [24], and cointegration-based approaches [25,26,27,28,29]. These techniques offer efficient and cost-effective solutions for reliable monitoring, enabling timely fault detection and reducing turbine downtime. A recently proposed stationarity-based approach, introduced in [30] and based on the Augmented Dickey-Fuller (ADF) test [31], enables fault detection by identifying abrupt changes in the stationarity of SCADA signals over time, eliminating the need for predefined normal behavior models of the turbine. This method employed a sliding data window scheme, where anomaly detection is achieved through the successive accumulation of variations in signal stationarity within the moving window.

A hybrid approach of the seasonal autoregressive integrated moving average and deep-learning Long-Short Term Memory (LSTM) networks was examined to improve predictive accuracy of the model [32]. Classical models are restricted to linear and stationary problems. Therefore, non-linear models like gated recurrent unit neural networks were employed to capture non-linear phenomena of wind turbine data. Scholars are experimenting with hybrid models and combine methods and techniques from different fields of study to improve the overall performance of fault detection models. In [33], the authors proposed machine learning with statistical process control approach to study fault diagnosis in wind turbine data. In the study, check lists, Pareto charts, control charts, and other techniques were combined with machine learning algorithms, mainly decision tree and random forest classifiers, to predict failures in the machines. The results of this hybrid approach showed more than 90% accuracy in damage diagnosis [33]. Another hybrid approach aimed at improving the accuracy and predictability of wind turbine faults was presented in [34]. In this method, the authors employed extreme gradient boosting (XGBoost) and LSTM neural networks to model the characteristic behavior of critical wind turbine components. To assess anomalies, statistical process control techniques were applied. Validated using SCADA data from the ENGIE La Haute Borne wind farm, the method demonstrated the capability to predict critical failures up to two weeks in advance [34]. Recently, a hybrid approach combining machine learning techniques with cointegration analysis has been proposed for wind turbine condition monitoring and fault detection [35].

Recent research on early wear and failure detection in wind turbine gearboxes has shown that traditional methods may be insufficient for accurately identifying wear under real operating conditions. To address this, an unsupervised machine learning approach using an autoencoder architecture was employed to filter noise from image data [36]. The autoencoder, consisting of encoder and decoder modules, compresses input data and then reconstructs it, enabling clearer signal extraction. The processed data was then input into various deep learning models for evaluation and comparison [36]. For time-series forecasting, transformer models were used due to their strong performance; however, their high memory and computational demands, resulting from their quadratic complexity, posed challenges. To overcome this, the periodic-enhanced informer model was introduced, leveraging the inherent periodicity in wind turbine operational data to reduce algorithmic complexity and improve viability [37].

It is clear that the effectiveness of each method depends heavily on the quality of the data used. Typically, SCADA data is collected from various wind turbines in an unprocessed form, and applying models directly to raw data can lead to inaccurate or misleading results. Therefore, data preprocessing is a critical step in any data flow. Raw datasets often present several challenges beyond limited volume, including missing values, skewed distributions, unknown or mislabeled data, sampling errors, and non-stationarity. Addressing these issues through in-depth preprocessing is essential to ensure the reliability and accuracy of analytical models. One of the most impactful data issues affecting model performance is the presence of outliers [38]. Outliers are data points that significantly deviate from the majority of the dataset due to sensor malfunctions, calculation errors, or software defects. Although they are part of the raw data, their statistical properties differ emphatically from normal observations, which can distort analysis and reduce the accuracy of predictive models.

To ensure high model performance, outliers are typically identified and removed from the dataset [39]. However, the decision to exclude outliers depends on the specific application. For example, forecasting models generally require clean data, free from anomalies, to produce accurate predictions. In contrast, for condition monitoring and fault detection, retaining certain outlier samples might be valuable. A typical example involves negative power values in wind turbine operational data. While removing these values can enhance power forecasting accuracy, their role in fault detection remains less clear and has not been thoroughly investigated in the literature. Negative power may indicate turbine downtime due to faults, or simply reflect periods when the turbine draws power from the grid during low wind conditions without any malfunction. This brings forward the critical question of whether including or excluding negative power values impacts the reliability, accuracy, and overall effectiveness of models or methods developed for detecting abnormal turbine behavior.

Current state-of-the-art outlier processing methods generally fall into two main categories: statistical techniques and image-based approaches. Statistical methods evaluate data based on characteristics such as mean, variance, standard deviation, Z-scores, and K-sigma rules. One of the most commonly used techniques is the Interquartile Range (IQR) method, which identifies a sample as an outlier if the sum of the third quartile and the interquartile range of the data multiplied by the factor of 1.5 is smaller than the point where the sample is defined [40]. This approach effectively filters out data points that deviate significantly from the majority, helping to improve data quality for further analysis. Another method for outlier detection is based on Chebyshev’s inequality theorem, which states that at least 88.89% of data samples lie within three standard deviations from the mean, regardless of the data distribution [41]. Therefore, any sample falling outside this range can be classified as an outlier. Unsupervised learning methods, particularly data clustering techniques, are commonly used to identify and filter out outliers. These methods include division-based clustering, density-based clustering, model-based approaches, and others [42]. Additionally, image processing techniques have been applied for outlier removal, utilizing methods such as morphological transformations, edge detection, and pixel value thresholding [43,44,45]. Advanced data processing techniques have been developed to enhance outlier detection by integrating statistical and image processing methods. In [46], the authors proposed a three-stage outlier removal approach that combines physical rule-based filtering, the random sample consensus algorithm, and IQR thresholding. The final refinement was achieved using mathematical morphological image transformations. This hybrid method demonstrated high accuracy in detecting outliers across multiple wind turbine datasets.

The method proposed in this paper uses a cluster-based outlier detection approach. It employs the K-Means++ unsupervised machine learning algorithm to group data points along the wind speed–power curve of a wind turbine. The number of clusters, a required parameter for the algorithm, was determined heuristically using the elbow method. Outliers were then identified by applying a threshold derived from Chebyshev’s inequality theorem. Data points falling outside the acceptable range, based on their Mahalanobis distance from cluster centers, were removed from the dataset. The proposed approach introduces a distinctive flexibility that allows users to include or exclude negative power values—an aspect rarely explored but potentially crucial for improving fault detection performance. The method was validated using real SCADA data from a Portuguese wind farm and evaluated through a CUSUM test-based structural change detection technique. The results offer new insights into how negative power values influence model reliability and the accuracy of early fault detection.

The remainder of this paper is organized as follows. Section 2 introduces a novel cluster-based filtering approach for SCADA data preprocessing, aimed at enhancing condition monitoring and fault detection in wind turbines. Section 3 presents the Energias de Portugal wind farm, and the publicly available SCADA dataset used to validate the proposed filtering method. The cleaned dataset, which includes a real generator fault event, is then used to demonstrate the effectiveness of a CUSUM-based fault detection technique, further validating the applicability of the preprocessing approach. The results are presented and discussed in Section 4. Finally, the conclusions are given in Section 5.

2. A Cluster-Based Filtering Approach to SCADA Data Preprocessing

The K-Means algorithm is an unsupervised machine learning method that partitions data points into clusters based on their similarity, measured using the Euclidean distance. For each data point, the distance to a cluster’s centroid is calculated, and points are assigned to the nearest centroid. The algorithm optimizes clustering by minimizing the sum of squared distances within each cluster. It operates iteratively, starting with randomly initialized centroids and updating their positions until convergence is reached. The algorithm requires the number of clusters to be specified in advance. Initialization of centroids can be done randomly or through more efficient sampling strategies, such as K-Means++. The method is widely applied because of its simplicity, scalability, and fast execution, achieving near O(log N) complexity. K-Means++ improves the initialization phase, leading to faster convergence and often better clustering results, while keeping the complexity close to that of standard K-Means. These features make it particularly suitable for processing SCADA data, which is collected at short time intervals. In this study, the method is applied to wind turbine SCADA data to address the challenge of outlier detection, thereby enhancing overall data quality. The number of clusters is typically determined heuristically using methods such as the elbow method [47,48] or silhouette analysis [49]. In this study, the elbow method was applied. This approach evaluates a scatter plot of the within-cluster sum of squares against the number of clusters. Since the algorithm always converges and the plot decreases monotonically, a point can usually be observed where the slope begins to flatten. These “elbow” points are used for determining the appropriate number of clusters. Choosing more clusters can improve accuracy, but it also increases computational cost.

Figure 1 presents the flowchart of the cluster-based outlier removal method used to preprocess wind turbine SCADA data. The algorithm is applied separately to the dataset of each turbine in the wind farm. A key feature of the proposed approach is its flexibility in either including or excluding negative power values, enabling an in-depth investigation of how such values affect the reliability, accuracy, and effectiveness of different methods for detecting abnormal turbine behavior.

• The workflow starts by identifying negative power samples in the generated power parameter of the dataset. These identified rows are stored separately for later use and removed from the main table to allow focused processing of variables relevant to outlier detection. The preserved negative samples retain their original indices, ensuring they can be seamlessly reintegrated with the processed dataset later without requiring reindexing.
• The data containing only nonnegative generated power samples is then grouped into clusters using the K-Means++ algorithm. The optimal number of clusters is determined using the “Elbow method,” where the within-cluster sum of squares is plotted against the number of clusters. This plot enables a heuristic decision, balancing cluster compactness with computational efficiency. The cutoff point at which the metric is achieving steady state is typically selected as the desired number of clusters. Each cluster is represented by its centroid, which corresponds to the mean value of all data points within that cluster.
• To detect outliers, a criterion is established to determine whether a data sample deviates abnormally from others in the same cluster. The measure used is the distance between the cluster’s centroid and the sample under examination. In this approach, the Mahalanobis distance is employed, as it accounts for potential skewness in the data distribution. A sample is classified as an outlier if its distance exceeds a predefined threshold.
• In this study, the outlier detection threshold is set to three standard deviations (three sigma). Any sample with a distance exceeding this threshold is classified as an outlier and subsequently removed from the dataset. To evaluate the impact of outliers on the fault detection process, this study applies two threshold settings: a 95% confidence interval, which excludes 5% of samples per cluster, and a 99% confidence interval, which excludes 1% of samples per cluster.
• Finally, for this study, the negative power samples are either merged back into the filtered dataset or excluded entirely. This process produces two distinct cleaned datasets—one including negative power values and one without—both free from outliers.

3. Case Study: Energias de Portugal Wind Farm

3.1. A Short Description of the Wind Farm

The wind farm SCADA data [50] under study consists of four identical wind turbines (T01, T06, T07, and T11), each rated at 2 MW and equipped with planetary gearboxes and asynchronous generators. These turbines operate within a wind speed range of 4 m/s (cut-in) to 25 m/s (cut-out). SCADA data were collected over a two-year period, spanning from 1 January 2016, to 31 December 2017, with measurements taken every 10 min. Some timestamps are without data, likely due to operational downtimes or events such as generator replacements. The dataset comprises 417,093 data samples across 84 variables, including turbine identifiers, timestamps, and a wide range of operational and environmental indicators, such as rotor and generator speeds, bearing and gearbox temperatures, power output, wind characteristics, ambient temperature, and so on. It also includes operational status, alarm codes, and failure logs. The failure records specify the turbine involved, the date and time of the incident, the affected components, and additional remarks. Over the two years, 27 failures were recorded, with a higher frequency occurring in the second half of each year.

3.2. Data Preprocessing Results for Wind Turbines

The cluster-based filtering procedure described in Section 2 was applied to the wind farm SCADA dataset. Figure 2 presents representative results obtained for wind turbine T07 based on its 2017 operational data. This turbine experienced a real generator fault, which is described in Section 4 and used as a case study to validate the proposed method. In this study, the number of clusters was determined heuristically using the elbow method, resulting in the selection of nine clusters, as demonstrated in Figure 2.

As discussed in the introduction, a key objective of this study is to investigate how including or excluding negative power values affects the reliability, accuracy, and effectiveness of methods (or models) for detecting abnormal turbine behavior. To address this, several combinations of confidence intervals were tested under two scenarios: with negative power values included and with them removed. Figure 3 and Figure 4 display the filtered power curves for T07 including negative power values, using 95% and 99% confidence intervals, respectively. Figure 5 and Figure 6 show the corresponding results when negative power values are excluded.

4. Validation Using a Real Generator Fault

This section validates the proposed cluster-based filtering approach using a real generator fault from wind turbine T07. The CUSUM test-based method for detecting structural changes in SCADA data, originally developed in [22], is applied to assess how the inclusion or exclusion of negative power values, together with different outlier removal levels (i.e., excluding 5% or 1% of samples per cluster), influence the method’s reliability, accuracy, and ability to detect the T07 generator fault at an early stage.

4.1. Description of a Real Generator Fault in Wind Turbine T07

This case study examines two consecutive generator faults that occurred in wind turbine T07. The fault data were extracted from historical SCADA failure logs supplied by the wind farm operator, which include detailed timestamps of reported failures along with the corresponding turbine identifiers and affected components. The first issue (a generator bearing failure) was recorded on 20 August 2017, at 06:08 (sample 32,730). Roughly 32 h later, a second and more critical generator fault was logged on 21 August 2017, at 14:47 (sample 32,920). Figure 7 illustrates the turbine’s power output for August 2017, with clear markers indicating the failure events and the recovery point. A detailed overview of both faults, including their designated fault IDs, is shown in

Table 1. Consecutive generator failures in wind turbine T07 during August 2017.

Fault ID	Fault Type	Occurrence Time	Sample Index
F1	Generator bearing damage	20 August 2017, at 06:08	32,730
F2	Generator damage	21 August 2017, at 14:47	32,920

. Notably, the recovery of the turbine is estimated to have occurred on 28 August 2017, at 20:30 (sample 33,930), which is about 8 days and 14 h after the initial fault (F1).

4.2. CUSUM Test-Based Method for Detecting Structural Changes in SCADA Data

The CUSUM test is a widely used technique in statistics and econometrics for detecting changes or structural breaks in the coefficients of a multiple linear regression (MLR) model. The general form of the MLR model is expressed as:

$$y_t={\beta }_0+{\beta }_{1t}{x}_{1t}+{\beta }_{2t}{x}_{2t}+\cdots +{\beta }_{pt}{x}_{pt}+{\epsilon }_t,\hspace{1em}t=1,\dots ,T,$$

(1)

This can be rewritten more compactly as:

$$y_t=x_t{\beta }_t+{\epsilon }_t,\hspace{1em}t=1,\dots ,T,$$

(2)

where $y_t$ is the dependent variable, $x_t=\left(1,x_{1t},\dots ,x_{pt}\right)$ is a vector of predictor variables, ${\beta }_t={\left({\beta }_0,{\beta }_{1t},\dots ,{\beta }_{pt}\right)}^T$ represents the time-varying regression coefficients, ${\epsilon }_t$ denotes the error term, assumed to follow a normal distribution with zero mean and constant variance ${\sigma }^2$. The intercept ${\beta }_0$ reflects the expected value of the output when all predictors are zero, while the slope coefficients ${\beta }_{1t},\dots ,{\beta }_{pt}$ indicate the effect of each predictor on the response variable.

The CUSUM test detects coefficient instability by examining the cumulative sum of recursive residuals, which are calculated from sequential regressions on growing subsets of the data. These recursive residuals are defined as follows [51]:

$$w_r={\displaystyle \frac{y_r-x_r{\widehat{\beta }}_{r-1}}{\sqrt{1+x_r^{\prime }{\left(X_{r-1}^{\prime }{X}_{r-1}\right)}^{-1}{x}_r}}},r=k+1,\dots ,T,$$

(3)

where $k$ is the number of regression coefficients, $X_r=\left(x_1,x_2,\dots ,x_r\right)$, and ${\widehat{\beta }}_r={\left(X_r^{\prime }{X}_r\right)}^{-1}{X}_r^{\prime }{y}_r$ represents the Ordinary Least Squares (OLS) estimate of the regression parameters based on the first $r$ observations.

The CUSUM test statistic is the cumulative sum of standardized recursive residuals [51]:

$$W_r=\sum _{t=k+1}^r{\displaystyle \frac{w_t}{\widehat{\sigma }}},r=k+1,\dots ,T,$$

(4)

where $\widehat{\sigma }$ is the estimated standard deviation of residuals, computed by:

$$\widehat{\sigma }=\sqrt{{\displaystyle \frac{{\sum }_{t=1}^T{\left(y_t-x_t{\widehat{\beta }}_r\right)}^2}{T-k}}}$$

(5)

The hypothesis test is defined as:

• Null hypothesis ($H_0$): The regression coefficients remain constant throughout the sample.
• Alternative hypothesis ($H_1$): The coefficients vary over time, implying structural instability.

The CUSUM test decision rule is:

• If the CUSUM statistic $W_r$ crosses the critical boundaries, reject $H_0$ (indicating instability of regression coefficients).
• Otherwise, do not reject $H_0$ (i.e., regression coefficients are stable).

The critical bounds are linear lines connecting the points $\left\{k,\pm a\sqrt{T-k}\right\}$ and $\left\{T,\pm 3a\sqrt{T-k}\right\}$, where $T$ is the total number of observations, $k$ is the number of coefficients, $a$ is a constant determined by the desired significance level (e.g., $a=0.948$ for 5%, $a=1.143$ for 1%). The sequence of CUSUM test statistics is utilized as a fault-sensitive indicator and displayed on a control chart. This visualization aids in interpreting the outcomes of hypothesis testing and supports effective real-time monitoring and fault detection. The chart is defined by upper and lower threshold boundaries that form a critical region—if the CUSUM statistic crosses these boundaries, it indicates the occurrence of a potential fault in the wind turbine system.

It is worth noting that the CUSUM test was originally introduced in the statistics and econometrics literature by Brown et al. in 1975 [52], although research on structural break or change-point detection dates back to the seminal work of Chow in 1960 [53]. More recently, the study in [54] has shown that combining change-point detection techniques with trend monitoring can improve diagnostic performance for wind turbines. This integration leverages the complementary strengths of both approaches while helping to overcome their individual limitations.

In this case study, SCADA data capturing power curve dynamics and critical subsystem temperatures were utilized for condition monitoring and fault detection in wind turbines. A multiple linear regression model was developed specifically for use with the CUSUM-based method. The model includes eight independent variables: generated power ($x_{1t}$), wind speed ($x_{2t}$), generator phase 1 temperature ($x_{3t}$), hydraulic oil temperature ($x_{4t}$), temperatures of generator bearings 1 and 2 ($x_{5t}$ and $x_{6t}$), gearbox oil temperature ($x_{7t}$), and gearbox bearing temperature ($x_{8t}$). Generator speed was selected as the dependent variable ($y_t$) to reflect the system’s operational response. The MLR model has the form:

$$y_t={\beta }_0+{\beta }_{1t}{x}_{1t}+{\beta }_{2t}{x}_{2t}+{\beta }_{3t}{x}_{3t}+{\beta }_{4t}{x}_{4t}+{\beta }_{5t}{x}_{5t}+{\beta }_{6t}{x}_{6t}+{\beta }_{7t}{x}_{7t}+{\beta }_{8t}{x}_{8t}+{\epsilon }_t$$

(6)

4.3. Fault Detection Results

To assess the effectiveness of the CUSUM test-based method in identifying early fault indications, SCADA data collected before the fault events were analysed. Specifically, samples in the range from T1 = 30,292 (3 August 2017, at 00:00) to T2 = 31,977 (14 August 2017, at 23:50)—covering the period from about 18.5 to 6.5 days prior to the occurrence of fault F2—were analysed by the method.

First, the results for the case including negative power values are presented in Figure 8 and Figure 9. The computation of the CUSUM test statistic (shown as the blue solid line) and the critical bounds (represented by the red dashed lines) is described in Section 4.2. As shown in Figure 8, when the outlier detection threshold was set to the 95% confidence interval (with the corresponding filtered power curve in Figure 3), the CUSUM test-based method identified the generator fault at sample 30,893. This point, calculated as T1 + 601 (i.e., 30,292 + 601), corresponds to an early detection occurring 338 h (approximately 14 days) before fault F2. However, when the outlier detection threshold was set to the 99% confidence interval (with the corresponding filtered power curve in Figure 4), the results in Figure 9 indicate that the CUSUM statistic crosses the critical region at the point 1309. This implies that the generator fault was detected at sample 31,601, calculated as T1 + 1309 (i.e., 30,292 + 1309), indicating that detection occurred 220 h (or 9 days) before fault F2. Recall that outliers were identified using 95% and 99% confidence intervals, corresponding to the removal of 5% and 1% of samples per cluster, respectively. The results show that the extent of outlier removal influences early fault detection: when only 1% of samples per cluster is excluded, the method performs worse in detecting the generator fault early compared to the case where 5% of samples are removed.

The results for the case without negative power values are shown in Figure 10 and Figure 11. Notably, under this condition, the CUSUM test-based method failed to detect the fault. This suggests that eliminating negative power observations may also discard important fault-related information.

4.4. Comparison and Discussion

This section compares the proposed cluster-based outlier removal approach for SCADA data preprocessing with the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) method [55,56]. DBSCAN, introduced by Ester et al. (1996) [55], is a widely used density-based, non-parametric clustering algorithm. It identifies clusters by grouping points that are densely packed together, while points located in low-density regions are classified as outliers. Due to its ability to automatically detect arbitrarily shaped clusters and isolate sparse data regions, DBSCAN has become one of the most commonly applied clustering algorithms in data analysis and anomaly detection.

Figure 12 shows the filtered power curve of wind turbine T07 based on the 2017 SCADA dataset processed using the DBSCAN method. In this case, DBSCAN identified and removed the negative power values by classifying them as outliers. Overall, 29.01% of the data points were eliminated, which is a comparable removal rate to that achieved by the proposed cluster-based outlier filtering approach when negative power values are excluded (as illustrated in Figure 5 and Figure 6).

The SCADA data filtered using the DBSCAN method were then evaluated using the CUSUM-based fault detection approach to identify the generator fault. The same dataset used previously in Section 4.3 was applied to ensure a consistent comparison. As shown in Figure 13, the resulting CUSUM statistic approaches the critical threshold but does not exceed it, indicating that the fault is at the borderline of detectability.

These results further demonstrate that removing negative power measurements can inadvertently eliminate valuable fault-related indicators. Consequently, such data filtering may degrade the reliability and accuracy of early fault detection.

5. Conclusions

This paper presented a cluster-based filtering approach for preprocessing wind turbine SCADA data, aimed at enhancing condition monitoring and fault detection. By integrating the K-Means++ clustering algorithm with Mahalanobis distance and Chebyshev-based thresholds, the method effectively identifies and removes outliers while allowing flexibility in handling negative power values.

The experimental validation, carried out using SCADA data from a Portuguese wind farm, highlighted two key findings. First, the extent of outlier removal directly influences early fault detection performance: excluding 5% of samples per cluster enabled earlier fault detection compared to removing only 1%. Second, excluding negative power values was shown to compromise the reliability of detection, as the CUSUM test-based method failed to identify the generator fault in this case. These results suggest that negative power observations, although often treated as erroneous, may carry fault-related information critical for accurate condition monitoring. A comparative analysis was conducted, and the results show that the DBSCAN method and the proposed cluster-based outlier filtering approach (when negative power values are excluded) exhibit a similar data removal rate, both eliminating approximately 29% of the SCADA observations.

Overall, the study demonstrates that preprocessing strategies—particularly decisions regarding outlier removal thresholds and treatment of negative power values—play a decisive role in the reliability, accuracy, and timeliness of wind turbine fault detection models. The findings provide practical guidance for SCADA data preprocessing in real-world applications and contribute to ongoing efforts in improving predictive maintenance strategies for wind turbines enabling accurate predictions while retaining majority of data. It is particularly useful for data sets containing large amounts of negative power samples which may come from geographical or meteorological factors. To facilitate further research, reproducibility, and industrial adoption, the Python implementation of the proposed cluster-based outlier removal method has been made publicly available at: https://github.com/kkijano/wtcbfdp (accessed on 15 October 2025).

While this study focused on detecting generator faults, future work will be extended to identify failures in other subsystems such as the gearbox, blades, or pitch control system. To strengthen its robustness and scalability, future research should validate the cluster-based outlier removal approach using SCADA data from diverse turbine models and wind farm environments, including offshore installations. Moreover, upcoming work will explore hybrid strategies that combine domain knowledge with data-driven preprocessing to further improve fault detection accuracy and interpretability.