Privacy-Preserving Fleet-Wide Learning of Wind Turbine Conditions with Federated Learning

Abstract

A wealth of data is constantly being collected by manufacturers from their wind turbine fleets. And yet, a lack of data access and sharing impedes exploiting the full potential of the data. Our study presents a privacy-preserving machine learning approach for fleet-wide learning of condition information without sharing any data locally stored on the wind turbines. We show that through federated fleet-wide learning, turbines with little or no representative training data can benefit from accuracy gains from improved normal behavior models. Customizing the global federated model to individual turbines yields the highest fault detection accuracy in cases where the monitored target variable is distributed heterogeneously across the fleet. We demonstrate this for bearing temperatures, a target variable whose normal behavior can vary widely depending on the turbine. We show that no member of the fleet is affected by a degradation in model accuracy by participating in the collaborative learning procedure, resulting in superior performance of the federated learning strategy in our case studies. Distributed learning increases the normal behavior model training times by about a factor of ten due to increased communication overhead and slower model convergence.

1. Introduction

Wind energy plays a pivotal role in climate change mitigation. A massive growth in the installed wind power capacity and grid infrastructure is required to decarbonize the power supply [1,2]. Wind energy is rapidly growing, with new wind parks being commissioned and planned across the globe [3,4,5]. Wind turbine (WT) condition monitoring plays a crucial role in minimizing downtimes and enabling predictive maintenance of wind farms [6,7]. Yet, manufacturers are averse to sharing the required condition monitoring information due to business strategic interests [8]. As a result, a severe lack of data persists [9,10], hindering the full potential and development of data-driven models for diagnostic and condition monitoring tasks.

We address this issue by presenting a distributed machine learning approach that enables sharing condition information within a fleet of WTs from different owners while still preserving the privacy of data stored on the wind turbines. Within our study, we define a fleet as a set of all WTs of the same model. We demonstrate how data-driven condition monitoring models can be trained collaboratively by a WT fleet in a manner that allows sharing of condition information among the WTs without sharing the WTs’ condition data. Specifically, we propose to train accurate turbine-specific models of each WT’s normal operation behavior for fault detection tasks by making use of the condition monitoring data of the entire WT fleet in a privacy-preserving manner. This is a highly relevant scenario because, in practice, only the manufacturer can access the condition data from all WTs of a fleet, whereas other stakeholders only have access to a small share of the fleet’s data from their own WTs or even no data at all [8]. Other stakeholder groups concerned may include operators, owners, third-party companies, regulators, and researchers.

Thus, our study demonstrates a path toward privacy-preserving sharing of condition information without any manufacturer, operator or owner having to grant anyone access to their WTs’ operation and condition data. Wind farm operators usually have no access to WT data from other operators and are, therefore, not able to make use of condition information from other operators’ WTs for their own wind farms. The lack of data sharing (among wind farms of different owners) within fleets is particularly unfortunate in situations where the relevant data are scarce, for example, when the operator or other stakeholders seek to establish a damaged database but have only a few (or even no) fault events of each fault type in their database, or when a new WT has been commissioned and the stakeholder has no condition data available yet for that WT type. In such situations, it would be highly desirable to benefit from fleet-wide information sharing. Manufacturers, on the other hand, usually have access to the operation and condition data of all operating WTs produced by them but do not share these data.

To address the data imbalance, we propose and investigate the potential of privacy-preserving federated learning [11] for condition monitoring and diagnostics tasks in wind farms based on WT data distributed among multiple owners. With the introduction of the FedAvg algorithm [11], federated learning has significantly gained relevance, especially in the area of IoT applications and mobile devices. Numerous contributions have recently proposed improvements building upon FedAvg, for instance, regarding efficiency [12,13] or in enhancing its security [14,15]. Recent works also employ encryption mechanisms within the federated learning environment (e.g., [16,17]), thereby adapting the privacy benefits of the alternative distributed approach of sharing and training models on encrypted data, as presented in, e.g., [18,19]. We refer the reader to [14,20,21,22,23,24,25] for more complete reviews of recent developments and algorithms for federated learning. An application of federated learning that is in use operationally is next-word predictions for virtual keyboards in mobile apps [26,27]. First applications have also been proposed in other fields, such as automotive systems [28,29,30]. The capabilities of federated learning are still largely unexplored in renewable energy applications. Recently, Zhang et al. [31] proposed a federated learning case study for probabilistic solar irradiance forecasting. Their presented FedAvg-based framework, enhanced by secure aggregation with differential privacy, was shown to achieve performance advantages over a setting in which data sharing between participants was unavailable. However, the authors noted that the shared federated learning model resulted in slightly inferior performance compared to a centralized setting with data sharing, as it is susceptible to data distribution deviations between clients. Lin et al. [32] presented a federated learning approach for community-level disaggregation of behind-the-meter photovoltaic power production. To address the data heterogeneity of each community, a layer-wise aggregation was introduced. Only the parameters of the shallow layers, learning community-invariant features, were exchanged, while the community-specific parameters of the deep layers remained local. This customization step was shown to result in improvements compared to a completely shared global model. With a focus on efficiency, Q. Liu et al. [33] demonstrated a successful federated learning application for collaborative fault diagnosis of photovoltaic stations. To address the inefficiencies of FedAvg, especially when computing capabilities and dataset sizes differ among participants, the authors proposed asynchronous, decentralized, federated learning. This framework without a central server resulted in significant reductions in communication overhead and training time.

In the area of wind energy, Cheng et al. [34] presented the first and, to our knowledge, so far, only study of a federated learning model for wind farms. The authors proposed an approach for detecting blade icing by classification. A blockchain-based architecture with a cluster-based learning module was introduced to address concerns regarding privacy and malicious attacks, as well as data imbalance. The authors remarked that, while not considered in their study, existing data heterogeneity may negatively affect the performance of the model.

Classification methods, such as Cheng et al. [34], are relatively uncommon for fault detection tasks in wind farms in practice due to the typically small number (or even absence) of fault observations. The more commonly used normal behavior models, which model the normal state of the WT, do not require an extensive collection of a fault database, as opposed to the fault classification-based approach [35]. Normal behavior modeling involves modeling the behavior of the monitored WT under normal fault-free operation conditions. The resulting normal behavior models (NBMs) characterize the normal operation behavior of the monitored WT as expected under the prevailing operating conditions. These NBMs can, for instance, output an expectation of the generator temperatures, given that the WT is running in a normal state. Consequently, these models allow for the differentiation of anomalous from normal behavior based on the residuals of the actual and its expected value, thereby indicating possible faults and signalizing the need for further inspection [36,37,38]. In the literature, NBMs based on SCADA data have been proposed for monitoring both single and multiple target variables [39,40,41].

For normal behavior modeling for fault detection and diagnosis, numerous sensor systems are typically available, ranging from sensor systems for internal temperatures and oil quality anemometers for environmental measurements to accelerometers for vibration responses [42,43,44,45]. The WT’s supervisory control and data acquisition (SCADA) data can also be used for condition monitoring tasks (e.g., [35,41,46,47,48], also in combination with vibration data (e.g., [49]). As no further sensor installations are required, condition monitoring based on SCADA data presents a cost-effective technique. On the other hand, the WT health information provided by SCADA data may be less informative in that it can be less component-specific, less timely and less accurate with regard to the fault diagnostics task than dedicated sensing systems, such as accelerometers. For example, gearbox faults can be identified from vibration measurements at an early stage of fault development (e.g., [50,51]), whereas associated SCADA data, such as from the gearbox temperature, would allow the fault to be detected only once it resulted in an unusual increase of the gearbox temperature, i.e., at a late development stage. Such temperature increases typically result from abnormal heat generation that can originate from excessive friction. Therefore, in SCADA-based fault detection, a fault can often be detected only at a relatively advanced fault development stage in which initial damage may have already occurred. Nonetheless, using SCADA data for condition monitoring remains popular due to its numerous benefits, such as its ease of use, cost-effectiveness, and the possibility to complement it with other fault detection techniques for WTs. We refer to [35,52,53,54,55] for detailed reviews regarding data-driven diagnostic and condition monitoring approaches.

The potential of collaborative fleet-wide learning of normal behavior models for fault detection tasks in WTs based on SCADA data has not been discussed or investigated despite its high relevance for practical applications. Our study addresses this research gap by proposing federated learning of normal behavior models in a data-privacy-preserving manner. We propose a solution to an important practical problem in wind farm monitoring and diagnostics: How to train NBMs for detecting developing faults in WT subsystems when SCADA and sensor data for training NBMs are missing or not representative of the WT’s current operation. This is a major challenge in newly commissioned wind turbines and in turbines whose operation behavior changed, for example, due to large hardware or software updates. We demonstrate the federated learning of NBMs in two case studies for gear-bearing temperatures and power curves in two wind farms.

The main contributions of our study are:

• A new privacy-preserving approach to wind turbine condition monitoring;
• A customization approach to tailor the federated model to individual WTs if the target variable distributions deviate across the WTs participating in the federated training;
• Federated training and customization are demonstrated in condition monitoring of bearing temperatures and active power.

Our study contributes to resolving a major problem: the “lack of data sharing in the renewable-energy industry [which] is hindering technical progress and squandering opportunities for improving the efficiency of energy markets” [8].

This study is structured as follows. Section 2 details our proposition for collaborative privacy-preserving learning for condition monitoring and diagnostics tasks in WT fleets. Section 3 presents two case studies of federated learning of normal behavior models in bearing temperatures and active power. We report and discuss our results in Section 4. Section 5 summarizes the conclusions from our study.

2. Federated Learning of Wind Turbine Conditions

2.1. Federated Learning

In conventional machine learning, all data on which a model is trained need to be available and accessible in a central system. If the data belong to different owners, such a centralized setting requires that the data owners give up their data privacy by sharing their data with others. In contrast, federated learning is a machine learning approach that learns a task from the joint data of different data owners without disclosing the data or sacrificing their privacy. In a federated learning environment, multiple industrial systems (clients, in our case, wind turbines) train a machine learning model in a collaborative distributed manner such that each client’s training data remains on its local client system, thereby preserving the privacy of the training data [11,56]. With federated learning, the training data are distributed across multiple client systems and are not located in one central system, as is the case with conventional machine learning. The parameters of a collaboratively trained model are learned from the distributed data without exchanging the training data among the client systems or transmitting them to a central system. Only updates of the locally computed model parameters are shared with and aggregated by the central system. The model training is collaborative in the sense that each client contributes to the joint model training task by using its locally stored data for that task.

We adopt the FedAvg federated learning approach of McMahan et al. [11] in our study. For a formal definition, it is assumed that a fixed number of $J$ client WTs are participating in the federated learning process. Each client WT $j$ has a fixed dataset $D_j$ of size $n_j=\left|D_j\right|$. In our case study, this is the dataset from the SCADA system used for training a normal behavior model of the WT normal operation behavior. Each dataset $D_j$ is stored locally in the client WT and not accessible to other client WTs or the central system. The FedAvg training proceeds in iteration rounds, at the start of which a central server transmits the initial model parameters of the current round to the $J$ client WTs (

Table 1. Steps in the training of a federated learning model based on McMahan et al. [11].

Federated model training process The central server selects the model architecture and initial weights. Then, it iterates the following steps:

The clients participating in the training are selected.

2. The central server transmits the starting weights of the current iteration to the participating clients.

3. Each client trains a local model with stochastic gradient descent on its local training data. The local training is performed by all clients in parallel.

4. The updated weights are transmitted to the server which averages them. The resulting average weights are sent to the clients to serve as the new model weights and starting weights of the next iteration.

). Then, each client WT $j$ updates the received model parameters by training on its local dataset $D_j$, and then transmits the update to the central server. The server updates the parameters of the global model by aggregating the updates received from all client WTs by averaging. The objective of the iterative FedAvg training process is to arrive at model parameters $w$ that minimize the sum of prediction losses ${\mathcal{L}}_i$ from the $J$ client WTs on all data points $(x_i, y_i$) of their local datasets $D_j$,

$$\underset{w}{\min }\sum _{j=1}^J\sum _{i=1}^{n_j}\frac{n_j}{n}{\mathcal{L}}_i\left(x_i,y_i,w\right)$$

(1)

In our case study, the model parameters $w$ will be weights of a feed-forward neural network. We compute the prediction losses ${\mathcal{L}}_i$ in terms of the mean squared errors. In each algorithm round $t$, the update step involves that the $J$ client WTs perform local weight updates in parallel, so each client WT performs a gradient descent step on its local data,

$$w_{t+1}^j=w_t^j-\eta \sum _{i=1}^{n_j}\frac{n_j}{n}\nabla {\mathcal{L}}_i\left(x_i,y_i,w_t^j\right)\hspace{1em}\forall j=1,\cdots ,J$$

(2)

wherein $\eta$ is the learning rate and $\nabla {\mathcal{L}}_i(x_i, y_i, w_t^j), i=1,\cdots ,n_j$ denotes the gradients on $D_j$ of client WT $j$ with regard to the model weights $w_t$. The central server then aggregates the received weights and returns an updated model state $w_{t+1}={\sum }_{j=1}^J\frac{n_j}{n}{w}_{t+1}^j$ to the client WTs, which ends the current training round. The training steps are repeated until a predefined stopping criterion is satisfied. The overall training process is summarized in Table 1.

In addition to preserving data privacy, further advantages of federated learning result from the fact that it does not require all client data to be stored in a central location. This can be highly beneficial when applied to complex remotely monitored power infrastructure, such as wind farms. Modern WTs are equipped with hundreds of sensors that can collect hundreds of gigabytes of data every day [57]. Transmitting and storing all those data in a central system (as would be common in conventional machine learning) is expensive and requires a high transmission bandwidth and data buffer. If the data are stored centrally, the data center managers of the central storage system are also responsible for protecting the data privacy and preventing unwanted third-party access, which entails an additional burden.

2.2. Federated Learning for Condition Monitoring

Condition monitoring of wind turbines is often based on normal behavior models in practice [35,40]. Normal behavior models (NBMs) can be used for applications in fault detection and diagnostics. We propose and demonstrate the federated learning of normal behavior models for such condition-monitoring tasks. In the following, we analyze how NBMs for condition monitoring can be trained collaboratively by a fleet of WTs in a manner that allows information sharing within the WT fleet without disclosing the data of any of the WTs. SCADA-based NBMs have been presented for fault detection tasks in [39,40,41]. Our case studies explore the application of the FedAvg method [11] for training accurate NBMs for fault detection applications in WTs that have few or no representative data. We focus on fault detection based on NBMs of drivetrain component temperatures and on active power production [58]. The drivetrain component temperatures exhibit more heterogeneous distributions across WTs. We investigate how federated learning can still be applied to extract accurate NBMs for condition monitoring despite significant inter-turbine differences in the distribution of the target variable, in our case, the gear-bearing temperature.

The temperature behavior of components and the active power form the basis of NBMs that are key for condition monitoring in WTs [38,59,60,61,62,63]. We demonstrate federated learning for NBMs of these applications.

Policies involved in the practical implementation of a federated learning process are beyond the scope of this study. There is certainly more than one setup and distribution of roles in the federated training process that can work in practice. For example, the federated learning process can be orchestrated by a regulatory entity that might define the process, the machine learning model structure, the aggregation, and distribute the software needed for the implementation. Federated learning can be organized in a centralized way, as presented here, but also in decentralized ways. Federated learning processes can be orchestrated by a central agency, such as a regulator. They may also be implemented and orchestrated by operators to enable data access across the fleet. Federated learning can, in principle, even be implemented by the manufacturer for customers who prefer not to give the manufacturer access to their turbines’ data. In the centralized learning process proposed in our study, the client WTs only need to be equipped with a client computer that can train neural networks on their local data, with computing capacity and storage similar to that of a laptop computer.

2.3. Customizing Federated Models to Individual WTs

A possible limitation of global federated learning models is that a single global model is trained for application in all client WTs of the fleet. Having a single non-customized model for all fleet members can limit the model’s performance in the fault detection task, especially in cases in which the client WTs’ SCADA and sensor datasets follow somewhat different statistical distributions in normal operation, requiring NBMs that are customized to each WT. Previous research investigating the effects of non-identically distributed data on the FedAvg algorithm has shown that data distribution differences can negatively impact the convergence and the performance of the global FedAvg model [64,65,66]. We investigate NBM customization in our case studies.

The NBM resulting from the federated training process (Table 1) is a global model trained on the data of all client WTs, so it is not customized to a specific client WT. We demonstrate the limitations of a single non-customized model in our case studies based on the example of WT gear-bearing temperatures and active power. Despite all WTs being the same model, each WT’s local dataset can present a somewhat different data distribution. The arising data heterogeneity can be described as domain shift [67,68,69], where the WTs’ local datasets form diverse domains with different feature distributions. For example, the temperature behavior of the gear bearing can differ across WTs because of differing thermal behaviors. A single global NBM without customization learned through the FedAvg training process can lead to poor generalizability across domains (i.e., WTs) and to situations where, for some client WTs, the global NBM outperforms a locally trained one, whereas, for other client WTs, the global NBM performs worse than a model trained only on their local data. A lack of generalizability can become especially critical when WTs that have little or no representative data are dependent on information contained in the data of other WTs with distinct domains. Shared global models may be inadequate under these circumstances.

Customized federated learning aims at alleviating this issue by customizing the global model to each client WT while still participating in the distributed learning process. Customization techniques that have been proposed for federated learning models range from customization layers in neural networks [70] to meta-learning with hypernetworks [71]. We refer to Kulkarni et al. [72] and Tan et al. [73] for an overview and taxonomy of customization techniques. In this study, we customize the global FedAvg model by means of local finetuning updates [73,74], which ensures that the participating client WTs can benefit from the federated learning process.

3. Case Studies: Federated Learning of Fault Detection Models

The goal of our case studies is to estimate WT-specific normal behavior models for WTs that lack representative observations and to perform the estimation in a collaborative privacy-preserving manner. A WT can suffer from a lack of representative training data for various reasons. A lack of representative data arises at the commissioning of a WT but can also occur after events that can affect the WT’s normal operation behavior, such as control software updates or hardware replacements.

In the first case study, normal behavior models of the active power are developed: Some of the WTs participating in the federated learning process have representative local training data covering all wind conditions, whereas the training data of other WTs are dominated by low wind speeds. The second case study focuses on federated learning of normal behavior models of bearing temperatures. Unlike in the first case study, the bearing temperatures exhibit heterogeneous distributions across the WTs participating in the federated training. We show that customizing the trained global model to individual WTs yields the highest fault detection accuracy under such conditions.

The case studies are performed with data from two wind farms. The two wind farms are in separate locations (with a distance of at least 900 km) with different geographical and environmental factors. In the following Section 3 and Section 4, we will describe, present, and discuss our case studies with regard to data from the first wind farm. We then apply the same case study design and validate our results on the dataset from the second wind farm, which is presented in Appendix A.4.

SCADA data from ten commercial onshore wind turbines are analyzed for the case studies. All ten WTs are of the same manufacturer and model. The WTs are a horizontal-axis variable-speed model with pitch control and share the same technical specifications (

Table 2. Technical specifications of the wind turbines employed in the case studies.

Parameter	Specification
Rotor diameter	112 m
Rated active power	3300 kW
Cut-in wind velocity	3 m/s
Cut-out wind velocity	25 m/s
Tower	Steel monopole
Control type	Pitch-controlled variable velocity
Gearbox	Two planetary stages, one helical stage

). The data were acquired from the WTs’ SCADA systems at a sampling rate of ten minutes over the course of 13 months. Each WT holds around 50,000 valid data points that contain wind speeds measured at the nacelle, the corresponding power generation, measured rotor speeds, and gear-bearing temperatures. The measurements are provided as average values over 10-min periods. All WTs are from the same wind farm, and we assume that no data sharing is allowed between the WTs. One randomly selected turbine out of the ten WTs is used only to define the network architecture with optimal hyperparameters, as explained in Appendix A.1. The NBMs of the remaining nine client WTs are estimated based on the SCADA data.

3.1. Federated Learning of Active Power Models

The first case study demonstrates the privacy-preserving collaborative learning of NBMs of a wind turbine’s active power generation. The trained NBMs enable the detection of underperformance faults in the monitored WTs. The normalized 10-min average wind speed serves as a regressor in the normal behavior model of the power generation. The wind speed was min-max normalized such that all normalized wind speeds are in the range of [0, 1]. In our case study setting, 5 WTs of 9 are subjected to a SCADA data scarcity. These WTs lack representative observations from periods of high wind speeds, as their training sets consist of only low to average wind speeds. Such a situation might happen in practice when data is predominantly collected during a period lacking high wind speeds, as it is possible, for instance, in central Europe, for a duration of up to months [75]. With this scenario, we demonstrate one of the many possibilities in which there exists a lack of representative SCADA data for training normal behavior models.

A lack of representative SCADA data from a particular WT means that accurate NBMs can hardly be estimated for that WT with conventional machine-learning approaches. It may take up to several months of SCADA data collection until a sufficiently representative dataset has been collected for training a new NBM from the WT’s own SCADA data. Active power monitoring and detection of underperformance faults are hardly possible during this time period. We demonstrate that collaborative learning of the nine WTs can mitigate this lack of training data and allow learning accurate power curve NBMs in a privacy-preserving manner.

For each WT, we set aside the last 30% of its SCADA data, i.e., the data gathered in months ~9–13 of the 13-month data collection period, as that turbine’s test set. Further, we assign nine randomly selected WTs as “client” turbines. The remaining WT is treated as a public turbine in the sense that its SCADA data will serve us for the model selection. The remaining 70% of the data of each client WT are split into a training set and a validation set in a manner that represents the data-scarce conditions as discussed above: We define the training set of each of the five WTs to be composed of the 10-min average wind speed and power generation values of the four weeks with the lowest average wind speed out of the considered 9-months measurement period. Thus, the training sets of the five WTs are characterized by low and moderate wind speed conditions. All other time periods form the validation set of that respective client WT. The training and validation set of the remaining four WTs comprises all wind resources, including low, moderate and high wind speeds. The last 30% of the training set data form the validation set for these clients. An illustration of the training, validation and test datasets is given in Figure 1 for one of the five data-scarce WTs and for one of the four WTs with representative training data. The accuracy of the power curves of the five WTs is limited due to the lack of observations of high wind speeds in their local training data.

Note that the data from the four WTs with representative training data are inaccessible to the data-scarce wind turbines. So, it is not possible to derive and transfer a power curve from any of those four WTs to any data-scarce WT because the data are local and, thus, unavailable to standard (non-federated) learning approaches.

3.2. Federated Learning of Bearing Temperature Models

Our second case study demonstrates the federated learning of NBMs of bearing temperatures for fault detection applications. Unusually high component temperatures can be caused, for example, by excessive friction or undesired electrical discharges, so excessive temperatures are key SCADA indicator variables of developing operation faults. The normal operation behavior of gear-bearing temperatures is modeled with normalized 10-min rotor speeds and power generation as regressor inputs. Again, five of the nine WTs are affected by a lack of representative SCADA data. Specifically, only one month of local training data is used to train the NBMs of these WTs. Such scarcity conditions regularly arise in newly installed WTs and after major software or hardware updates. As in the first case study, the last four months (30% of the SCADA data) serve as the WT’s test set. Nine randomly selected WTs are assigned as client WTs, whereas the remaining WT is used for the model selection. The remaining 70% of the SCADA data of each client WT are split into a training set and a validation set in accordance with the data scarcity conditions: The training set of each of the five WTs is defined to be composed of the 10-min average gear bearing temperatures, rotor speeds, and active power generation values of one month, i.e., four randomly chosen consecutive weeks. All other time periods form the validation set of the respective WT. The datasets of the remaining four WTs comprise a longer, more representative data collection, where the last 30% of the training set data form the validation set for these clients. Figure 2 illustrates the training, validation and test datasets for one of the five data-scarce WTs and for one of the four WTs with representative training data.

3.3. Heterogeneously Distributed Target Variables

Deviations in the data distributions across the local datasets of the participating client WTs can negatively affect the FedAvg learning process, as discussed in Section 2. Our case studies exhibit different degrees of distribution shifts in the target variables, enabling us to investigate the effects of deviating distributions of the monitored variables. Figure 3 shows the distributions of the active power generation and the gear-bearing temperatures across the nine WTs participating in the federated training. The distributions of active power (i.e., the target variable of the first case study) display only minimal differences across the client WTs. Consequently, one expects that a global federated model should be able to capture information that can be generalizable across WTs. We assess how this global knowledge can be shared and utilized by WTs with scarce training datasets in the case studies. We also evaluate how the loss of WT-specific information in the global model affects WTs with representative training datasets and the utility of customized models under these conditions.

The distributions of the gear-bearing temperatures show distinct differences across all client WTs. A globally shared model may have difficulties capturing global information that is generalizable across WTs. Customization to individual WTs may improve the model performance in the case of non-identically distributed datasets. We assess the effect of the observed distribution shifts on the performance of the global model in the second case study and whether collaborative condition information sharing across the fleet is still possible and beneficial for the participating WTs under these conditions.

4. Results

4.1. Federated Learning Strategies and Model Architecture

The scarcity of training data is addressed by privacy-preserving information sharing between all WTs active in the federated training: The local data of each client WT contribute to training the global NBM and fine-tuning it to the respective client WT. Yet, the local data remain stored in the respective client WT without exposing them to other client WTs or the central server in the federated training process (Table 1). We compare three privacy-preserving learning strategies (Figure 4):

A.

Conventional machine learning of NBMs using only the local data of each WT,

B.

Federated learning of a single global NBM for all WTs,

C.

Customized federated learning of WT-specific NBMs.

4.1.1. Conventional Machine Learning

We evaluate the training of a NBM in a conventional non-distributed machine learning environment. Each client WT individually learns an NBM based on its own past operation data and without any access to data from other WTs of the fleet. This constitutes the default situation in practice. We typically lack access to data from other fleet members because they have other owners, and no data sharing is in place.

4.1.2. Federated Learning of a Single Global Model

Our second training strategy for the NBM is a federated learning environment. In this setting, a central server communicates with the client WTs in a privacy-preserving manner. We implement the federated averaging approach of McMahan et al. [11], as outlined in Table 1. First, the server broadcasts the model architecture, determined with the model search over the server-accessible public WT, and further information, such as the optimizer, loss, and metrics, to the client WTs in the initialization step. The iterative update step consists of the client WTs first updating their models in parallel—which we implemented as running three epochs over their private local training sets—and then sending their model weights back to the server. Next, the server averages the collected client weights and broadcasts the averaged model weights to the client WTs. The averaged model weights represent the global FedAvg model. An additional sidestep involves that all clients evaluate the updated global model on their validation set and send their validation losses to the server. We repeat the update step until the average validation loss of the clients has not improved within five repetitions, representing 15 local epochs by each client. The global federated learning model is then evaluated by calculating the root mean squared error (RMSE) on the test set of each client WT.

4.1.3. Customized Federated Learning of Turbine-Specific Models

A possible disadvantage of the presented federated learning approach (B) is that it results in a single global model that is not customized to a specific client WT. The individual client WTs may exhibit somewhat different data distribution characteristics depending, for example, on their sites, maintenance history, or local ambient conditions. The feature distributions of the monitored target variable may differ significantly across the fleet (Figure 3). Such differences are not represented by the global NBM, which may result in performance losses of the model for some client WTs. Some turbine operators might be incentivized to opt out of the federated learning process if they find that a local NBM trained only on their local data with conventional machine learning (A) outperforms the global NBM (B). Training WT-specific NBMs can make it attractive for all client WTs to join the training, so we customize the MLP that represents the global NBM to specific client WTs. After training a single global MLP model based on FedAvg (B), we achieve the customization by having each client WT finetune a subset of the trained layers of the global MLP on its local dataset (Figure 5 and Appendix A.3). This turbine-specific finetuning resembles transfer learning methods in which neural network layers of a previously trained model are finetuned on a separate dataset [72,73,74,76,77]. Based on the validation set losses, each client WT optimizes the number of layers to finetune its customized MLP. The weights of the other layers remain fixed with the weights of the global federated learning model. The resulting model performances are presented in Appendix A.3. The customized model with the lowest RMSE on the client WT’s validation set was finally evaluated on each test set.

4.2. Federated Learning of Active Power Models

In case study 1, a feedforward neural network is trained as a NBM for the power generation. The inputs to the model are the normalized SCADA wind speeds. The model outputs a prediction of the active power generation in MW. In each case study, all client WTs and federated learning strategies make use of the same feedforward multilayer perceptron (MLP) model architecture to ensure fair comparisons among experiments. The respective MLP architecture will be determined by applying a random search model selection algorithm on the SCADA dataset of the public turbine. For the first case study, the resulting model architecture is summarized in

Table 3. The model architecture of the Active Power NBM was used in all experiments of the first case study.

Model Architecture (First Case Study)

Input layer: Normalized wind speed value Layer 1: Fully connected, 12 units, exponential linear unit activation function Layer 2: Fully connected, 8 units, exponential linear unit activation function Output layer: Fully connected, 1 unit, Rectified Linear Unit activation function [78] Output: Predicted Power Value [MW] Number of parameters: 137 Loss: Mean Squared Error Optimizer: Stochastic Gradient Descent (learning rate = 0.013, batch size = 32)

. The search algorithm and model architecture selection are outlined in Appendix A.1. Each client WT minimizes the mean squared error loss over the training set by applying stochastic gradient descent (SGD) in the conventional machine learning according to strategy A. Training is stopped once the client WT’s validation set loss has not improved within 15 epochs. The model performance is finally evaluated as the RMSE on the test set over the client WT. The results are summarized in Figure 6. Detailed results for each WT are presented in Appendix A.2.

Model performance. The performances of the three privacy-preserving NBM training strategies (A–C) are compared with regard to the accuracy of the resulting active power NBMs on the test sets of each of the nine client WTs and with regard to the model training time. In conventional machine learning (strategy A), we find a significant difference in model performance depending on the type of the training dataset. The client WTs trained on the four weeks with the lowest wind speed average of the considered 13-month period show a significantly higher error (mean: 0.231) than those WTs with training datasets of all wind speed conditions (mean: 0.104), as shown in Figure 6. Due to the scarcity of high-wind speed observations, conventional machine learning on the local client’s data cannot train a sufficiently accurate power curve NBM. An example of such a client WT dataset is shown in Figure 7 (1). The corresponding model trained with gradient descent on only the local training data (strategy A) does not capture the power curve behavior correctly at higher wind speeds (Figure 7 (3)). For the four WT clients with representative wind speed data, the power curve can be fit accurately even with conventional machine learning with only the local training data.

The results of the global federated learning model (strategy B) show a contrast between the client WTs with scarce high wind speed observations and the client WTs with representative wind speeds. For the WTs with scarce high wind speed observations, the RMSE of the active power NBM is significantly reduced by the global federated learning (mean: 0.125) compared to the conventional machine learning setting. By receiving shared model parameters from all client WTs through the server aggregation step, the client WTs with scarce high wind speed observations are now able to also model the upper wind speed ranges by means of the shared global model. Therefore, these client WTs benefit from the federated learning process through a significant improvement in model performance. Figure 7 (4) shows the accordingly improved power curve of one of the five client WTs with few or no high wind speed data with realistic behavior in the upper ranges despite not having any reference data points available in its own local training set.

Conversely, the model performance has slightly but noticeably decreased for all but one of the four client WTs with representative wind speed observations (mean: 0.113) by the global federated learning as compared to the conventional machine learning setting. The averaging step of the global federated learning leads to a loss of individual characteristics contained in the local models of those client WTs. Therefore, as these clients were already locally capable of fitting a model tailored to their individual turbine characteristics, also in the upper wind speed ranges, the averaged global federated learning model leads to a performance loss by incorporating individual information from other turbines.

Such performance losses could discourage operators of client WTs with sufficiently representative training data from joining the federated learning process. These client WTs should not drop out of the federated learning, though, because they are essential to the performance increase of the client WTs with scarce data in this example. Indeed, our results show that a customized federated learning implementation can counteract this issue. The local finetuning of the global federated learning model manages to revert the impact of the global averaging and re-introduces individual characteristics into the models. Thus, the active power NBMs include both global information as well as customized adjustments. Figure 7 (5) shows that the active power NBM from the customized federated learning model is very similar to but somewhat deviates from the global federated learning model to correct for local dataset characteristics.

Comparing the average performances of the three learning strategies (A–C), the customized federated learning approach (C) accomplished the lowest RMSEs for the clients with scarce high wind speed observations (mean: 0.117) and achieved the same performance as the conventional machine learning strategy (A, mean: 0.104) for clients with representative wind speed observations. Our results suggest that a customization method should be applied for possible performance improvements of the trained NBMs and as an incentive for all client WTs to join the federated training process.

Compared to conventional machine learning (A), a distributed learning process, such as federated learning, requires additional computational costs due to the communication between server and clients, overhead operations, and slower model convergence. Figure 6 shows the measured computational time taken to accomplish the training process for the three learning strategies. All client WTs finish training within less than three minutes in a conventional machine learning setting (A). With a global federated learning strategy (B), the clients require more than 9 min for the learning to be accomplished. Given this increase, the training time needs to be investigated when considering federated learning applications for more complex models and for training with a larger number of client WTs. In customized federated learning (C), the computational costs are dominated by the global learning step, as the customization step requires a finished federated learning process. The added time taken by the actual customization step, i.e., the local finetuning, becomes negligible (on average +10.4 s) in comparison. Thus, a local finetuning step is a very cost-efficient improvement.

4.3. Federated Learning of Bearing Temperature Models

The privacy-preserving learning strategies A–C are also investigated in a second case study. A feedforward neural network is trained to model the normal behavior of the gear-bearing temperature using SCADA data. To ensure fair comparisons between the strategies, all client WTs and federated learning strategies use the same feedforward MLP model architecture, described in

Table 4. The model architecture of the Bearing Temperature NBM used in all experiments of case study 2 was determined through a model search (Appendix A.1).

Model Architecture (Second Case Study)

Input layer: Normalized rotor speed and power Layer 1: Fully connected, 8 units, exponential linear unit activation function Layer 2: Fully connected, 16 units, exponential linear unit activation function Output layer: Fully connected, 1 unit, linear activation Output: Predicted Gear Bearing Temperature [°C] Number of parameters: 185 Loss: Mean Squared Error Optimizer: Stochastic Gradient Descent (learning rate = 0.00035, batch size = 32)

.

Model performance. The accuracies of the NBMs trained with non-collaborative strategy A are shown in Figure 8. WTs with scarce datasets (mean RMSE: 4.29) have a higher average RMSE than WTs with representative datasets (mean: 3.93) with this strategy. The models trained on scarce datasets with strategy A are not capable of fully capturing the temperature behavior. An example of this is shown in Figure 9, in which the trained model is unable to adequately estimate temperatures in underrepresented ranges (very low and high temperatures, as shown in Figure 9 (1)), which leads to larger errors in the lowest and highest observed temperature values on the unseen test dataset (Figure 9 (2)).

Comparing the performance of global federated learning (B) to conventional machine learning (A) for the five client WTs with scarce datasets, the global model leads to performance increases in only two client WTs but raises the prediction errors of the NBMs in three other WTs. The global models result in worse NBM performance even though the three WTs lack representative data and receive shared model parameters. This result suggests that the substantially differing bearing temperature behavior across clients strongly affects the generalizability of the global model, such that one global model trying to combine all individual characteristics cannot always offer a satisfactory fit. Therefore, despite receiving information about temperature ranges not represented in their training set, these values do not necessarily reflect the actual bearing temperature behavior of that WT. An example is shown in Figure 9 (3), where the global model introduces a strong overestimation of the lower bearing temperatures.

For the four clients with a fully representative training set, the global federated learning model leads to a noticeable increase in the RMSE in all cases. The global model incorporates information from all turbines, leading to a loss of individual characteristics within the model and, thus, to a loss in performance, as already observed in case study 1.

A customized federated learning strategy can encourage operators of client WTs without data scarcity to participate in the federated learning process because a customized strategy can revert potential performance degradation introduced by the global model. Both case studies show that the customization step is a necessity to encourage clients without data scarcity to join the federated learning process. For clients with scarce datasets, the customized federated learning strategy achieves the best performance across all strategies. The local finetuning enables the customized models to retain and transfer usable knowledge from the global model (for data not represented in the scarce dataset) and additionally incorporate individual characteristics from the private local dataset. An example is shown in Figure 9 (4), where the bearing temperature estimates from the customized model are now improved in the unseen low and high-temperature ranges. Our results suggest that a customized federated learning strategy can enable fleet-wide learning of condition information even in the presence of a significant domain shift.

The computational time taken to train the NBMs following the three learning strategies (Figure 8) confirms the results of case study 1. We observe a strong increase in training time of the global federated learning model compared to conventional machine learning. Training a model according to the conventional machine learning strategy takes, on average, 43 s, while the federated learning process requires more than 10 min. In contrast, the increase in time for the local finetuning of the global model (customization part of strategy C) remains negligible as it only requires an additional 12.9 s of training on average. The results of case study 2 reinforce that a disadvantage of the federated learning process is its additional computational costs and that customized federated learning (strategy C) is a very time-efficient model improvement strategy. Detailed results for all WTs are shown in Appendix A.2.

All experiments were run on an Intel Xeon CPU @ 2.20 GHz with implementations using TensorFlow v2.8.3, Keras v2.8, and the TensorFlow-federated v.0.20.0 framework [79,80].

4.4. Second Wind Farm

We further validate our findings by replicating our case studies using data from the second wind farm. The wind turbines in the two farms belong to different fleets. They have different manufacturers, different rated powers, and major constructional differences. Details and results are provided in Appendix A.4. The transfer across different fleets is not in the scope of our study.

5. Conclusions

A wealth of data is being constantly collected by manufacturers from their wind turbine fleets. Stakeholders interested in those data can include operators, owners, manufacturers, third-party companies, regulators, and researchers. There are various reasons why different stakeholders want access to information contained in a fleet’s operation data. Benefits of making the information accessible include technological progress, for example, through new and improved data-driven applications and economic advantages resulting from increased transparency and competition. For example, improved machine learning models can be trained based on a fleet’s data to provide better decision support to wind farm operators. This may involve improved predictions of failure events and estimations of the remaining useful lifetime of critical parts.

Conventional machine learning on local wind turbine datasets is often applied in practice, but it cannot exploit the information contained in the operation data of distributed wind turbine fleets. Conventional machine learning cannot overcome the lack of access to fleet-wide data because it is incompatible with data privacy needs.

We have demonstrated a distributed machine learning approach that enables fleet-wide learning on locally stored data of other participants in the federated learning process without sacrificing the privacy of those data. We have investigated the potential of federated learning in case studies in which a subset of wind turbines was affected by a lack of representative data in their training sets. The case studies involve the collaborative learning of normal behavior models of bearing temperatures and power curves for condition monitoring and fault detection applications.

The results of our case studies suggest that a conventional machine learning strategy fails to adequately train normal behavior models for fault detection when representative training data are lacking. The presented privacy-preserving federated learning strategy significantly improves the accuracy of normal behavior models for wind turbines lacking representative training data, as they can benefit from the training on the data of other turbines.

However, when the distributions of the monitored variable differ strongly across the fleet, a single global model shared by all turbines can deteriorate the performance of the normal behavior models, compared to conventional machine learning, even if representative training data are lacking. We have presented a customized federated learning strategy to address this challenge of heterogeneously distributed target variables. By customizing the global model to each client WT by local finetuning of neural network layers, we first successfully revert the performance losses of the global model so that no turbine suffers from a performance loss by participating in the federated learning process. Customized federated learning yields the best model performance across all compared learning strategies. Our case studies suggest that fleet-wide learning and sharing of condition information can be achieved even where the monitored target variable is distributed heterogeneously across the fleet. Client WTs with scarce training sets were able to extract and customize knowledge from other fleet members. The federated learning process increased the average model training time by factors of seven and 14 in the presented case studies, which can be attributed to more comprehensive communication and overhead operations and slower model convergence in the federated learning process.

Our proposed federated learning method proposes a solution to a major problem in energy and power system fleets: The lack of data sharing, which “is hindering technical progress (…) in the renewable-energy industry” [8]. Future research directions may involve investigating further applications of federated learning in renewable energy domains, various customization strategies, and different characteristics and effects of heterogeneously distributed target variables. It should also investigate how model training times scale with fleet size for large fleets and possibly more complex models, such as multi-target normal behavior models.