Fault-Tolerant Cooperative Control of Large-Scale Wind Farms and Wind Farm Clusters

: Large-scale wind farms and wind farm clusters with many installed wind turbines are increasingly built around the world, and especially in offshore regions. The reliability and availability of these assets are critically important for cost-effective wind power generation. This requires effective solutions for online fault detection, diagnosis and fault accommodation to improve the overall reliability and availability of wind turbines and entire wind farms. To meet this requirement, this paper proposes a novel active fault-tolerant cooperative control (FTCC) scheme for large-scale wind farms and wind farm clusters (WFCs). The proposed scheme is based on a signal correction method at wind turbine level that is augmented with two innovative “control reallocation” mechanisms at wind farm and network operator levels. Applied to a WFC, this scheme detects, identiﬁes and accommodates the effects of both mild and severe power-loss faults in wind turbines. Various simulation studies on an advanced WFC benchmark indicate the high efﬁciency and effectiveness of the proposed solutions.


Introduction
Wind power generation is booming and will continue to do so for the foreseeable future. Increasingly offshore wind turbines are installed in groups called wind farms, and several wind farms in direct vicinity constitute a wind farm cluster (WFC).
Cost-effective large-scale wind power generation by an offshore WFC is particularly challenging due to the increased complexity of offshore assets, limited accessibility, and harsher environmental loadings, all of which result in soaring rates of faults and failures. Given that, effective solutions for online fault detection and diagnosis (FDD) and faulttolerant control (FTC) are necessary to enhance reliability and availability of wind turbines and wind farms. According to the technical literature, such solutions can be designed, depending on the type and nature of faults, either at an individual wind turbine level or at an entire wind farm (or even WFC) level.
Reviewing the literature, most of the published papers have studied the applications of FDD and FTC at the wind turbine level (e.g., see [1][2][3][4][5][6][7][8][9]). Conversely, the number of published papers on FDD and FTC at the wind farm level is relatively small. For instance, various data-driven methods are presented in [10][11][12] to develop suitable models for fault detection and prediction in wind farms. In [13,14], different machine learning techniques are used to model and monitor wind farm power curves. Work in [15] employs interval parameter-varying parity equations to achieve fault diagnosis in a wind farm. Furthermore, authors in [16] present an active FTC scheme which uses a model-based FDD system. In [17], the problem of active power control in a wind farm is converted to the rotor speed tracking problem, and an adaptive FTC scheme is presented to track the power signal of each wind turbine with an acceptable transient performance and robustness to the actuator faults. The particle swarm optimization (PSO) approach is used in [18] for optimal active power dispatch in a wind farm during generator cooling system faults. A PSO-based active power dispatch method is introduced in [19] to accommodate the fault effects (i.e., inter-turn short circuit faults in wind turbines generators) at the farm level.
In the case of decreased power production because of blade erosion/debris build-up, different FTC solutions under a cooperative framework, called "fault-tolerant cooperative control (FTCC)" schemes, are presented in [20,21]. Having no dependence on the information of wind speed or direction, the said schemes can simultaneously handle multiple faults in wind turbines of a wind farm with any arbitrary layouts. Badihi et al. in [22] extends the authors' works in [20,21] by developing another FTCC scheme which relies on an effective "model reference adaptive proportional-integral control reconfiguration" algorithm that is integrated with a "control reallocation" mechanism. Compared with the solutions in [20,21], the more recently developed scheme in [22] can handle both mild and severe power-loss faults because of different levels of icing or debris build-up on wind turbine rotor blades. Table 1 presents a categorized list of the references on FDD, FTC, and FTCC of wind farms with their corresponding design approaches.

Design Approaches Objectives References
Various data-mining-based algorithms Prediction of wind turbine faults [10,11] Principal component analysis Reducing the dataset for turbine FDD [12] Various machine learning methods Modeling and estimation of a wind farm power curve [13,14] Interval nonlinear parameter-varying Fault diagnosis of a wind farm [15] Fuzzy model-based FDD with signal correction Active FTC for a wind farm against decreased power generation [16] Adaptive FTC using barrier Lyapunov function Active power control during actuator fault [17] Particle swarm optimization Accommodation of the generator fault effects at farm level [18,19] FTCC using fuzzy logic at farm level Accommodation of mild power-loss faults caused by turbine blade erosion and debris build-up on the blades [20,21] FTCC using adaptive PI control at turbine level and control reallocation at farm level Accommodation of mild and severe power loss faults caused by turbine blade erosion and debris build-up on the blades [22] It should be noted that the effective implementation of the presented scheme in [22] for a large-scale wind farm or WFC, where the number of wind turbines becomes quite large, turns to be complex or computationally expensive due to the increased number of modules required in the scheme (i.e., each module monitors the power consistency between two different turbines in the wind farm). To overcome these issues when dealing with very large-scale wind farms or WFCs, the present paper proposes a novel clustering approach to enable a computationally efficient multilayer FTCC scheme which provides effective fault identification and accommodation performance with simplified computer programming. For this purpose, it is reasonable to suppose that a very large-scale wind farm or WFC can be divided into a group of smaller scale wind farms, each including a smaller number of wind turbines. In other words, a very large-scale wind farm can be studied as a WFC consisting of several smaller wind farms that are directly/indirectly connected to a node of transmission network. In the presented scheme, a distributed FDD system is applied on each smaller wind farm in the cluster that reduces the total number of computational modules. Accordingly, a multilayer FTCC scheme is designed and implemented at three control levels (i.e., turbine, farm, and cluster levels) to accommodate mild and severe power-loss faults in any wind turbine. The mild power-loss faults are accommodated at a primary control level (turbine level) using a signal correction approach, while the secondary and tertiary control levels (farm and cluster levels, respectively) are used for accommodating severe power-loss faults. In this regard, control reallocation mechanisms for each smaller wind farm, and also the network operator, are introduced. Various simulation studies on an advanced WFC benchmark, including 100 wind turbines, reveals the notable effectiveness of the proposed solutions in the presence of realistic fault scenarios, as well as wind turbulences and measurement noises.
The remainder of this paper is ordered as follow. The WFC benchmark as well as the considered faults are discussed in Section 2. Section 3 presents the proposed FTCC scheme to tackle the faults in Section 2. Section 4 illustrates and discusses the obtained simulation results. Finally, Section 5 draws some important conclusions and proposes future work.

WFC Benchmark and Power-Loss Faults
Using the "SimWindFarm" toolbox in [23], an advanced dynamic simulation model was developed to represent a WFC benchmark. The original toolbox created a nonlinear wind farm model that allows studying wind farm controls and diagnostics in the presence of realistic operating conditions. This paper considers WFC Q that includes N wind farms each of which referred to as Q i consisting of N i wind turbines. In other words, WFC Q with N wind farms is represented as Q = {Q i | i ∈ N, i ≤ N} and wind farm Q i with N i turbines is represented as Q i = q j | j ∈ N, j ≤ N i (Note: N is the set of natural numbers, and q j is the jth turbine in wind farm Q i ). The considered layout of a WFC with 100 wind turbines (i.e., ten wind farms, each having ten 5-MW wind turbines), and the benchmark block diagram under consideration are illustrated in Figures 1 and 2, respectively.

Benchmark Components
As shown in Figure 2, there are four major components in the benchmark model, namely the "wind farm controller", "wind turbines" and "wind field" for each wind farm in WFC, and the "network operator". The following paragraphs provide more information about these components: Network Operator: This component relates to the tertiary control level and can work in different operational modes such as "absolute", "delta", or "frequency regulation" to specify the "total demanded active power" P D of the WFC Q (i.e., P D = ∑ N i=1 P D,Q i where P D,Q i is the demanded power of wind farm Q i in the cluster) required for reliable connection of the WFC to the power grid. Among the above-mentioned operational modes, this paper employs the "frequency regulation mode", using which the measured grid frequency f mes is considered as a feedback signal to enable online "active power control (APC)". The aim of APC is to maintain the essential balance between generated power in WFC and demanded power by grid load, which thus regulates the frequency to its reference value f r , despite a changing load. The baseline network operator in frequency regulation mode includes a "dead-band proportional gain control" that uses the frequency error f e in Equation (1) to compute P D at the time-step (k + 1) from Equation (2). Such a simple control system regulates the frequency to any specified reference value; for example, 50 Hz that is commonly used in large areas of the world [21].
In Equation (2), the constants c and d respectively denote the "control-band" and the "dead-band" values (c > d) defined by user. In addition, P min and P max denote the prescribed minimum and maximum limits for the total power produced by the WFC, respectively. Finally, the network operator distributes the WFC's P D among wind farms in the cluster via the proportional algorithm in Equation (3).
where P D,Q i denotes the demanded power from wind farm Q i in the WFC Q, and P A,Q i denotes the estimated available power in the same wind farm. P A is the estimated total available power of the WFC. It is worth noting that the simple control law in Equation (3) is designed for a WFC in which all wind farms are healthy. Otherwise, the network operator's fault-tolerant control solutions should be considered, as explored in Section 3.4. Wind Farm Controller: Each wind farm in the cluster has a wind farm controller that corresponds to the secondary control level. The ith wind farm controller is an interface between the already-explained network operator and wind turbines in the wind farm Q i . At each k (time-step), it provides the operator with an estimate of the total available power P A,Q i in the wind farm Q i . In addition, it distributes the wind farm total demanded power P D,Q i (received from the network operator) among wind turbines based on the proportional algorithm below: where N i is the number of wind turbines in the wind farm Q i , and P d,q j (k + 1) denotes the demanded power from wind turbine q j in the considered wind farm. Moreover, P a,q j (k) denotes the estimated available power in turbine q j that is obtained by [23]: where ρ is the air density, R is the rotor radius, C Pmax is the maximum power coefficient, and V mes,q j (k) is the measured nacelle wind speed of wind turbine q j in the wind farm Q i . Similar to that mentioned for Equation (3), note that the control algorithm in Equation (4) is designed for a healthy wind farm where wind turbines are all healthy with no faults or failures. Otherwise, fault-tolerant wind farm control solutions should be considered as discussed in Section 3.3.
Wind Turbines: This component simulates the real-life dynamics of N i wind turbines installed in wind farm Q i using nonlinear models of an offshore 5-MW turbine that is already available by the U.S. National Renewable Energy Laboratory (NREL) [24]. For outputs, the wind turbines component generates the active power along with a set of measurements for use by the wind farm controller. Using a wind turbine control system, each individual wind turbine q j inside this component acts upon the power demand P d,q j in Equation (4). The wind turbine control system comprises a blade-pitch controller and a generator torque controller (as primary control level) to compute proper reference values for blade-pitch angle β r,q j and generator torque τ r,q j , respectively. It is worth noting that the generator torque controller is programed to be active during both below and above rated wind speeds.
Wind Field: Using an appropriate ambient field model along with a wake model, each wind farm has a wind field model that simulates the aerodynamic interactions between wind turbines in the farm.

Power-Loss Faults
Power-loss faults in wind turbines are highly probable and can be caused by different reasons such as debris build-up or icing on rotor blades [25]. As discussed in [26,27], even mild levels of this type of fault can lead to a substantial off-normal turbine operation and loading and reduced power generation by the wind turbine. If not handled in a timely and effective way, serious problems may arise with optimal tracking of electric grid loads and maintaining a safe frequency at the point of interconnection to the electric grid. Addressed in Section 3, this type of fault can be effectively tackled by implementing a novel FTCC scheme that identifies and accommodates the effects of mild to severe power-loss faults. For wind turbines, wind farms, and the entire WFC, this scheme improves reliability and availability while lowering operation and maintenance costs.

Active FTCC Design
This section presents a novel active FTCC scheme based on a signal correction approach at wind turbine (or primary control) level that is augmented with two innovative control reallocation mechanisms at the wind farm (or secondary control) and WFC (or tertiary control) levels. The differences between active and passive fault-tolerant methods are explained in several papers [28][29][30]. In summary, compared with passive approaches, active strategies rely on diagnosis information from an FDD system to deal with a larger variety of faults. As shown in Figure 3, this cooperative framework covers the entire WFC, regardless of wind directions and layouts. The architecture of the proposed FTCC scheme with communication links between different components in the WFC is illustrated in Figure 4.
Similar to the works in [20][21][22], the basic idea for designing an FDD system is to monitor and analyze the consistency of the generated powers from wind turbines altogether in real time. However, in contrast to the above-cited works, the proposed scheme in this paper acts against both mild and severe power-loss faults while using distributed FDD systems, which operate in parallel to detect and identify power-loss faults in each wind farm in a cluster. Indeed, the FDD process is separately carried out at each small-scale wind farm inside a WFC so that each wind farm includes its own FDD system working in parallel with the rest of FDD systems in other wind farms and is responsible for detection and identification of mild and severe faults in the relevant wind farm.
As shown in Figures 3 and 4, the active FTCC scheme includes N FDD systems, N automatic signal correction (ASC) units, N fault-tolerant controllers, and one faulttolerant network operator, all of which work together to identify the power-loss faults and accommodate the adverse impacts of faults at turbine level (for mild faults), and farm and WFC levels (for severe faults). Indeed, wind turbines in WFC are classified into "healthy", "mildly faulty" and "severely faulty" ones. About the two latter cases, mildly faulty wind turbines continue working by signal corrections using ASC units at the wind turbine level. However, severely faulty wind turbines need reallocation of control signals at the farm level using the farm controller, so that healthy and mildly faulty turbines (with extra or unused available power) produce more power. In case of a severe power loss fault in each wind farm, if the extra available power in the farm's healthier turbines (i.e., healthy, and mildly faulty turbines) does not adequately compensate the missing power of the severely impaired wind turbines, control reallocation is performed at the WFC level using faulttolerant network operator. This reallocates control commands among the farms to require healthier ones (with an extra available power) to produce more power to compensate for the farm's diminished power loss.

Distributed FDD Systems
Before providing more information about the design of the distributed FDD system, it should be pointed out that handling the considered power loss faults at the turbine level is a difficult task. To further explain, a lower generated power can occur because of debris build-up on the turbine's blades, or simply that the real wind speed is less than the measured/estimated wind speed. It should be pointed out that at the farm level, it is possible to monitor and compare the performance and other dynamic features of different turbines in the farm (as a part of the cluster) [20]. In the present paper, the FDD approach introduced in [20] is extended to a larger level at the WFC level. More precisely, the proposed FDD approach in [20] is applied on each wind farm in the WFC. As discussed in this section, the wind farm clustering approach results in a more efficient programming and substantially lower computational cost, all of which are important when implementing an active FTCC scheme in large-scale wind farms or WFCs.
Without loss of generality, the distributed FDD system in Figures 3 and 4 is used to provide real-time information about the true status of each turbine in the cluster. More precisely, the FDD systems deal with the considered power-loss faults at the WFC level by monitoring the consistency of produced power from all turbines in each farm in real time. Therefore, any inconsistencies in produced power are detected and identified to obtain reliable FDD information. Such a fault-monitoring and FDD scheme can be achieved by the model-based algorithm presented in [20]. As a matter of fact, real-time consistency monitoring of the powers from different wind turbines (in a farm or cluster) is not a straightforward procedure even in the case of adjacent wind turbines where wind properties might be almost similar. In fact, for different turbines in a farm, the power reference signals are not necessarily the same, and thus the turbines' generated power might be different even during fault-free or normal conditions. Given that, in order to correctly monitor the power consistency generated from any two arbitrary turbines, it is important to consider not only their generated power, but also their power reference signals.
An effective approach is to monitor the consistency of produced power online from any individual turbine and all other turbines in the farm. To further explain, consider wind farm Q i with N i turbines that are labeled as q 1 , q 2 , . . . , q N i . Thus, the simultaneous consistency monitoring of generated powers needs R i similar modules as shown in Equation (6). Table 2 shows the formation of required modules (each denoted by an M) for the farm Q i with N i wind turbines. Each module from this table monitors the consistency of the generated power from two different wind turbines in the farm. For instance, M α,β with α = β represents the module that monitors the consistency of the produced power from turbine q α and turbine q β in a wind farm. Considering a farm with ten wind turbines (N i = 10), 45 modules (R i = 45) are needed to monitor the entire wind farm. However, for a large-scale wind farm or WFC, for example, with 100 wind turbines, the FDD scheme in [20] requires 4950 modules (R i = 100(100−1) 2 = 4950) that would be computationally expensive to implement. To address such a problem, this study suggests a novel clustering approach which significantly reduces the number of modules involved in the FDD computations. Applying the clustering approach in this paper, a large-scale wind farm or WFC is divided into a group of smaller scale wind farms for which a significantly lower number of modules would be required. For instance, in case of the WFC with 100 wind turbines, this can be divided into a group of small-scale wind farms (e.g., 10 wind farms with equal number of 10 wind turbines in each wind farm) for which the required number of modules is significantly reduced from 4950 to 450 modules. That is, each of the 10 wind farms has an FDD system with 45 modules (R i = 10(10−1) 2 = 45) which makes a total of 450 modules, i.e., more than 10 times less than the required 4950 modules in [20]. Table 2. Required R i modules for a farm with N i wind turbines.
An FDD system's general structure is illustrated in Figure 5. As seen, it includes R i modules (already listed in Table 2) and a group of decision making (DM) blocks (DM 1 , DM 2 , . . . , DM N i ) that each relates to an individual wind turbine. Each DM block analyses the relevant module output (MO) signals to make decision about the status of its corresponding wind turbine in real-time. In Figure 5, Mes denotes a performance vector including measured variables as well as control commands (or references) in the wind turbines. The FDD system in each wind farm only requires the sets of reference and generated power values that are defined in Equations (7) and (8), respectively. Figure 5. FDD system i for wind farm Q i with N i wind turbines. Each module M (from R i modules) monitors the consistency of generated powers from any two specific wind turbines in the wind farm. The module outputs, denoted by MO with appropriate subscripts, are analyzed and final decisions are made by the relevant decision making (DM) processes. Note that, q Z with (Z ∈ N and 1 < Z < N i ) represents any wind turbine except q 1 and q N i .
Here, the reference powers denoted by P r,q j are computed according to [20] as: P r,q j (k) = τ r,q j (k)ω g,q j (k) (9) where τ r,q j and ω g,q j denote the reference generator torque and the angular speed of generator for wind turbine q j in the farm Q i , respectively. Each module in Figure 5 has its respective inputs and outputs. For instance, Figure 6 illustrates module M α,β with its appropriate inputs and outputs. This module with α = β monitors the consistency of generated powers from turbine q α and turbine q β in the farm Q i . In Figure 6, the module's inputs include the reference powers (P r,q α , P r,q β ) and the generated powers (P g,q α , P g,q β ) in the relevant wind turbines q α and q β , whereas the module's output vector MO α,β includes inconsistency detection (ID α,β ) and inconsistency magnitude (I M α,β ) signals. ID α,β and I M α,β signals indicate, respectively, the occurrence and the magnitude of any possible inconsistencies in the produced powers from turbines q α and q β . Under two possible cases for the power consistency by turbines q α and q β , the module's output vector MO α,β may attain the values indicated in Table 3. As observed in the table, if power production is inconsistent between turbines q α and q β , the ID α,β , I M α,β signals indicate the faulty turbine along with its absolute value of the estimated power-loss ∆P α,β , respectively. Conversely, if power production is consistent between turbines q α and q β , both ID α,β and I M α,β signals become zero. Note that obtaining such zero signals (in the latter case) does not necessarily mean that both turbines q α and q β are healthy. As a matter of fact, if power production is consistent between two considered turbines they may either be both healthy or faulty. Given that, as can be seen in Figure 5, the FDD system employs a set of DM blocks to analyse the modules outputs all together in real-time and identify any faulty turbine(s) accordingly.
As mentioned, Figure 5 shows the appropriate distribution and contribution of module outputs in the decision-making procedure within each DM block. Each individual wind turbine has a DM block that is exclusively used to analyse the MO signals related to that individual wind turbine and, accordingly, make an appropriate decision about the wind turbine's real-time health status. The simple logic behind such a decision is that as long as at least one of the relevant modules detects a turbine as a faulty turbine, then that turbine is faulty and its power-loss due to the fault (i.e.,P q j ) is conservatively the maximum power-loss estimated by the modules [20]. As per the given information in Figures 5 and 6 and Table 3, the aforementioned decision-making logic can be formulated as in Rule 1, Rule Z and Rule N i for turbines q 1 , q Z and q N i , respectively [20]. Note that q Z with (Z ∈ N and 1 < Z < N i ) denotes any wind turbine except q 1 or q N i .
Outcomes obtained from DM blocks (see Figure 5) include the FDD information vector F i in Equation (10) for the wind farm Q i in the cluster.
where F q j (k) includes the number of wind turbine q j and its power-loss magnitudeP q j . Apparently, the functionality of the FDD system described here depends on the monitoring process in the modules. From a design perspective, the entire modules utilized in the distributed FDD system are the same, although each one corresponds a particular pair of wind turbines. Every module uses a dynamic model of the nominal system, which is developed by a data-driven modelling technique based on fuzzy modeling and identification (FMI) approach [31]. Indeed, the dynamic model estimates the nominal relative performance between wind turbines q α and q β in wind farm Q i . The following paragraphs provide more information about the internal structure of the modules. Module Structure: Figure 7 illustrates the internal structure of module M α,β . As can be seen, the module uses a dynamic model that is represented by a Takagi-Sugeno (T-S) type fuzzy model. More details about T-S type fuzzy modeling approach are available in the literature [20,31]. The mentioned model estimates the nominal relative behavior between turbines q α and q β in wind farm Q i . The generated powers from turbines q α and q β are defined as consistent when P g α,β andP g α,β are approximately equal (or the residual signal r α,β is almost zero). In practice, this equality can not happen at all times, mainly due to measurement noises, possible disturbances and model uncertainties. For this reason, a post-processing of inconsistency signature unit is used to achieve absolute inconsistency I α,β . It should be noted that I α,β has binary values (i.e., 0 for consistent power generation and 1 for inconsistent power generation). The post-processing unit utilizes a threshold test technique to obtain a reliable conclusion for I α,β . In more detail, in the case of any inconsistency occurrence in the power generation, I α,β switches from 0 to 1 if r α,β remains outside a predefined threshold T α,β for more than t in consecutive time-steps. Then, I α,β switches from 1 to 0 if r α,β remains within T α,β for more than t out consecutive time-steps. The appropriate values for T α,β , t in , and t out can be determined by the designer. It should be noted that I α,β only indicates the consistency or inconsistency between considered turbines. In other words, I α,β does not include any information about the magnitude of the powerloss fault. Therefore, the inconsistency analysis and isolation unit is used to isolate the faulty turbine and identify the magnitude of fault. During any inconsistency (i.e., I α,β = 1), this unit computes the running mean of ∆P α,β as ∆P α,β . Note that if (∆P α,β < 0) then q α is faulty, and if (∆P α,β > 0) then q β is faulty.

Integrated ASC Unit
At the turbine level, and in the event of possible mild power loss faults, the proposed active FTCC scheme enables fault-tolerant capability for generator torque controller in any faulty wind turbine(s) inside each farm. More precisely, this control solution accommodates fault effects in faulty wind turbine(s) whose loss of generated power is/are mild (normally less than 3%). Such mildly faulty turbines are still healthy enough to continue their operations although an appropriate signal correction is required in their generator torque controllers. As previously discussed in Section 2.2, the mentioned power loss fault results in a lower power production that adversely affects the total active power produced by all the turbines in WFC as a whole. When a mild power loss fault occurs in one or more than one turbine in a cluster's wind farms, the relevant wind farm controller should follow the network operator's demanded power P D,Q i in Equation (4), no matter which turbine or turbines are mildly faulty. However, the compensation of power loss in mildly faulty wind turbine(s) is preferred to prevent overloading the remaining healthy wind turbines. Instead, the power reference signal(s) to the faulty wind turbine(s) is/are corrected in order to accommodate the mild fault effects. As shown in Figure 3, ASC units are responsible for the described accommodation of mild power loss faults at the turbine level inside the cluster's individual wind farms and according to their respective FDD information. Indeed, providing the most up-to-date information about the status of WFC, the distributed FDD systems enable ASC units to automatically safeguard the entire WFC. To explain further, any inconsistencies in the generated powers are identified to generate reliable FDD information. Finally, this online information is utilized for ASC and accommodation of the mild power loss faults in faulty wind turbines. The ASC mechanism does not interfere with the normal operation of baseline controllers at the turbine or farm levels, but only corrects the output of torque control system in faulty wind turbine(s) based on the online fault information received from the FDD system. The ASC units in Figure 3 are not described explicitly here since they are very simple algorithms. According to the provided FDD information in each wind farm, the relevant ASC unit identifies the faulty wind turbines along with their relevant fault magnitudes (estimated mild power losses). Accordingly, the estimated fault magnitudes are used to correct the generator reference torque signals τ r,q j computed by the torque control system. This simple signal correction mechanism is shown in Equation (11) for the q j th turbine in the farm Q i . τ r,q j ,corr (k + 1) = τ r,q j (k) +P q j (k) ω g,q j (k) , q j ∈ Q i In Equation (11), τ r,q j and ω g,q j denote the generator reference torque and generator angular speed for the q j th turbine in the farm Q i , respectively. The corrected reference generator torque is denoted by τ r,q j ,corr , whereasP q j is the estimated fault magnitude, both in the q j th turbine in the farm Q i . Under fault-free conditions,P q j is assigned to be zero to keep the system's nominal performance. To avoid excessive loadings on the actuators, the recommended rate and magnitude limiters are used for τ r,q j ,corr computed by Equation (11).

Fault-Tolerant Wind Farm Controller
In the event of severe power loss faults (normally more than 3%) in wind turbines, the ASC mechanism at the turbine level is not sufficient. Thus, a hierarchical control reallocation mechanism in two higher levels is required. The first control reallocation level, referred to as "farm-level control reallocation", relies on the turbines' health factors obtained by Algorithm 1. In order to compensate the missing power of severely faulty wind turbines, this farm-level control reallocation mechanism automatically reallocates control commands among turbines in each wind farm to demand healthier wind turbines to produce more power. If healthier wind turbines in a respective farm do not have enough available power to accommodate severe fault effects, a higher control reallocation level, referred to as "WFC-level control reallocation" (inside the fault-tolerant network operator in Figures 3 and 4) is activated automatically. This top layer of control is discussed in the next subsection.
In line with the proposed solution in [22], the online health status of turbine q j in farm Q i is represented using the binary health factor H f q j = {0, 1}. That is, for healthy or mildly faulty wind turbines, the health factor attains 1, whereas for severely faulty turbines this factor attains 0. The farm-level control reallocation mechanism uses these health factors to appropriately switch between nominal control and control reallocation modes. In more detail, when all health factors of wind turbines are 1, the farm controller operates under nominal control mode as per Equation (4). However, the signal correction by ASC units (at wind turbine level) might be needed to accommodate mild power loss faults as discussed in previous subsection. In the event of any severe fault in any turbine of farm Q i , the turbine's respective health factor changes to 0. Accordingly, the farm controller switches to the reallocation mode. As such, the wind farm's healthier wind turbines with extra (unused) available power take over the pending duties of the severely faulty wind turbines. This strategy ensures safe and continuous operation of farm Q i following the farm's demanded power.
According to [22], under the farm-level control reallocation mode, a wind farm controller initially computes the missed power in severely faulty wind turbines (with H f = 0) as follows: P m,q j (k) = P d,q j (k) − P g,q j (k) (12) P m,q j (k) (13) in which P d,q j (k), P g,q j (k), and P m,q j (k) are, respectively, the demanded, generated, and missed powers in turbine q j from farm Q i at the time-instant (time-step) k. Moreover, P M,Q i (k) in Equation (13) is the total missed power in severely faulty turbines at the timestep k. This amount of total missed power must be compensated for by demanding the healthier turbines (with H f = 1) to produce more power if they have extra available power computed by: P u,q j (k) = P a,q j (k) − P g,q j (k) (14) P U,Q i (k) = ∑ q j ∈{Q i | H f q j =1} P u,q j (k) (15) in which P a,q j (k) and P u,q j (k) respectively denote the estimated available power and the extra available power in turbine q j from farm Q i at the time-step k. Note that, P U,Q i (k) in Equation (15) represents the total extra available power in healthier turbines in the same farm. A wind turbine's available power is estimated according to Equation (5), and the estimated total available power in farm Q i is given by: Lastly, the demanded power from healthier turbines (with H f = 1) is obtained as: where P d,q j (k + 1) and P D,Q i (k) are the already defined variables for Equation (4). Algorithm 1 presents the entire details of the described farm-level control reallocation mechanism. Algorithm 1. Wind Farm Q i Control Reallocation (Active FTCC at Secondary Control Level).
N: natural numbers set N: number of wind farms in a wind farm cluster, N i : number of wind turbines in wind farm Q i , k: time-step, P D,Q i (k + 1): total demanded power from wind farm Q i for the next time-step, P g,q j (k): generated (output) power of wind turbine q j , V mes,q j (k): measured nacelle wind speed of wind turbine q j , and F q j (k): fault information including the number of wind turbine q j and its power loss magnitudeP q j .

Outputs:
set of wind turbines in wind farm Q i , P d,q j (k + 1): demanded power from wind turbine q j for the next time-step, P A,Q i (k): total available power in wind farm Q i , P G,Q i (k): total generated power in wind farm Q i , and HF Q i (k): total health factor of wind farm Q i . Constants and Variables: P rated,q j , ρ, R, C Pmax , T Q i , T q j , P d,q j (k), P a,q j (k), P m,q j (k), P u,q j (k), P M,Q i (k), P U,Q i (k), H f q j (k) P rated,q j : rated power of wind turbine q j , ρ: air density, R: rotor radius, C Pmax : maximum power coefficient, T Q i : threshold for severe fault detection in wind farm Q i , T q j : threshold for severe fault detection in wind turbine q j , P d,q j (k): demanded power for turbine q j , P a,q j (k): available power in turbine q j , P m,q j (k): missed power of turbine q j , P u,q j (k): extra available power in turbine q j , P M,Q i (k): total missed power of wind farm Q i , and P U,Q i (k): total extra available power of farm Q i , and H f q j (k) = {0, 1}: health factor for turbine q j . 1. for each q j ∈ Q i do 1.1. ifP q j ≥ T q j then

Fault-Tolerant Network Operator
According to Algorithm 1, the health status of farm Q i (inside the WFC Q) is represented by its respective binary health factor HF Q i (k) = {0, 1} that is determined by a threshold-based detection method. In fact, for fault-free wind farms or faulty wind farms with mild power loss fault, the health factor is 1, whereas for faulty wind farms with severe power loss fault, the health factor is 0. In other words, when HF Q i = 1, the faults effects can be accommodated at the farm level using the ASC unit and fault-tolerant wind farm controller in farm Q i . However, when HF Q i = 0, the fault accommodation at the farm level is not sufficient due to the lack of enough available power in the farm. In this case, the control commands is automatically reallocated among the farms at the WFC level to require healthier farms (with an extra available power) to produce more power and compensate the missing power of the farms impacted by severe power loss.
Here, the network operator employs the health factors HF Q i from wind farm Q i to appropriately switch between nominal control and control reallocation modes, but at the WFC level. When all wind farms have health factors of 1, the network operator works under the nominal control mode as in Equation (3). In such a case, only signal correction at the turbine level and control reallocation at the wind farm level might be required to accommodate possible power loss faults as previously described. Conversely, as soon as any severe power loss fault is detected in any wind farm of the WFC, the wind farm's respective health factor changes to 0. As a result, the network operator switches to the reallocation mode in which healthier wind farms (with extra available power) take over the pending duties of the faulty farm(s). Thus, the safe and uninterrupted operation of the WFC as a whole is guaranteed.
Under the WFC-level control reallocation mode, the network operator first computes the missed power in wind farms impacted by severe fault (HF = 0) as shown below: where P D,Q i (k), P G,Q i (k), and P M,Q i (k) are, respectively, the demanded, generated, and missed powers for farm Q i at the time-instant (time-step) k. In addition, P M (k) in Equation (19) denotes the total missed power in farms with severe power loss at the time-step k. This amount of total missed power can be compensated for by requiring the healthier farms (with HF = 1) to produce more power if they have extra available power. The extra available power in healthier farms is given by where P A,Q i (k) and P U,Q i (k) are, respectively, the estimated available power and the extra available power in farm Q i at the time-step k. Moreover, P U (k) in Equation (21) denotes the total extra available power in healthier farms. Note that the available power in wind farm Q i is P A,Q i (k) (received from the farm controller), and the estimated total available power in the WFC Q is obtained by: The demanded powers from healthier farms with HF = 1 are as follows: where P D,Q i (k + 1) and P D (k + 1) represent the already defined variables for Equation (3). Algorithm 2 presents the details of the mechanism described for WFC-level control reallocation.

Algorithm 2 Network Operator Control Reallocation (Active FTCC at Tertiary Control Level)
Inputs: P A,Q i (k), P G,Q i (k), HF Q i (k), f mes (k)| i ∈ N, i ≤ N N: natural numbers set, N: number of farms in a wind farm cluster, k: time-step, P A,Q i (k): total available power in wind farm Q i , P G,Q i (k): generated (output) power of wind farm Q i , HF Q i (k) = {0, 1}: health factor for wind farm Q i , and f mes (k): measured grid frequency.
set of wind farms in wind farm cluster, P D,Q i (k + 1): demanded power from wind farm Q i for the next time-step. Constants and Variables: f r , P min , P max , c, d, f e (k), P D (k + 1), P A (k), P D,Q i (k), P M,Q i (k), P U,Q i (k), P M (k), P U (k) f r : reference grid frequency, P min : minimum limit for total power generated by the wind farm cluster, P max : maximum limit for total power generated by wind farm cluster, c: control band used for active power control, d: dead band used for active power control (c > d), f e (k): grid frequency error, P D (k + 1): total demanded power from wind farm cluster for the next time-step, P A (k): total available power in wind farm cluster, P D,Q i (k): total demanded power from wind farm Q i , P M,Q i (k): total missed power of wind farm Q i , P U,Q i (k): total extra available power of wind farm Q i , P M (k): total missed power in wind farm cluster, and P U (k): total extra available power in wind farm cluster. 6. else if f e (k) ≤ −c then 6.1. P D (k + 1) = P max 7. end if

end for
Algorithm 2 Network Operator Control Reallocation (Active FTCC at Tertiary Control Level) 9.9. end for 10. end if 11. return P D,Q i (k + 1)| Q i ∈ Q

Simulation Results
The nonlinear benchmark model for WFC and the power loss faults in Section 2 are used here to evaluate the effectiveness and performance of the proposed active FTCC scheme. The considered power loss faults are mathematically modelled through scaling the output powers of faulty wind turbines, representing their decreased power generation. In reference to the WFC of 100 wind turbines shown in Figure 1, a typical fault scenario is designed for a total simulation time of 1000 s during which mild and severe power loss faults occur in some wind turbines. Among the said faults it is assumed that some simultaneously occur in more than one wind turbine. The timeline for the occurrence of mild and severe faults in a designated number of wind turbines in the WFC is shown in Figure 8. Figure 9 illustrates the layout of the WFC under the considered faults.
For wind farms Q 2 and Q 5 (i.e., WF 2 and WF 5 in Figure 9) that are only impacted by mild power loss faults, no control reallocation at wind farm or WFC levels would be necessary (see Algorithms 1 and 2). Indeed, the previously discussed ASC mechanism at wind turbine level is adequate to automatically accommodate mild faults effects. However, for wind farms Q 8 and Q 9 (i.e., WF 8 and WF 9 in Figure 9), the procedure is similar to wind farms Q 2 and Q 5 except for T 8,4 , T 9,1 , and T 9,2 that are impaired by severe power loss faults. The magnitude of such faults in the mentioned severely faulty turbines develops to a severe total power loss as shown in Figure 10. In general, a total power loss condition can occur either because of a shutdown command by the farm operator or an unpredicted failure. However, the proposed active FTCC scheme accommodates the adverse impacts of any severe power loss faults (no matter partial or total). As per Algorithm 1, severe power loss effects in faulty wind farms Q 8 and Q 9 can be accommodated by farm-level control reallocation. In addition, ASC at the turbine level still accommodates the remaining impacts of mild power loss faults in T 8,3 , T 8,7 , T 9,3 , and T 9,5 . Unlike the previously discussed cases, for wind farm Q 6 (i.e., WF 6 in Figure 9), the fault accommodation at the wind turbine or wind farm level will not be effective (due to the number of severely faulty wind turbines and the lack of enough available power in the farm (see Figures 9 and 10)). Therefore, the control commands must be reallocated among the wind farms at WFC level as per Algorithm 2. The following subsections provide more information about the conducted simulation results.

Mild Power Loss Fault Accommodation at the Turbine Level
In reference to fault-accommodation at the turbine level in wind farms Q 2 and Q 5 , Figure 11 shows the generated power during fault-free (nominal) and faulty operations with or without considering the FTCC scheme. As shown by the black lines (i.e., faulty operation with FTCC scheme) following the green lines (i.e., nominal operation), the proposed FTCC scheme (i.e., FDD integrated with ASC at turbine level) can effectively accommodate the faults effects. In addition, this figure illustrates that the FTCC scheme under fault-free operation has almost no effects on the performance of the wind turbines. To demonstrate clearly, Figure 11 only shows generated power around the faults' activity periods in the turbines with considered mild faults. Similarly, for the other turbines impacted by mild power loss faults in wind farms Q 6 , Q 8 and Q 9 (i.e., WF 6 , WF 8 and WF 9 in Figure 9), the corresponding ASC units (i.e., ASC units 6, 8 and 9 integrated with FDD systems 6, 8 and 9, respectively) accommodate the faults impacts at wind turbine level.

Severe Power Loss Fault Accommodation at Farm and Cluster Levels
As per already depicted fault scenario in Figure 8, some wind turbines in farms Q 6 , Q 8 , and Q 9 encounter severe power loss faults. Specifically, the severe fault effects in wind farms Q 8 and Q 9 are accommodated by the farm-level control reallocation mechanism based on Algorithm 1 since there is enough available power in the farms' healthy wind turbines.
However, for the accommodation of severe fault effects in farm Q 6 , the farm-level control reallocation is not sufficient due to the lack of enough available power in the farm's healthy wind turbines (i.e., T 6,1 and T 6,4 ). Therefore, the entire active FTCC scheme, including the WFC-level control reallocation (Algorithm 2), automatically tackles severe fault effects in addition to the other mild fault effects.
In Figure 12, comparing the generated powers in wind farms during fault-free (nominal) and faulty operations with and without the active FTCC scheme, it is observed that the faulty operations (for the same wind farms) with farm-level control reallocation (blue lines) are different from the faulty operations with both farm-level and WFC-level control reallocations (black lines). This is due to the severe power loss faults in Q 6 , changing HF Q 6 to zero (at about 450 sec), which activates the WFC-level control reallocation mode to compensate the total missed power in Q 6 . The resultant control commands are obviously different. This fact is also seen in Figure 13 that shows the demanded powers sent by the network operator to the respective wind farms in the cluster. As seen, the power demanded from Q 6 (severely impaired wind farm) is almost the same as that under nominal operation. However, for the healthy wind farms, the demanded power is increased to accommodate the overall fault effects at the cluster level.     The WFC's characteristics, including the grid frequency and the total active power (tracking the considered grid load), are illustrated in Figures 14 and 15, respectively. In these figures, the adverse fault impacts (because of severe power loss faults) are clearly observed as significant deviations in the grid frequency response and total generated power (compare the red lines with the green lines in the figures). However, the black lines (i.e., faulty operation with active FTCC) in the figures show that the proposed active FTCC scheme successfully accommodates the adverse fault effects at the farm and cluster levels. The observed fluctuations in the black lines correspond to the moments of major disturbances in the wind speed or the occurrence of mild power loss faults. As can be seen, the proposed active FTCC scheme can handle all the mentioned moments effectively.  Moreover, the blue lines in Figures 14 and 15 indicate faulty operation with the active FTCC scheme but by ASC and farm-level control reallocation only. Predictably, the accommodation of considered power loss faults still enhances the grid frequency response but not as efficiently as with the complete active FTCC scheme that accommodates the severe faults at the cluster level.
As seen in Figure 16, illustrating the total demanded power from the WFC during fault-free (nominal) and faulty operations with or without FTCC, the WFC under faulty operation without active FTCC is required to provide more power than the same WFC but with active FTCC. Indeed, the total demanded power under faulty operation with active FTCC is almost similar to that under fault-free operation. The faulty operation with the active FTCC scheme (black line in Figure 16) shows that at about 450 s, the network operator automatically switches to the control reallocation mode.
Finally, to evaluate the robustness of the proposed FTCC scheme against measurement uncertainties and communication delays, extensive Monte Carlo simulations were performed. In the considered benchmark model, sensors (i.e., anemometer, rotor speed, generator speed, and blade pitch angle sensors) are considered as noise-contaminated measurement systems, and communication links between network operator and wind farm controllers, and between each wind farm controller and its corresponding turbines, contain time delays. The measurement noises in each sensor and the time delay in each communication link are generated as band-limited white noise. Tables 4 and 5 present the considered sensors and communication links with relevant parameters used during the simulations. Overall, 100 Monte Carlo simulations were performed for the designed FTCC scheme. Tables 6 and 7 provide the best, average, and worst values of the performance characteristics obtained from the simulations for severe and mild power loss fault scenarios, respectively. In summary, all the simulation results confirm that the proposed active FTCC scheme can successfully maintain reliable wind farm and WFC performance under faulty conditions and in the presence of possible uncertainties.  Various simulation studies on an advanced WFC benchmark model indicate the high effectiveness of the proposed schemes and solutions over an entire range of considered wind profiles and in the presence of wind profile turbulences, possible measurement noises and other realistic fault scenarios. The noticeable fault-tolerance capability obtained by the active FTCC scheme makes it superior to a typical WFC control system. The fact that the proposed scheme employs a type of signal correction technique makes it more industry-friendly compared to other methods cited in the technical literature. Moreover, the active FTCC scheme addresses off-normal power reductions due to both partial and total power loss conditions. This is done without interfering with the main duties of a WFC operator, including the shutdown of the turbines with severe power loss. In addition to the mentioned benefits, the proposed active FTCC scheme requires less computational power compared with the previous methods, since the proposed clustering approach in the scheme reduces the number of modules needed for computations significantly. However, future research should investigate for how long a severely faulty turbine is prescribed to continue working and contribute to farm power production before being structurally endangered. Different control techniques at each level (i.e., turbine, farm, and cluster levels) as well as passive FTCC approaches can also be investigated. Finally, when clustering a very large-scale wind farm, determining the optimal sizes of small-scale wind farms before activating the control reallocation mechanism at network operator level is another interesting topic that remains for future work.  Data Availability Statement: This study did not report any data.