^{1}

^{*}

^{2}

^{3}

^{4}

^{3}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (

This work is aimed at optimizing the wind turbine rotor speed setpoint algorithm. Several intelligent adjustment strategies have been investigated in order to improve a reward function that takes into account the power captured from the wind and the turbine speed error. After different approaches including Reinforcement Learning, the best results were obtained using a Particle Swarm Optimization (PSO)-based wind turbine speed setpoint algorithm. A reward improvement of up to 10.67% has been achieved using PSO compared to a constant approach and 0.48% compared to a conventional approach. We conclude that the pitch angle is the most adequate input variable for the turbine speed setpoint algorithm compared to others such as rotor speed, or rotor angular acceleration.

This paper presents the application of intelligent optimization techniques in wind turbine rotor speed setpoint control algorithms. Setpoint algorithms are compared in order to achieve two main objectives: to capture as much mean power as possible and to avoid reaching the tower resonance, which involves security stops with the corresponding mechanical fatigue and efficiency loss.

The rotor speed setpoint is highly related to power generation dispatch algorithms. In our case, a 100 kW Wind Turbine, the control must capture as much power as possible, but in other wind turbine systems such as [

In a first approach we have applied a Reinforcement Learning (RL) scheme. In wind turbine control, the most important variable is wind speed. This variable has stochastic behaviour, as the environment is changing all the time. In consequence, when the RL algorithm is applied to a stochastic dynamic, we must model the problem as a Markov Decision Process (MDP). MDP is a framework for optimal system control modelling in uncertain dynamic environments. MDP was first studied by Bellman [

There are different works on RL-MDP applied to wind turbines [

An MDP framework is defined by states, actions and probability of transition from one state to another. MDP states may be continue or discrete. In [

After multiple experiments, the choice of the proposed RL scheme was rejected, because it does not improve the conventional algorithm results. This question is further detailed later. As an alternative intelligent algorithm a Particle Swarm Optimization (PSO) based adjustment scheme is proposed. PSO is a bio-inspired computational technique based on the idea of natural swarm learning mechanisms, where living organisms remember successful positions of the swarm and use them to improve future rewards. This algorithm has been applied in Wind Turbine design with success [

The paper is organized as follows: Section 2 establishes the problem statement. Section 3 describes the wind turbine model: aerodynamics, power train and electrical machine. Wind regimes are explained in Section 4. Section 5 describes the conventional algorithm. Section 6 is devoted to the first proposed control system based on Reinforcement Learning. Section 7 is dedicated to the second proposed control algorithm based on Particle Swarm Optimization. Section 8 provides results and comparison. Finally, the paper is concluded in Section 9.

Wind turbine control algorithms adjust the setpoint in order to capture as much mean power as possible while preventing from reaching the tower resonance speed. These two objectives are achieved maximizing a reward function which is bounded between zero and one. This function is defined as the product between two sigmoid function form terms.

It is important to capture as much mean power as possible but, for example, if the control surpasses a certain power limit, the reward saturates. This effect has been taken into account in the first term of the reward function. The second term is related to rotor speed. A low rotor speed is a consequence of low wind speed. However this term is more relevant when the rotor speed increases up to the setpoint speed. Above this speed the corresponding term saturates. This reward function is described in Section 6.1.

The wind turbine plant is described by a classical model. The most important parts of the model are the power train, the power stage, the blades' aerodynamics and the electrical machine.

When an airfoil moves through a fluid, it supports three types of stress: lift, drag and pitch moment. Specifically the lift is responsible for turning the turbine and its azimuthal torque. However, from the Control Engineer's point of view, the parameter that best defines the wind turbine behaviour is the power coefficient. This coefficient indicates the percentage of kinetic energy of the passing air captured by the wind turbine per time unit.

In fact, it is common to express the aerodynamics torque T_{w} in the following way [

_{w}

_{p}

_{w}

Manufacturers usually provide the aerodynamic power coefficient _{p}_{w}

There exist many other more complex models that may be consulted in [

In pitch control the most interesting variables are the flapping and bending moments at the blade roots, together with the gyroscopic effects such as the furling forces. All these effects are described in [

The tower and the nacelle dynamics are modelled by finite element techniques given by CAD platforms (for example Catia). When these dynamics are taken into account in control algorithms design, NREL software platforms are frequently used, mostly FAST [

The dynamics of the power train model is one of easiest part to be described, apart from the torsion elasticity effects. The torsion elasticity is mainly defined by the elastic link of commercial elements and their technical characteristics are often not well known. Another frequent problem is the gear losses dynamics. In fact, these losses change the curve torque

_{w}

_{em}

_{tot}

_{turbine}

_{generator}

_{gear}

100 kW wind turbine generators usually have full converter asynchronous machine topologies. The main reasons for that are the cost of power electronics, the cost of electrical machine and the power performance behaviour

The permanent magnet machines are utilized in medium and high power wind turbines without gear. This characteristic makes these wind turbines cheaper with higher performance. In order to make compatible the speed spread of the electrical machine and the speed spread of the wind turbine a gear is set in the power train.

The main criteria to select the electrical machine for a wind turbine are:

The power stage has been modelled by a first order system. The input is the torque demand; the output is the real torque given by the electrical machine. A _{ps}

_{ps}

This equation only models the torque dynamics because our objective is not to develop the power stage control.

The wind speed realizations utilized in our simulations, as shown in

Mean wind speed hub: 20 m/s;

Class: III-A;

Hub height: 36 m;

Height of the low-level: 70–490 m;

Power law exponent: 0.2.

This model has been chosen because it forces the control to move the pitch angle. In this kind of wind speed realizations the pitch has more “nervous” temporal series. The structural fatigue is very high in these conditions and the pitch activity is a good measure to limit this fatigue.

Another wind speed realization utilized in this paper is the Extreme Operating Gust (EOG), as shown in

The EOG model parameters are the following:

Wind turbine radius: 11.25 m;

Class: IIIA.

A variable speed, variable pitch wind turbine has two main control loops: The torque loop and the pitch loop. The torque loop tries to follow these objectives:

To capture the maximum power at low wind speeds;

To obtain constant power from the wind when the turbine is above the rated rotor speed;

To make a smooth torque transition between these two modes.

According to these objectives, the relationship between torque setpoint and rotor speed is presented in

When the wind turbine works below the rated speed, the torque demand is calculated as follows:

_{p_max}

_{optimal}

When the wind turbine works above the rated speed, the torque demand is calculated as follows:

_{nom}

The transition between these two curves is made linearly. This allows a smooth torque transition between them. The first curve tries to obtain the maximum power from the wind. In this stage the pitch angle is set to zero. The second curve tries to keep the rated power when the wind turbine is above the rated rotor speed. In this stage the pitch angle is variable, and it is calculated by a pitch controller. It is important to notice that the performance has to be taken into account.

The pitch controller has the objective of keeping the wind turbine speed at a certain setpoint. A typical controller structure is a PI with gain scheduling. A good technique can be found in [

^{*}

_{p}

^{*}

_{i}

The pitch actuator model is described as follows:

_{pa}

A τ_{pa} time constant has been identified after several trials made with the pitch actuator. In this work, the pitch time rate has been limited to 10°/s. This limit is very common in this kind of applications. In fact, a greater pitch time rate can dramatically increase the structural fatigue. The PI controller parameters can be calculated by the formulae given at [

_{n}

_{o}

_{1}_{2}

The wind turbine power sensibility can be estimated from the power coefficient surface. Once this sensibility has been estimated the control parameters can be calculated following

The conventional rotor speed setpoint algorithm is based on real pitch values. Basically, the rotor speed setpoint is set to its rated value. With higher wind speed, the pitch angle will take higher values. In this situation, when sudden falling wind gusts appear, rotor speed decreases causing loss of electrical power. This loss of production can be avoided when there is a high wind speed, if the rotor speed setpoint is slightly increased. This increment is usually set depending on the pitch angle. A more detailed explanation of this algorithm can be found in [

The first new solution experimented in this work is based on a Reinforcement Learning (RL) based rotor speed setpoint. The main reason for this choice is that the rotor speed setpoint has to be calculated following several opposed criteria and the decision scenario is a stochastic one: the wind speed is a very stochastic process. The two main criteria are:

To capture as much power as possible. To achieve this objective, it is necessary to maintain the rotor speed at high values when the wind speed is above the rated value. So, the rotor speed has to be maintained at high values although a wind speed gust decreases during a short time.

To stay as far as possible from the resonance and reduce the load fatigue. So, the rotor speed has to be reduced to avoid a structural damage.

This kind of problems can be posed as a Markov Decision Process (MDP), and the chosen optimization method is the Q-learning. This technique is applied in stochastic scenarios for multi elements control systems with high success [

_{state}

_{rated}

_{state}

_{rated}

The defined actions are three: High, Medium and Low rotor speed setpoint. When the control system is working with high wind speed and the angular acceleration is negative, high rotor speed setpoint prevents the power from falling. On the other hand, if the rotor speed is high but the acceleration is positive, low rotor speed setpoint prevents the rotor speed from getting closer to resonance speeds. There are different discrete states, and in each state different actions can be chosen. As it may not be clear which is the best action in each state, the reinforcement matrix

_{t}

_{t}

The _{t}

Finally the learning algorithm updates the

The reward function can be expressed as follows:

_{rated}

This function depends on the captured power and on the rotor speed error. The function is the product between two sigmoid terms. The first term gives a low value when the captured power is low, but has a horizontal asymptote that goes to one when the power arrives to infinity. In fact, this term is 0.73 at the rated power. The second term has the same sigmoid function, but in this case the positive rotor speed errors are not penalized because they appear when the wind speed is low. When the rotor speed error is negative, as when there is an over speed, the term decreases because it is closer to the resonance speed. This reinforcement function is always between zero and one. This mathematical property is very desirable because it prevents numerical instabilities in the learning process. The algorithms are compared according to the mean value given by this function along the different wind speed regimes.

The second new solution experimented in this work is based on a metaheuristic optimization using the Particle Swarm Optimization (PSO) method. PSO seeks the best parameters for a proposed pitch based rotor speed setpoint function. This proposed function is used to control the wind turbine by pitch angle, which is the only input variable utilized as proposed in the literature [

PSO is a metaheuristic technique [

The PSO computational method iteratively optimizes by proposing new solutions based on information of previous results of each particle and the neighbour particle's experiences. The mathematical implementation of PSO method applied to this problem follows. More details can be found in [

Mathematically, the search space is defined as A⊂R^{n}, where n is the number of variables. In our case, there are two dimensions or operation variables:

A particle _{i}(k)_{i}(k+1)_{i}(k)

The best positions that have ever been visited until iteration

The best position that

In PSO there are some parameters to be set:

_{min}_{max}_{min}_{max}

_{i}

_{1}

_{2}

High values of _{1}_{2}_{1}_{2}

_{1}_{2}_{1}_{2}

The position is updated from iteration

The novelty in this paper resides in the function set

In order to maintain the continuity of the setpoint function,

_{max}_{max}

The proposed rotor speed setpoint function can express the conventional algorithm when

A comparison between the four main rotor speed setpoint policies is carried out:

The proposed

The proposed

These algorithms are going to be evaluated in two mains aspects:

In this comparison the wind speed series utilized are based on EOG models and turbulence wind speed series generated by TurbSim program [

PSO simulations have been carried out with a number of particles in between 40 and 400. _{1}_{2}

0.2·

Other tested variable spans were:

0.2·

0.2·

0.2·

All simulations lead us to the same optimal point.

Result values are summarized in

The algorithm has more input variables (rotor speed and acceleration) than the other ones (pitch angle). This increases the complexity of RL adjustment process.

The RL states have been discretized into a very low number of possibilities in order to reduce the tail of

Mean Reward is evaluated along 800 s time span following the equation below:

In this paper we have presented intelligent optimization techniques in wind turbine rotor speed setpoint control algorithms. Setpoint algorithms were compared in order to capture as much mean power as possible while avoiding the tower resonance on a 100 kW wind turbine. RL and PSO algorithms were used, together with constant and conventional rotor speed setpoint algorithms for comparison purposes.

Results show that the proposed PSO based rotor speed setpoint algorithm is the best approach in order to improve the proposed reward function. This algorithm is easy to implement in Wind Turbine Control systems and it does not require much computational power. It improves a constant approach, which is very common, in 10.67%. It also improves the more sophisticated conventional approach in 0.48%.

We estimate that RL algorithm has a good potential in this problem but it needs a more complex state definition in order to improve its reward mean value. This implies a heavy increase of computational requirements. Further exploration of these alternatives is envisaged in a near future.

Another important potential improvement is to change the proposed reward function in

Wind speed realization calculated with TurbSim.

Extreme Operation Gust.

Torque setpoint

Mean Reward

Mean reward result obtained with each setpoint algorithm.

| ||||
---|---|---|---|---|

Mean reward | 0.2625 | 0.2891 | 0.2500 | 0.2905 |

Improvement PSO | 10.67% | 0.48% | 16.20% |

This research was supported by the Basque Government through the projects IG-2011/0000794, S-PE11UN061 and S-PE12UN015, and by the Argolabe Ingeniería SL Company. Some of the authors belong to Computational Intelligence Group of the University of the Basque Country (UPV/EHU), supported by the Basque Government.