Wind Power Prediction Using a Dynamic Neuro-Fuzzy Model

Abstract

A Dynamic Neuro-fuzzy Model (Dynamic Neuro-fuzzy Wind Predictor, DNFWP) is proposed for wind power prediction. The fuzzy rules in DNFWP consist of a typical antecedent part with static inputs, while the consequent part is a small three-layer neural network, incorporating unit feedback connections at the outputs of the neurons of the hidden layer. The inclusion of internal feedback targets to capture the intrinsic temporal relations of the dataset, while maintaining the local modeling approach of traditional fuzzy models. Each rule in DNFWP represents a local model, and the fuzzy rules operate cooperatively through the defuzzification process. The fuzzy rule base is extracted employing the Fuzzy C-means clustering algorithm, and the consequent neural networks’ weights are tuned by the use of Dynamic Resilient Propagation. Two cases with datasets of different volumes are tested and the performance of DNFWP is very promising, according to the results attained using a series of metrics like Root Mean Squared Error, Mean Absolute Error, and the r-squared statistic. The dynamic nature of the predictor allows it to operate effectively with a single input, thus rendering a feature selection phase unnecessary. DNFWP is compared to Machine Learning-based and Deep Learning-based counterparts, such that its prediction capabilities along with its reduced parametric complexity are highlighted.

1. Introduction

Wind power capacity has become a critical issue in recent years, as the shift to renewable energy sources from fossil fuels seems to be an essential demand of developed societies. Both the U.S. and the European Union are making significant efforts to transform their economies into green ones, while simultaneously requiring continuously greater amounts of energy in order to fulfill their needs in the energy-intensive sectors like the computer and internet facilities sector [1,2]. For example, in 2023, the global wind power capacity was reported to attain 1047 GW [3]. Accurate wind prediction is a major challenge to operators of Energy Management Systems (EMS) since reliable predictions are crucial inputs to EMS’s fundamental operating functions [4].

In recent years, there has been growing interest in developing reliable methods for wind power forecasting, primarily driven by the increased availability of relevant data [5,6,7,8,9]. For short-term predictions, traditional methods, such as the autoregressive moving average (ARMA) model, have been explored [10,11]. Their advantage is the simplicity in model-building; however, they perform less effectively when wind power variations are irregular. The key challenges in wind power data are nonlinearity, nonstationarity, and high dimensionality. To tackle these challenges, Machine Learning- and Computational Intelligence-based models stepped into the spotlight: in an attempt to effectively handle the data, a two-stage method was proposed in [12], where wind time-series data are first processed using wavelet decomposition, followed by the prediction phase, which is implemented by an adaptive wavelet neural network (AWNN). Another neural network approach, based on radial basis functions, was presented in [13], with promising results. As far as the fuzzy pillar of Computational Intelligence is concerned, a number of static fuzzy systems were proposed to offer more interpretable alternatives to the black-box behavior of neural networks [14,15].

Additional studies include traditional Machine Learning methods, such as support vector machines (SVM) and regression (SVR) or random forests (RF) [16,17]. In an attempt to mine the information of temporal dependencies, data-driven Machine Learning methods were proposed, including lagged-ensemble Machine Learning [18,19] and dynamic principal component regression [20], having shown satisfactory results.

Further improvements were accomplished with the introduction of Gaussian Process Regression (GPR) [21], a nonparametric kernel-based model, as well as Ensemble Learning approaches (Boosted Trees (BS), Bagged Regression Trees, and eXtreme Gradient Boosting (XG-Boost)) [22,23,24]. These developments have firmly established Machine Learning as a key contributor to the field [25,26,27].

Over the past decade, Deep Learning has been revolutionizing Computational Intelligence, enriching the application fields, and taking advantage of the explosion in computational resources of modern computing systems. Therefore, Deep Learning predictors became a key tool for wind power prediction. All the established deep models (Long Short-term Memory (LSTM), Gated Recurrent Unit (GRU), recurrent neural networks, Convolutional Neural Networks (CNNs), transformers) [28,29,30,31,32,33,34], as well as alternative ones like Temporal Collaborative Attention and Deep Graph Attention Networks [35,36], have been proposed to address such a demanding problem and considerable advances have been accomplished.

One of the most significant issues in time-series prediction problems is the selection of an appropriate input vector from an extensive pool of candidate inputs. In the case of wind power generation where dependencies on temporal and spatial factors, like weather conditions, climate variables, and seasonal patterns, exist, the feature selection step is crucial for the predictor to be effective. In this perspective, the Dynamic Neuro-fuzzy Wind Predictor (DNFWP) is proposed as a wind power predictor. The model has the form of a typical fuzzy system, with the neural components being the fuzzy rules’ consequent parts. They are three-layer neural networks, with the hidden layer consisting of neurons with internal output feedback, thus introducing dynamics to the overall model. No external feedback connections exist, maintaining the classic structure of fuzzy systems, with the rules operating separately from each other and being interconnected through the defuzzification part. A single input is used, in an attempt to alleviate the aforementioned issues of feature selection, as well as to examine whether the time series can be identified without using external variables linked to weather conditions.

The Dynamic Resilient Propagation (DRPROP) is selected to be the tuning algorithm. It is a learning scheme that aims to alleviate the disadvantages existing in methods based on gradient descent and takes into consideration the recurrent connections of the proposed predictor.

The rest of this paper is organized into four sections: Section 2 outlines the architecture of DNFWP, while in Section 3, the model-building process is illustrated. In Section 4, two experimental examples and a comparative analysis with well-known forecasters are hosted, such that the proposed model’s characteristics and forecasting capability are highlighted. Finally, Section 5 summarizes the conclusions.

2. The Structure of Dynamic Neuro-Fuzzy Wind Predictor

The fuzzy rule base is the core of a fuzzy system with m inputs and a single output, comprising rules expressed as conditional statements:

$$IF x_1(k) \mathrm{is} A_1 \mathrm{AND} \dots \mathrm{AND} \hspace{0.17em}{x}_m(k) \mathrm{is} A_{m}THEN y(k\hspace{0.17em}) \mathrm{is} B$$

(1)

where $\mathit{x}={[x_1,\dots ,x_m]}^T$ is the input vector, k represents the sample (time step) index, A_j, and B correspond to the fuzzy sets of the j-th input and the output, respectively.

The introduction of the Takagi–Sugeno–Kang (TSK) fuzzy rule [37], in which the consequent part is expressed as a linear combination of the input variables, marked a significant advancement over classic fuzzy systems, by enhancing their learning capabilities. The rule can be considered as a subsystem of the overall fuzzy system, operating in the region where the membership function of the premise part reaches high degrees of membership. In such regions, the consequent part is responsible for modeling the desired output sequence. In this perspective, the linear polynomial in the consequent can be replaced by any continuous and differentiable function, $g(\mathit{x}(k)\hspace{0.17em})$, offering greater modeling potential. A direct implementation of this idea is the fuzzy polynomial systems or the CANFIS model, with a wide variety of applications [38,39,40]. Stemming from the fact that the inclusion of feedback connections has been proven to be an effective approach in the case of dynamic systems, several fuzzy models have been developed with recurrent components in their consequents [41,42,43].

In this perspective and taking into consideration that neural networks are universal approximators—particularly recurrent networks, which are effective in capturing the model’s dynamics [44,45,46]—the present approach integrates a recurrent neural network into the consequent part of each fuzzy rule. This type of fuzzy rule was initially proposed in [46] for modeling telecommunications data and later demonstrated effective results in electric load forecasting [47]. The operation of DNFWP, with a rule base comprising R fuzzy rules, is described as follows:

• The fuzzy rules have premise parts, which include the fuzzy sets that correspond to the inputs and are linked via the AND operator, as depicted in Equation (1). Therefore, the premise part of the l-th rule can be regarded as a m-dimensional hypercell:

$${\mu }_l(k)=f_{\mu }(\mathit{x}(k); {\mathit{m}}_l,{\mathit{\sigma }}_l)$$

(2)

composed by m single-dimensional Gaussian membership functions. The Gaussian type is selected:

$${\mu }_{A_{\hspace{0.17em}i}^{\hspace{0.17em}l}}(x_i(k))=\hspace{0.17em}\hspace{0.17em}\exp \left\{-{\displaystyle \frac{1}{2}}\cdot {\displaystyle \frac{{(x_i(k)-m_{li})}^2}{{\sigma }_{li}^2}}\right\}$$

(3)

Equation (2) describes the degree of fulfillment of the l-th rule, i.e., the degree to which the particular rule is activated by the input vector $\mathit{x}(k)$. The Gaussian membership function has two adaptable parameters: the mean value, $m_{li}$, and the standard deviation, ${\sigma }_{li}$, regarding the i-th input axis. Thus, each rule has two parameter vectors for the premise part: ${\mathit{m}}_l={[m_{l1},\dots ,m_{lm}]}^T$ and ${\mathit{\sigma }}_l={[{\sigma }_{l1},\dots ,{\sigma }_{lm}]}^T$. The degree of fulfillment is calculated by the algebraic product of the membership functions:

$${\mu }_l(k)={\displaystyle \prod _{i=1}^m{\mu }_{A_{\hspace{0.17em}i}^{\hspace{0.17em}l}}(x_i(k))}$$

(4)

The premise part is static and is fed with the input vector at the current time step, k.

• The consequent part of each rule is a three-layer neural network, with H neurons in the hidden layer.

  ➢

  The input layer is static and it simply forwards the weighted input vector to the hidden layer, as shown in Figure 1. The synaptic weights are noted as $w_{lji}^{(1)}$, $l=1,\dots ,R$, $i=1,\dots ,m$, $j=1,\dots ,H$.

  ➢

  The output layer is also static, containing a single neuron that receives input from the weighted outputs of the hidden layer’s neurons. The synaptic weights connecting the hidden layer with the output layer are noted as $w_{lj}^{(4)}$, $l=1,\dots ,R$, $j=1,\dots ,H$, while $w_l^{(5)}$ are the weights of the bias terms. The activation function of the output layer’s neuron is the hyperbolic tangent $f(q)={\displaystyle \frac{e^q-e^{-q}}{e^q+e^{-q}}}$. Noting $s_{li}(k)$ as the output of the j-th neuron of the hidden layer, the rule’s output is given by the following:

  $$O_l(k)=f\left({\displaystyle \sum _{j=1}^H\left[w_{lj}^{(4)}\cdot s_{lj}(k)\right]}+w_l^{(5)}\right)$$

  (5)

  ➢

  The hidden layer comprises neurons with unit output feedback connections, as presented in Figure 1. The particular type of internal feedback is called local output feedback and does not include external feedback connections of the rule’s output neuron or even of the model’s output (global output feedback). The bias terms are noted as $w_{lj}^{(3)}$, $l=1,\dots ,R$, $j=1,\dots ,H$, and the feedback weights are noted as $w_{lj}^{(2)}$. The outputs of the hidden neurons are extracted as follows:

  $$s_{lj}(k)=f\left({\displaystyle \sum _{i=1}^m\left[w_{lji}^{(1)}\cdot x_i(k)\right]}+w_{lj}^{(2)}\cdot s_{lj}(k-1)+w_{lj}^{(3)}\right)$$

  (6)

• The DNFWP’s output is the output of the defuzzification part. This part is static, since it averages the rules’ outputs, which are weighted by the values of the respective degrees of fulfillment:

$$y(k)=\hspace{0.17em}{\displaystyle \frac{{\displaystyle \sum _{l=1}^R{\mu }_l(k)\cdot O_l(k)}}{{\displaystyle \sum _{l=1}^R{\mu }_l(k)}}}$$

(7)

The block diagram of DNFWP is presented in Figure 2.

As can be seen from Figure 1 and Figure 2, the proposed architecture is a locally recurrent fuzzy neural network, belonging to a greater group of recurrent neurofuzzy models [48,49]. The particular architecture, however, is clearly distinguished from models with external output feedback [50] or recurrent premise parts [51], since in the former cases, the use of feedback connections interconnects the parts of the fuzzy system in multiple ways, thus altering the local-identification approach of the classic TSK model, as well as its interpretability. On the contrary, the proposed scheme retains the discrete connectivity of the rules, which can be regarded as local subsystems, cooperating via the defuzzification part. At each rule, the hypercell, formed by the respective fuzzy sets along each input axis, constitutes an operating region, in which the neural network of the consequent part aims to apply the potential of its recurrent nature in order to identify the temporal dependencies of the wind power time series. Since the fuzzy rules overlap each other through their premise parts, the recurrent consequent parts cooperate intrinsically, and provide an enhanced input–output mapping.

The nature and enhanced modeling capabilities of recurrent neural networks with local output feedback were explored in [44,45], following their initial application in a fuzzy system described in [52]. That implementation, however, was computationally intensive, since it included the following: (a) complex synapses in the recurrent neurons, feeding the input of the neuron with multiple delayed output values, and (b) dynamic neurons at the output layer of the antecedent part of the rule, as well. In contrast, [46] demonstrated that even simple neural networks using unit feedback in hidden neurons can effectively perform system identification tasks with much lower computational demands. Therefore, the proposed DNFWP aims to be regarded as an economical wind power predictor, especially when compared to Deep Learning-based predictors, as will be illustrated in the sequel.

3. The Model-Building Process

The model-building process aims at determining the structure of DNFWP, including (a) the number and kind of inputs, (b) the number of fuzzy rules and their operating regions, and (c) the size of the neural networks’ hidden layers. It also includes the consequent parameter tuning, where the weights of the neural networks form the consequent parameter vector.

As far as input selection is concerned, in the present work, wind power modeling based on a single input is attempted; therefore, there is no need to employ a feature selection mechanism. The issue of forming the fuzzy rule base is closely related to the aim of producing an economical predictor with a reduced size compared to the computationally intelligence-based systems existing in the literature. Therefore, the FCM clustering algorithm is applied [53,54] in order to identify the areas of discrete concentrations of data and, consequently, determine the most appropriate centers of the fuzzy sets pertaining to the premise parts of the rules. In this way, a moderate and concise rule base will be built. According to FCM, if R is the size of the rule base (i.e., the number of clusters), for a single-dimensional dataset (as is the case of DNFWP) containing N samples, each input sample, $x(k)$, does not belong to a single cluster but to all clusters, up to a certain degree, $u_l(k)$, where l is the cluster index:

$$u_l(k)={\displaystyle \frac{1}{{{\displaystyle \sum _{n=1}^R\left[\frac{\left|m_l-x(k)\right|}{\left|m_n-x(k)\right|}\right]}}^{\frac{2}{c-1}}}}$$

(8)

The cluster centers are extracted by the following equation:

$$m_l=\hspace{0.17em}{\displaystyle \frac{{\displaystyle \sum _{k=1}^N{\left(u_l(k)\right)}^c\cdot x(k)}}{{\displaystyle \sum _{k=1}^N{\left(u_l(k)\right)}^c}}}$$

(9)

where c > 1 is a scale parameter, controlling the fuzziness introduced to the algorithm.

The centers in Equation (9) constitute the mean values of the Gaussian-type membership function of the premise part’s fuzzy sets. The standard deviations are considered common to the rules comprising the fuzzy rule base and are derived as follows [55]:

$${\sigma }_l={\displaystyle \frac{{\displaystyle \sum _{k=1}^N{u}_l(k)\cdot {\left(m_l-x(k)\right)}^2}}{{\displaystyle \sum _{k=1}^N{u}_l(k)}}}$$

(10)

As mentioned above, FCM is based on the assumption that the number of clusters is determined in advance, meaning that the size of the fuzzy rule base should be set somehow. In order to do so, the Davies–Bouldin cluster validity index [56] is applied in order to provide an internal validation measure, taking into consideration both the compactness of the clusters, as well as the separation of each cluster from the others. The lower the value of the index the higher the compactness and the separation of the resulting clusters.

The aforementioned model-building scheme results in the determination of the size of the rule base and the derivation extraction of the values of the fitting parameters of the premise part’s fuzzy sets. As far as the size of the recurrent neural networks for the consequent part of the fuzzy rules is concerned, the number of hidden neurons is determined by trial and error, while the consequent parameters remain to be tuned.

Due to the feedback connections of the neural networks that form the consequent parts of the fuzzy rules, there exist temporal relations that need to be taken into consideration by the training algorithm. Therefore, consequent parameter tuning is performed using the training method DRPROP introduced in [46], which is based on Resilient Propagation (RPROP [57]) that was devised for static structures, but it has been modified to be applied to dynamic structures of fuzzy rules with internal feedback. As it will be shown in the sequel, it overcomes the problems inherent to standard gradient-based training algorithms.

Let w be any of the consequent synaptic weights in Equations (5) and (6), and E be the error function that consists of the errors of the actual value of wind power, $d(k)$, and the predictor’s output. The Mean Square error (MSE) is selected as E:

$$MSE={\displaystyle \frac{1}{N}}\cdot {\displaystyle \sum _{k=1}^N{\left(y(k)-d(k)\right)}^2}$$

(11)

${\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}$ and ${\displaystyle \frac{{\partial }^{+}E(t-1)}{\partial w}}$ are the partial derivatives of E with respect to w at the present and the previous iterations, t and t − 1, respectively. At each iteration, each weight is updated by an amount $\Delta (t)$, called weight update factor (WUF), using the following update scheme (Table 1):

Table 1. The DRPROP weight update scheme.

(a)  Initialize WUF for all the consequent weights:  $\Delta (1)={\Delta }_0$

Repeat

(b)  For each w, compute ${\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}$

(c)  For each w, update its step size:     (c.1) If ${\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}\cdot {\displaystyle \frac{{\partial }^{+}E(t-1)}{\partial w}}>0$

Then $\Delta (t)=\min \hspace{0.17em}\left\{\hspace{0.17em}{\eta }^{+}\cdot \Delta (t-1),{\Delta }_{\max }\right\}$

(c.2) Else if ${\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}\cdot {\displaystyle \frac{{\partial }^{+}E(t-1)}{\partial w}}<0$

Then

$\Delta (t)=\max \hspace{0.17em}\left\{\hspace{0.17em}{\eta }^{-}\cdot \Delta (t-1),{\Delta }_{\min }\right\}$ If

(c.3) Else $\Delta (t)=\Delta (t-1)$

$$w(t)=w(t-1)-sign\left({\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}\right)\cdot \Delta (t)$$ (12)

Until convergence

Table 1. The DRPROP weight update scheme.

(a)  Initialize WUF for all the consequent weights:  $\Delta (1)={\Delta }_0$

Repeat

(b)  For each w, compute ${\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}$

(c)  For each w, update its step size:     (c.1) If ${\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}\cdot {\displaystyle \frac{{\partial }^{+}E(t-1)}{\partial w}}>0$

Then $\Delta (t)=\min \hspace{0.17em}\left\{\hspace{0.17em}{\eta }^{+}\cdot \Delta (t-1),{\Delta }_{\max }\right\}$

(c.2) Else if ${\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}\cdot {\displaystyle \frac{{\partial }^{+}E(t-1)}{\partial w}}<0$

Then

$\Delta (t)=\max \hspace{0.17em}\left\{\hspace{0.17em}{\eta }^{-}\cdot \Delta (t-1),{\Delta }_{\min }\right\}$ If

(c.3) Else $\Delta (t)=\Delta (t-1)$

$$w(t)=w(t-1)-sign\left({\displaystyle \frac{{\partial }^{+}E(t)}{\partial w}}\right)\cdot \Delta (t)$$ (12)

Until convergence

It becomes evident that in DRPROP the size of the error gradient does not directly affect the weight update, thus the disadvantages of gradient-based methods tailored to the gradient values do not appear in the particular learning method. Moreover, each WUF is calculated based on the behavior of the sign of the error gradient, allowing a more rapid and efficient search in the weight space. In the initial stages of training, where the reduction in the error value is usually significant, the gradients do not change their sign in consecutive iterations, indicating that big updates are necessary. Step c1 is activated and WUF increases by $n^{+}\in \left[1.01, 1.2\right]$, thus accelerating learning. At later stages, c2 takes charge, decreasing WUF by $n^{-}\in \left[0.5, 0.9\right]$, with the aim of avoiding oscillations. Additionally, WUF has an upper bound of ${\Delta }_{\max }$ in order to avoid missing minima, and a lower bound of ${\Delta }_{\min }$, to prevent learning from becoming stalled.

Since the output layer of the network’ is static, the derivatives are easily extracted using the standard chain rule:

$${\displaystyle \frac{\partial E}{\partial w_{lj}^{(4)}}}={\displaystyle \frac{2}{N}}\cdot {\displaystyle \sum _{k=1}^N\left\{\left(y(k)-d(k)\right)\cdot \frac{{\mu }_l(k)\cdot f^{\prime }(.)\cdot s_{lj}(k)}{{\displaystyle \sum _{i=1}^R{\mu }_i(k)}}\right\}}$$

(13)

$${\displaystyle \frac{\partial E}{\partial w_l^{(5)}}}={\displaystyle \frac{2}{N}}\cdot {\displaystyle \sum _{k=1}^N\left\{\left(y(k)-\widehat{y}(k)\right)\cdot \frac{{\mu }_l(k)\cdot f^{\prime }(.)}{{\displaystyle \sum _{i=1}^R{\mu }_i(k)}}\right\}}$$

(14)

where $f^{\prime }(.)$ is the derivative of $O_l(k)$, with respect to its arguments.

Due to the feedback connections of the hidden layer’s neurons, their operation should be unfolded in time. Thus, the calculation of the ordered derivatives is performed [58], using Lagrange multipliers [59]. In the following, the ordered derivatives for the single-input case are calculated:

$${\displaystyle \frac{{\partial }^{+}E}{\partial w_{lj}^{(1)}}}={\displaystyle \sum _{k=1}^N{\lambda }_{lj}(k)\cdot f^{\prime }(.)\cdot x(k)}$$

(15)

$${\displaystyle \frac{{\partial }^{+}E}{\partial w_{lj}^{(2)}}}={\displaystyle \sum _{k=1}^N{\lambda }_{lj}(k)\cdot f^{\prime }(.)\cdot s_{lj}(k-1)}$$

(16)

$${\displaystyle \frac{{\partial }^{+}E}{\partial w_{lj}^{(3)}}}={\displaystyle \sum _{k=1}^N{\lambda }_{lj}(k)\cdot f^{\prime }(.)}$$

(17)

$${\lambda }_{lj}(k)={\lambda }_{lj}(k+1)\cdot f^{\prime }(.)\cdot w_{l.}^{(2)}+{\displaystyle \frac{2}{N}}\cdot {\displaystyle \sum _{k=1}^N\left\{\left(y(k)-d(k)\right)\cdot \frac{{\mu }_l(k)\cdot f^{\prime }(.)\cdot w_{lj}^{(4)}}{{\displaystyle \sum _{i=1}^R{\mu }_i(k)}}\right\}}$$

(18)

Equation (18) is a backward difference equation, subjected to the following boundary condition:

$${\lambda }_{lj}(N)={\displaystyle \frac{2}{N}}\cdot {\displaystyle \sum _{k=1}^N\left\{\left(y(N)-d(N)\right)\cdot \frac{{\mu }_l(N)\cdot f^{\prime }(.)\cdot w_{lj}^{(4)}}{{\displaystyle \sum _{i=1}^R{\mu }_i(N)}}\right\}}$$

(19)

Equation (19) is calculated first, and then the Lagrange multipliers are extracted via Equation (18) for $k=N-1,\dots ,1$. In Equations (15)–(19), $f^{\prime }(.)$ is the derivative of $s_{lj}(k)$, with respect to its arguments.

From the above, it is concluded that the parameter tuning of the consequent weights is an iterative procedure, where at each iteration, firstly, the updated WUFs are calculated through the DRPROP algorithm. Next, the new values of the weights are extracted by Equation (12), and the MSE is derived and compared to a predefined threshold. If the threshold is attained, the learning ends successfully; otherwise, the algorithm proceeds to the next iteration.

Table 2. Tunable parameters (weights) of DNFWP.

Parameter	Number
m	Rules
σ	Rules
$w_{lji}^{(1)}$	Rules ∙ Hidden neurons
$w_{lj}^{(2)}$	Rules ∙ Hidden neurons
$w_{lj}^{(3)}$	Rules ∙ Hidden neurons
$w_{lj}^{(4)}$	Rules ∙ Hidden neurons
$w_l^{(5)}$	Rules
Premise parameters	2∙Rules
Consequent parameters	Rules ∙ (4 ∙ Hidden neurons + 1)
Total	Rules ∙ (4 ∙ Hidden neurons + 3)

hosts the DNFWP’s parameters.

In conclusion, the process of building DNFWP can be divided into two phases, as shown in the flow-chart of Figure 3: in the first phase, a data cleansing scheme is applied to the available raw data, where missing and irregular values are handled and the whole dataset splits to a training dataset and a testing one. The first part of the second phase is dedicated to forming the rule base and calculating the fitting parameters of the fuzzy sets of the premise parts, while in the second part, the DRPROP algorithm tunes the weights of the consequent recurrent neural networks.

4. Experimental Results

4.1. The Wind Prediction Problem—Data Preprocessing Stage

DNFWP is applied to the problem of one-step ahead wind power prediction, based exclusively on the present value of wind power. Neither weather variables are used, nor a series of past values. The main scope of the present work is to assess the forecasting capabilities of a single-input predictor, where the intrinsic temporal relations of the wind power data are identified by the dynamic nature of the forecaster. This approach stems from the fact that the feature selection process is a cumbersome activity, and one cannot apply a systematic procedure in all circumstances. Statistical methods like auto-correlation and cross-correlation can contribute, along with the domain knowledge of experts; however, the suggested modeling approach is an attempt to overcome this issue and to provide a predictor with reduced complexity. Hence, the actual wind power at time-step k, $d\left(k\right)$ (kW), is the single input of DNFWP, which predicts the power at the next time-step k + 1, thus performing the mapping:

$$y\left(k+1\right)=f\left(d\left(k\right)\right)$$

(20)

The intrinsic temporal relations are attempted to be modeled through the recurrent connections of the neural networks that implement the rules’ consequent parts. The identification process aims at developing and training DNFWP such that it will be capable of performing the input–output data matching to an acceptable level of accuracy. The experiments were conducted on a desktop equipped with an INTEL i7-10700 processor, clocked at 2.9 GHz and having 16 GB RAM. The implementation of the predictors employed Jupyter Notebook 7.4.1 and Keras 3.0 libraries.

In both examples, data were scaled in the interval $\left[-0.8,0.8\right]$, allowing power values to lie within the neurons’ active region. However, denormalization took place prior to metrics’ calculation, such that the results would be consistent with previous findings that existed in the literature.

The model’s performance will be assessed based on the following evaluation metrics:

(a)

The Root Mean Squared Error (RMSE), which is the square root of Equation (11) and measures in a quadratic manner the difference between the actual and the predicted wind power values.

(b)

The Mean Absolute Error (MAE), which averages the absolute errors:

$$MAE={\displaystyle \frac{1}{N}}\hspace{0.17em}{\displaystyle \sum _{k=1}^N\left|y(k)-d(k)\right|}\hspace{0.17em}$$

(21)

(c)

The r-squared statistic, $R^2$, which indicates how well the regression line approximates the actual data:

$$R^2=1-{\displaystyle \frac{{\displaystyle \sum _{k=1}^N{\left(y(k)-d(k)\right)}^2}}{{\displaystyle \sum _{k=1}^N{\left(y(k)-\overline{y}\right)}^2}}}\hspace{0.17em}$$

(22)

where $\overline{y}$ is the mean value of the actual data.

4.2. Example 1: The Kaggle Dataset

The Kaggle dataset [60] consists of data collected between January 2018 and March 2020, with a time resolution of 10′. The full set comprises meteorological features, turbine and rotor features, wind direction, and wind speed. The dataset contained a considerable number of missing values, which needed to be taken care of. When missing values appeared sporadically, linear interpolation was applied to fill the blanks. In case of consecutive missing values, cubic spline interpolation was used to complete the dataset. Finally, in a small number of periods where the gap was extensive, these data were removed.

Two datasets are employed, adopting the partition used in [61]: the training dataset is used during the model-building process and contains historical data from 1 January to 30 March 2020, while the next three days constitute the testing dataset, used for examining the forecasting capabilities of DNFWP.

The fuzzy rule base was determined by applying the FCM algorithm. Several partitions of up to 18 clusters were examined. The selection of the most appropriate fuzzy rule base was extracted on the basis of the Davies–Bouldin index and its evolution versus the number of clusters is presented in Figure 4. Taking into consideration that the lowest index value corresponds to the best clustering, it becomes evident that, in terms of structural complexity, the optimal fuzzy rule base comprises six rules. Figure 5 hosts the partition of the input space; it is obvious that the six-rule fuzzy rule base adequately covers the entire input axis, with its left part being covered more densely, thus requiring more rules to operate in this subspace. It has been noticed that the same cluster distribution appears in different partitions. For example, when 18 clusters are extracted, 9 of them are placed at the left third of the input axis, proving that the FCM algorithm effectively identifies the areas where data are concentrated. It should be noted that once the fuzzy sets at the premise parts of the rules are extracted, their mean values and standard deviations are set.

As far as the formation of the fuzzy rules’ consequent parts is concerned, the number of hidden neurons was decided on the basis of (a) the influence of the size of the hidden layer on the predictor’s performance and (b) the aim to produce models of decreased complexity. Hidden layers with 2 to 10 neurons were examined and the most suitable size was set to 6. It should be mentioned that the higher the number of rules, the quicker the performance of the predictor’s stalled with respect to the neural networks’ size. Therefore, the best structure with 6 fuzzy rules and 6 hidden neurons at each consequent neural network comprises 12 fixed premise parameters and 150 adaptable consequent parameters, being tuned by the DRPORP training algorithm.

Table 3. Learning parameters of DRPROP ($n^{+}$ and $n^{-}$ are the increase and the attenuation factor, respectively, of WUF).

$n^{+}$	$n^{-}$	${\Delta }_0$	${\Delta }_{\min }$	${\Delta }_{\max }$
1.05	0.5	0.01	0.0001	0.5

hosts the learning parameters of the DRPROP algorithm. The maximum number of epochs is set to 1000.

The performance of DNFWP is initially evaluated in terms of the performance indices.

Table 4. DNFWP’s performance indices on training and testing datasets.

Number of Rules	Training Dataset			Testing Dataset
	RMSE (kW)	MAE (kW)	r-Squared	RMSE (kW)	MAE (kW)	r-Squared
2	31.65	22.31	0.996	37.28	25.62	0.996
3	25.76	17.87	0.997	29.97	21.18	0.997
6	24.24	17.14	0.998	29.92	20.76	0.997
8	23.59	17.28	0.998	30.85	21.72	0.997
10	24.69	17.79	0.998	30.38	20.48	0.997
12	25.98	18.21	0.997	36.01	23.73	0.996
14	25.26	18.60	0.997	34.90	23.32	0.996
16	26.00	18.26	0.997	37.08	24.44	0.996
18	26.23	17.77	0.997	37.09	23.13	0.996

summarizes the performance that various structures of the proposed model attained. From the results, the following is concluded:

• The three indices are complementary and provide a thorough insight into the predictor’s performance. The lower the values of RMSE and MAE are, the more accurate the predictions become, while a value of 1 in $R^2$ indicates the best possible performance.
• In all the DNFWP structures, DPROP provides effective training, with the case of two fuzzy rules appearing to be inferior compared to the rest of the structures.
• The most appropriate fuzzy rule base that was derived according to the Davies–Bouldin index attains the best results in all three indices and both in the training and the testing datasets, leading to the conclusion that the clustering algorithm significantly facilitates the model-building process and provides a concrete and rather small fuzzy rule base. Moreover, as the complexity of the predictors increases, the performance of the testing set deteriorates, indicating the occurrence of overfitting.
• The absence of weather features or past wind power values does not affect the forecasting performance. Therefore, the goal of letting the internal recurrent links reveal and model the temporal relations of the wind power time series seems to have been fulfilled.

The DNFWP’s performance is further assessed with a series of graphs hosted in Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10. The overall testing dataset and some parts of it, which have a special interest, are shown. The blue line corresponds to the actual wind power time series and the red line to the predictor’s output, respectively. As can be seen, the predictor is capable of identifying the nature of the time series, since it tracks the waveform effectively for most of the time steps (Figure 6), even when there are big (Figure 7) or quick fluctuations (Figure 8).

A special case exists when the power attains very low values, as shown in Figure 9. DNFWP accurately tracks the waveform but provides slightly higher positive values compared to the very small negative values existing in the dataset. Another case takes place when there are consecutive peaks, as depicted in Figure 10. DNFWP continues to identify the dynamics but, in some time steps, the output levels are lower than the actual ones. This discrepancy is smoothed when structures with considerably broader fuzzy rule bases are used since the particular subspaces can be covered more efficiently. Nevertheless, as mentioned above, the overall performance of more complex structures is not ameliorated.

As mentioned in Section 1, DNFWP has a classic structure of fuzzy systems, with the rules operating separately from each other and being interconnected through the defuzzification part. In this way, the operation of the predictor offers a degree of interpretability, since the input determines which rules will be fired, and, thus, which sub-models of DNFWP will operate each time. In order to highlight this characteristic of fuzzy modeling, three different input samples are selected, corresponding to different areas of the input space: 22.17 (datum no. 10), 489.12 (datum no. 53), and 1685 kW (datum no. 64), as shown in the upper left graph of Figure 11, were the input dataset is depicted. These samples belong to the clusters of small, medium, and large values and are marked with a red, blue, and black circle, respectively.

The membership functions of the premise parts of the fuzzy rule base are plotted in the upper left graph of Figure 11. As can be seen, the three input samples fire different combinations of the fuzzy rules, according to the part of the input space that each sample belongs to.

Therefore, the first input sample fires the first and second rules, with their degrees of fulfillment appearing in the last graph of Figure 1. The dominant rule is Rule 1, with Rule 2 contributing to the overall output in a ratio of approximately 1 to 4. According to Equation (7), the output of DNFWP is calculated as follows, indicating that the active rules cooperate at the defuzzification part:

$$y(10)=\hspace{0.17em}{\displaystyle \frac{0.99993\cdot O_1(10)+0.23809\cdot O_2(10)}{0.99993+0.23809}}$$

(23)

In the case of the second sample, which belongs to the middle area of the input space, Rules 2, 3, and 4 are fired, with Rule 3 being dominant. Now, the respective three sub-models cooperate to calculate the predictor’s output:

$$y(53)=\hspace{0.17em}{\displaystyle \frac{0.22348\cdot O_2(53)+0.97816\cdot O_3(53)+0.15366\cdot O_4(53)}{0.22348+0.97816+0.15366}}$$

(24)

Finally, the third sample, which is close to the upper boundary of the input space, practically activates solely the sixth rule, since the fifth being is activated by 6.6%:

$$y(64)=\hspace{0.17em}{\displaystyle \frac{0.965\cdot O_5(64)+0.066232\cdot O_6(64)}{0.965+0.066232}}$$

(25)

In the sequel, the proposed predictor is compared to four Machine Learning-based models, along with the Long Short-term Memory (LSTM) and the Gated Recurrent Unit (GRU), which represent Deep Learning-based approaches. The Machine Learning models are GPR [21], SVR [16], RF [17], and BS [22]. These four models provided the best performances when they were applied to the same prediction problem in [61]. All six competitors are dynamic models, since LSTM and GRU are inherently dynamic and the other four counterparts have input vectors consisting of time-lagged values of wind power of length 5 [61]. The SVR uses a linear kernel. The hyperparameters of GPR, BS, and RF have been optimized using the Bayesian Optimization algorithm [62] and the parameter set of Machine Learning models is presented in [61]. A series of setups of the LSTM and the GRU schemes scheme were examined, and are summarized in the following:

• Number of layers: two and three;
• Number of units per layer: 25, 50 and 60;
• Dropout: 0.1 to 0.4, with a step size of 0.05;
• Batch size: 16 to 80, with a step size of 8;
• Learning rate: 0.0001 to 0.005, with a step size of 0.0001.

The characteristics of the LSTM and GRU setups that provided the best results are given in

Table 5. Parameters of LSTM-2L/3L and GRU forecasters.

LSTM-2L/3L
Number of neurons	Optimizer	Dropout	Batch size	Learning rate	Activation function
50	Adam	0.2	24	0.001	Hyperbolic tangent
GRU
Number of neurons	Optimizer	Dropout	Batch size	Learning rate	Activation function
60	Adam	0.2	32	0.001	Hyperbolic tangent

, where LSTM-2L and LSTM-3L are LSTMs with two and three layers, respectively. The GRU contains three layers.

Table 6. Performance indices for Machine Learning-based and Computational Intelligence-based predictors for the Kaggle dataset.

Predictor	RMSE (kW)	MAE (kW)	r-Squared
BS	129.26	87.08	0.92
RF	133.70	90.41	0.91
GPR	129.06	84.94	0.92
LSTM-2L	29.83	22.11	0.997
LSTM-3L	28.54	22.54	0.998
GRU	32.07	22.33	0.996
DNFWP	29.92	20.76	0.997
SVR	130.89	85.36	0.91

hosts the performance indices of the comparing rivals. It becomes evident that LSTMs, GRU, and DNFWP outperform the Machine Learning counterparts in all indices, confirming the dominant opinion that the new era of intelligence is based on this kind of model. As far as the proposed predictor is concerned, it exhibits a similar performance with LSTMs and GRU, requiring, however, only a fraction of the enormous parameter set of Deep Learning forecasters, as clearly shown in

Table 7. Complexity of DNFWP and Deep Learning predictors.

Model	No. of Parameters
DNFWP	162
GRU	54,781
LSTM-2L	30,651
LSTM-3L	50,851

.

In summary, the application of DNFWP to the Kaggle dataset leads to the conclusion that, beyond achieving very low MAE and RMSE values, along with a r-squared value that is close to 1, the proposed predictor offers some additional benefits:

• It does not require a feature selection stage, either to include past wind power values or to incorporate seasonal and climate-related inputs, thus reducing preprocessing costs.
• It maintains a low model complexity, compared to Deep Learning-based forecasters.

4.3. Example 2: The Greek Dataset

The Greek dataset refers to renewable energy sources data that were collected by the European Network of Transmission Systems Operators for the forty-two countries participating in the centralized European energy market [63,64]. In particular, the dataset contains hourly data from 1 January 2017 to 31 December 2020, from 18 sources in Greece, leading to a size of 631,134 data. The aim of the task is to perform a one-hour-ahead prediction of wind power under the constraint of using solely the current value of wind power. The set was divided into a training set, hosting 70% of the available data, and a testing set with the remaining 30% of the data. Since the results of the first example clearly favored DNFWP and the Deep Learning-based forecasters, these models participate in the present experiment. The Greek dataset is two orders of magnitude greater than the Kaggle dataset, and the employment of the proposed predictor aims to explore its behavior when applied to larger datasets.

The FCM method and the Davies–Bouldin index provided a six-rule fuzzy rule base, as shown in Figure 12 and Figure 13. Once again, DNFWP has 162 tuning parameters (12 premises and 150 neural network weights). The learning parameter set hosted in Table 3 is used for operating DRPROP, and the maximum number of epochs is set to 1000 as well.

RNFMWP is compared to a LSTM and a GRU with three layers. These structures performed efficient forecasting in the previous example and their performance indices were similar to those of DNFWP. In this example, the models with 50 neurons in each layer attain the best performance. The parameters of the forecasters are given in Table 4 and the results are depicted in

Table 8. Performance indices and sizes of tuning parameter vectors for RNFM-WP and LSTM-3L for the Greek dataset.

Predictor	RMSE (MW)	MAE (MW)	r-Squared	No of Parameters
GRU	51.85	42.28	0.988	38,151
LSTM-3L	32.72	24.82	0.995	50,851
DNFWP	32.05	22.31	0.996	162

.

It is concluded from Table 8 that in the case of the larger dataset, RNFMWP and LSTM exhibit a similar performance with respect to RMSE and $R^2$, with the proposed predictor attained a slightly better MAE than its Deep Learning competitor. GRU has a decreased number of parameters compared to LSTM, but it was clearly outperformed by its rivals. With regard to the size of the parameter set, the conclusion drawn in the first example also holds, pointing out the reduced complexity of DNFWP.

The quality of time-series modeling is highlighted in Figure 14 and Figure 15, where two instances of the time series at different output levels are shown. The blue line corresponds to the actual wind power time series and the red line to the predictor’s output, respectively. Despite the different evolution of the time series in these time frames, RNFMWP is capable of tracking the wind power curve very closely.

5. Conclusions

A recurrent fuzzy neural model for wind load prediction has been presented. The predictor integrates fuzzy logic with recurrent neural networks in the consequent part of each fuzzy rule. Unlike conventional approaches, DNFWP employs a single-input framework and relies on its dynamic internal feedback structure to capture the temporal dependencies inherent in wind power data.

DNFWP was applied to two datasets—a smaller, high-resolution Kaggle dataset and a significantly larger Greek national dataset—demonstrating effective forecasting accuracy in both cases. The model’s performance was evaluated using three metrics (RMSE, MAE, and r-squared), and DNFWP delivered results comparable to or better than state-of-the-art Machine Learning and Deep Learning approaches, including LSTM and GRU models. Moreover, it achieved this while requiring significantly fewer parameters, thus offering an alternative with reduced complexity.

Key contributions and findings of this study include the following:

• The elimination of the feature selection stage. Due to the single-input model architecture, there is no feature selection stage, thus simplifying preprocessing without sacrificing prediction quality.
• Effective modeling of temporal dependencies using localized recurrent neural networks embedded in the consequent parts of fuzzy rules. The strong generalization performance on both moderate and large-scale datasets confirms the robustness of DNFWP across diverse real-world scenarios.
• An overall modular structure that retains the interpretability and transparency of fuzzy systems, while benefiting from the approximation power of neural networks.

In conclusion, DNFWP constitutes a promising tool for wind power prediction, particularly in applications requiring low computational complexity, interpretability, and reliable short-term forecasts without the need for extensive feature engineering. Future research could include the establishment of an automated model-building process using evolutionary computation algorithms and a multiple-step-ahead prediction strategy.