Forecasting of Day-Ahead Natural Gas Consumption Demand in Greece Using Adaptive Neuro-Fuzzy Inference System

: (1) Background: Forecasting of energy consumption demand is a crucial task linked directly with the economy of every country all over the world. Accurate natural gas consumption forecasting allows policy makers to formulate natural gas supply planning and apply the right strategic policies in this direction. In order to develop a real accurate natural gas (NG) prediction model for Greece, we examine the application of neuro-fuzzy models, which have recently shown signiﬁcant contribution in the energy domain. (2) Methods: The adaptive neuro-fuzzy inference system (ANFIS) is a ﬂexible and easy to use modeling method in the area of soft computing, integrating both neural networks and fuzzy logic principles. The present study aims to develop a proper ANFIS architecture for time series modeling and prediction of day-ahead natural gas demand. (3) Results: An e ﬃ cient and fast ANFIS architecture is built based on neuro-fuzzy exploration performance for energy demand prediction using historical data of natural gas consumption, achieving a high prediction accuracy. The best performing ANFIS method is also compared with other well-known artiﬁcial neural networks (ANNs), soft computing methods such as fuzzy cognitive map (FCM) and their hybrid combination architectures for natural gas prediction, reported in the literature, to further assess its prediction performance. The conducted analysis reveals that the mean absolute percentage error (MAPE) of the proposed ANFIS architecture results is less than 20% in almost all the examined Greek cities, outperforming ANNs, FCMs and their hybrid combination; and (4) Conclusions: The produced results reveal an improved prediction e ﬃ cacy of the proposed ANFIS-based approach for the examined natural gas case study in Greece, thus providing a fast and e ﬃ cient tool for utterly accurate predictions of future short-term natural gas demand.


Introduction
The increasing technological advancements and the rapid global population growth have led to a remarkable increase in energy consumption all over the world and especially in the developed and developing countries. After all, energy consumption is an index of a society's economical welfare and represents the economic development of a city or country [1]. Due to this unexpected boost in energy cases that the results are good and the ANFIS model performed better than regression, neural network and fuzzy hybrid systems. ANFIS was also used in [87] for short-term electricity demand forecasting, using weekly electricity load data, as well as in [88], to estimate possible improvement of electricity consumption. Also, for electricity load forecasting, ANFIS was used in [89] to highlight its superiority to the ANN model, while it was furthermore applied in the field of transportation, forecasting the corresponding energy demand for the years 2010 to 2030, in the country of Jordan, revealing the efficiency of the examined model. Another study regarding the energy domain, where the ANFIS model was applied, is that of [90]. A long-term prediction of oil consumption was studied, further examining the interrelationship between oil consumption and economic growth in Turkey, for the years 2012 to 2030.

Related Work on ANFIS in Natural Gas Consumption Forecasting
Casting a view on the literature that refers to NG consumption forecast, the authors came across only one study that devised solely an ANFIS model. Specifically, ANFIS was used in [91] in order to estimate the daily NG demand in Iran, which actually used an extremely small dataset of historical data for both testing and training (December 2007-June 2008. Models trained on a small dataset tend to overfit, which results in high variance and very high error on a test set, producing inaccurate results. In this case, the predicting error decreases monotonically with the size of training set [92]. The rest of the studies dealt with approaches that combine ANFIS with other methods. For example, in [21], statistical time series analysis along with ANN and ANFIS methods were applied in order to predict weekly NG consumption in Turkey. Moreover, an ANFIS-fuzzy data envelopment analysis (FDEA) was developed in [93] for long-term NG consumption forecasting and analysis. In this study, 104 ANFIS were constructed and tested and six models were proposed to forecast annual NG consumption. The same approach was proposed in [94] for accurate gas consumption estimation in South America with noisy inputs. An ANFIS-stochastic frontier analysis (ANFIS-SFA) approach was formulated in [95] for long-term NG consumption prediction and analysis. Three patterns of the hybrid ARIMA-ANFIS model were tested in [2] to predict the annual energy consumption in Iran, using a set of data like population, GDP, export and import. Finally, a hybrid model of adaptive neuro fuzzy inference system and computer simulation for the prediction of NG consumption was developed in [96].

Related Work on Fuzzy Cognitive Maps (FCMs) in Energy and Natural Gas Consumption Forecasting
Moreover, other soft computing techniques, like evolutionary fuzzy cognitive maps (FCMs) have been applied for the modeling and prediction of time series problems. The dynamic modeling structure of FCMs inheriting the learning capabilities of recurrent neural networks works properly for modeling and time series prediction. Salmeron and Froelich further investigated the applicability of FCMs in univariate time series prediction by proposing an FCM simplification approach with the removal of nodes and weights [97]. Regarding the task of multivariate time series prediction, Froelich and Salmeron proposed a nonlinear predictive model based on an evolutionary algorithm for learning fuzzy grey cognitive maps [98], while Papageorgiou et al. [99] and Poczeta et al. [100] applied a new type of evolutionary FCM enhanced with the Structure Optimization Genetic Algorithm (SOGA) in energy for electricity load forecasting. Through the SOGA algorithm, an FCM model can automatically be constructed by taking into consideration any available historical data. A two-stage prediction model for multivariate time series prediction, based on the efficient capabilities of evolutionary FCMs and enhanced by structure optimization algorithms and ANNs, was introduced in [101]. In the first stage of the prediction model, SOGA-FCM was applied for selecting the most significant concepts and defining the relationships between them. Next, that model was fed into the second stage to define the initial features and weights of the training ANN. This generic prediction approach was applied in four common prediction problems, one of which dealt with electric power consumption.
In [102], Poczeta and Papageorgiou conducted a preliminary study on implementing FCMs with ANNs for NG prediction, showing for the first time the capabilities of evolutionary FCMs in this domain.
Furthermore, the research team in [13] recently contacted a study for time series analysis devoted to NG demand prediction in three Greek cities, implementing an efficient ensemble forecasting approach through combining ANN, real coded genetic algorithm (RCGA)-FCM, SOGA-FCM, and hybrid FCM-ANN. In this research study, the advantageous features of intelligent methods through an ensemble to multivariate time series prediction in NG demand forecasting are explored.

Research Gap and the Novelty of This Study
Based on the reported literature survey and reviews [7,15], regarding the application of ANN-based and hybrid forecasting methods, a research gap has been identified in the field of NG day-ahead demand prediction. The observed gap mainly refers to the lack of model simplicity and flexibility, the insufficient exploration of certain modelling aspects, and inadequacy to cope with the inherent fuzziness in data handling. Most of these forecasting methods need a large dataset to be trained and a relatively large number of features to make accurate predictions. Furthermore, they are complex in their structure, time consuming and difficult to be used by non-experienced AI users. There has been hardly any research on successfully applying the ANFIS technique on the field of NG demand prediction, having performed a deep exploration process for determining the best model configuration, thus producing a highly accurate model with generalization capabilities.
Considering the aforementioned limitations, this work aims to fill the observed research gap and seeks to develop an easy to use, robust and flexible ANFIS model, which is at the same time fast, simple in structure and able to cope with fuzziness. More specifically, the proposed ANFIS architecture uses as model's inputs the most important and commonly used input variables according to the literature [7,15], such as day, month and daily average temperature, along with past NG consumption data. Moreover, a relatively large dataset was used for both testing and training the model, resulting in a systematic improvement of the model's predictive accuracy [92]. The current work pays great attention to the generalization of the proposed method and tries to properly evaluate the model's generalization capabilities in two ways: (i) by applying the model on a city level, where 10 different cities were properly examined, and (ii) by carrying out an exploration process, where 94 different models' configuration sets were examined for each one of the cities that participated in this research.
To sum up, the innovations offered in this paper are as follows, highlighting the contribution of this work to the research community: • The creation and demonstration of a simple, fast, robust ANFIS prediction tool to forecast NG demand using historical time series data. The proposed model is characterized by high flexibility, especially in large datasets, easiness of use and low execution time requirements.

•
The rigorous ANFIS fine-tuning for determining the most appropriate architecture for an enhanced prediction performance.

Aim of This Research Work
The motivation of this work is to propose an ANFIS-based forecasting approach with generalization capabilities for short-term (day-ahead) city-level NG prediction in Greek areas. Also, a comparative analysis is conducted, applying ANNs, evolutionary FCMs and hybrid combinations of them on the same dataset to show the capabilities of the proposed ANFIS architecture.
The objectives of the present paper regarding NG demand forecasting, are briefly summarized in the following: (a) To develop a robust ANFIS model to provide accurate short-term forecasts for a number of cities in Greece, using a relatively large dataset. At the same time, the authors perform model fine-tuning that can lead to high accuracy in most distribution points. The proposed model is characterized by high flexibility, easiness of use and low execution time requirements.
(b) To apply FCMs, ANNs and hybrid combinations of them to forecast NG demand in the same dataset, since these approaches have been proved as efficient techniques for NG demand forecasting according to the relevant literature. (c) To assess the performance of these soft computing methods in terms of prediction accuracy using well-known evaluation metrics. (d) To compare forecasting accuracy results of the proposed approach with those of the other soft computing and ANN methods that were examined, and finally decide on which model offers the best forecasting accuracy.
The outline of this paper is as follows. Section 2 describes the datasets of NG demand, collected for 10 Greek distribution points, as well as the proposed methodology of ANFIS for NG demand prediction using a well-defined set of evaluation performance metrics. Section 3 presents the results of the investigated ANFIS architectures. In the same section, a comparative analysis with other traditional neural networks and soft computing methods was performed for the same dataset. The discussion of results, which is also included in Section 3, presents the main outcomes of a meticulous ANFIS exploration analysis, along with ANFIS advantageous features. These are compared with ANNs and FCMs, and their overall contribution in NG forecasting is presented. Section 4 summarizes the paper, presenting future challenges in energy demand forecasting and highlighting further research directions.

Materials and Methods
This study aims to develop an ANFIS architecture capable of forecasting short-term NG consumption demand of the 10 main cities in Greece using the dataset that was provided by the Hellenic Gas Transmission System Operator S.A. (DESFA) [103]. The developed ANFIS approach deployed the aforementioned dataset along with other variables like the average daily temperature data for all the examined cities to accomplish forecasting. The results produced were further compared with those calculated by ANN and other soft computing techniques like FCM and hybrid-ANN to prove the prediction performance of the ANFIS prediction tool. Details on the dataset and its features, as well as the proposed methodology, are provided below. MATLAB M-file environment version 9.3.0.71 (R2017b) was used to program ANFIS networks and develop ANFIS models.

Dataset
The dataset covers ten different prediction datasets of historical data referring to ten cities all over Greece (Alexandroupoli, Athens, Drama, Karditsa, Larissa, Markopoulo, Serres, Thessaloniki, Trikala and Volos) and was linked to the values of gas demand for eight (8) previous years, in total. It should be mentioned that the time period for each dataset (city) was not the same in duration and did not correspond to the same years of data with all the other datasets collected. Table 1 depicts the duration in years that is linked to each dataset collected and used in this case study. The historical datasets for 15 Greek cities were initially provided by the NG grid company of Greece, DESFA, which is responsible for the operation, management, exploitation and development of the Greek NG system and its interconnections. However, the authors, after thoroughly reviewing the available datasets, decided to include only 10 out of 15 cities in their case study, since these datasets contained less outliers and missing values than the other 5 datasets that were finally rejected, for data consistency purposes. For the datasets that were finally included in this work, a preliminary preprocessing phase was performed, where the insignificant outliers were removed, and any missing values were substituted with the average real value of the previous two days demand. The real data that were used for ANFIS modeling, performance evaluation and comparison with other popular forecasting methods were then split into training and testing samples. For all cities, the last year of each dataset (from November 2017 up to October 2018) was devoted to testing, whereas the rest of the years were used for training the developed ANFIS model. In order to properly forecast day-ahead NG consumption demand of Greece, the proper number and type of input parameters should be selected. So, five factors were carefully considered as input parameters and the amount of one-day-ahead NG consumption demand of each distribution point was the output parameter. The prediction model was based on observations of past NG consumption, weather data, and calendar indicators, which are all among the most important input variables for prediction of NG consumption [15]. In particular, the dataset contains historical data of NG consumption of each city's distribution point, the daily average temperature of the area in Celsius degrees, a month indicator and a day indicator. As regards the previous NG consumption data, these are linked to two different input variables: demand of a day before and current day demand. The temperature data are obtained by the nearest to the distribution gas point meteorological station. Concerning the calendar indicators (month and day), they need to undergo certain data form preprocessing before their use. Specifically, two different input indicators need to be considered for each one of the two variables. We define k = 1,2, . . . , 12 as the month index (1 January, 2 February, . . . , 12 December) and l = 1,2, . . . , 7 as the day index (1 Monday, 2 Tuesday, . . . 7 Sunday). Following the coding procedure as presented in [104], the index for the month is scaled to the range [1/12, 1] in which the months of the year from January to December take successive values of the scaled index. That is, January has the value of 1/12 and December the value of 1. Similarly, the days of the week take successive values in the scaled range [1/7, 1], in which Monday and Sunday take the values of 1/7 and 1, respectively. All these parameters constituting the actual recorded data are briefly presented in Table 2. All data that compose the investigated dataset underwent a normalization process. This was necessary because all entries needed to have the same limited range of values so the model produces meaningful results [105].
The algorithm that was used for data normalization is the Min-Max, which scales the values of the dataset linearly over a specific range. As described in previous works [13,105], each variable was normalized to [0,1] before the forecasting model was applied. The normalized variable took its original value when the testing phase was implemented. Data normalization was carried out mathematically, as follows: where x (new) is the normalized value of the variable x, and x (min) and x (max) are, respectively, the minimum and maximum values of the concerned variable x.

Adaptive Neuro-Fuzzy Inference System (ANFIS)
The adaptive neuro-fuzzy inference system (ANFIS) uses an architecture that is based on both ANN and fuzzy logic principles and takes advantage of the benefits of both in a single framework. It can be described by the fuzzy "IF-THEN" rules from the Takagi and Sugeno (TS) type [106] as follows: where A i,k is the membership function associated with input variables x k and n is the number of inputs. A typical ANFIS network is a five-layer structure consisting of the fuzzy layer, the product layer, the normalized layer, the de-fuzzy layer and the total output layer [3,107,108], as depicted in Figure 1.

Adaptive Neuro-Fuzzy Inference System (ANFIS)
The adaptive neuro-fuzzy inference system (ANFIS) uses an architecture that is based on both ANN and fuzzy logic principles and takes advantage of the benefits of both in a single framework. It can be described by the fuzzy "IF-THEN" rules from the Takagi and Sugeno (TS) type [106] as follows: where , is the membership function associated with input variables and n is the number of inputs.
A typical ANFIS network is a five-layer structure consisting of the fuzzy layer, the product layer, the normalized layer, the de-fuzzy layer and the total output layer [3,107,108], as depicted in Figure 1. In the first layer, every node i represents a linguistic label and is described by the following membership function, as given in Equation (3).
where , is the membership function which is considered to be Gaussian and is described by the center ν and the spread σ.
In the second layer, the firing strength of the rule is computed using multiplicative operator, as presented in Equation (4). Firing strength is the weight degree of the IF-THEN rule and determines the shape of the output function for that rule.
In the third layer, the i-th node calculates the ratio of the i-th rule's firing strength to the sum of the firing strength of all rules. This is the normalization layer which normalizes the strength of all rules and the output of each node is given by Equation (5).
In the fourth layer, each node is an adaptive node with a function given by Equation (6). In this layer, each node calculates a linear function where its coefficients are adapted by using the error function of the multilayer feed-forward neural network.
In the fifth layer, there is only a fixed node indicated as the sum of the net outputs of the nodes in Layer 4. It computes the overall output as the sum of all incoming inputs and is expressed by Equation (7). In the first layer, every node i represents a linguistic label and is described by the following membership function, as given in Equation (3).
where A i,k is the membership function which is considered to be Gaussian and is described by the center ν and the spread σ.
In the second layer, the firing strength of the rule is computed using multiplicative operator, as presented in Equation (4). Firing strength is the weight degree of the IF-THEN rule and determines the shape of the output function for that rule.
In the third layer, the i-th node calculates the ratio of the i-th rule's firing strength to the sum of the firing strength of all rules. This is the normalization layer which normalizes the strength of all rules and the output of each node is given by Equation (5).
In the fourth layer, each node is an adaptive node with a function given by Equation (6). In this layer, each node calculates a linear function where its coefficients are adapted by using the error function of the multilayer feed-forward neural network.
In the fifth layer, there is only a fixed node indicated as the sum of the net outputs of the nodes in Layer 4. It computes the overall output as the sum of all incoming inputs and is expressed by Equation (7).
ANFIS uses a hybrid learning algorithm to train the model. The back-propagation algorithm is used to train the parameters in Layer 1, whereas a variation of least-squares approximation or back-propagation algorithm is used for training the parameters of the fourth layer [108,109].

Proposed ANFIS Architecture Applied in Natural Gas Consumption Forecasting
In order to develop an efficient ANFIS model for NG demand forecasting, the authors needed to follow a certain process regarding the design of model's architecture as well as an exploration process that will properly configure the input and training parameters of the examined model. Priority was given to the definition of the FIS architecture before the training of the network [110]. Among various fuzzy inference system (FIS) models, the Sugeno fuzzy model is the most widely used because of its higher interpretability and computational ability, that includes embedded optimal and adaptive techniques [111]. In order to create a fuzzy rule, the input space needs first to be divided. Two methods are used to divide space, comprised by input variables: the grid partitioning method and the subtractive clustering method. The main difference between these two functions refers to the way the partition of the input space is created.
In grid partitioning [109], the input space is divided into a grid-like structure without overlapping parts. Grid partitioning performs partitioning of the input space using all possible combinations of membership functions of each variable. This method is used when the number of input variables is small. For example, for 10 input variables and two membership functions for each input variable, then the input space is divided into 2 10 = 1024 specific areas, representing one rule for each specific area, and the total number of rules is 1024, which is a very complicated structure. Therefore, the grid partitioning method is mainly used when the number of input variables is small.
On the other hand, the subtractive clustering method divides the input space into appropriate clusters, even if the user does not specify their number. If the size of the cluster becomes small, then the number of clusters increases, thus increasing the number of fuzzy rules. A rule is created for each cluster, whereas different values for parameters, like range of influence, squash factor, accept ratio and reject ratio, need to be explored for determining an efficient architecture, which will keep the balance between the total number of ANFIS parameters and the total number of rules.
Considering the above specifications, the authors used the Grid partition option to define the FIS architecture due to its simplicity, less time-consuming performance as well as it can easily explore the number and type of membership function (MF). In this stage, the number and type of membership functions of each input variable, along with the rules and values of parameters that belong to these functions, were determined using the option of Grid partition.
When implementing an ANFIS architecture, researchers should have in mind that there is one main restriction: the number of input variables. When these are more than five, then the number of the IF-THEN rules and the computational time also increase, hindering ANFIS to model output with respect to inputs [110]. Thus, in this study, five variables were chosen as input parameters, i.e., month, day, temperature, demand of a day before and demand of current day. As described above, a day-ahead consumption demand was selected as the output variable whose value can be produced by choosing between the option of linear or constant type of MF.
Finding the most efficient ANFIS architecture is a demanding task and entails a rigorous exploration process. Since our concern focuses on the increment of network's accuracy and decrement of the errors, five necessary configurations should be considered in this direction: (i) the number of membership functions, (ii) types of MF (triangular, trapezoidal, bell-shaped, Gaussian and sigmoid), (iii) types of output MF (constant or linear), (iv) optimization methods (hybrid or back propagation) and (v) the number of epochs [112]. For the convenience of readers, these steps are visually represented in the flowchart in Figure 2. The aforementioned set of configurations needs to be deployed in order to generate FIS and next to train the ANFIS model. Accordingly, the dataset that included the five input variables (i.e., month, day, temperature, demand of a day before, demand of current day) was selected to determine the only output (day-ahead demand). Initially, the training dataset was loaded in the ANFIS tool, as shown in Figure 3a. The next step was the design of the neuro-fuzzy model using the option "Generate FIS". The Grid partition option was selected according to the description above (see Figure  3a). These two settings, concerning the fuzzy input variables along with their membership functions, are the most important parts to design the ANFIS. An example of selecting the number and type of MFs is illustrated in Figure 3b. The number and type of membership function were assigned to the input parameters following the trial-and-error approach. The different types of MF that are offered by the MATLAB ANFIS editor include the triangular, trapezoidal, generalized bell (Gbell), Gaussian curve, Gaussian combination, difference between two sigmoid functions and product of two sigmoid functions (see Figure 3b). Regarding the type of output MFs, in the Sugeno-type fuzzy system, there are two options: a constanttype conclusion or a linear-type conclusion function. In the case of linear function, the output y is defined as: where x1, x2, …, xn are the n inputs. In this case, ANFIS needs to define k0, k1, k2 up to kn, and it is very time consuming to efficiently calculate the outputs when a large number of parameters are considered. On the other hand, when a constant MF is selected, the algorithm needs to define only one parameter to provide a reliable forecasted value. Thus, the computational time is really low. The selected configuration also includes the hybrid optimization method, while the number of epochs selected to train the model was between 10 and 50. The hybrid optimization method uses the The aforementioned set of configurations needs to be deployed in order to generate FIS and next to train the ANFIS model. Accordingly, the dataset that included the five input variables (i.e., month, day, temperature, demand of a day before, demand of current day) was selected to determine the only output (day-ahead demand). Initially, the training dataset was loaded in the ANFIS tool, as shown in Figure 3a. The next step was the design of the neuro-fuzzy model using the option "Generate FIS". The Grid partition option was selected according to the description above (see Figure 3a). These two settings, concerning the fuzzy input variables along with their membership functions, are the most important parts to design the ANFIS. An example of selecting the number and type of MFs is illustrated in Figure 3b. The aforementioned set of configurations needs to be deployed in order to generate FIS and next to train the ANFIS model. Accordingly, the dataset that included the five input variables (i.e., month, day, temperature, demand of a day before, demand of current day) was selected to determine the only output (day-ahead demand). Initially, the training dataset was loaded in the ANFIS tool, as shown in Figure 3a. The next step was the design of the neuro-fuzzy model using the option "Generate FIS". The Grid partition option was selected according to the description above (see Figure  3a). These two settings, concerning the fuzzy input variables along with their membership functions, are the most important parts to design the ANFIS. An example of selecting the number and type of MFs is illustrated in Figure 3b. The number and type of membership function were assigned to the input parameters following the trial-and-error approach. The different types of MF that are offered by the MATLAB ANFIS editor include the triangular, trapezoidal, generalized bell (Gbell), Gaussian curve, Gaussian combination, difference between two sigmoid functions and product of two sigmoid functions (see Figure 3b). Regarding the type of output MFs, in the Sugeno-type fuzzy system, there are two options: a constanttype conclusion or a linear-type conclusion function. In the case of linear function, the output y is defined as: where x1, x2, …, xn are the n inputs. In this case, ANFIS needs to define k0, k1, k2 up to kn, and it is very time consuming to efficiently calculate the outputs when a large number of parameters are considered. On the other hand, when a constant MF is selected, the algorithm needs to define only one parameter to provide a reliable forecasted value. Thus, the computational time is really low. The selected configuration also includes the hybrid optimization method, while the number of epochs selected to train the model was between 10 and 50. The hybrid optimization method uses the The number and type of membership function were assigned to the input parameters following the trial-and-error approach. The different types of MF that are offered by the MATLAB ANFIS editor include the triangular, trapezoidal, generalized bell (Gbell), Gaussian curve, Gaussian combination, difference between two sigmoid functions and product of two sigmoid functions (see Figure 3b). Regarding the type of output MFs, in the Sugeno-type fuzzy system, there are two options: a constant-type conclusion or a linear-type conclusion function. In the case of linear function, the output y is defined as: where x 1 , x 2 , . . . , x n are the n inputs. In this case, ANFIS needs to define k 0 , k 1 , k 2 up to k n , and it is very time consuming to efficiently calculate the outputs when a large number of parameters are considered. On the other hand, when a constant MF is selected, the algorithm needs to define only one parameter to provide a reliable forecasted value. Thus, the computational time is really low. The selected configuration also includes the hybrid optimization method, while the number of epochs selected to train the model was between 10 and 50. The hybrid optimization method uses the back propagation learning algorithm for parameters associated with input MF and the least-square estimation algorithm for parameters associated with output MF; thus, it was selected as the most proper one [113]. Various sets of ANFIS configurations are presented in Table 3, regarding different sets of number for MFs, as considered by the authors of this work.  21  trimf  3-3-3-3-3  Constant  10  hybrid  22  trimf  3-3-3-3-3  Constant  10  backpropa  23  trapmf  3-3-3-3-3  Constant  10  hybrid  24  trapmf  3-3-3-3-3  Constant  10  backpropa  25  gbellmf  3-3-3-3-3  Constant  10  hybrid  26  gbellmf  3-3-3-3-3  Constant  10  backpropa  27  trimf  3-3-3-3-3  Constant  30  hybrid  28  trimf  3-3-3-3-3  Constant  50  hybrid  29  trapmf  3-3-3-3-3  Constant  30  hybrid  30  trapmf  3-3-3-3-3  Constant  50  hybrid  31  gbellmf  3-3-3-3-3  Constant  30  hybrid  32  gbellmf  3-3-3-3-3  Constant  50  hybrid  33  Regarding the output MFs, constant and linear MF were accordingly investigated after certain numbers of experiments conducted. From these experiments and for the linear output, it was observed that the number of rules increases significantly, as well as the computational time, even in the case of problems with a small number of inputs (see Table A1 in Appendix A). Thus, the linear type was not considered as an appropriate parameter of output MF since it is extremely time consuming. In this context, the trial-and-error approach was followed for the selection of the input-output type of MFs. Figure 4 illustrates an indicative ANFIS model, which was constructed with the following configuration set: 3-3-3-2-2, gbell MF, constant output MF, 10 epochs, hybrid.

Testing and Evaluation
The testing process for the ANFIS model was accomplished by using the testing data, which were completely unknown to the model. The predictor makes predictions on each day and finally compares the calculated predicted value with the real value. For example, considering the city of Volos, the predicted values that are illustrated in red in Figure 5 are compared with the real values (in blue color). In order to evaluate the prediction of NG demand, five well known and commonly used statistical indicators were introduced, i.e., mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and coefficient of determination (R 2 ). The mathematical equations of the statistical indicators are described below.

Testing and Evaluation
The testing process for the ANFIS model was accomplished by using the testing data, which were completely unknown to the model. The predictor makes predictions on each day and finally compares the calculated predicted value with the real value. For example, considering the city of Volos, the predicted values that are illustrated in red in Figure 5 are compared with the real values (in blue color).

Testing and Evaluation
The testing process for the ANFIS model was accomplished by using the testing data, which were completely unknown to the model. The predictor makes predictions on each day and finally compares the calculated predicted value with the real value. For example, considering the city of Volos, the predicted values that are illustrated in red in Figure 5 are compared with the real values (in blue color). In order to evaluate the prediction of NG demand, five well known and commonly used statistical indicators were introduced, i.e., mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and coefficient of determination (R 2 ). The mathematical equations of the statistical indicators are described below.
1. Mean squared error: 2. Root mean squared error: 3. Mean absolute error: In order to evaluate the prediction of NG demand, five well known and commonly used statistical indicators were introduced, i.e., mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE) and coefficient of determination (R 2 ). The mathematical equations of the statistical indicators are described below.

1.
Mean squared error: 2. Root mean squared error: 3. Mean absolute error: 4. Mean absolute percentage error: 5. Coefficient of determination: where X(t) is the forecasted value of the NG at the t-th iteration, and Z(t) is the actual value of the NG at the t-th iteration, t = 1, . . . , T, where T is the number of testing records.
Higher values of R 2 , i.e., closer to 1, mean better model performance and the regression line fits the data well. A coefficient of determination value of 1.0 points out that the regression curve fits the data perfectly.

Results
This section presents the exploration analysis results for the various ANFIS architectures as proposed in Section 2.2.2. Considering the steps proposed in Section 2.2, the initial dataset is split into training and testing. During the training process, the ANFIS model is designed for each one of the suggested configurations. After the training process of the ANFIS finishes, NG consumption demands for the next day (one day ahead prediction) are calculated from the generated FIS. The NG consumption results only for the city of Athens (as an indicative example) regarding all configurations tested, are presented in Table 4, whereas Table 5 gathers the best three results of NG consumption demand obtained from ANFIS for each of the 10 cities. To evaluate the performance of the models, the statistical indicator mean absolute percent error (MAPE) was used [91]. MAPE is a relative measurement, independent of scale, and it is the most common performance metric in time series forecasting, due to being reliable and valid [21].   The corresponding graphical representation of the results regarding the best three out of 47 total ANFIS architectures for the city of Athens is illustrated in Figure 6. Also, the best ANFIS model for each city can be found in Table 6, which provides the most reliable ANFIS architecture results for each city. All the results have been previously ranked, based on the minimum value of MAPE and subsequently the minimum values of MSE, RMSE and MAE. The priority was given to MAPE as one of the most crucial evaluation metrics, according to the literature [91,96], which was used in this study to compare various models obtained from ANFIS and other soft computing and neural networks methods. As a relative and easy to interpret measurement, MAPE is reliable, valid and independent of scale. The smaller the values of MAPE are, the closer the forecasted values are to the actual values.   As illustrated in Table 6, ANFIS models appear to perform best mostly when triangular MFs are used for the input variables: three MFs for the first three input variables (month, day of week and mean temperature) and two or three MFs for the other two input variables (daily demand for current day and one day before). Also, constant MFs are selected for the output variable and hybrid optimization method. The graphical representation of the best ANFIS models for each city is illustrated in Figure 7.   As illustrated in Table 6, ANFIS models appear to perform best mostly when triangular MFs are used for the input variables: three MFs for the first three input variables (month, day of week and mean temperature) and two or three MFs for the other two input variables (daily demand for current day and one day before). Also, constant MFs are selected for the output variable and hybrid optimization method. The graphical representation of the best ANFIS models for each city is illustrated in Figure 7.   As illustrated in Table 6, ANFIS models appear to perform best mostly when triangular MFs are used for the input variables: three MFs for the first three input variables (month, day of week and mean temperature) and two or three MFs for the other two input variables (daily demand for current day and one day before). Also, constant MFs are selected for the output variable and hybrid optimization method. The graphical representation of the best ANFIS models for each city is illustrated in Figure 7.  It is worth mentioning that all three most efficient ANFIS architectures with respect to MAPE values have triangular or gaussian MFs and 2-2-2-2-2 or 3-3-3-2-2 number of input MFs, whereas the output MF is constant and the learning algorithm is hybrid. In addition, the application of other MFs combinations does not seem to give results that could be on top of the list. Due to the limitation of the total number of parameters that should not exceed the number of training data pairs, the number of MFs was chosen based on the number of input parameters. Figure 8 shows the exponential increase It is worth mentioning that all three most efficient ANFIS architectures with respect to MAPE values have triangular or gaussian MFs and 2-2-2-2-2 or 3-3-3-2-2 number of input MFs, whereas the output MF is constant and the learning algorithm is hybrid. In addition, the application of other MFs combinations does not seem to give results that could be on top of the list. Due to the limitation of the total number of parameters that should not exceed the number of training data pairs, the number of MFs was chosen based on the number of input parameters. Figure 8 shows the exponential increase in the number of rules when the number of MFs increases, whereas

Comparison with ANNs, FCMs and Hybrid FCM-ANN
To further investigate the performance of the proposed ANFIS architectures, an extensive comparative analysis between the state of the art ANNs, soft computing methods of FCMs and their hybrid combination of FCMs with ANNs was performed.
The architecture of the analyzed ANN was a multilayer feed forward network with an input layer containing five inputs (month, day, temperature, demand of a day before, current demand), a hidden layer with 10 neurons, and an output layer with one output (a day-ahead demand prediction). The authors used the sigmoidal activation function in all layers and implemented Levenberg-Marquardt algorithm to train the network.
The soft computing method of fuzzy cognitive map with evolutionary learning capabilities, such as the real-coded genetic algorithm (RCGA-FCM) and structure optimization genetic algorithm (SOGA-FCM) [100], were used for time series modeling and prediction of day-ahead NG energy demand. For FCM learning, we implemented RCGA-FCM and SOGA-FCM. A short description on the applied evolutionary-based FCM approaches is given in Appendix B. The implementations of FCMs differ from ANFIS, even though they both belong to the soft computing family.
In this research study, we used the dynamic model type (Equation (B1)) which is found in Appendix B, with sigmoidal transformation function. FCMs learned with the use of RCGA and SOGA algorithm contain five concepts (month, day, temperature, demand of a day before, current demand) [114].
The applied hybrid approach for time series prediction is based on FCMs and ANNs and was previously proposed in [18,19]. It allows us to select the most significant concepts for FCM using SOGA. These concepts are used as the inputs for ANN. In the hybrid approach, we used artificial neural networks with an input layer with five inputs selected by the SOGA-FCM approach, a hidden layer with 10 neurons and an output layer with one output (one day-ahead demand prediction). Sigmoidal activation function and Levenberg-Marquardt learning algorithm were used. All the simulations for FCMs and hybrid FCM-ANN configurations were performed with the software tool ISEMK [115] which has been developed for time series forecasting purposes. An analytical description of FCM-based models and hybrid FCM-ANN can be found in [13,101,102,116], whereas they are used in this work only for comparison purposes.
In what follows, Table 7 gathers the results of the explored ANN and soft computing models, which are straightforward compared with our best performed ANFIS configuration, for each one out of the 10 cities, suggested in this research work. In Figure 9, three indicative graphs of the cities Alexandroupoli, Athens, and Drama are illustrated regarding the predicted values of NG demand for all the best proposed architectures. In Appendix A, the corresponding graphs of the rest of the cities are presented (see Figure A3).

Comparison with ANNs, FCMs and Hybrid FCM-ANN
To further investigate the performance of the proposed ANFIS architectures, an extensive comparative analysis between the state of the art ANNs, soft computing methods of FCMs and their hybrid combination of FCMs with ANNs was performed.
The architecture of the analyzed ANN was a multilayer feed forward network with an input layer containing five inputs (month, day, temperature, demand of a day before, current demand), a hidden layer with 10 neurons, and an output layer with one output (a day-ahead demand prediction). The authors used the sigmoidal activation function in all layers and implemented Levenberg-Marquardt algorithm to train the network.
The soft computing method of fuzzy cognitive map with evolutionary learning capabilities, such as the real-coded genetic algorithm (RCGA-FCM) and structure optimization genetic algorithm (SOGA-FCM) [100], were used for time series modeling and prediction of day-ahead NG energy demand. For FCM learning, we implemented RCGA-FCM and SOGA-FCM. A short description on the applied evolutionary-based FCM approaches is given in Appendix B. The implementations of FCMs differ from ANFIS, even though they both belong to the soft computing family.
In this research study, we used the dynamic model type (Equation (B1)) which is found in Appendix B, with sigmoidal transformation function. FCMs learned with the use of RCGA and SOGA algorithm contain five concepts (month, day, temperature, demand of a day before, current demand) [114].
The applied hybrid approach for time series prediction is based on FCMs and ANNs and was previously proposed in [18,19]. It allows us to select the most significant concepts for FCM using SOGA. These concepts are used as the inputs for ANN. In the hybrid approach, we used artificial neural networks with an input layer with five inputs selected by the SOGA-FCM approach, a hidden layer with 10 neurons and an output layer with one output (one day-ahead demand prediction). Sigmoidal activation function and Levenberg-Marquardt learning algorithm were used. All the simulations for FCMs and hybrid FCM-ANN configurations were performed with the software tool ISEMK [115] which has been developed for time series forecasting purposes. An analytical description of FCM-based models and hybrid FCM-ANN can be found in [13,101,102,116], whereas they are used in this work only for comparison purposes.
In what follows, Table 7 gathers the results of the explored ANN and soft computing models, which are straightforward compared with our best performed ANFIS configuration, for each one out of the 10 cities, suggested in this research work. In Figure 9, three indicative graphs of the cities Alexandroupoli, Athens, and Drama are illustrated regarding the predicted values of NG demand for all the best proposed architectures. In Appendix A, the corresponding graphs of the rest of the cities are presented (see Figure A1).  For a deeper analysis of the examined architectures (ANFIS, ANN, FCM, hybrid FCM-ANN), the authors report on further details regarding the parameters of each model used in this study. The ANN and FCM models were previously applied for NG demand prediction in several research works, such as those in [13,101,102,116]. The models were sufficiently described and the hyperparameters were properly configured to offer optimum performance of the investigated FCM models. Table 8 depicts the optimum parameters for all cities considering the neural and FCM evolutionary methods (ANN, RCGA-FCM, SOGA-FCM, Hybrid), compared with the proposed best performed ANFIS. The average running time is also presented in Table 8, which was calculated for each soft computing architecture for all models. It is worth mentioning that we have conducted a rigorous exploratory analysis for all the investigated neuro-fuzzy, soft computing techniques and ANNs, with different parameters, for training and model optimization, to reach the highest prediction accuracy with respect to the evaluation metrics. For a deeper analysis of the examined architectures (ANFIS, ANN, FCM, hybrid FCM-ANN), the authors report on further details regarding the parameters of each model used in this study. The ANN and FCM models were previously applied for NG demand prediction in several research works, such as those in [13,101,102,116]. The models were sufficiently described and the hyperparameters were properly configured to offer optimum performance of the investigated FCM models. Table 8 depicts the optimum parameters for all cities considering the neural and FCM evolutionary methods (ANN, RCGA-FCM, SOGA-FCM, Hybrid), compared with the proposed best performed ANFIS. The average running time is also presented in Table 8, which was calculated for each soft computing architecture for all models. It is worth mentioning that we have conducted a rigorous exploratory analysis for all the investigated neuro-fuzzy, soft computing techniques and ANNs, with different parameters, for training and model optimization, to reach the highest prediction accuracy with respect to the evaluation metrics.

Discussion of Results
In this work, several ANFIS architectures were investigated, with respect to all the variables that were carefully determined in the developed model as reported in Section 2.1 and after different sets of model configurations were tested. However, only one ANFIS architecture reached the optimum performance, in terms of forecasting accuracy, considering the minimum value of MAPE and subsequently the minimum values of MSE, RMSE and MAE values produced. In particular, it emerged that the optimum ANFIS configuration is 2-2-2-2-2 with triangular MFs for input variables, which produces the most simple (concerning the number of rules), fast (see Table A2 in Appendix A) and accurate model for this energy forecasting problem. In general, it is observed that the best results are produced from the combination of triangular or gaussian MFs regarding the input variables, and the constant MFs regarding the output layers.
To further discuss the results produced and to show the effectiveness of the proposed forecasting methodology of ANFIS, the authors conducted a comparative analysis regarding the forecasting performance between the proposed technique, and other ANN and soft computing methods too, such as FCM, which were reported in the literature and have already been applied in the specific domain. The MAPE criterion was used to compare various models from ANN, evolutionary FCM, Hybrid FCM-ANN and ANFIS. The results are given in Table 7 Considering the same dataset linked to only three cities (Athens, Thessaloniki, and Larissa) out of the ten that participated in our study, a day-ahead NG consumption prediction was investigated in [117], applying ANN and LSTM approaches and in [102], implementing the SOGA-FCM method and a hybrid combination of it. Furthermore, an ensemble FCM prediction methodology concerning the same dataset was presented in [13], in which a recent soft computing technique for time series forecasting, using evolutionary fuzzy cognitive maps and their ensemble combination was compared to ANNs, as benchmark forecasting methods. These methods and their results in terms of MSE and MAE values for three benchmark cities are all gathered in the following table and certain results can be concluded. The main reason for selecting the statistical indicators MSE and MAE in the following figure is to accomplish a straightforward comparison with the results published in previous works.
In Figure 10, it can be noted that all methods achieve high accuracy in NG consumption predictions, using the same dataset. The best ANFIS approach seems to excel over the ensemble and hybrid methods. Consequently, the proposed ANFIS architecture, which handles the fuzziness of data more efficiently, outperforms all the other examined methods in most cases, with a rather remarkable difference. ANFIS is less time consuming and more flexible than ANN, and as it employs fuzzy rules and membership functions incorporating with real-world systems, it can be used as alternate method to ANN forecasting. Energies 2020, 13, x FOR PEER REVIEW 21 of 32 Figure 10. Comparison of results between machine learning and soft computing methods for three benchmark cities.
It is observed that the proposed method exhibits better or similar performance to other wellknown ANN, FCM or hybrid FCM-ANN architectures for the ten cities under investigation. The produced results highlight the significance and superiority of neuro-fuzzy methods over the other examined methods in terms of prediction accuracy, when they deal with time series forecasting problems in energy. This is in accordance with the main advantageous features of ANFIS models, which are their ability to capture the nonlinear structure of a process, their adaptation capability, and fast training characteristics. As reported in the literature, ANFIS models are able to cope with the uncertainty and fuzziness that characterize the energy domain [118,119] when other intelligent methods cannot tackle them.
The main outcomes of this study can be summarized as follows: i. The proposed ANFIS method exhibits the best performance when certain configuration settings are selected for the examined datasets which are linked to ten cities of Greece. The authors concluded that a certain configuration is best for the examined ANFIS model, after having conducted a number of experiments and following a trial-and error approach. The best ANFIS model is based on a distinct architecture that features a 2-2-2-2-2 triangular or gaussian MF. ii.
The proposed ANFIS architecture is superior to the four benchmark and well-known ANN and FCM methods (ANN, SOGA-FCM, RCGA-FCM, Hybrid FCM-ANN), which have been efficiently used in NG consumption forecasting. The results presented in Table 7, which gathers various error indicators and the R 2 , as prediction accuracy indices for all five architectures, show that the best ANFIS model holds the best prediction accuracy among all the methods that were included in this comparative analysis. iii.
The proposed ANFIS model shows significant capacity when applied to forecasting NG demand, since it exhibits better performance (see Table 7) with less running time (see Table 8) and more flexibility to handle fuzziness than other well-known ANN and FCM architectures.

Conclusions
This study proposes the ANFIS method to predict short-term demand of NG consumption. This approach is applied on the Greek territory and uses 10 different datasets provided by DESFA, that regard previous energy consumption historical data, for ten main cities. To decide the model's proper architecture, the authors follow an exploration process regarding the best configuration of input and training parameters. The best ANFIS model is then compared to other well-known ANN and soft computing models that are commonly used for energy demand prediction purposes. The ANFIS method demonstrates significant performance in the field of energy demand prediction, outweighing the traditional ANN and FCM architectures. In addition, the running time of the proposed architecture is much less than those of other examined models, making it the right decision for dayahead demand forecasting of NG. The findings of this study reveal that the highest forecasting It is observed that the proposed method exhibits better or similar performance to other well-known ANN, FCM or hybrid FCM-ANN architectures for the ten cities under investigation. The produced results highlight the significance and superiority of neuro-fuzzy methods over the other examined methods in terms of prediction accuracy, when they deal with time series forecasting problems in energy. This is in accordance with the main advantageous features of ANFIS models, which are their ability to capture the nonlinear structure of a process, their adaptation capability, and fast training characteristics. As reported in the literature, ANFIS models are able to cope with the uncertainty and fuzziness that characterize the energy domain [118,119] when other intelligent methods cannot tackle them.
The main outcomes of this study can be summarized as follows: i. The proposed ANFIS method exhibits the best performance when certain configuration settings are selected for the examined datasets which are linked to ten cities of Greece. The authors concluded that a certain configuration is best for the examined ANFIS model, after having conducted a number of experiments and following a trial-and error approach. The best ANFIS model is based on a distinct architecture that features a 2-2-2-2-2 triangular or gaussian MF. ii.
The proposed ANFIS architecture is superior to the four benchmark and well-known ANN and FCM methods (ANN, SOGA-FCM, RCGA-FCM, Hybrid FCM-ANN), which have been efficiently used in NG consumption forecasting. The results presented in Table 7, which gathers various error indicators and the R 2 , as prediction accuracy indices for all five architectures, show that the best ANFIS model holds the best prediction accuracy among all the methods that were included in this comparative analysis. iii.
The proposed ANFIS model shows significant capacity when applied to forecasting NG demand, since it exhibits better performance (see Table 7) with less running time (see Table 8) and more flexibility to handle fuzziness than other well-known ANN and FCM architectures.

Conclusions
This study proposes the ANFIS method to predict short-term demand of NG consumption. This approach is applied on the Greek territory and uses 10 different datasets provided by DESFA, that regard previous energy consumption historical data, for ten main cities. To decide the model's proper architecture, the authors follow an exploration process regarding the best configuration of input and training parameters. The best ANFIS model is then compared to other well-known ANN and soft computing models that are commonly used for energy demand prediction purposes. The ANFIS method demonstrates significant performance in the field of energy demand prediction, outweighing the traditional ANN and FCM architectures. In addition, the running time of the proposed architecture is much less than those of other examined models, making it the right decision for day-ahead demand forecasting of NG. The findings of this study reveal that the highest forecasting accuracy emerged when the same model configuration was used for most of the cities, highlighting the generalization capabilities of the proposed architecture.
This work can be widely used in short-term demand forecasting for other countries too, with the same or similar input parameters, and can be also useful especially for distribution operators, providing them with the ability to make long-term planning decisions and apply the correct strategic policies in this direction. Following the literature, the ANFIS approach can be applied in various other domains such as medicine, environmental modelling, various energy systems, like solar and wind, as well as other engineering applications. As can be seen, the ANFIS application area is wide, and as regards the energy sector, this method finds great applicability due to its high prediction accuracy, robustness, and easiness to use.
The results show that the proposed algorithm, which was proven to be efficient, fast and robust, can be adopted by regulatory authorities and decision makers to perform rigorous forecasting of natural gas demand for the respective case cities and other cities in Greece too. The investigated approach is an accurate estimation method as it makes efficient short-term predictions in natural gas demand, showing minor deviations between the real and the predicting values. Since short-term natural gas forecasting is mostly used for the timely reservation of transport, storage capacity optimization, timely purchase of natural gas deliveries and capacity allocation, this method becomes critical to determine the energy policy for Greece and the wider area too, having overall a positive impact in natural gas consumption.
Future work is oriented in developing more advanced neuro-fuzzy models providing explainability and transparency in prediction tasks in diverse research domains, in order to evaluate the generalization capabilities of this approach. Furthermore, new forecast combination architectures of efficient deep learning and regularized recurrent neural networks for time series modelling and prediction in the energy sector will be investigated.     2-3-3-3  Constant  10  Hybrid  108  11 s  trimf, trapmf, gbell, gauss, pim, sigm  3-3-3-2-2  Constant  10  Hybrid  108  19 s  trimf, trapmf, gbell  3-3-3-3-3  Constant  10  Hybrid  243  68  where Xi(t) is the value of the i-th concept at the t-th iteration, wj,i is the weight of the causal relationship between concepts Xj and Xi taking values from the range [−1,1], t is discrete time, i,j = 1, 2, …, n, n is the number of concepts, and F is the transformation function normalizing the factor values to the range [0,1] or [−1,1]. Fuzzy cognitive maps can be constructed based on expert knowledge or with the use of machine learning algorithms. The aim of fuzzy cognitive map learning is to determine the weights of the causal relationships between concepts on the basis of available time series.
An effective method for fuzzy cognitive map learning is the real-coded genetic algorithm (RCGA) [100]. RCGA defines each individual in the population based on a floating-point vector containing the causal relationships. Each individual is decoded into a candidate map and evaluated with the use of proper fitness function. We used the following fitness function: where a is a parameter, l is the number of generations, l = 1, …, L, L is the maximum number of generations, p is the number of individuals, p = 1, …, P, P is the population size, and is the learning error that can be in the following form: Fuzzy cognitive maps (FCMs) are an effective tool for modeling and predicting time series. The structure of the FCM model is based on a directed graph, the nodes of which denote concepts significant for the analyzed problem, and the links are the causal relationships. Values of concepts can change over time according to the adopted dynamics model, for example the nonlinear dynamics model: where X i (t) is the value of the i-th concept at the t-th iteration, w j,i is the weight of the causal relationship between concepts X j and X i taking values from the range [−1,1], t is discrete time, i,j = 1, 2, . . . , n, n is the number of concepts, and F is the transformation function normalizing the factor values to the range [0,1] or [−1,1]. Fuzzy cognitive maps can be constructed based on expert knowledge or with the use of machine learning algorithms. The aim of fuzzy cognitive map learning is to determine the weights of the causal relationships between concepts on the basis of available time series.
An effective method for fuzzy cognitive map learning is the real-coded genetic algorithm (RCGA) [100]. RCGA defines each individual in the population based on a floating-point vector containing the causal relationships. Each individual is decoded into a candidate map and evaluated with the use of proper fitness function. We used the following fitness function: f itness p (erorr l ) = 1 a·erorr l + 1 (A2) where a is a parameter, l is the number of generations, l = 1, . . . , L, L is the maximum number of generations, p is the number of individuals, p = 1, . . . , P, P is the population size, and erorr l is the learning error that can be in the following form: where X(t) is the predicted value of the decision concept at the t-th iteration, Z(t) is the real normalized value of the decision concept at the t-th iteration, t = 1, . . . , T, and T is the number of learning records. Another way of learning fuzzy cognitive maps is the structure optimization genetic algorithm (SOGA) [100,120]. This allows one to simplify the structure of the FCM model by selecting the most significant concepts and causal relationships during the learning process. In this approach, the fitness function is based on the modified learning error including an additional penalty for highly complexity of the candidate fuzzy cognitive map, understood as a large number of concepts and non-zero relationships, described as follows: erorr l = erorr l + b 1 n r n 2 erorr l + b 2 n c n erorr l (A4) where b 1 , b 2 are the learning parameters, n c is the number of the concepts in the candidate FCM model, n r is the number of the non-zero relationships between concepts, n is the number of all possible concepts, and erorr l is the learning error type (Equation (A3)).
In this paper, we also used the hybrid approach for time series prediction based on fuzzy cognitive maps with the structure optimization genetic algorithm and artificial neural networks [101]. In the first stage of this approach, the most important concepts are selected with the use of FCMs and the SOGA algorithm. In the second stage, these concepts are used as inputs for the artificial neural network in order to increase the prediction accuracy. The above algorithms have been implemented in the developed ISEMK system [101].