Measuring Country Sustainability Performance Using Ensembles of Neuro-Fuzzy Technique

: Global warming is one of the most important challenges nowadays. Sustainability practices and technologies have been proven to signiﬁcantly reduce the amount of energy consumed and incur economic savings. Sustainability assessment tools and methods have been developed to support decision makers in evaluating the developments in sustainable technology. Several sustainability assessment tools and methods have been developed by fuzzy logic and neural network machine learning techniques. However, a combination of neural network and fuzzy logic, neuro-fuzzy, and the ensemble learning of this technique has been rarely explored when developing sustainability assessment methods. In addition, most of the methods developed in the literature solely rely on fuzzy logic. The main shortcoming of solely using the fuzzy logic rule-based technique is that it cannot automatically learn from the data. This problem of fuzzy logic has been solved by the use of neural networks in many real-world problems. The combination of these two techniques will take the advantages of both to precisely predict the output of a system. In addition, combining the outputs of several predictors can result in an improved accuracy in complex systems. This study accordingly aims to propose an accurate method for measuring countries’ sustainability performance using a set of real-world data of the sustainability indicators. The adaptive neuro-fuzzy inference system (ANFIS) technique was used for discovering the fuzzy rules from data from 128 countries, and ensemble learning was used for measuring the countries’ sustainability performance. The proposed method aims to provide the country rankings in term of sustainability. The results of this research show that the method has potential to be effectively implemented as a decision-making tool for measuring countries’ sustainability performance.


Introduction
Sustainability is defined the use of resources to "meets the needs of the present without compromising the ability of future generations to meet their own needs" [1].Environmental and societal sustainability have become important issues nowadays.Sustainable development has attracted increasing attention from organizations in different scopes [2,3].Sustainability assessment tools have been developed to support the decision makers in evaluating sustainable technology developments, and are nowadays a distinct and important realm of research.According to Devuyst, et al. [4] cited by Tan et al. [5], sustainability assessment is defined as "... a tool that can help decision makers and policy makers decide which actions they should or should not take in an attempt to make society more sustainable".The sustainability development has been based on three major focus areas: the economy, society, and the environment [6,7].These dimensions have been more important for sustainability, and the majority of research studies have been conducted on these dimensions.
Machine learning techniques have been successful in developing the decision support systems [8] and prediction learning tools [9].Sustainability assessment tools commonly rely on the indexes and indicators that are more suitable for the evaluation of the sustainability in various domains.Accordingly, based on these indexes and indicators, several research studies have been conducted to develop sophisticated methods for sustainability assessment.Arushanyan et al. [10] developed a methodological framework for the qualitative sustainability assessment of future scenarios with transformative changes.Their framework suggested qualitative assessment with a consumption perspective and a life cycle approach.Houshyar et al. [11] used a combination of geographic information system (GIS), analytical hierarchy process (AHP), and fuzzy for the assessment of the sustainability of silage corn production in Fars province.Cavallaro [12] used Takagi-Sugeno fuzzy inference in developing a synthetic index for the sustainability assessment of production of the biomass for energy purposes.Zhao and Li [13] proposed a hybrid framework for the evaluation of the Strong Smart Grid (SSG) performance using AHP and technique for order of preference by similarity to ideal solution (TOPSIS).Phillis et al. [14] developed SAFE (sustainability assessment by fuzzy evaluation), which is a measuring scheme of sustainability as well as a decision support tool for policymaking that uses the fuzzy rule-based system to measure the overall sustainability of a country.This decision support tool is based on basic indicators of sustainability.The SAFE model is mainly based on two broad components: human sustainability (HUMS) and ecological sustainability (ECOS).Tan, Shuai, Jiao, and Shen [5] proposed an approach using an adaptive neuro-fuzzy inference system (ANFIS) technique for assessing countries' sustainability levels.Their method was based on the dimensions and indications that have been used in SAFE, which was developed by Phillis and Andriantiatsaholiniaina [15] and Phillis, Grigoroudis, and Kouikoglou [14].Our method is also developed based on these two dimensions of sustainability, and includes their basic indicators in each dimension.
From the literature, it was found that there has been no implementation of the ensemble learning of the ANFIS technique in developing the sustainability assessment tools.It has been shown that solely using a single inductive learning method cannot achieve a generalization performance in all of the possible prediction tasks [16].In addition, the accuracy of using supervised learning methods more depends on the nature of the problem investigated and observed data.Despite this consideration, ensemble learning has proved to have an excellent performance in many prediction tasks of real-world applications [17].In the ensemble learning paradigm, several predictors are combined to construct the learning models [18].Therefore, this will result in improved prediction accuracy in the problem under investigation.Specifically, the generalization performance of the ensemble learning has been proved to be much better than a single individual ensemble member.In fact, these improvements in performance and accuracy are because of the combination of accurate predictors, whose errors are complementary.In this research, multiple components of the ANFIS learner are trained for measuring countries' levels of sustainability assessment.Accordingly, integration by an average approach was applied for final ensemble decision [19].As a straightforward approach, the final decision by integration by the average approach is obtained from the sum of the results generated by each ANFIS model divided by the number of ANFIS models used for the prediction task.
Accordingly, in this research, attempts shall be made to develop a new method using ensembles of the ANFIS technique for measuring the sustainability performance of countries.It is believed that the combination of these techniques will help improve the prediction accuracy of previous methods in assessing the sustainability performance.
Our study at hand is organized as follows: Section 2 focuses on literature review.In Section 3, the ANFIS technique and its ensemble are explained.In Section 4, the method is evaluated.The discussions and recommendations are provided in Section 5. Finally, we conclude our work in Section 6.

Sustainability and Fuzzy Neural Networks
In recent years, the concept of sustainability has broadly been acknowledged as an interesting topic for numerous disciplines.Accordingly, several techniques, models, and approaches have been developed and extended for the assessment of sustainability performance.Previous studies have categorized these techniques into five different groups including: data environment analysis, systems dynamics, fuzzy sets theories, comprehensive evaluation, and ecological footprint [5].Among these groups, fuzzy sets theory has widely been used for the assessment of various applications of sustainable development.There are some advantages of integrating the neural networks with fuzzy sets theory, including its relative ease of use, its capability to approximate any function regardless of its linearity, and its applicability for complex/abstract problems.
The ANFIS technique for fuzzy rules induction and fuzzy membership function was introduced by Jang [20], and it has been a robust supervised machine learning technique for prediction tasks.This technique is a specific kind of neuro-fuzzy system that takes the advantages of fuzzy logic [21] and neural networks to automatically learn from the data.There are many studies that have examined the ANFIS and neural network techniques.Some of these studies are introduced as follows.The ANFIS approach has extensively been used in different application areas of energy and sustainability.Tan, Shuai, Jiao, and Shen [5] used the ANFIS technique for assessing the sustainability of 128 countries.The outcomes of this paper indicated that ANFIS is an efficient technique for evaluating the sustainability performance of countries.Halkijevic et al. [22] proposed a new model, quasi-strong sustainability, and used the ANFIS model for the assessment of water supply sustainability.The findings of this paper showed that the combination of fuzzy logic and neural networks was an appropriate approach for the assessment of water supply sustainability.Cavallaro [23] used the neural network for a load application prediction of a small electric grid.The outcomes of this paper demonstrated that the proposed model was effective and had better performance compared with the traditional statistical approaches.Ruben et al. [24] presented a conceptual model for assessing the performance of the sustainable systems of automotive manufacturing firms by using the ANFIS method.The outcomes of this paper showed that the presented performance model has the ability to assess lean performance in the manufacturing firms.Tan, Shuai, Jiao, and Shen [5] introduced the ANFIS method for the assessment of urban sustainability in 185 cities in China.The outcomes of this paper indicated that the ANFIS method is suitable for evaluating urban sustainability.Pousinho et al. [25] introduced a novel hybrid model to estimate short-term wind power.The results of this paper showed that the proposed model enhanced forecasting accuracy and outperformed the other models.Altin and Sefa [26] proposed a new model for the implementation of dSPACE by using the ANFIS model based on grid interactive voltage source inverter.The outcomes of this paper indicated that the presented inverter is faster to respond to the reference variations and lower steady-state error than a PI controller.Shamshirband et al. [27] applied the ANFIS model to achieve the maximal net in a wind farm.The finding of this paper showed that the proposed optimization strategy was effective.Osório et al. [28] proposed a novel hybrid methodology for predicting the short-term electricity market price by using the ANFIS model, mutual information (MI), wavelet transform (WT), and evolutionary particle swarm optimization (EPSO).The results of this paper found that uncertainty related to market prices has decreased.Liu et al. [29] proposed two novel wind speed prediction approaches based on Fast ensemble empirical mode decomposition (FEEMD)-multi layer perceptron (MLP) and FEEMD-ANFIS.The results of this paper showed that the proposed hybrid method was suitable for the prediction of wind speed.Mohammadi et al. [30] applied the ANFIS model to forecast the daily global solar radiation; the finding of this paper showed that the ANFIS technique was suitable for predicting the daily horizontal global solar radiation.Al-Ghandoor et al. [31] applied the ANFIS technique for predicting the transport energy demand by the use of socio-economic and transport-related indicators.Zahedi et al. [32] designed a new intelligent controller based on the ANFIS technique to maximize the output power of wind turbines.The findings of this article demonstrated that the proposed model was effective for wind power turbines.Nikolić et al. [33] applied the ANFIS technique for predicting the wake wind speed.Nikolić et al. [34] also used the ANFIS technique for enhancing the performance of the wind turbines.Naji et al. [35] later designed and adapted the ANFIS technique to estimate the energy consumption of buildings based on some parameters, including line insulation, K-value, and material thickness.The results of this paper showed that the ANFIS results are more accurate than the results of the artificial neural network (ANN) and genetic programming (GP) techniques.Petković et al. [36] constructed the ANFIS model to evaluate the investment risk of a wind power project.The findings of this paper demonstrated that the ANFIS model can provide better performance compared to the regression of the net present value based on the coefficient of determination (R 2 ).Shamshirband et al. [37] applied the ANFIS model to calculate and predict the monthly wind power density.The outcomes of this study showed that all of the values of wind power prediction were between the acceptable range of −10% to 10%.They found that the ANFIS model had an excellent performance for the prediction of wind power density.Mellit and Kalogirou [38] implemented the ANFIS model for modeling the photovoltaic power supply (PVPS).The findings of this paper indicated that the ANFIS model performed better than the ANN approach.Yang et al. [39] used the ANFIS model for measuring the wind data as one of the key input data for wind farm design and planning.The findings of this paper proved that the ANFIS results were effective.Fernandez-Jimenez et al. [40] used ANFIS, autoregressive integrated moving average (ARIMA), and ANN for the estimation of the grid-connected photovoltaic plant.The findings of this paper showed that the presented forecasting system would be useful for the owners or managers of photovoltaic plants that are connected to offering forecast values of the hourly electrical energy production and electricity networks.

ANFIS
The ANFIS model that was proposed by Jang [20] has been a robust supervised machine learning techniques for prediction tasks [41].According to Zadeh [21], this technique is a specific kind of neuro-fuzzy system that takes the advantages of fuzzy logic and the neural network to automatically learn from data [42].The ANFIS model is functionally equivalent to a Takagi-Sugeno fuzzy inference systems (FIS) [43,44].Takagi-Sugeno FIS is also called Sugeno FIS Chen [45].It is a special kind of FIS.Its fuzzy rules are different from the common fuzzy rules.Usually, the premise conditions of fuzzy rules and the consequent conclusions of fuzzy rules are fuzzy linguistic values [46].They are shown as follows: where are the input fuzzy linguistic values, and B is the output fuzzy linguistic value.However, in Sugeno FIS, the fuzzy rules are shown as follows: where are the input fuzzy linguistic values, and c i are the parameters of certain values.Sugeno FIS is shown in Figure 1.In this study, a five-layer ANFIS model [47] is used for sustainability performance prediction (see Figure 2).In Layer 1, the membership grades of a linguistic label are generated.In Layer 2, the firing strength of each rule is generated.In Layer 3, normalized firing strength is calculated.In Layer 4, the parameter function on Layer 3 output is computed.Finally, in Layer 5, the overall output as the summation of all of the incoming signals is calculated.In this research, ensembles of predictors are used for assessing countries' sustainability performance.It has been shown that solely using a single inductive learning method cannot achieve a generalization performance in all of the possible classification tasks [16].In addition, the accuracy of using supervised learning methods more depends on the nature of the problem investigated and observed data.Despite this consideration, ensemble learning has proved to have an excellent performance in many of the prediction tasks of practical applications [17,18].In this research, ensembles of predictors are used for assessing countries' sustainability performance.It has been shown that solely using a single inductive learning method cannot achieve a generalization performance in all of the possible classification tasks [16].In addition, the accuracy of using supervised learning methods more depends on the nature of the problem investigated and observed data.Despite this consideration, ensemble learning has proved to have an excellent performance in many of the prediction tasks of practical applications [17,18].In this research, ensembles of predictors are used for assessing countries' sustainability performance.It has been shown that solely using a single inductive learning method cannot achieve a generalization performance in all of the possible classification tasks [16].In addition, the accuracy of using supervised learning methods more depends on the nature of the problem investigated and observed data.Despite this consideration, ensemble learning has proved to have an excellent performance in many of the prediction tasks of practical applications [17,18].It has been shown that the generalization performance of the ensemble learning is much better than a single individual ensemble member [20].In fact, these improvements in performance and accuracy are because of the combination of accurate predictors, whose errors are complementary.The final ensemble decision can be reached by combining the individual predictions of the ensemble members.To do so, different approaches, for example, average prediction, majority voting, and non-linear decision functions can be used.It should be noted that in general, there is no strong evidence supporting that more complex combination approaches can perform better.
In this research, multiple components of the ANFIS learner are trained for measuring the sustainability performance of countries.Accordingly, for each ANFIS model, the prediction accuracy is calculated for output, and then integration by an average approach was applied for the final ensemble decision [19].As a straightforward approach, the final decision using the integration by average approach is obtained from the sum of the results generated by each ANFIS model divided by the number of ANFIS models used for the prediction task.The proposed ensemble structure is depicted in Figure 3. From this figure, it can be found that the ensembles of the ANFIS technique are used to predict the sustainability performance.In each ANFIS model, the SAFE data is used for training the ANFIS models by a specific membership function, and the performance of the prediction is calculated.Then, the final performance is obtained by integrating the performance of the individual predictors.It has been shown that the generalization performance of the ensemble learning is much better than a single individual ensemble member [20].In fact, these improvements in performance and accuracy are because of the combination of accurate predictors, whose errors are complementary.The final ensemble decision can be reached by combining the individual predictions of the ensemble members.To do so, different approaches, for example, average prediction, majority voting, and nonlinear decision functions can be used.It should be noted that in general, there is no strong evidence supporting that more complex combination approaches can perform better.
In this research, multiple components of the ANFIS learner are trained for measuring the sustainability performance of countries.Accordingly, for each ANFIS model, the prediction accuracy is calculated for output, and then integration by an average approach was applied for the final ensemble decision [19].As a straightforward approach, the final decision using the integration by average approach is obtained from the sum of the results generated by each ANFIS model divided by the number of ANFIS models used for the prediction task.The proposed ensemble structure is depicted in Figure 3. From this figure, it can be found that the ensembles of the ANFIS technique are used to predict the sustainability performance.In each ANFIS model, the SAFE data is used for training the ANFIS models by a specific membership function, and the performance of the prediction is calculated.Then, the final performance is obtained by integrating the performance of the individual predictors.

ANFIS Ensemble Evaluation
The aim of this study is to measure the countries' sustainability performance using a set of input indicators in the SAFE dataset.To do so, the ANFIS technique was used to discover the relationship between the inputs and output parameters (Y = f (X 1 , X 2 , . . ., X n )) in the SAFE dataset for the accurate prediction of the countries' sustainability performance.For the SAFE dataset, X 1 , X 2 , . . ., X n stand for the input parameters of the dataset, and Y stands for the output parameter.In our research, the countries' sustainability performance is measured using four levels of ANFIS ensemble models.
As can be seen in Figure 4, in the first level, a country sustainability performance is measured.In the second level, ANFIS ensemble models are developed for ECOS and HUMS to measure the overall sustainability performance (OSUS).In the third level, the ANFIS ensemble models are developed to measure the ECOS and HUMS sustainability of each country.Each of these dimensions has its own indicators, which are used as inputs for the ANFIS models of ECOS and HUMS.The ECOS dimension includes four criteria: biodiversity (BIOD), land integrity (LAND), air quality (AIR), and water quality (WATER).The HUMS dimension also includes health (HEALTH), education (KNOW), economic welfare (WEALTH), and political aspects (POLICY).Each of these criteria is also categorized into tertiary indicators in the fourth level, which are defined as STATE (ST), RESPONSE (RE), and PRESSURE (PR).The components and basic indicators of the SAFE model are presented in Table A1 of Appendix A. For the tertiary indicators, a total of 75 indicators are used as inputs for the ANFIS ensembles of the fourth level.Hence, the main relationships in Figure 4 can be defined as follows.
Y OSUS = f (ECOS, HUMS) Y HEALTH = f (ST(HEALTH), RE(HEALTH), PR(HEALTH)) Y KNOW = f (ST(KNOW), RE(KNOW), PR(KNOW)) Y POLICY = f (ST(POLICY), RE(POLICY), PR(POLICY)) Y BIOD = f (ST(BIOD), RE(BIOD), PR(BIOD)) To evaluate the prediction models constructed by the ANFIS technique, a 4-GHz processor PC with a Windows 7 operation system was used.First, 20% of the data was initially considered for the test set, 20% of the data was considered for the checking set, and 60% of the data was considered for the training set.In each model, different types of membership functions (MFs) were designed and considered for fuzzification tasks.The types of MFs that were considered were: Triangular, Generalized Bell-Shaped, Gaussian, and Π-Shaped.In Figure 5, the four MFs are shown for ECOS and HUMS.As can be seen from this figure, three linguistic variables (low, moderate and high) are used as MFs degrees for each feature.Hence, ANFIS ensemble models are developed using four types of MFs with these linguistic variables.To evaluate the prediction models constructed by the ANFIS technique, a 4-GHz processor PC with a Windows 7 operation system was used.First, 20% of the data was initially considered for the test set, 20% of the data was considered for the checking set, and 60% of the data was considered for the training set.In each model, different types of membership functions (MFs) were designed and considered for fuzzification tasks.The types of MFs that were considered were: Triangular, Generalized Bell-Shaped, Gaussian, and Π-Shaped.In Figure 5, the four MFs are shown for ECOS and HUMS.As can be seen from this figure, three linguistic variables (low, moderate and high) are used as MFs degrees for each feature.Hence, ANFIS ensemble models are developed using four types of MFs with these linguistic variables.For each ANFIS ensemble, integration by average approach was then applied for the final decision.To evaluate the results and measure the accuracy of the method, the root mean square error (RMSE) and coefficient of determination (  2 ) were used.RMSE and the adjusted coefficient of determination (  2 ) are presented in Equations ( 14) and ( 15): where  is the number of observations,   is the predicted value,   is the actual output,  ̅  is the actual mean value,  ̅  is the predicted mean value, and  is the number of independent variables.In the following figures, the training errors of the ANFIS models are presented.From Figure 6, it can be seen that the ANFIS technique has accurately predicted the output of the models.The errors of prediction for OSUS, ECOS and HUMS are less than 0.001 with   For each ANFIS ensemble, integration by average approach was then applied for the final decision.To evaluate the results and measure the accuracy of the method, the root mean square error (RMSE) and coefficient of determination (R 2 adjusted ) were used.RMSE and the adjusted coefficient of determination (R 2 adjusted ) are presented in Equations ( 14) and ( 15): where N is the number of observations, P k is the predicted value, A k is the actual output, A m is the actual mean value, P m is the predicted mean value, and m is the number of independent variables.
In the following figures, the training errors of the ANFIS models are presented.From Figure 6, it can be seen that the ANFIS technique has accurately predicted the output of the models.The errors of prediction for OSUS, ECOS and HUMS are less than 0.001 with R 2 adjusted = 0.921, R 2 adjusted = 0.937, and R 2 adjusted = 0.945, respectively.Combining both the input MFs with the rules discovered by the ANFIS model, the visualization was better performed for the relationship between the inputs and output in a three-dimensional curve.Figure 7a illustrates the interdependency of overall sustainability and human and ecological sustainability through a control surface obtained from the combination of membership functions, fuzzy rules, and trained models in ANFIS for the SAFE dataset.In fact, this figure represents the mapping from each of two parameters of the dataset to the overall sustainability performance.In addition, Figure 7b measures the overall sustainability using generated fuzzy rules by ANFIS and Gaussian MFs.This figure shows that when the human and ecological sustainability are respectively 0.998 and 0.855, the overall sustainability that can be achieved is about 0.926.It should be noted that based on these input values for human and ecological sustainability, the overall sustainability achieved by SAFE [14] was 0.927.The difference between the two predictions was about 0.001, and this indicates that the error of prediction by the ANFIS model is very minor.the actual mean value, is the predicted mean value, and is the number of independent variables.
In the following figures, the training errors of the ANFIS models are presented.From Figure 6, it can be seen that the ANFIS technique has accurately predicted the output of the models.The errors of prediction for OSUS, ECOS and HUMS are less than 0.001 with 0.921, 0.937, and 0.945, respectively.Our method also is compared with the other prediction machine learning techniques on the SAFE dataset.These techniques are: neural network (NN) [48], support vector regression (SVR) [49], and multiple linear regression (MLR).After performing these techniques, the RMSE of their prediction is calculated and presented in the results.These techniques were applied on the same dataset without incorporating ensemble learning.The radial basis function (RBF) kernel was used and the kernel elements (C and γ) were selected by the use of exhaustive search and 10-fold cross-validation [49].Specifically, the exponentially growing sequences of C from 2 −15 to 2 10 and γ from 2 −10 to 2 9 are selected to find the optimal values.For NN, feedforward back-propagation with a single output is used for the prediction task.The model used in this research has three layers.For training NN, there are different back-propagation techniques, which are conjugate gradient resilient back-propagation [50], back-propagation [51], and Levenberg [52].In this research, the back-propagation training algorithm is used to train the NN technique.The result is presented in Table 1.It can be seen from the result that the method that uses ensemble learning (ANFIS-Ensembles) is more accurate compared with the other methods, which use solely prediction techniques.This table further demonstrates that the ANFIS ensemble, which uses a combination of Triangular, Gaussian and Π-Shaped MFs, works better than combination of other MFs in measuring the overall sustainability performance.Combining both the input MFs with the rules discovered by the ANFIS model, the visualization was better performed for the relationship between the inputs and output in a three-dimensional curve.Figure 7a illustrates the interdependency of overall sustainability and human and ecological sustainability through a control surface obtained from the combination of membership functions, fuzzy rules, and trained models in ANFIS for the SAFE dataset.In fact, this figure represents the mapping from each of two parameters of the dataset to the overall sustainability performance.In addition, Figure 7b measures the overall sustainability using generated fuzzy rules by ANFIS and Gaussian MFs.This figure shows that when the human and ecological sustainability are respectively 0.998 and 0.855, the overall sustainability that can be achieved is about 0.926.It should be noted that based on these input values for human and ecological sustainability, the overall sustainability achieved by SAFE [14] was 0.927.The difference between the two predictions was about 0.001, and this indicates that the error of prediction by the ANFIS model is very minor.
Our method also is compared with the other prediction machine learning techniques on the SAFE dataset.These techniques are: neural network (NN) [48], support vector regression (SVR) [49], and multiple linear regression (MLR).After performing these techniques, the RMSE of their prediction is calculated and presented in the results.These techniques were applied on the same dataset without incorporating ensemble learning.The radial basis function (RBF) kernel was used and the kernel elements (C and  ) were selected by the use of exhaustive search and 10-fold cross- validation [49].Specifically, the exponentially growing sequences of C from 2 −15 to 2 10 and  from 2 −10 to 2 9 are selected to find the optimal values.For NN, feedforward back-propagation with a single output is used for the prediction task.The model used in this research has three layers.For training NN, there are different back-propagation techniques, which are conjugate gradient resilient backpropagation [50], back-propagation [51], and Levenberg [52].In this research, the back-propagation training algorithm is used to train the NN technique.The result is presented in Table 1.It can be seen from the result that the method that uses ensemble learning (ANFIS-Ensembles) is more accurate compared with the other methods, which use solely prediction techniques.This table further demonstrates that the ANFIS ensemble, which uses a combination of Triangular, Gaussian and Π-Shaped MFs, works better than combination of other MFs in measuring the overall sustainability performance.
(a) Interdependency inputs and output.In Table A2 presented in Appendix B, the ranks of 128 countries' sustainability performance by SAFE, ANFIS, and ANFIS ensembles are presented.In addition, in this table, the ranking difference between ANFIS-Ensemble and SAFE is also presented.For this result, the combination of Gaussian, Triangular, and Π-Shaped MFs were considered for ANFIS ensemble learning, as this combination provided better estimation accuracy compared to the other MFs.From Figure 8, it can also be seen that there are minor differences between the ANFIS ensemble and the SAFE model in ranking the countries' sustainability performance.The largest difference for the ANFIS ensemble is the value of 3 for Estonia and Gambia.Compared with the ANFIS model, the analysis also demonstrates that, since our method uses the ensemble learning, it can better rank the countries' sustainability performance.In Table A2 presented in Appendix B, the ranks of 128 countries' sustainability performance by SAFE, ANFIS, and ANFIS ensembles are presented.In addition, in this table, the ranking difference between ANFIS-Ensemble and SAFE is also presented.For this result, the combination of Gaussian, Triangular, and Π-Shaped MFs were considered for ANFIS ensemble learning, as this combination provided better estimation accuracy compared to the other MFs.From Figure 8, it can also be seen that there are minor differences between the ANFIS ensemble and the SAFE model in ranking the countries' sustainability performance.The largest difference for the ANFIS ensemble is the value of 3 for Estonia and Gambia.Compared with the ANFIS model, the analysis also demonstrates that, since our method uses the ensemble learning, it can better rank the countries' sustainability performance.

Discussion and Recommendations
The advantage of the proposed method is the automatic generation of decision rules for sustainability assessment from the real-world data for input and output parameters without human intervention.In addition, the ANFIS ensembles could effectively estimate the sustainability performance, which is a very complex problem with non-linear relationships between its variables (Hjorth and Bagheri [53]; Tan, Shuai, Jiao and Shen [5]).Overall, in the ANFIS ensemble learning paradigm, it was found that the collection of different MFs for prediction models results in improved accuracy in measuring countries' sustainability performance.This was because of the combination of accurate predictors whose errors are complementary.
The proposed method is also flexible enough to accept a large number of input and output indicators to be used for estimating the sustainability performance.However, in this situation, as the rule number and computation time of the method will be increased, the ANFIS model may not be able to appropriately construct the prediction models of sustainability.Note that as the ANFIS technique may perform the prediction in large offline training time for large datasets, which restricts the use of this technique in real time applications, the incremental version of the ANFIS model can be more suitable and efficient for the assessment of sustainability performance.The incremental updates will allow the system to incrementally update the membership functions, which can in general reduce the computation time of constructing the models of sustainability performance in the ANFIS model.Accordingly, this can be a limitation of ANFIS-based methods that should be taken into account in future studies.
In addition, it was observed that in the previous methods based on rule-based techniques, discovering fuzzy rules and defining the membership functions are performed with human

Discussion and Recommendations
The advantage of the proposed method is the automatic generation of decision rules for sustainability assessment from the real-world data for input and output parameters without human intervention.In addition, the ANFIS ensembles could effectively estimate the sustainability performance, which is a very complex problem with non-linear relationships between its variables (Hjorth and Bagheri [53]; Tan, Shuai, Jiao and Shen [5]).Overall, in the ANFIS ensemble learning paradigm, it was found that the collection of different MFs for prediction models results in improved accuracy in measuring countries' sustainability performance.This was because of the combination of accurate predictors whose errors are complementary.
The proposed method is also flexible enough to accept a large number of input and output indicators to be used for estimating the sustainability performance.However, in this situation, as the rule number and computation time of the method will be increased, the ANFIS model may not be able to appropriately construct the prediction models of sustainability.Note that as the ANFIS technique may perform the prediction in large offline training time for large datasets, which restricts the use of this technique in real time applications, the incremental version of the ANFIS model can be more suitable and efficient for the assessment of sustainability performance.The incremental updates will allow the system to incrementally update the membership functions, which can in general reduce the computation time of constructing the models of sustainability performance in the ANFIS model.Accordingly, this can be a limitation of ANFIS-based methods that should be taken into account in future studies.
In addition, it was observed that in the previous methods based on rule-based techniques, discovering fuzzy rules and defining the membership functions are performed with human intervention.However, this can be a time-consuming process.Accordingly, it is suggested that in future studies, some other machine learning techniques (e.g., decision tree-based techniques) be employed to automatically induce the rule to be used in the rule-based system.Hence, the combination of decision trees and fuzzy rule-based techniques can be also evaluated for sustainability performance problems.In addition, as the rule discovery of a fuzzy rule-based system is one of the important stages of prediction, the use of decision trees can be helpful for automatic rule discovery to be used in a rule-based system.
Furthermore, for large datasets with huge indicators, clustering techniques can be used as an unsupervised classification method to cluster the data of datasets into similar groups.Accordingly, it is suggested that the data be clustered in similar groups before it can be used for training in ANFIS models.This will improve the readability and handling of the data.In addition, for big datasets, the clustering of data will improve the complexity issue of data processing.Furthermore, the data that was used for training the ANFIS models cannot be considered as a large dataset.The number of tuples in the datasets is relatively small, and the efficiency of method should be tested on large datasets to show whether the method is capable of processing the data and giving the results in real time.As a dimensionality reduction technique, the clustering techniques need to be incorporated into the sustainability assessment methods to reflect its broad appeal and usefulness as one of the steps in exploratory real-world data analysis.

Conclusions and Future Work
In this paper, a new method was proposed to measure the countries' sustainability performance.The method was developed using ensembles of neuro-fuzzy technique.Different combinations of membership functions (Triangular, Generalized Bell-Shaped, Gaussian, and Π-Shaped) were used for the ANFIS ensemble method.In the ANFIS models, centroid of area (COA) was used for the defuzzification task to measure the countries' sustainability performance.To evaluate the method, the SAFE dataset was used to measure the accuracy of all of the prediction models using root mean square error (RMSE).The accuracy of the proposed method was compared with neural network (NN), multiple linear regression (MLR), and support vector regression (SVR).Our analysis on the SAFE dataset demonstrated that the proposed method-which uses Gaussian, Triangular, and Π-Shaped membership functions-is more accurate in relation to the other combinations.Specifically, the ANFIS ensemble method, which uses Gaussian, Triangular, and Π-Shaped membership functions, provides the lowest RMSE value (0.00038).The data analysis further demonstrated that the ANFIS ensembles outperform the other prediction methods-NN, MLR, and SVR-in measuring the countries' sustainability performance.Compared to the method that solely relies on the ANFIS technique, the data analysis also demonstrates that the method that uses ensemble learning can better rank the countries' sustainability performance.
In this study, a non-incremental ANFIS model has been implemented for measuring the countries' sustainability performance.In addition, the method developed by the ANFIS technique does not support the incremental learning, and it requires recomputing all of the training data in constructing the prediction models.Accordingly, in order to improve the computation time of measuring the countries' sustainability performance, it is suggested to develop this method to incrementally update the trained models when new information is available, which can be more efficient regarding memory requirements.
In future studies, more attention should be paid to the ensemble learning techniques in developing the systems for sustainability performance assessment to exploit all of their potential and usefulness.Future research can extend the method presented in this study for incremental learning.The proposed method also can be further evaluated on large datasets, which include other indicators of sustainability to show their effectiveness for accuracy and computation time.In addition, future research can

Figure 3 .
Figure 3. Sustainability performance by the ANFIS model.Figure 3. Sustainability performance by the ANFIS model.

Figure 3 .
Figure 3. Sustainability performance by the ANFIS model.Figure 3. Sustainability performance by the ANFIS model.

Figure 7 .
Figure 7.The results of the ANFIS model for measuring overall sustainability performance.

Figure 7 .
Figure 7.The results of the ANFIS model for measuring overall sustainability performance.

Figure 8 .
Figure 8. Ranking difference of the ANFIS-sustainability assessment by fuzzy evaluation (SAFE) and the ANFIS ensemble-SAFE in estimating the sustainability performance of 128 countries.

Table 1 .
Root mean square error (RMSE) of different methods.MLR: multiple linear regression, NN: neural network, SVR: support vector regression.

Table 1 .
Root mean square error (RMSE) of different methods.MLR: multiple linear regression, NN: neural network, SVR: support vector regression.

Table A2 .
Countries sustainability performance ranking by SAFE, ANFIS, and ANFIS ensemble.UK: United Kingdom, USA: United States.