Performance Evaluation of Residential Demand Response Based on a Modiﬁed Fuzzy VIKOR and Scalable Computing Method

: For better utilizing renewable energy resources and improving the sustainability of power systems, demand response is widely applied in China, especially in recent decades. Considering the massive potential ﬂexible resources in the residential sector, demand response programs are able to achieve signiﬁcant beneﬁts. This paper proposes an effective performance evaluation framework for such programs aimed at residential customers. In general, the evaluation process will face multiple criteria and some uncertain factors. Therefore, we combine the multi-criteria decision making concept and fuzzy set theory to accomplish the model establishment. By introducing trapezoidal fuzzy numbers into the Vlsekriterijumska Optimizacijia I Kompromisno Resenje (VIKOR) method, the evaluation model can effectively deal with the subjection and fuzziness of experts’ opinions. Furthermore, we ameliorate the criteria weight determination procedure of traditional models via combining the fuzzy Analytic Hierarchy Process and Shannon entropy method, which can incorporate objective information and subjective judgments. Finally, the proposed evaluation framework is veriﬁed by the empirical analysis of ﬁve demand response projects in Chinese residential areas. The results give a valid performance ranking of the ﬁve alternatives and indicate that more attention should be paid to the criteria afﬁliated with technology level and economy beneﬁts. In addition, a series of sensitivity analyses are conducted to examine the validity and effectiveness of the established evaluation framework and results. The study improves traditional multi-criteria decision making method VIKOR by introducing trapezoidal fuzzy numbers and combination weighing technique, which can provide an effective mean for performance evaluation of residential demand response programs in a fuzzy environment.


Introduction
During the past decades, with the rapid population growth and accelerated industrialization and urbanization construction, the energy demand is increasing sharply in China [1].Wide exploitation and utilization of fossil energy has caused more and more serious environmental issues, such as greenhouse effects and large scale fog and hazy weather [2].To deal with the conflicts between environment protection and economy development, a safe, effective and sustainable energy system is necessary for China [3].Increasing the proportion of renewable energy sources (RES) in electricity generation and developing smart grids [4], energy internet [5] and some other new concepts have raised more and more concerns of successive Chinese governments [6].In this respect, demand response (DR) is often viewed as a particularly suitable and effective way of coping with potential imbalances in power systems caused by RES as well as one of the main components in smart grids and energy internet [7,8].DR is a specific type of demand-side management, which is focus on the instantaneous adjustment of electricity consumption pattern of end users at a given time point [9].The implementations of DR can be classified into two main types: dispatchable and non-dispatchable [10].Dispatchable DR resources refer to direct remote control of interruptible electricity consumers to reduce or shift demand according to the grid's real-time operation [11].Non-dispatchable DR is defined as consumers' initiative control of their own electricity demand based on price changes.In the non-dipatchable DR situation, end-users can choose to respond the price change or not [12].DR programs can not only reduce peak loads and traditional power generation, but also handle the variability issue caused by RES power generation [13].
In the past, implementation experience of DR has mainly concentrated on the industrial and commercial sectors, where some programs achieved significant demand reduction [14].For instance, in the FSC Group's investigation, non-residential DR programs implemented in California (USA).showed large aggregate load reduction in the industrial sector and substantial differences in business categories [15].Some DR programs aimed at industrial or commercial customers in Korea [16], United Kingdom [17], Norway [18], Denmark [19], Germany [20], Italy [21] are reported as having achieved considerable contributions for power systems.In China, a few demand side management programs were firstly implemented in the 1990s.Then in 2010, the National Development and Reform Commission (NDRC) announced the "Electricity Demand-side Management Measures" mainly aimed at the industrial segment [22].For further promoting demand side management (DSM) and DR application, in 2012, Suzhou, Beijing, Foshan, Tangshan and Shanghai were selected as related pilot cities [23].Compared with rich experience in industrial and commercial sectors, the residential DR implementation in China is still at the initial stage.However, in recent years, there has been a growing interest in residential DR programs because of their enormous application potential.As reported, in 2012, the energy consumption in Chinese residential sector has accounted for 10.90% of the total energy use [24].Because residential loads are flexible and have great potential to be adjusted, more DR programs aimed at residential customers should be designed and carried out to ensure a stable and sustainable electricity system.It is noteworthy that performing DR programs not only needs the support of advanced technologies and devices, but also a comprehensive evaluation framework for the program performance.To better give a valuable decision references, some researchers have conducted related assessments on DR implementation in different countries or regions.From the point view of economy, Siano assessed the benefits of DR programs conducted in the residential sector mainly from the aspects of cost measurement and energy savings [25].Moghaddam established an economic model to analyse the change trend of residential demand elasticity when implementing DR programs [26].From the power system perspective, Dupont analyzed the impact of flexible loads on the demand side and quantified the operational benefits of DR [27].Besides their influence on economy and electric system operation, the environmental value of DR projects has also aroused much attention in the current development situation.Reduction of pollutant and CO 2 emissions as well as increase of RES penetration rate have become the new focuses when promoting DR.Rodríguez-García established an economic evaluation model for DR programs in which CO 2 emission reduction was transformed into an economic value according to emission factors and market price [28].For the whole cost-minimization of the system, Behboodi thought the intra-hourly DR is much more helpful in promoting additional RES than inter-hourly DR [29].Although the previous studies gave us much inspiration for DR performance evaluation, there still are some limitations and room for improvement.First of all, a completed measurement index hierarchy needs to be established which is not only limited to a certain aspect.Secondly, most of the calculation methods or evaluation indexes depend on the data acquisition, which increases the difficulty and uncertainty for DR implementation evaluation in reality.From a sustainable perspective, the index system should include economic, environmental and social dimensions [30].Moreover, because the smart devices play an important role when implementing DR programs in residential consumers, the completed evaluation index system should consider the technical factors as well.Hence, the final evaluation indexes for residential DR performance should be composed of economic, environmental, social and technical criteria.
For an evaluation model, considering the performance of residential DR programs involves many aspects, a multiple criteria decision making (MCDM) approach is suitable to be applied in the evaluation procedure [31].MCDM has found its grounding application in many projects and enterprises in recent decades.Various MCDM techniques and algorithms were proved to be effective in the performance evaluation, such as elimination and choice translating reality (ELECTER), analytical hierarchy process (AHP), weighted sum model, technique for order preference by similarity to ideal solutions (TOPSIS) and its modified framework VlseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) [32,33].In the DSM and DR research field, MCDM approaches have also seen extensive use during the last few years.Sanjay [34] conducted an evaluation of DSM implementation strategies in India with an AHP method.Mokhtar [35] analyzed and gave a selection framework for DSM options employing a combination of the AHP-PROMETHEE and TOPSIS methods.However, due to some unavoidable vagueness and uncertainty existing in the evaluating process, traditional MCDM methods may have limitations to accurately express experts' opinions [36].Therefore, some studies showed solicitude to express the human's subjective languages and judgments with imprecise numeric values.Lin [37] introduced fuzzy set theory into TOPSIS to expand the traditional MCDM approach in the fuzzy environment.Dong incorporated fuzzy set, AHP and VIKOR methods to evaluated DSM [38] and DR [39] programs implemented in the commercial sector.Nevertheless, to the best of the authors' knowledge, no fuzzy MCDM method application was found regarding the evaluation of residential DR performance.
For better evaluating the implementation situation of residential DR programs in reality, we organized the paper as follows: based on the previous researches, we established a comprehensive index system for residential DR programs and innovated the evaluation framework by integrating VIKOR, fuzzy set theory and combination weighting approaches.Specifically, we used trapezoidal fuzzy numbers (TrFNs) to express all the linguistic ratings from experts rather than often used triangular fuzzy numbers (TFNs).TrFN can encompass more uncertainty compared with TFN which is only a special case of TrFN [40].Moreover, to improve the rationality of the weighting determination process in fuzzy VIKOR, a new weighting method combining fuzzy-AHP and Shannon Entropy approaches was introduced in the paper.Through the proposed evaluation framework and calculation results, we can clearly learn which alternative has the best performance or achieves the most benefits.The results can give some inspiration for the decision makers when conducted related programs and offer references for policies formulation.
The primary contributions of the paper are: 1.From a sustainability perspective, the comprehensive evaluation index system aimed at residential DR programs was formed.The four pillars of criteria including economy, environment, technology and society can reflect the implementation effect of DR programs especially in residential customers completely.
2. A modified decision making model with multi-criteria was proposed to appraise the implementation of residential DR projects.We introduced TrFNs into the VIKOR method and at the same time improved the weighting determination by applying the combination weighing method.In virtue of the fuzzy set theory and TrFNs, the established model has its advantages to capture and deal with the fuzziness of decision makers effectively.The combination weighting technique can obtain more information containing subjective judgements as well as objective data for all alternatives.
3. To further examine the robustness and effectiveness of the model, we performed a series of sensitivity analyses.By changing the weights of each criterion and the parameters in the model, we observed whether the fluctuation would affect final ranking results.This is the first study to compare the importance of residential DR evaluation criteria for performance ranking results.The findings can give significant references during decision making about DR implementation.
The paper contains seven sections, and the remaining structure is as follows: Section 2 introduces the basic theory and methodology.The framework of the proposed model is given in Section 3. Then Section 4 chooses and describes the evaluation criteria for residential DR performance.The empirical analysis of five residential DR programs in Beijing is conducted based on the proposed framework in Section 5. Findings and discussion are given in Section 6.Finally, we draw the conclusions in Section 7.

Methodology
For better understanding the proposed evaluation framework, we briefly introduce some related mathematical methods, including fuzzy set theory, fuzzy-VIKOR, fuzzy-AHP and Shannon Entropy.The fuzzy set theory is the fundamental for fuzzy-VIKOR and fuzzy-AHP, and TrFN as one of its expression forms is applied to deal with the linguistic terms in the calculation process.In addition, the combination method of fuzzy-AHP and Shannon Entropy improves the traditional weighting process of the fuzzy-VIKOR method.

Fuzzy Sets Theory
In the real world, decision makers are always faced with some uncertainties or doubts.They may describe a judgment by some vague words.To solve the vagueness and ambiguity, Zadeh [41] introduced fuzzy sets theory in 1965 which improved the linguistic judgments expression of decision making.Then fuzzy sets are developed to resolve the problem of lacking precise description in criteria importance weights assigning and alternatives rating during MCDM issues.A fuzzy set is constituted of a series of functions denoting the membership degree of an element.The degree membership of every factor is able to be expressed by a real number between [0,1].When the element belongs to the fuzzy set totally, the degree number is "1".Analogously, "0" stands for the criterion not belonging to the set at all.A value among [0,1] means the element belongs to the set partly.Based on the fuzzy set theory, some linguistic expression like "excellent", "good", "ordinary", "inferior" can be turned into a battery of interval values.
Fuzzy membership function has several types.This paper uses TrFNs to deal with the linguistic term.A positive TrFN can be defined as a quaternion A = (a 1 , a 2 , a 3 , a 4 ) that respectively represent minimum possible value, most promising interval value, and maximum possible value [42].The membership function µ A (x) is represented as Equation (1): A = (a 1 , a 2 , a 3 , a 4 ) and B = (b 1 , b 2 , b 3 , b 4 ) are positive TrFNs, r is a positive real number, some common rules of operation are as follows: A ⊗ r ∼ = (a 1 r, a 2 r, a 3 r, a 4 r) Humans always give indistinct answers rather than precise numerical values during MCDM processes.Fuzzy set theory and qualitative linguistic values have more advantages to assess performances than traditional numerical methods in a fuzzy environment.Thus, traditional MCDM methods combined with fuzzy set theory get more application in recent years, like fuzzy-AHP, fuzzy-TOPSIS, fuzzy-VIKOR etc.For comparing alternatives and getting intuitive results, Central Value (CV) method is introduced to defuzzy the fuzzy number A [43,44].The best non-fuzzy performance (BNP) number B( A) denotes the defuzzying result which can be obtained using Equation (5) as follows [45]:

Fuzzy VIKOR Method
Opricovic [46] put forward the VIKOR method for multi-criteria decision making.Its central theory is to construct an aggregated function for calculating the distances between evaluation point and positive or negative ideal points.For the purpose of dealing with the subjectivity and non-determinacy during deciding procedure, the fuzzy set theory could be applied to traditional VIKOR.The fuzzy VIKOR is more reliable and rational at getting alternative rankings than the conventional method with respect to various criteria and complexed decision environment [47].
We suppose that there are L decision makers who give rankings of m alternatives based on n criteria.Each alternative will be evaluated as regards the n criteria.Because of vagueness existing in subjective assessments, Table 1 gives the comparisons between linguistic variables and corresponding fuzzy numbers for decision makers to measure the uncertainty.Let x ij = x ij1 , x ij2 , x ij3 , x ij4 be the fuzzy rating, in which i denotes the alternatives, j stands for criteria and l represents evaluators, i = 1, 2, . . ., m; j = 1, 2, . . ., n and l = 1, 2, . . ., L.
Detailed calculation steps of the fuzzy VIKOR method are as follows: Step 1: Set up the original matrix integrating fuzzy decisions.
These fuzzy ratings of alternatives in terms of each criterion are aggregated as: where , and x ij4 = max{x ijl4 }.Then the aggregated fuzzy ratings can be further integrated into a matrix X, as in Equation (7).
Step 2: Normalize the original fuzzy decision matrix.
Given that the criteria may have differences in measuring dimension, it is essential to standardize the aggregated fuzzy ratings.Among the evaluation indexes, those criteria whose higher values are favorable, are benefit-type criteria and those criteria whose smaller values are desirable, are cost-type criteria.The criteria normalization in the paper is based on linear scaling transformation method [48].The benefit-type criteria are divided by the maximum value in the decision matrix with Equation (8).For cost-type criteria, use the criterion value to divide minimum value with Equation (9) to accordance with benefit-type measuring means.Y = y ij m×n represents a standardized fuzzy decision matrix.The normalization equations are shown as follows: Energies 2018, 11, 1097 6 of 27 Step 3: Define the fuzzy positive and negative ideal solutions with respect to each criterion.
Step 4: Defuzzify the trapezoidal fuzzy numbers with Equation ( 5) and compute the crisp value distances from the positive ideal solutions with Equation (11): where d ij represents the normalized distance between alternative i in related with criterion j and the positive ideal solution R + j .
Step 5: Compute utility S i and regret value R i for each alternative: where w j denotes the criterion weight of j. S i indicates the maximum of the overall utility.Smaller values of S i show larger overall benefits.R i stands for the dissatisfactory degree of individual criteria presentations.Lower values of R i indicate less individual regret.
Step 6: Measure the compromise index Q i : where η reflects the weight of maximum overall benefits and 1 − η measures the importance of individual regret.
Step 7: Compare S i , R i , Q i and obtain the ranking results of alternatives.
According to the decreasing order of Q i values, rank all alternatives.An alternative is more optimal if its Q i value is lower.The new ranking of all alternatives is marked as A (1) , A (2) , . . ., A (m) .In addition, only when satisfying the following conditions, the alternative A (1) with a lowest Q i value is the first best.
Condition 1: where m denotes the number of evaluation alternatives.

Condition 2:
A (1) should be the optimal alternative according to the S i or R i in a decreasing order.

Combination Weight Technique
In order to give more accurate weights in the calculation process of fuzzy VIKOR, it is recommended to apply a combination weight technique, including subjective and objective weighting Energies 2018, 11, 1097 7 of 27 determine methods.The subjective weights can be obtained by fuzzy-AHP method on the basis of expert comments.Shannon Entropy method will be used to calculate the objective weights according to performance data.

Fuzzy-AHP Method
The Analytic Hierarchy Process (AHP) was first proposed by Saay [49] and developed by Marsh and Moran [50] who applied the method to decision making problems and gave detailed steps.As an effective method for dealing with the MCDM problem, AHP can clarify complicated relationships of criteria and handle a complex problem by decomposing it into some relatively simple sub-problems.However, the conventional AHP has some shortcomings in coping with subjective and ambiguous perception of an exact number.Therefore, Buckley developed the fuzzy-AHP model to deal with the issue [51].In virtue of linguistic variables, fuzzy-AHP has more advantages to compute criteria weights than traditional AHP during an uncertain decision making processes.Most fuzzy AHP application models often only use TFNs.Nevertheless, TrFNs could better capture the most possible situation when concerning much of uncertainty compared with TFNs.In the paper, TrFNs is used in fuzzy AHP to determine the criteria weights.
Suppose that a respectively denote the pairwise comparison of main criteria and  Based on the comparison judgments and TrFNs, the computing processes are as follows: Step 1: Obtain the pair-wise comparison matrix.
Suppose that a k I J and a k ij respectively denote the pairwise comparison of main criteria and sub-criteria using TrFNs given by expert k according to Table 2.A single fuzzy judgement matrix W k of expert k can be described as: In the above matrix, if i > j, a k ij equals a TrFN according to the judgement of expert k; Step 2: Verify the consistency of judgement matrices.
For checking the consistency of judgement matrix intuitively, TrFNs should be converted into crisp numbers using Equation (5).The consistency of judgement matrix can be expressed by consistency ratio (CR) which is calculated by Equations ( 17) and (18): where CI is the consistency index and RI is the random index as shown in Table 3. λ max denotes the largest eigenvalue of fuzzy matrix, and n is the matrix size.As a rule, a fuzzy comparison matrix consistent threshold is set to 0.2 of CR as the upper limit [53].If CR is less than 0.2, we can consider the matrix is consistent.
Step 3: Form the aggregated fuzzy degree of each criterion.
Let W main represent aggregated fuzzy judgement matrices of main criteria and W sub represent the matrices of sub-criteria, shown as follows: Energies 2018, 11, 1097 9 of 27 where I4 denote the fuzzy aggregated degree at the main criteria level, as: where , h s i4 at the sub-criteria level is: where h s . Then use Equation ( 5) to transform H main Considering the hierarchy structure, normalize the global weight W SG i for sub-criteria i as:

Shannon Entropy Method for Objective Weights
Entropy put forward by Shannon in 1947 [54] is a technique which can measure non-determinacy during information formulation.Thus, the method has its advantages when applied in the decision making process to obtain objective weights [55,56].Shannon entropy is capable of assessing the importance of different criteria based on the quantity of information provided by data [57].Suppose a MCDM problem has m alternatives and n criteria, then the decision matrix can be described as: The calculation steps of Shannon entropy are as follows: Step 1: Normalize the decision matrix with Equation ( 27): where P ij denotes the projection value of each criterion.
Energies 2018, 11, 1097 10 of 27 Specially, if the decision making matrix is based on fuzzy numbers, we need to defuzzy the matrix before normalization.
Step 2: calculate the entropy value of each criterion with Equation ( 28): where e j is the entropy value for criteria j; k = (ln(m)) −1 .
Step 3: Determine the divergence degree with Equation ( 29): where div j is the divergence degree of criterion j.A large value of div j indicates criterion j is important.
Step 4: obtain the objective weights for each criterion using Equation ( 30): where w o j is the objective weight for criterion j.

The Combination Method
Based on the subjective and objective weights obtained above, the final combination weights of each criterion can be calculated using multiplicative synthesis, as: in: where, w c j stands for combination weight of criterion j. w s j denotes subjective weight of criterion j using fuzzy AHP method.w o j is the objective weight of criterion j obtained from the Shannon entropy technique.

Calculation Framework for the Proposed Model
The proposed MCDM model to assess the DR project performance in residential areas is based on the modified fuzzy VIKOR and combination weights method.The detailed evaluation process consists of three phases, as shown in Figure 2: Phase 1: Establish a comprehensive evaluation criteria system and identify the alternatives to be assessed.
In the first phase, experts from DR and relevant research fields are invited to participate in decision making groups.Based on the experts' opinions and industry characteristics, determine the criteria for evaluating the performance of residential DR programs according to relevant research literatures.Considering the reality, the alternatives to be assessed are selected.
Phase 2: Obtain the combining weights for each criterion in virtue of fuzzy AHP and Shannon entropy methods.
Based on the index system, determine the weights for criteria, aggregated by subjective and objective dimensions.In virtue of the fuzzy-AHP approach, experts' opinions can be processed for the subjective weights.Firstly, relevant experts need to assign linguistic ratings to each criterion  2. Then check each judgement matrix's consistency and aggregate the individual matrices.Finally, after computing the fuzzy synthetic extent values, we can obtain the subjective weights.On the other hand, the Shannon entropy method is applied to obtain the objective weights.During the calculation process, each alternative in regard to criteria will be allocated a TrFN which can be calculated based on the vague linguistic ratings according to the comparison Table 1.Then defuzzify the decision matrix by CV method with Equation ( 6) and get the objective weights with Shannon entropy approach.Based on the results above, the last step to obtain the combining weights is using multiplicative synthesis to aggregate the subjective and objective weights.to integrate more information from subjective weights and objective weights.Lastly, the modified fuzzy VIKOR with TrFNs and combination weights can efficiently tackle the vagueness and ambiguity of experts' judgements during the multi-criteria decision making process.Accordingly, the hybrid evaluation model is applicable and advantageous to deal with issues in reality.

Establishing an Index System for Residential DR Performance Evaluation
Evaluation criteria are very vital when evaluating residential DR programs.It is necessary to build a comprehensive evaluation index system to reflect the inherent characteristics of DR programs.Nonetheless, because DR programs are still at an early development stage in China, there is a lack of a consistent criteria system to appraise the residential DR performance.DR program application can not only provide short term benefits such as optimum allocation or operation of electrical power systems, but also achieve the long term goals of environmental protection or management [58].From the perspective of sustainable development, we try to establish an evaluation index system considering economy, environment, technology and society factors.Furthermore, criteria affiliated with the above four dimensions are collected from experts and the literature, which are illustrated below.The completed evaluation indexes are shown in Figure 3.In this phase, the first step is to standardize the initial fuzzy decision matrix of alternatives in phase 2.Then, determine a set of fuzzy positive and negative ideal solutions.Next, compute the crisp value distances from the positive ideal solutions of each alternative.Finally, rank the performances of alternatives following the order of S i , R i and Q i from high to low.
The established MCDM frame in virtue of the modified fuzzy VIKOR and combination weighting method has three advantages as follows when applied to the comprehensive evaluation of DR performance.First of all, the framework can effectively capture and make use of the fuzziness of human judgements.Secondly, the combination weighting method has the outstanding capability to integrate more information from subjective weights and objective weights.Lastly, the modified fuzzy VIKOR with TrFNs and combination weights can efficiently tackle the vagueness and ambiguity of experts' judgements during the multi-criteria decision making process.Accordingly, the hybrid evaluation model is applicable and advantageous to deal with issues in reality.

Establishing an Index System for Residential DR Performance Evaluation
Evaluation criteria are very vital when evaluating residential DR programs.It is necessary to build a comprehensive evaluation index system to reflect the inherent characteristics of DR programs.Nonetheless, because DR programs are still at an early development stage in China, there is a lack of a consistent criteria system to appraise the residential DR performance.DR program application can not only provide short term benefits such as optimum allocation or operation of electrical power systems, but also achieve the long term goals of environmental protection or management [58].From the perspective of sustainable development, we try to establish an evaluation index system considering economy, environment, technology and society factors.Furthermore, criteria affiliated with the above four dimensions are collected from experts and the literature, which are illustrated below.The completed evaluation indexes are shown in Figure 3.

Economic Benefit (Z1)
DR programs applied in residential areas can obtain considerable economic benefits.The criteria associated with economy aspect for DR performance of residential users are summarized as below: 1) Redundant investment suspension (C1): Refers to reducing the need to build extra generation or transmission infrastructure.DR programs can postpone or avoid some peaking plants or system capacity investment by adjusting residents' electricity usage habits [59,60].2) Price elasticity (C2): Refers to the elasticity of electricity demand with regard to electricity price changes in the residential sector.The elasticity can reflect the sensitivity of residents to electricity prices which can directly influence the performance of DR programs [60].3) Electricity bill reduction (C3): DR programs can guide residents to choose a most economical plan to use energy by combining different prices schemes.Through adjustment of energy use habits, users can also reduce their electricity bills [61].

Environmental Benefit (Z2)
The critical criteria affiliated with environmental benefits for DR performance evaluation are summarized as below: 1) Pollutant emission reduction (C4): Because DR can decrease peak loads and improve electric equipment utilization, it can be considered as a kind of clean energy resource.Thus, applying DR programs in residential areas can effectively reduce the emissions of harmful substances

Economic Benefit (Z1)
DR programs applied in residential areas can obtain considerable economic benefits.The criteria associated with economy aspect for DR performance of residential users are summarized as below:

Environmental Benefit (Z2)
The critical criteria affiliated with environmental benefits for DR performance evaluation are summarized as below: (1) Pollutant emission reduction (C4): Because DR can decrease peak loads and improve electric equipment utilization, it can be considered as a kind of clean energy resource.Thus, applying DR programs in residential areas can effectively reduce the emissions of harmful substances such as SO 2 , NO x , and decrease greenhouse gas emissions [60].(2) Renewable resources use (C5): As reserve resources, DR can stabilize renewable energy output, increase its consumptive ability, and promote renewable resource use [62].

Technology Level (Z3)
After stepping into an automation development stage, it is necessary for DR to apply new technologies as soon as possible [14].The technology level refers to advanced terminal equipment or system application for DR programs.The critical criteria affiliated with technology level for DR performance evaluation are summarized as below: (1) Smart meter use rate (C6): Smart meters can record electricity consumption in intervals of an hour or less and provide energy use information for monitoring centers, which can help residents and management departments to implement DR plans [61,63].(2) Smart home application degree (C7): Smart homes include the control and automation of heating, lighting, air conditioning, ventilation and security.Smart home application can help customers find an optimized energy use scheme that maximizes their economic savings as well as fulfills DR program goals [64,65].

Societal Benefit (Z4)
Developing DR programs can promote stable and sustainable development of society.The critical criteria affiliated with society benefits for DR performance evaluation are summarized as below: (1) Power supply reliability and sustainability (C8): DR is an effective manner to improve electricity supply service.By reducing demand peaks and filling load valleys, it can alleviate electricity network stresses, and increase the supply reliability and sustainability of power supply [60,66].(2) Stabilizing electricity market (C9): DR program implementation can reduce spot price volatility, mitigate market power and increase the electricity market efficiency [67].(3) Reduction in service quality or residential comfort (C10): Refers to when participating in the DR programs, residential comfort reduction or depreciation of the quality of the energy service provided [62].
Based on the description of the index system, we can know that except C10, the other criteria all pertain to benefit-type criteria where larger values represent better performance in the following evaluation process.

Case Statement
Beijing, as one of the first DR pilot sites in China, has implemented a series of DR programs aimed at residential customers and has relatively rich experience in this field.Residential consumers or load integrators who participate in DR programs allow the temperature settings of air conditioners and brightness of lighting to be adjusted via intelligent sockets when it is necessary to reduce peak loads and ensure electric reliability.Based on this background, the proposed framework can be applied to the performance evaluation of DR programs for management promotion and sustainable development.We chose five residential DR alternatives Ui (i = 1, 2, . . ., 5) located in the Changping, Haidian, Chaoyang and Fengtai districts in Beijing as the empirical analysis cases; the location description is shown in Figure 4.The five alternatives are all residential areas, and their power supplier is a coal-fired power plant with 300 MW located in Hebei Province.U1 is an ordinary residential district build in 1998 Energies 2018, 11, 1097 14 of 27 with a total of 800 households.Most of the inhabitants of U1 have been living there a long time and the intelligence degree of their devices is not very high.Demand response application is mainly aimed at the heating and air conditioning systems of each household.U2 is a newly-built high-end district with 1000 households and equipped with smart home systems.Its DR service areas are lighting, ventilation and air condition systems.U3 is an ordinary district with more than 20 years of construction history.The intelligence degree is close to that of U1, but its DR services are concentrated on lighting and air conditioning.U4 and U5 are both middle grade districts and their DR control implemented in lighting, ventilation and air conditioning systems.Because the five alternatives are at different levels of intelligence, the DR implementation types are various.U1 and U3 belong to direct load control program type, which means the program operator can remotely shut down the participators' electrical equipment (air conditioner or water heater) at short notice by installing load intervention devices.U2, U4 and U5 are price-based DR programs.Through smart home systems, the residents of U2, U4, and U5 can set up the power output of their electric equipment according to the time-of-use electricity rates.More detailed introduction of the five alternatives is presented in Table 4.  Four groups of experts in the DR research area are selected to make decisions.The characteristics of the experts are described in the Table 5.They are familiar with the DR programs and will give linguistic ratings of criteria and alternatives by mail or phone surveys.Four groups of experts in the DR research area are selected to make decisions.The characteristics of the experts are described in the Table 5.They are familiar with the DR programs and will give linguistic ratings of criteria and alternatives by mail or phone surveys.The selected four groups of experts gave significance comparisons of the different layer criteria including main and sub-criteria based on linguistic ratings and TrFNs in Table 2. Then assemble the linguistic ratings of criteria importance as fuzzy comparison matrices and check their consistency by computing CR values, the results shown in Table 6.As we can see, the CR values are all below 0.2.That means the experts' judgments are consistent and credible.(2) Aggregate the information of the four group experts' comparison matrices to obtain an integrated fuzzy comparison matrix for each level criteria.
Assume that ) is a component in the integrated fuzzy comparison matrix.We give an instance to illustrate how A 12 represents the importance comparison between the economic main criterion (Z1) and environmental main criterion (Z2).The linguistic ratings given by the four expert groups are Z2 WI Z1, Z1 EI Z2, Z1 WI Z2 and Z1 WI Z2.They can be aggregated as: Thus, A 12 = (1.25,1.45, 1.89, 2.13).Analogously, we can calculate the other criteria importance in each integrated fuzzy comparison matrix and the results are listed in Tables 7-11.
Table 9. Integrated fuzzy comparison matrix for Z2 sub-criteria weights.
Through Equations ( 19)-( 22), fuzzy aggregated extent values H can be received, and furthermore transformed to the corresponding BNP values.Via Equation ( 25), the results of subjective weights W SG i will also be obtained.The results are presented in Table 12.

Objective Weight Calculation Based on Shannon Entropy
Based on the performance of five alternatives, the four group experts should give their linguistic evaluation according to Table 1 firstly.The experts' linguistic judgements of alternatives for each criterion is summarized in Table 13.
Table 13.Linguistic judgments of five alternatives for criteria performances.
Then the TrFNs of five alternatives for ten criteria can be aggregated in Table 14 according to Equation (6).We take the TrFN of U1 with respect to criterion C1 for example to explain the calculation process: X 121 = MAX(0.6,0.5, 0.6, 0.5) = 0.6 With Equation ( 5), the BNP numbers matrix of fuzzy evaluation can be aggregated as in Table 15.
Energies 2018, 11, 1097 Using Equations ( 27)-( 30), the normalized decision matrix P ij and objective weights w o j of the criteria in line with entropy method can be obtained, as shown in Table 16.

Subjective and Objective Weights Combination
Integrate the virtues of fuzzy AHP and Shannon Entropy to get the combination weights w c j with Equation (31).The combination weights of each criterion are listed in the Table 17.

Residential DR Performance Evaluation Based on Fuzzy VIKOR
Step 1: Standardize the fuzzy evaluation matrix shown in Table 14 with Equations ( 8) and ( 9).For benefit-type criteria C1, C2, C3, C4, C5, C6, C7, C8 and C9, they are divided by the maximum value in the fuzzy evaluation matrix respectively with Equation ( 8); and for cost-type criterion C10, divide the minimum value by C10 value with Equation (9).The normalization results are presented in Table 18.Step 2: Define the series of positive ideal solutions R + j and negative ideal solutions R − j involved with evaluation criteria by Equation (10), presented in Table 19.Step 3: Defuzzify the fuzzy numbers in Tables 18 and 19, and the results are shown in Table 20.Then calculate the crisp value distances from the positive ideal solutions of five alternatives with respect to each criteria by Equation (11), as listed in Table 21.Step 4: Compute the utility value S i , regret value R i and compromise index Q i , and give the ranks of all alternatives.
Based on Equations ( 12)-( 14), the values of S i , R i and Q i can be obtained as shown in Table 22.Then rank alternatives by the decreasing order of S i , R i and Q i , with η = 0.5, and check the two conditions about the two preferred alternatives ranking.The final performance ranking is U2 > U4 > U1 > U5 > U3, as shown in Table 23.

Findings and Discussion
The performances of five residential DR programs are ranked with the method proposed in this paper.Based on the Q i values from low to high, we can sort the performance as the order of U2, U4, U1, U5, U3.U2 as the optimal program is followed by U4 and U1.From the results we can see that the proposed research framework has its advantages to select a best alternative and give rankings with reference value for decision makers.To further check the effectiveness of the modified fuzzy VIKOR, we compare the evaluation results with traditional weighting procedure results.Moreover, a series of sensitivity analyses are performed in this section for verifying the model steadiness and robustness.
The traditional fuzzy VIKOR is based on one subjective or objective weighting methods, which may ignore some important information.Table 24 shows the comparison results of fuzzy VIKOR with different weighting procedures including fuzzy AHP, entropy and combination weighting method.The evaluation ranks of five alternatives with entropy weighting method and the proposed combination weighting method are coincident.However, in the entropy weighting situation, the Q i values of the first two alternatives U2 and U4 is too close, which doesn't satisfy the selection Condition 1 (Equation ( 15)).Therefore, we can only think U2 and U4 are both optimal alternatives.In the proposed model, the combination weighing method tackles the problem effectively by integrating more information from subjective and objective aspects.Based on the comparison of different weighting methods applied in fuzzy VIKOR, we can find that the combination weighting has its advantages during MCDM.The proposed model can avoid some uncertainty caused by subjective or objective factors and give a more comprehensive evaluation result.Focusing on the above cases, Table 17 presents the initial combination weights allocation for ten sub-criteria.C7, C6 and C2 had the first three weights allocation.That implies C6 and C7, affiliated with technology level of terminal monitoring and control equipment, play an important role in the residential DR programs.Moreover, sub-criterion C2 affiliated with price elasticity also has relative great influence to the DR implementation.As far as we know, DR is still in a starting phase in China, especially in the residential area.Thus, the application of intelligent devices and technology in homes and economic incentives may promote residential DR programs more smoothly.However, with the maturity and extended application of DR in China, the weights of evaluation criteria may fluctuate accordingly.For possessing a profound understanding of the assessment results and check the ranking rationality, we conduct a series of sensitive analyses for criteria weights.Ten sub-criteria involved with the four evaluation aspects reduce their weights by 10%, 20%, 30%, 40%, 50% and increase 10%, 20%, 30%, 40%, 50% than basic weights.Table 17 gives the basic weights.
In the respect of the economic benefits criteria weights change, no matter how sub-criterion C1 changes, the Q i values of the five alternatives have little fluctuation (Figure 5).With the increase of C2, the Q i values of U4 and U5 show swings and even a reversal of ranking.As for C3, its weight increase the influence on Q i of U4 and U5 more obviously.Therefore, C2 and C3 are the sensitive factors, but no matter how the weights change, the optimal alternatives are still U2 and U4.For the sub-criteria affiliated with environmental benefits aspect, the i Q values of five alternatives remain stable when corresponding weights changing, shown as Figure 6, which means C4 and C7 are stable criteria.For the sub-criteria affiliated with environmental benefits aspect, the Q i values of five alternatives remain stable when corresponding weights changing, shown as Figure 6, which means C4 and C7 are stable criteria.
The weight sensitivity analysis for sub-criteria C6 and C7 in the technology main aspect is presented in Figure 7.As C6 and C7 are more important, the Q i values of U1, U3, U4 and U5 fluctuate strongly with the changing weights.We can see that except for U2, the other alternatives are sensitive to the technology level criteria.Although C6 and C7 weights may affect Q i values obviously, the top three alternatives are still U2, U4 and U1.
For the sub-criteria affiliated with environmental benefits aspect, the i Q values of five alternatives remain stable when corresponding weights changing, shown as Figure 6, which means C4 and C7 are stable criteria.The weight sensitivity analysis for sub-criteria C6 and C7 in the technology main aspect is presented in Figure 7.As C6 and C7 are more important, the i Q values of U1, U3, U4 and U5 fluctuate strongly with the changing weights.We can see that except for U2, the other alternatives     After the sensitivity analysis of criteria weights, for further examining the robustness and rationality of the evaluation framework, we conduct another sensitivity analysis for parameter η, which represents the weight of maximum group benefits in the fuzzy VIKOR method.The η value is changed from 0.0 to 1.0 by steps of 0.1 to disclose the influence of ultimate order.The analysis conclusion is presented graphically in the Figure 9 and the alternatives final evaluation rankings are shown in the Figure 10.The best option is still U2 followed by U4 and U1.
Above all, the series of sensitivity studies prove that the evaluation results given by the established method are reliable and effective.It verifies the modified fuzzy VIKOR and combination weighting method is robust and valid to deal with a MCDM problem.After the sensitivity analysis of criteria weights, for further examining the robustness and rationality of the evaluation framework, we conduct another sensitivity analysis for parameter , which represents the weight of maximum group benefits in the fuzzy VIKOR method.The  value is changed from 0.0 to 1.0 by steps of 0.1 to disclose the influence of ultimate order.The analysis conclusion is presented graphically in the Figure 9 and the alternatives final evaluation rankings are shown in the Figure 10.The best option is still U2 followed by U4 and U1.Above all, the series of sensitivity studies prove that the evaluation results given by the established method are reliable and effective.It verifies the modified fuzzy VIKOR and combination weighting method is robust and valid to deal with a MCDM problem.

Conclusions
For the purpose of evaluating DR programs in the residential area, a hybrid framework is proposed in the paper.We can select optimal alternatives and promote the management of residential DR programs in virtue of the method.Based on the sustainable development perspective, an index system containing economy, environment, technology and society aspects is established for program performance evaluation.Then, to deal with the vagueness of experts during decision making process, the fuzzy VIKOR method with TrFNs is applied to give a comprehensive

Conclusions
For the purpose of evaluating DR programs in the residential area, a hybrid framework is proposed in the paper.We can select optimal alternatives and promote the management of residential DR programs in virtue of the method.Based on the sustainable development perspective, an index system containing economy, environment, technology and society aspects is established for program performance evaluation.Then, to deal with the vagueness of experts during decision making process, the fuzzy VIKOR method with TrFNs is applied to give a comprehensive assessment result.The method has advantages in grasping the fuzziness information and coping with uncertainty of subjective judgements.Moreover, the paper modified the fuzzy VIKOR by introducing a combination weighting technique, in which fuzzy AHP and Shannon entropy methods played their own function in obtaining the subjective and objective weights.Compared with the traditional weighting procedure in fuzzy VIKOR, the proposed combination weighting method can integrate more information and give more rational weight allocation.The compound framework has clear calculation procedures and is proven reliable and effective in the empirical analysis.In the case study, sub-criteria C6, C7 and C2 affiliated with technology and economy dimensions garnered more attention from experts.To confirm the robustness and reliability of the model and rankings, we conduct a set of sensitivity analyses.The results indicate that however the ten sub-criteria weights and η value adjust, the optimal alternatives are always U2 and U4 at the first and second preference respectively.
Although the hybrid approach is proven effective and suitable to deal with multi-criteria decision making issues, there still are some limitations to be improved.Since the index system is determined according to some research references and experts' opinions, as the development situation for DR programs changes, the evaluation criteria need to be updated in a timely way.Thus, it is necessary to perform the calculation procedures again based on the new index system.Moreover, from a methodological perspective, we will check the proposed framework by comparing with other approaches in a further study.

I
and H sub i to BNP values B m I and B s i .Step 4: Compute the main criteria and sub-criteria weights Consequently, let W main I and w sub i represent the weight of main criterion and local sub-criterion respectively, as:

Figure 2 .
Figure 2. The research framework of the hybrid MCDM model for evaluating residential DR performance.

Figure 2 .
Figure 2. The research framework of the hybrid MCDM model for evaluating residential DR performance.

29 Figure 3 .
Figure 3. Comprehensive criteria for residential DR performance evaluation.

Figure 3 .
Figure 3. Comprehensive criteria for residential DR performance evaluation.

( 1 )
Redundant investment suspension (C1): Refers to reducing the need to build extra generation or transmission infrastructure.DR programs can postpone or avoid some peaking plants or system capacity investment by adjusting residents' electricity usage habits [59,60].(2) Price elasticity (C2): Refers to the elasticity of electricity demand with regard to electricity price changes in the residential sector.The elasticity can reflect the sensitivity of residents to electricity prices which can directly influence the performance of DR programs [60].(3) Electricity bill reduction (C3): DR programs can guide residents to choose a most economical plan to use energy by combining different prices schemes.Through adjustment of energy use habits, users can also reduce their electricity bills [61].

Figure 4 .
Figure 4. Location description of the selected alternatives in Beijing.

Figure 4 .
Figure 4. Location description of the selected alternatives in Beijing.

Energies 2018 , 29 Figure 5 .
Figure 5. Weight sensitivity analysis for economic benefits criteria.(a) Weight sensitivity analysis for sub-criterion C1.(b) Weight sensitivity analysis for sub-criterion C2.(c) Weight sensitivity analysis for sub-criterion C1.

Figure 5 .
Figure 5. Weight sensitivity analysis for economic benefits criteria.(a) Weight sensitivity analysis for sub-criterion C1.(b) Weight sensitivity analysis for sub-criterion C2.(c) Weight sensitivity analysis for sub-criterion C1.

Figure 6 .
Figure 6.Weight sensitivity analysis for environmental benefits criteria.(a)Weight sensitivity analysis for sub-criterion C4. (b)Weight sensitivity analysis for sub-criterion C5.

Figure 6 .
Figure 6.Weight sensitivity analysis for environmental benefits criteria.(a) Weight sensitivity analysis for sub-criterion C4.(b) Weight sensitivity analysis for sub-criterion C5.

Figure 7 .
Figure 7. Weight sensitivity analysis for technology level criteria.(a) Weight sensitivity analysis for sub-criterion C6.(b) Weight sensitivity analysis for sub-criterion C7.

Figure 8
Figure 8 presents the case in which the weights of sub-criteria affiliated with social benefits aspect fluctuate up and down.The results indicate that C8, C9 and C10 are not the sensitive criteria in the evaluation process.

Figure 7 .
Figure 7. Weight sensitivity analysis for technology level criteria.(a) Weight sensitivity analysis for sub-criterion C6.(b) Weight sensitivity analysis for sub-criterion C7.

Figure 8
Figure8presents the case in which the weights of sub-criteria affiliated with social benefits aspect fluctuate up and down.The results indicate that C8, C9 and C10 are not the sensitive criteria in the evaluation process.After the sensitivity analysis of criteria weights, for further examining the robustness and rationality of the evaluation framework, we conduct another sensitivity analysis for parameter η, which represents the weight of maximum group benefits in the fuzzy VIKOR method.The η value is changed from 0.0 to 1.0 by steps of 0.1 to disclose the influence of ultimate order.The analysis conclusion is presented graphically in the Figure9and the alternatives final evaluation rankings are shown in the Figure10.The best option is still U2 followed by U4 and U1.Above all, the series of sensitivity studies prove that the evaluation results given by the established method are reliable and effective.It verifies the modified fuzzy VIKOR and combination weighting method is robust and valid to deal with a MCDM problem.

Figure 8
Figure8presents the case in which the weights of sub-criteria affiliated with social benefits aspect fluctuate up and down.The results indicate that C8, C9 and C10 are not the sensitive criteria in the evaluation process.

Figure 8 .
Figure 8. Weight sensitivity analysis for social benefits criteria.(a) Weight sensitivity analysis for sub-criterion C8.(b) Weight analysis for sub-criterion C9.(c) Weight sensitivity analysis for sub-criterion C10.

Figure 8 .
Figure 8. Weight sensitivity analysis for social benefits criteria.(a) Weight sensitivity analysis for sub-criterion C8.(b) Weight sensitivity analysis for sub-criterion C9.(c) Weight sensitivity analysis for sub-criterion C10.

Figure 10 .
Figure 10.Alternatives final rankings of  sensitivity analysis.

Table 2 .
Linguistic terms and TrFN numbers comparison.

Table 2 .
Linguistic terms and TrFN numbers comparison.

Table 3 .
RI values for different matrix size.

Table 4 .
The key features of the five DR projects in residential areas.

Table 5 .
Expert group characteristics description.

Table 4 .
The key features of the five DR projects in residential areas.

Table 5 .
Expert group characteristics description.

Table 6 .
CR values of main criteria and sub-criteria comparative matrices.

Table 7 .
Integrated fuzzy comparison matrix for main-criteria weights.

Table 8 .
Integrated fuzzy comparison matrix for Z1 sub-criteria weights.

Table 11 .
Integrated fuzzy comparison matrix for Z4 sub-criteria weights.

Table 12 .
Fuzzy aggregated extent values and evaluation criteria weights.

Table 14 .
Fuzzy evaluation matrix of alternatives according to criteria performance.

Table 15 .
BNP numbers of the evaluation criteria.

Table 16 .
P ij and evaluation criteria objective weights according to entropy method.

Table 17 .
Combination weights of the evaluation criteria.

Table 18 .
Standardization for fuzzy evaluation matrix of five alternatives.

Table 19 .
Fuzzy ideal solutions for evaluation criteria.

Table 20 .
The BNP numbers for the standardization evaluation matrix and ideal solutions.

Table 21 .
The standardized distances for five alternatives on each criterion.

Table 22 .
Calculation results of S i , R i and Q i .

Table 23 .
The performance orders of all the alternatives.

Table 24 .
The performance ranks of five alternatives using different weighting methods.