Efficiency Evaluation of Operation Analysis Systems Based on Dynamic Data Envelope Analysis Models from a Big Data Perspective

The operating environment of power grid enterprises is complex with a range of monitoring indicators. To grasp the overall operational status in time and find the key affecting factors, Balanced Scorecard Card (BSC), Interpretive Structural Model (ISM), Principal component analysis (PCA) should be applied. This paper proposed several grid enterprise operators and monitoring indicator systems (which include achievement indicators and driver indicators), and applied PCA for verification and evaluation. The achievement indicators mainly reflected the effectiveness of business operations, which included corporate value, social value, customer value, and so on. Driver indicators mainly reflected the core resources and operations process management of business operations, which have a direct impact on the achievement indicators. The driver and achievement indicators were used as input and output indicators for the provinces to assess the efficiency of operations, and appropriate measures were proposed for improvement. The results showed that the dynamic data envelopment analysis (DEA) model could reflect the time lag of the grid enterprises operating investment and income much better than the other two methods, and the static changes compared to assess efficiency had an average around 4%.


Introduction
As distributed energy has developed, regional energy grids have gradually formed.For example, in industrial parks, the Combined Cooling Heating and Power (CCHP), which has formed a network structure of district energy power generation, heating, and cooling, has been adopted.Energy efficiency has been improving with the development of distributed energy, but has also brought new challenges to district heating and cooling networks, especially for district power grid operation and management, which calls for greater requirements.Enterprise operational efficiency evaluation is an important method used to evaluate the investment behavior and effectiveness of managers.An efficiency evaluation of power grid enterprises is mainly conducted to evaluate the efficiency of grid investment behavior made by power grid enterprise managers, and to sort the evaluation object based on the evaluation results to provide the theoretical basis for judging the investment behavior of power grid enterprises.
First, a grid operational efficiency evaluation should involve both design and management issues, which are inseparable.At present, however, evaluations of the operating efficiency of power grid enterprises mainly focus on financial indicators, and ignore the entire process of operations management, which leads to some existing problems of evaluation as the evaluation method is limited to the feasibility study of the single power grid construction project, and the non-financial indicators are always despised.Furthermore, these types of evaluation mainly use the financial internal rate of return and financial net present value to reflect the performance of a single power grid construction project, so lack indicators that can accurately reflect supply reliability, voltage, and cannot completely reflect power grid development and efficiency.Therefore, this paper puts forward the analysis of operational performance indicators based on the whole process of grid operation perspective, extracts key indicators through scientific and reasonable methods, and uses them as input-output indicators to analyze the efficiency of the assessment objects.
Second, a proposal has been made for the use of the Cobb Douglas production function as an intelligent technology evaluation method in the analysis of the technical progress of a grid.This method reflects the development level of an intelligent power grid through the contribution of technology for economic benefits [1].To evaluate the technical and scale efficiency of a power grid, integrated data envelopment analysis was adopted [2].The Stochastic Frontier Model (SFM) was proposed to evaluate the efficiency of a smart grid [3]; furthermore, some scholars have used data envelopment analysis (DEA) to evaluate the technical efficiency of hydraulic power generation enterprises and transmission-distribution systems [4,5].However, the results of these studies were based on static data, and power grid operation is unique as it exhibits a strong time lag in investment and income, which traditional DEA and SFM methods cannot solve.To fuse the time factor problem, big data theory may provide some favorable resources.
To resolve the above-mentioned issues, this paper proposed the selection of monitoring indicators that reflected the operational efficiency of these enterprises.The Interpretive Structural Model (ISM) was used to filter the driving and achievement indexes, and the improved DEA method was used to evaluate the operational performance of the power grid enterprises to improve the efficiency of the evaluation.The methodology proposed in this paper can be applied in two ways: first, for power grid enterprises, the results can be used directly in the drive and achievement indicators, and Dynamic Data Envelopment Analysis (D-DEA) can be used to calculate operational efficiency.For non-grid companies, the drive and performance indicators should first be sorted through the ISM before D-DEA is used to calculate operational efficiency.

Operational Monitoring Index System of Power Grid Enterprises
The monitoring index is the main mean in power grid enterprises that reflects business transaction and efficiency [6,7].Related indexes include human resources, financial resources, material resources, power grid planning, infrastructure, operations, maintenance, marketing and other aspects.In such a large group, a comprehensive evaluation index system not only evaluates the operation state, but also analyzes the internal driving factors of enterprise development.
The Balanced Scorecard (BSC) theory [8] can be used to adjust four aspects of finance: customers, internal operations, learning and growth [9]; however, one must also consider the public interest in power grid enterprises.As operational performance can be reflected in social value, the financial index was adjusted for the operating results index.The second was to consider the non-quantifiable index in the learning and growth index.As the power enterprise belongs to a traditional energy enterprise that pays more attention to resource investment, this type of index was adjusted to a core resource index.
The improved index system included core resources, internal operations, customer market, and operating results.The core resource index reflected the basic condition of enterprise operation and resource investment; the internal operation index reflected the collaboration and effectiveness of internal operations; the customer market index reflected the effect and recognition degree of operation activities; and the operating result reflected the final value of the enterprise operations, including financial performance, energy saving, and emission reduction performance.

Index Identification Method
This paper proposed an operational monitoring index system that identified key indicators within huge amounts of data that produced very significant results to identify operational changes and monitor operational risk.For power grid enterprises, the key indicator is one of duality: the result indicator reflects the operational situation, and the driving indicator has the largest effect on the performance indicator.
The Interpretative Structural Model is a special research method used to analyze the correlation structure of complex elements in educational technology research [10], with the role of revealing the internal structure using known relationships among the system elements.The ISM was improved in this paper (based on the relation graph of the reachability matrix) to select the operational outcome index from the top down and the operation driving index from the bottom up, before checking the selection index using the principal component analysis method.The working process of the ISM can be divided into the decision making stage and calculation process, which is shown in Figure 1.

Index Identification Method
This paper proposed an operational monitoring index system that identified key indicators within huge amounts of data that produced very significant results to identify operational changes and monitor operational risk.For power grid enterprises, the key indicator is one of duality: the result indicator reflects the operational situation, and the driving indicator has the largest effect on the performance indicator.
The Interpretative Structural Model is a special research method used to analyze the correlation structure of complex elements in educational technology research [10], with the role of revealing the internal structure using known relationships among the system elements.The ISM was improved in this paper (based on the relation graph of the reachability matrix) to select the operational outcome index from the top down and the operation driving index from the bottom up, before checking the selection index using the principal component analysis method.The working process of the ISM can be divided into the decision making stage and calculation process, which is shown in Figure 1.

Operational Efficiency Evaluation Model Based on Dynamic Data Envelopment Analysis (D-DEA)
Data Envelopment Analysis, which is based on the concept of relative efficiency, can be used in efficiency evaluation as a non-parametric statistical method to evaluate the efficiency of decisionmaking units with the same type of multiple input and output [11][12][13].The idea is to regard an economic system as a unit (called a Decision Making Unit (DMU)), and make a "product" by putting in production factors on the grandest scale possible.The evaluation group is composed of DMUs where through the analysis of input or output ratio, evaluation variables are index weights of the DMU.Next, the efficient production frontier is determined and the validity of each DMU can be known by the distance between the DMU and the frontier.At the same time, the projection method can be used to point out the improved direction.
For model selection, this study focused on two aspects.First, compared to other models, DEA models are not only used to obtain the efficiency of each DMU, but also to solve problems where the data contain negative values due to their translation invariance.Second, Tone and Tsutsui [14] classified the constraints of linking two types: the free link and the fixed link.With respect to the free link case, the linking activities were freely determined whilst continuity between the input and output was maintained.This case demonstrated whether the current link flow was appropriate in light of the other DMUs.With respect to the fixed link case, the linking activities were kept unchanged.
This model also considered the time continuity of enterprise development, which means that the grid construction effect of the last time-period is the initial condition of the next time.The management efficiency evaluation model is dynamic, and so defined as the Dynamic Data Envelopment Analysis.At the same time, it considers the relevance of different attributes, that is, the

Operational Efficiency Evaluation Model Based on Dynamic Data Envelopment Analysis (D-DEA)
Data Envelopment Analysis, which is based on the concept of relative efficiency, can be used in efficiency evaluation as a non-parametric statistical method to evaluate the efficiency of decision-making units with the same type of multiple input and output [11][12][13].The idea is to regard an economic system as a unit (called a Decision Making Unit (DMU)), and make a "product" by putting in production factors on the grandest scale possible.The evaluation group is composed of DMUs where through the analysis of input or output ratio, evaluation variables are index weights of the DMU.Next, the efficient production frontier is determined and the validity of each DMU can be known by the distance between the DMU and the frontier.At the same time, the projection method can be used to point out the improved direction.
For model selection, this study focused on two aspects.First, compared to other models, DEA models are not only used to obtain the efficiency of each DMU, but also to solve problems where the data contain negative values due to their translation invariance.Second, Tone and Tsutsui [14] classified the constraints of linking two types: the free link and the fixed link.With respect to the free link case, the linking activities were freely determined whilst continuity between the input and output was maintained.This case demonstrated whether the current link flow was appropriate in light of the other DMUs.With respect to the fixed link case, the linking activities were kept unchanged.This model also considered the time continuity of enterprise development, which means that the grid construction effect of the last time-period is the initial condition of the next time.The management efficiency evaluation model is dynamic, and so defined as the Dynamic Data Envelopment Analysis.At the same time, it considers the relevance of different attributes, that is, the benefit of construction investment from a property may become the investment of another property.Thus, the target function of the D-DEA was as follows: where x t ijk is the value of the input index i in t year; y t rjk is the value of output index r in t year; s t− ijk is the slack variable of input index i in t year; and s t+ rjk is the slack variable of output index r in t year.The constraint conditions were as follows: where indicates the associated value of decision-making unit j from attribute k to attribute h during t period.z t,t+1 jk i represents the influence value of attribute k from t to t + 1.Through optimization and adjustment, the setting parameters were changed as follows: The efficiency value of each DMU was obtained with Equation ( 5): Among them, the efficiency value of each attribute of the DMU was calculated:

Operational Efficiency Evaluation Model Based on Stochastic Frontier Function
Combined with cost and production functions, SFM can be used to realize the allocation efficiency evaluation.The Douglas's cost function with stochastic frontier characteristics can be defined as [15][16][17]: where C it is the total cost of grid enterprise i in the t period; p s it is the price of input factor s for grid enterprise i in the t period; X s it is the input factor s for grid enterprise i in the t period and is only k it or l it ; and s is the type of input element.
From Equation ( 7), if the cost function was used to evaluate the allocation efficiency, the known price information of the input elements is the primary condition.To avoid the price information demand of the input elements as per the inherent duality between the Cobb-Douglas production and cost function, the following optimization model as established by combining Equation (8).
The objective function of Equation ( 8) was to minimize the cost, and the constraint condition indicated that the input-output met the condition of the Cobb-Douglas production function.Taking the production function of the input element into consideration, the extremum problem of this condition was solved with Lagrange's function.
By obtaining the exponential function from the production function, we can see The coefficient k it was calculated by The obtained parameter function was brought into the original formula, and the unknown parameters were eliminated, so the new cost function became Based on the general form of the Cobb-Douglas production function, we know that the cost efficiency is the exponential term of cost function.Therefore, the cost efficiency, technical efficiency, and allocation efficiency were obtained by

Results and Discussion
By using the actual operation monitoring index of the State Grid Corporation as a case study, 32 relevant indicators with the highest attention were selected in operation, and the specific corresponding relationships of the indicators are shown in Table 1.

Identification and Verification of Dual Index in Power Grid Enterprises
First, the index system was analyzed, and the element relation was constructed as per the relationship among indexes determined by expert grading.Next, an adjacency matrix was established based on this relationship (Figure 2), and a reachability matrix was calculated through the matrix operation (Figure 3). ]

Results and Discussion
By using the actual operation monitoring index of the State Grid Corporation as a case study, 32 relevant indicators with the highest attention were selected in operation, and the specific corresponding relationships of the indicators are shown in Table 1.

Identification and Verification of Dual Index in Power Grid Enterprises
First, the index system was analyzed, and the element relation was constructed as per the relationship among indexes determined by expert grading.Next, an adjacency matrix was established based on this relationship (Figure 2), and a reachability matrix was calculated through the matrix operation (Figure 3).Finally, the reachability matrix was decomposed under the condition of   ( ) ( ) and the relation diagram was constructed based on the result (Figure 1).As seen in Figure 4, the 32 indexes were distributed at two ends of the level, which meant that the element closer to the top was more likely to be affected, so had great dependence; and the element closer to the bottom was more likely to impact others, so had great driving force.Two core indicators were divided under different requirements: the achievement index group and the driving index group (specific indicators are shown in Table 2).The data was analyzed by SPSS, and to test whether 16 core indicators from the structural model could achieve a contribution rate of more than 85%.Based on the software results (Table 3), the contribution rate was more than 85%, which meant that the extracted index by the structural model could represent the operational monitoring index of a power grid.
Finally, the reachability matrix was decomposed under the condition of R(S i ) ∩ Q(S i ) = R(S i ), and the relation diagram was constructed based on the result (Figure 1).
As seen in Figure 4, the 32 indexes were distributed at two ends of the level, which meant that the element closer to the top was more likely to be affected, so had great dependence; and the element closer to the bottom was more likely to impact others, so had great driving force.Two core indicators were divided under different requirements: the achievement index group and the driving index group (specific indicators are shown in Table 2).Finally, the reachability matrix was decomposed under the condition of   ( ) ( ) and the relation diagram was constructed based on the result (Figure 1).As seen in Figure 4, the 32 indexes were distributed at two ends of the level, which meant that the element closer to the top was more likely to be affected, so had great dependence; and the element closer to the bottom was more likely to impact others, so had great driving force.Two core indicators were divided under different requirements: the achievement index group and the driving index group (specific indicators are shown in Table 2).The data was analyzed by SPSS, and to test whether 16 core indicators from the structural model could achieve a contribution rate of more than 85%.Based on the software results (Table 3), the contribution rate was more than 85%, which meant that the extracted index by the structural model could represent the operational monitoring index of a power grid.The data was analyzed by SPSS, and to test whether 16 core indicators from the structural model could achieve a contribution rate of more than 85%.Based on the software results (Table 3), the contribution rate was more than 85%, which meant that the extracted index by the structural model could represent the operational monitoring index of a power grid.We comprehensively evaluated eight driving indicators, and the results using SPSS software are shown in Table 4. Based on the processing results, we calculated the comprehensive evaluation model, as shown in the formula: where F 1 is comprehensive score; and Z i indicates the standard value of driving indicator i.
(2) Achievement index evaluation We also comprehensively evaluated eight achievement indicators, and the results using SPSS software are shown in Table 5.Based on the processing results, we calculated the comprehensive evaluation model shown in the formula: F 2 = −0.321Z9 + 0.291Z 10 + 0.031Z 11 − 0.112Z 12 + 0.3Z 13 + 0.294Z 14 + 0.109Z 15 − 0.18Z 16 (17) where F 2 is the comprehensive score; and Z i indicates the standard value of achievement indicator i.
Based on the comprehensive evaluation model of the driving and achievement indexes, the index scores were calculated in combination with nearly six years of standard data.The trend chart is shown in Figure 5. From Figure 5, the driving index of operational performance had a rising trend.Although the achievement index as basically the same, there were some differences, especially shown by the different trends in 2011 and 2012.This difference showed that the driving index was not only the influencing factor of the achievement index, but the most important one.Thus, the index analysis could be replaced by the driving and achievement indexes, which could greatly reduce the amount of computation and improve efficiency.Combined with the general trend and efficiency evaluation of the driving and achievement indexes, it could release an early warning signal and make recommendations.

Operational Efficiency Evaluation of Power Grid Enterprises
(1) Comparison of operating efficiency over the years We used the driving index as the input index, and the achievement index as the output index.When considering the availability of the indicators, the output index was added, including the increased supply load of unit investment, the increased consumption of unit investment, and the transmission cost of unit quantity.Next, a company's operational efficiency was analyzed and some improvements made based on the evaluation results.
First, actual data were collected from 2008 to 2013.The mixed distance function and bidirectional priority of input-output were used in the calculation.The result showed that annual efficiency evaluations were 1 from 2008 to 2013, and the scale returns were stable.Thus, the technical efficiency and scale benefit scores of this enterprise were 1, which meant it was well controlled in input-output proportion.At the same time, it also showed that input decision and post management were reasonable and had achieved good operating efficiency.
(2) Operational efficiency comparison among enterprises As the contrast of operational efficiency can improve the investment decision, we selected 14 provincial companies to calculate their efficiency based on data from 2013.The mixed distance function and bidirectional priority of input-output were also selected, and the operation results are shown in Table 6   From Figure 5, the driving index of operational performance had a rising trend.Although the achievement index as basically the same, there were some differences, especially shown by the different trends in 2011 and 2012.This difference showed that the driving index was not only the influencing factor of the achievement index, but the most important one.Thus, the index analysis could be replaced by the driving and achievement indexes, which could greatly reduce the amount of computation and improve efficiency.Combined with the general trend and efficiency evaluation of the driving and achievement indexes, it could release an early warning signal and make recommendations.

Operational Efficiency Evaluation of Power Grid Enterprises
(1) Comparison of operating efficiency over the years We used the driving index as the input index, and the achievement index as the output index.When considering the availability of the indicators, the output index was added, including the increased supply load of unit investment, the increased consumption of unit investment, and the transmission cost of unit quantity.Next, a company's operational efficiency was analyzed and some improvements made based on the evaluation results.
First, actual data were collected from 2008 to 2013.The mixed distance function and bidirectional priority of input-output were used in the calculation.The result showed that annual efficiency evaluations were 1 from 2008 to 2013, and the scale returns were stable.Thus, the technical efficiency and scale benefit scores of this enterprise were 1, which meant it was well controlled in input-output proportion.At the same time, it also showed that input decision and post management were reasonable and had achieved good operating efficiency.
(2) Operational efficiency comparison among enterprises As the contrast of operational efficiency can improve the investment decision, we selected 14 provincial companies to calculate their efficiency based on data from 2013.The mixed distance function and bidirectional priority of input-output were also selected, and the operation results are shown in Table 6 and Figure 6.From Table 7 and Figure 7, when considering the delay of grid investment, the dynamic and static evaluation results changed with the improved DEA method, and was relatively reasonable for the certain particularities of power grid investment, which include the large scale of fixed assets, long investment cycle, influence of grid structure, and regional economy.Therefore, it was not scientific to evaluate the static efficiency of power grid enterprises, and instead, dynamic efficiency evaluation should be adopted.
Results with a high efficiency declined and increased with low assessment results.Although there was more investment in a low assessment region in 2013, the income was not obvious, and the efficiency evaluation results were low.When considering the delay of investment benefit, the dynamic calculation obtained a higher evaluation efficiency.
Through the comparison of static and dynamic evaluation efficiency, the improved DEA model considered the impact of the previous year's investment on the operating result of the following year.Using Henan Province as an example, by comparing the static and dynamic efficiency evaluation of the two methods, the dynamic efficiency evaluation result improved with the modified DEA method that considered power grid structure and regional economic factors; the results of the dynamic efficiency evaluation did not change with the SFM model, which indicated that these factors were not taken into account.Therefore, the improved DEA model was more consistent with the factual data.From Table 7 and Figure 7, when considering the delay of grid investment, the dynamic and static evaluation results changed with the improved DEA method, and was relatively reasonable for the certain particularities of power grid investment, which include the large scale of fixed assets, long investment cycle, influence of grid structure, and regional economy.Therefore, it was not scientific to evaluate the static efficiency of power grid enterprises, and instead, dynamic efficiency evaluation should be adopted.
Results with a high efficiency declined and increased with low assessment results.Although there was more investment in a low assessment region in 2013, the income was not obvious, and the efficiency evaluation results were low.When considering the delay of investment benefit, the dynamic calculation obtained a higher evaluation efficiency.
Through the comparison of static and dynamic evaluation efficiency, the improved DEA model considered the impact of the previous year's investment on the operating result of the following year.Using Henan Province as an example, by comparing the static and dynamic efficiency evaluation of the two methods, the dynamic efficiency evaluation result improved with the modified DEA method that considered power grid structure and regional economic factors; the results of the dynamic efficiency evaluation did not change with the SFM model, which indicated that these factors were not taken into account.Therefore, the improved DEA model was more consistent with the factual data.

Conclusions
The operational monitoring service of a power grid enterprise is pioneering work in the control practice of large enterprise groups.This paper evaluated the operational efficiency of power grid enterprises, and obtained the following conclusions: 1.
The index concerned with power grid enterprises may reach hundreds of items.The monitoring index system and identification method used in this study played a guiding role in simplifying the index.

2.
The key indicator identification method in this paper is feasible and applicable to power grid enterprises, and plays a supporting role in improving operations monitoring and the decision making technology of power grid enterprises.

3.
This paper compared the effectiveness of SFM, traditional DEA, and D-DEA for operational efficiency evaluations.Results showed that D-DEA had a stronger rationality when compared with the static efficiency evaluation where the average change ratio was 4% of the dynamic efficiency evaluation results.
The method proposed in this paper not only is applicable to the efficiency evaluation of regional power grids, but can also be applied to efficiency evaluations of district heating and cooling networks.And, there is still plenty of improved space for model design methods in further study, and is the subject of future research.

Figure 1 .
Figure 1.Basic workflow of the Interpretive Structural Model (ISM).

Figure 1 .
Figure 1.Basic workflow of the Interpretive Structural Model (ISM).

Figure 4 .
Figure 4. Relationship diagram of the operational monitoring index.

Figure 4 .
Figure 4. Relationship diagram of the operational monitoring index.

Figure 4 .
Figure 4. Relationship diagram of the operational monitoring index.

Figure 5 .
Figure 5. Comprehensive trend of operational performance driving index and achievement index.

Figure 5 .
Figure 5. Comprehensive trend of operational performance driving index and achievement index.

Figure 7 .
Figure 7. Evaluation comparison of static efficiency and dynamic efficiency.

Figure 7 .
Figure 7. Evaluation comparison of static efficiency and dynamic efficiency.

Table 1 .
Correspondence relation for index definition.

Table 1 .
Correspondence relation for index definition.

Table 2 .
Achievement index group and driving index group.

Table 2 .
Achievement index group and driving index group.

Table 2 .
Achievement index group and driving index group.

Table 3 .
Contribution rate of core index and total amount.

Table 4 .
Processing results of the driving index.

Table 5 .
Processing results of the achievement index.