Modelling Efficiency of Electric Utilities Using Three Stage Virtual Frontier Data Envelopment Analysis with Variable Selection by Loads Method

Abstract

Electric utility regulators and policy makers implement incentive-based regulation to improve electric utilities efficiency or to manage the cost of electricity. However, poorly implemented regulation may produce undesired results such as low reliability or poor quality of service. Moreover, the competition within the electricity sector is likely to be low because of the high barriers to entry, vertically integrated electric utilities, and high capital requirements. Therefore, benchmarking exercises allow policy makers and regulators to gauge the relative efficiency of electric utilities and help them to reward or penalize the electric utilities accordingly. In this study, we examined the variables that significantly influence the efficiency of electric utilities and developed an optimum method to measure the efficiency of the electric utilities. The results of the efficiency measurement were then used to rank the electric utilities. The result of this study indicates that there are 13 variables that significantly affect the efficiency score of electric utilities and three stage virtual frontier data envelopment analysis (3S-VF-DEA) is the optimum method to measure the efficiency of the electric utilities.

1. Introduction

The electricity generation industry is often regarded as an essential service for a nation since a reliable electricity supply is vital for economic growth. Traditionally, this industry tends to be vertically integrated and has a high barrier of entry for new players. Therefore, policy makers and regulators around the world have implemented various regulations and legislations to regulate the electric utilities and improve the competition in this industry [1]. Vertically integrated electric utilities manage or operate the generation, transmission, and distribution of electricity supply. However, regulators in some countries such as the United States of America [2], United Kingdom [3] and Germany [4] have unbundled vertically integrated utilities into electricity generation, transmission, and distribution businesses.

This purpose of this paper is to measure the efficiency of electric utilities and benchmark the electric utilities against their peers. Benchmarking of electric utilities is an under-researched subject, and this study may improve our understanding of this subject. Apart from that, this study also to tries to improve existing benchmarking studies by eliminates bias in variables selection by incorporating variable selection by loads method. Lastly, this study also tries to determine the optimum method to measure the efficiency of electric utilities by exploring methods such as data envelopment analysis (DEA), virtual frontier DEA (VF-DEA), three stage DEA (3S-DEA) and three stage virtual frontier DEA (3S-VF-DEA).

Section 1 of this paper briefly introduces regulation in electric industry and the remainder of this paper is organized as follows, Section 2 describes some of the efficiency measurement methods, followed by methodology used in this study in Section 3. The findings of this study are discussed in Section 4 and concluded in Section 5. Section 5 describes the variables that significantly affect the efficiency score of electric utilities, the optimum method that can be used to measure the efficiency of the electric utilities and summarizes the benchmarking results.

1.1. History of Electricity Industry

Electricity was developed by scientists such as Girolamo Cardano in 1550, William Gilbert in 1600, Sir Thomas Browne in 1646 and Benjamin Franklin in 1752 [5]. Nikola Tesla was a visionary inventor that developed this field further by inventing alternating current energy transmission, systems of motors and generators [6]. He constructed a brushless alternating current induction motor in 1887 and successfully demonstrated it to the American Institute of Electric Engineers (AIEE) in 1888 [7].

Apart from that. Thomas Edison created long lasting bulbs and demonstrated them to the public in 1879. After developing reliable and safe bulbs for homes, cities, and businesses, he founded Edison Electric Illuminating company one year later. He then built a power plant in New York City in 1882 and named it Pearl Street Station [8]. Thomas Edison also expanded his business to United Kingdom by establishing an agreement with City Corporation to provide street lighting in Goldaming [9].

Meanwhile, the electricity industries grew rapidly in United States from year 1800 to 1920 and these early electric utilities were funded by private investors. The rapid growth also prompted policy makers in New York and Wisconsin to regulate the electric utilities [10].

1.2. Regulation in Electricity Industry

Regulation of private industries began in United States of America in 1877 following a Supreme Court case between the Munn and Scott Warehouse Company and the State of Illinois. The company challenged Illinois for introducing a legislation that set the maximum fees for transportation and storage of agriculture goods, but the company lost their case at a lower court and at the Supreme Court. The Supreme Court opined that governments may regulate private companies if such regulations benefit the public [11].

Initially, legislators and policy makers believed that vertically integrated electric utilities will be able to produce, transmit and distribute electricity at the lowest cost to the consumers. This led to a situation where 90% of the electricity supply in the United States of America was supplied by only 19 holding companies. However, some large holding companies abused a loophole in inter-state electricity supply to achieve higher profits by imposing exorbitant charges and acquiring other electric utilities [10].

Such abuses forced legislators to implement legislation such as Public Utility Holding Company Act (PUCHA) in 1935 [12]. This legislation allowed the government to manage the abuses by restructuring large holding companies into smaller electric utilities. Apart from that, this legislation also allowed smaller electric utilities to own and manage some portion of generation, transmission, and distribution of electricity [3].

The policy makers in United States of America continued to liberalize the electricity industry by implementing Energy Policy Act (EPACT) in 1992 and amending it further in 2005. This legislation liberalized the electricity industry further by allowing the electric utilities to sell their services anywhere in the world and by promoting more competition in the electricity generation businesses. Apart from that, this legislation also allowed Federal Energy Regulatory Commission to regulate wholesale electricity market and give incentive to certain fuels [13].

Apart from United States of America, other countries also have reformed their electricity market by introducing various legislations. For example, lawmakers in India legislated several acts such as Electricity Act 2003 and amended it in 2004, 2007, 2008, and 2020 to reform the electric industry by addressing issues such as accountability, competition, and corporatization [14].

Similarly, the United Kingdom Electricity Supply Industry (ESI) in England and Wales was under public ownership from 1948 to 1990. It was vertically integrated and was operated by Central Electricity Generating Board (CEGB) during this period. CEGB was restructured and privatized in 1990 by dividing it into four (4) companies such as National Power, PowerGen, National Grid and Nuclear Electric. The successful reform of ESI later became a model for power sector reform for the rest of the world [15].

Likewise, Norway also reformed their electricity market by enacting the Energy Act in 1990. Alongside New Zealand and United Kingdom, Norway was one the first to deregulate and liberalize its electricity sector. The main driver for electricity market deregulation in Norway was an increasing dissatisfaction with the performance of the electricity sector, particularly with regard to an excessive investment in capacity that exceeded the demand [16].

1.3. Relationship between Regulation and Efficiency of Electric Utilities

Legislation alone may not be sufficient to improve the efficiency or performance of electric utilities. Therefore, regulators and policy makers have also been implementing incentive-based regulations to encourage the electric utilities to improve their efficiency and deliver expected quality of service to the customer. The regulators are implementing such measures to improve the overall cost competitiveness, increase technical efficiency and strengthen operational performance [17].

Even though regulators and policy makers implement incentive-based regulation to improve the electric utilities efficiency or to manage the cost of electricity, poorly implemented regulation may produce undesired results such as low reliability or poor quality of service. For example, regulations that encourages the electric utilities to lower the cost of electricity may unintentionally encourage the electric utilities to reduce their spending on maintenance that may lead to lower service quality or reliability. Studies have shown that such poorly implemented regulations may have caused major power outage to parts of Canada and United States of America in 2003 [18].

Similarly, the Brazil Electric Authority Agency (ANEEL) enforced a regulation where Brazilian electric distribution companies are compelled to produce zero Economic Value Added (EVA). EVA means that the electric utilities’ earnings shall be equal to their capital costs and ANEEL may reduce the electricity tariff if the electric utilities are generating higher than expected profit [19]. Although ANEEL may have good intentions to manage the cost of electricity, however such regulations will reduce the profitability of the electric utilities and will not attract new investors.

Likewise, Pakistan implemented the Water and Power Development Authority (WAPDA) Act in 2008 to meet future energy demands, improve operational performance and increase efficiency. However, this legislation did not improve the efficiency of electric utilities as expected because the legislation did not address underlying issues such as poor collection, technical losses, non-technical losses, undersized grid, low retail price and, inadequate capital investment [17].

2. Modelling the Efficiency Measurement and Benchmarking

There are various methods or models such as data envelopment analysis (DEA), DEA-Malmquist index, stochastic frontier analysis (SFA) and slack-based measure (SBM) that can be used to measure the efficiency of electric utilities.

DEA is a linear programming model that was introduced by Charnes, Cooper and Rhodes in 1978 and was later improved to account for variable return to scale (VRS) in 1984. DEA is a deterministic method because it measures efficiency as distance from the efficient frontier without considering statistical noise [20].

The Malmquist index was developed by Malmquist (Wu et al., 2017). It is a distance function that uses technical efficiency change (TEC) and technical change (TC) components. The Malmquist index can be calculated from both DEA and SFA methods [21].

Meeusen and Brook developed SFA in 1977 as a parametric method that integrates error terms such as inefficiency and statistical noise. The main advantage of SFA is its ability to measure efficiency while simultaneously considering the presence of statistical noise [22].

Traditional DEA models such as Charnes-Cooper-Rhoder (CCR) and Banker-Charnes-Cooper (BCC) models ignore slack adjustments and cannot distinguish efficient DMUs effectively. Tone (2001) developed a non-radial SBM model based on DEA-CCR model that are more accurate because it can eliminate deviation of efficiency measurement caused by the difference of radial selection. In contrast to CCR or BCC, SBM deals directly input excess and output shortfalls. Tone (2001) also stated that SBM is a product of input and output inefficiencies [23].

However, the selection of efficiency measurement methods and variables are often influenced by quality and availability of data [24]. Therefore, the efficiency measurement methods and variables are being improved continuously and existing efficiency measurement methods may not be perfect for all conditions.

2.1. Variables Selection Methodologies

Selection of variables is critical because it can significantly influence electric utilities efficiency measurement and score. The efficiency score may vary significantly if regulators or policy makers do not consider all pertinent variables or if they decide to give higher weightage to certain variables [25].

Moreover, the variables chosen by the policy makers or regulators may not be perfect and may include extraneous variables. Therefore, it is important that the variables are selected correctly because inclusion of irrelevant variables or exclusion of significant variables may lead to inaccurate efficiency score [26]. Additionally, the selection of variables may affect methods such as DEA by altering its frontier and may produce inaccurate efficiency scores [27].

Similarly, studies have shown that accurate selection of variables will eliminate or reduce inclusion irrelevant variables and will reduce the size of efficiency measurement method. The reduction of variables may also produce more accurate efficiency scores [28]. Apart from that, exclusion of significant variables may also produce unexplained gap between estimated and measured efficiency score [29].

Some of the variables that were used in energy sector related efficiency measurement studies are listed in

Table 1. Variables Used in Previous Energy Sector Related Efficiency Measurement Studies.

Objective	Variables	Author, Year, [Reference]
To measure and evaluate the performance of electricity generation systems	Energy produced, labor, fuel consumption, capacity of transfer stations, length of transmission lines, energy delivered, total energy sold, number of consumers and total distribution transformers	Alizadeh et al., 2020, [31]
To measure and evaluate the efficiency of 32 thermal power plants	Energy produced, generation capacity, number of employees, CO2 emission and water pollution	Khodadadipour et al., 2021, [32]
To evaluate the impact of energy reforms on energy efficiency	GDP, labor, energy consumption and CO2 per capita	Mohsin et al., 2021, [33]
To measure and identify significant factors of China’s coal resources	GDP, total coal consumption, resident population, GDP, pollution control investment, science & technology expenditure, and industry structure	Xue et al., 2021, [34]
To measure the efficiency of coal fires power plants in India	Installed capacity, GDP, capacity utilization, availability factor, coal consumption, secondary fuel oil consumption and auxiliary power consumption.	Jindal & Nilakantan, 2021, [35]

below. Although, studies have shown that variables must be selected carefully because it will affect the efficiency score and influence incentive-based regulation efforts [30], but the selection of the variables are often influenced by socio-economic structure of a country and availability or quality of data [12].

There are several variables selection method that can be used to select the significant variables. For example, researchers such as Omrani, Beiragh, and Kaleibari investigated the efficiency of 37 electric distribution companies in Iran using DEA, principal component analysis (PCA) and game theory. Initially, the number of variables was too big compared to the number of decision making units (DMUs) and therefore, the researchers used PCA to reduce the number variables and improve the accuracy of the DEA method [36].

Another method that can be used to select the significant variables is efficiency contribution measure (ECM). ECM method works by comparing two DEA methods; first a DEA method with the test variable and a second DEA method without the test variable. The efficiency scores of both methods will be evaluated using binomial statistical test to determine if the test variable significantly affects the efficiency score. The process is repeated ether via addition or removal of the other variables. Apart from ECM, regression-based test can also be used to select significant variables. In this method, the initial efficiency score will be calculated using known variables. After that, the efficiency is regressed against potential variables to determine if those variables are statistically significant. These steps are repeated for all potential variables until all significant variables are found [27].

Researchers such as Madhanagopal and Chandrasekaran used a genetic algorithm (GA) to reduce the number of variables to measure the efficiency of the Indian banking sector. They used DEA with GA to reduce the number of variables from 13 to 6 [37].

More recently, Fernandez-Palacin, Lopez-Sanchez and Munoz-Marquez used variable selection by loads method to reduce the number of variables in DEA. They used this method to measure and rank research activities carried out by Spanish public universities. Compared to other variables reductions methods, the variable selection by loads method can detect bias and eliminate them. Therefore, this method may prevent inclusion of irrelevant variables in the efficiency calculation and produce a more accurate efficiency score. The authors of this method have created an online application at http://knuth.uca.es/DEA (accessed on 8 June 2021) [26].

Apart from that, researchers have utilized multiplier restrictions such as assurance regional analysis and the cone ratio approach to reduce the number of efficient DMUs and improve the discriminatory factor. However, such multiplier restrictions require prior information such as previous experience to restrict the multiplier in a specific range and may not produce feasible solution without such prior information [38].

The variable selection by loads method (ADEA Selection Method) will be used in this study to measure the efficiency of electric utilities since it is able to reduce the number of variables without bias and improve DEA method’s discriminatory factor.

2.2. Efficiency Measurement Methods

DEA is one of the earliest efficiency measurements methods, developed by Charnes, Cooper and Rhodes in 1978 and published in the European Journal of Operations Research. This model is known as CCR model and assumes constant return to scale (CRS) assumption. In CRS assumption, production output increases or decreases proportionally to the changes in the inputs.

DEA measures the efficiency of DMUs by creating a non-parametric envelopment frontier over data points and uses it to identify efficiency and inefficient DMUs. Let’s assume that we have n firms which are represented as DMUj (j = 1,2,…,n). Each firm produces s outputs yrj (r = 1,2,…,s) and uses m inputs xij (i = 1,2,…,m). Therefore, efficiency of DMU_o can be formulated as shown below [39]:

$$\mathrm{maximize}\mathsf{\theta }={ }_{}\frac{{{\displaystyle \sum }}_{r=1}^s{u}_r{y}_{r0}}{{{\displaystyle \sum }}_{i=1}^m{v}_i{x}_{i0}}$$

subject to $\frac{{{\displaystyle \sum }}_{r=1}^s{u}_r{y}_{rj }}{{{\displaystyle \sum }}_{p=1}^m{v}_{i}x{i}_{ij }}$ ≤ 1; j = 1…n, r = 1,…,s

$u_r, v_p\ge 0$, i = 1,…,m, r = 1,…,s

This formulation above can be re-expressed in linear programming as shown below:

$$maximize \mathsf{\theta }={{\displaystyle \sum }}_{r=1}^s{u}_r{y}_{r0 }{}_{ }$$

subject to ${{\displaystyle \sum }}_{r=1}^s{u}_r{y}_{rj }-{{\displaystyle \sum }}_{i=1}^m{v}_i{x}_{ij } \le 0;j=1\dots n$

${\displaystyle \sum }_p^m{v}_i{x}_{i0 }$ = 1

$u_r, v_p\ge 0$, i = 1,…,m, r = 1,…,s

Referring to Figure 1 below, assume that a production functions requires two (2) kinds of resources. These resources are labelled as x1 and x2. D1–D5 are efficient DMUs because they are on the production frontier.

The BCC model [40] was developed by Banker, Charles and Cooper in 1984. This model revised the CCR model by adopting VRS assumption [41]. In VRS assumption, production output does not increase proportionally to the changes in the inputs. In this study, an input oriented radial DEA-BCC model with VRS was used to calculate the electric utilities’ efficiency. The linear programming for the BCC input-oriented model can be formulated as shown below:

$$Min \mathsf{\theta }$$

subject to $\mathsf{\theta }{x}_{i0 }\ge$ ${\displaystyle \sum }_{j=1}^n{\lambda }_j{x}_{ij } ,i=1\dots m$

$y_{r0 }\le$ ${\displaystyle \sum }_{j=1}^n{\lambda }_j{y}_{rj } ,r=1\dots s$

${{\displaystyle \sum }}_{j=1}^n{\lambda }_j$ = 1

${\lambda }_j\ge 0$, j = 1,…,n

Efficiency of the DMUs can also be calculated using input-oriented model or output-oriented model [42]. The output-oriented model shall be used when we have more control over the outputs and input oriented model shall be used when we have more control over the inputs.

Although DEA can measure the efficiency of the DMUs, however DEA will not be able to differentiate or rank the DMUs on the production frontier [43]. In such situations, a virtual frontier can be added to overcome these limitations as shown in Figure 2. The virtual frontier is formed from a set of virtual or dummy DMUs that are created by reducing the input variables by a factor of 0.95 and increasing the output variables by a factor of 1.05 [44]. In this example, V1–V5 will form a new production frontier where D1–D5 will be inefficient, and this will allow DEA to differentiate the efficiency of D1–D5.

Researchers such as Cui and Li have used the so-called virtual frontier DEA (VF-DEA) method to measure the carbon efficiency of transportation industry. They collected data from 15 countries from year 2003–2010 and concluded that management and technology factor significantly influence the carbon efficiency. However, structure factors do not significantly influence the transportation industry’s carbon efficiency [44].

The same researchers improved the VF-DEA method and used the virtual frontier benevolent DEA cross efficiency model (VFB-DEA) to measure the energy efficiency of airlines industry from 2008–2012. The collected data from 11 airlines worldwide and concluded that capital efficiency significantly influences the energy efficiency of the airlines [43].

Although VF-DEA method can differentiate efficient DMUs, however this method is not able to eliminate environmental or other random factors. Therefore, researchers have developed Three Stage DEA (3S-DEA) to eliminate these environmental or random factors [30]. The steps for calculating efficiency using 3S-DEA method is as shown below:

(1)

First stage: Measure efficiency of DMUs using DEA method,

(2)

Second stage: Using the DEA efficiency score from the previous step, conduct stochastic frontier analysis (SFA) to exclude environmental or random factors and adjust the inputs,

(3)

Third stage: Measure the efficiency of DMUs using DEA methods and adjusted inputs [45].

In the first stage, the efficiency of the DMUs is measured using DEA-BCC method. However, the efficiency scores are affected by effects such as managerial inefficiencies environmental affects, and statistical noise [46].

In the second stage, regression analysis is conducted with SFA to adjust these effects. The total input slack is identified as S_ni and it is the dependent variable in the SFA regression model. The independent variables in the SFA regression model are the element s of the t observable environmental variables, Z_i = Z_1i, Z_2i,…,Z_ti. The N separate regression model is expressed as follows:

S_ni = fⁿ (Z_i; βⁿ) + (v_ni + µ_ni), n = 1,…,N; i = 1,…,I

Referring to the model above, N is number of firms and I is the number of inputs, fⁿ(Z_i; βⁿ) is the deterministic feasible slack frontier, βⁿ is the environmental vector for estimation and, (v_ni + µ_ni) is the composite error that includes statistical noise and managerial inefficiencies. The input variables of each DMU are then adjusted to SFA regression model results. The adjusted model is expressed as follows:

$$X_{ni}^A = X_{ni} + \left[{max}_i \left\{Z_i{\widehat{\beta }}^n \right\} - Z_i\widehat{\beta }n \left] +\right[{max}_i \left\{{\widehat{v}}_{ni}\right\} - {\widehat{v}}_{ni} \right], n = 1,2,\dots ,N; i = 1,2,\dots ,I$$

In this model, $X_{ni}^A$ is the adjusted input value from the observed input quantity X_ni. In the third stage, the efficiency of the DMUs is re-measured using the adjusted inputs and DEA-BCC method.

Researchers such as Shyu and Chiang used the 3S-DEA method to measure efficiency of 123 bank branches in Taiwan and discovered that environmental factors or statistical noise significantly affects the efficiency of the bank branches [47].

Likewise, Iparraguirre and Ma measured the efficiency of senior people’s social care in 148 English councils using 3S-DEA method. They found that the efficiency of the social care services is very high but the efficiency is negatively influenced by eligibility criteria and eligibility assessment costs variables [48].

Like the DEA method, 3S-DEA method cannot discriminate or differentiate between efficient DMUs. Therefore, researchers such as Cuia and Li have improved 3S-DEA by adding a virtual frontier. The improved method is known as three stage virtual frontier DEA (3S-VF-DEA). The steps for calculating the efficiency of DMUs are like VF-DEA as shown below:

• First stage: Measure efficiency of DMUs using VF-DEA method,
• Second stage: Using the DEA efficiency score from the previous step, conduct stochastic frontier analysis (SFA) to eliminate environmental or random factors and adjust the inputs,
• Third stage: Measure the efficiency of DMUs using VF-DEA methods and adjusted inputs [44].

Cui and Li used 3S-VF-DEA method to examine the efficiency of transportation energy in 30 Provisional Regions in China. They found that the efficiency of transportation energy is significantly affected by management measures and transport structure.

Some of the recent efficiency measurement methods that have been employed by the researchers are shown in

Table 2. Efficiency Measurement Methods Used in Previous Studies.

Objective	Method	Author, Year [Reference]
To evaluate the operational cost efficiency of electric distribution utilities.	DEA (Adjusted Contingent Weight Restrictions), DEA (Constant Return to Scale) and DEA (Non-Decreasing Return to Scale).	Pereira de Souza et al., 2014, [49]
To select the appropriate variables for DEA to assess the relative efficiency of Decision-Making Units (DMU).	DEA and Genetic Algorithm (GA)	Madhanagopal & Chandrasekaran, 2014, [37]
To evaluate the operational efficiency of power distribution utilities.	DEA and Corrected Ordinary Least Squares (COLS) with Cobb Douglas production function.	Costa et al., 2014, [50]
To measure the efficiency of electric distribution in terms of cost reduction and also with respect of overall regulatory framework.	Two stage DEA.	Cambini et al., 2014, [13,51]
To benchmark an electric distribution company’s outage repair and maintenance activities.	Value Based DEA.	Gouveia et al., 2015, [52]
To measure the performance of electric distribution companies in Iran.	Data Envelopment Analysis (DEA), Game theory and, Principal Component Analysis (PCA).	Omrani et al., 2015, [36]
To measure the efficiency of hydroelectric power plants in Angola.	Two stage virtual frontier dynamic DEA	Barros et al., 2017, [53]
To analyze the eco-efficiency of coal-fired plants in China.	DEA cross efficiency model	Liu et al., 2017, [54]
To measure the efficiency of thermal power plants.	DEA cross efficiency model	Khodadadipour et al., 2021, [32]

below. The table shows that researchers have employed improved DEA methods such as value-based DEA, DEA with game theory and PCA, two stage virtual frontier dynamic DEA, and DEA cross efficiency methods.

2.3. DEA Window Analysis

DEA window analysis calculates the average efficiency of CCR and BCC models and it can be used to measure efficiency of DMUs over time [21]. In window analysis, the same DMU in a different period is treated as a different DMU and a moving average method is used to choose different reference set to determine the relative efficiency of each DMU. This method is beneficial when there are limited number of DMUs because this method can increase the number of DMUs and improve discriminatory power [55].

Consider a set of N (n = 1,2,…N) DMUs in T (t = 1,2,…T) period of time and each DMU has r kind of inputs and s kind of outputs. The input vector ($X_n^t$) and output vector ($Y_n^t$) are represented as follows:

$$X_n^t=\left[\begin{array}{c}{y}_n^{1t}\\ .\\ .\\ .\\ y_n^{rt}\end{array}\right] Y_n^t=\left[\begin{array}{c}{y}_n^{1t}\\ .\\ .\\ .\\ y_n^{st}\end{array}\right]$$

If the window starts at the time point of k (1 $\le$ k $\le$ T), and the window width is w (1 $\le w\le$ T–k), then the input X_kw and output Y_kw are represented as follows:

$$X_{\mathrm{kw}}=\left[\begin{array}{c}\begin{array}{c}{x}_1^k\\ x_1^{k+1}\\ .\\ .\\ .\\ x_1^{k+w}\end{array} \begin{array}{c}{x}_2^k\\ x_2^{k+1}\\ .\\ .\\ .\\ x_2^{k+w}\end{array} \begin{array}{c}\dots \\ \dots \\ \\ \\ \dots \\ \end{array}\end{array} \begin{array}{c}{x}_N^k\\ x_N^{k+1}\\ .\\ .\\ .\\ x_N^{k+w}\end{array} \right]Y_{\mathrm{kw}}=\left[\begin{array}{c}\begin{array}{c}{y}_1^k\\ y_1^{k+1}\\ .\\ .\\ .\\ y_1^{k+w}\end{array} \begin{array}{c}{X}_2^k\\ y_2^{k+1}\\ .\\ .\\ .\\ y_2^{k+w}\end{array} \begin{array}{c}\dots \\ \dots \\ \\ \\ \dots \\ \end{array}\end{array} \begin{array}{c}{y}_N^k\\ y_N^{k+1}\\ .\\ .\\ .\\ y_N^{k+w}\end{array} \right]$$

Replacing the above inputs and outputs in the DEA models will generate the results of DEA window analysis.

2.4. Benchmarking in Electricity Industry

Benchmarking is an exercise that includes measuring the efficiency of electric utilities and ranks them based on the measured efficiency scores. The competition within the electricity sector tends to be low because of the high barriers to entry, vertical integration, and high capital requirements. Therefore, the benchmarking exercise will allow the policy makers and regulators to gauge the relative efficiency of electric utilities and ensure that the electric utilities are operating in an efficient manner [56].

Other than that, policy makers and regulators can implement incentive-based revenue and use benchmarking exercise to reward efficient utilities and penalize inefficient electric utilities. Such exercises will encourage the electric utilities to continuously improve themselves and provide the best quality of service to the consumer [25].

There are several studies that evaluates the efficiency of electric utilities and benchmarks them against their peers. Researchers such as Giannakis, Jamasbb, and Pollitt measured the efficiency of electric utilities in United Kingdom using hybrid DEA and Malmquist index using information from year 1991–1992 and 1998–1999. Their study revealed that there is no significant relationship between high quality service and electric utilities that are cost efficient. They also concluded that benchmarking exercise should include quality of service variables since cost only approaches may not produce accurate results [1].

Apart from measuring and comparing the efficiency of electric utilities against their peers, regulators and policy makers can also use benchmarking exercise as a tool to determine electricity rates or tariffs. However, as highlighted by the researchers above, the efficiency measurement and benchmarking must be accurate so that it will allow the policy makers and regulators to determine an accurate electricity tariff [57].

For example, the Finnish Energy Market Authority (EMV) is one the pioneers that implemented benchmark regulation. Initially EMV employed DEA as part of its regulatory model in 2005 and adopted SFA in 2008. However, EMV revised its regulatory model and adopted stochastic semi-nonparametric envelopment of data (StoNED) after its DEA and SFA method was criticized by electricity distribution utilities and other energy industry players [58].

Similarly, Norway implemented its Energy Act in 1991 that laid the foundation for its electricity sector deregulation. However, in Norway, the electricity market was deregulated but not privatized and largely remains under public ownership. The electricity sector in Norway is regulated by Norwegian Water Resources and Energy Directorate (NVE) and this regulator has mainly used DEA for its efficiency measurement [59].

Researchers also have used DEA methods to measure the efficiency and benchmark thermal power plants in India. The study shows that stated owned power plants are less efficiency compared to privately owned power plants. Apart from that, the study also concludes that smaller power plants are less efficient compared to medium and large thermal power plants [60].

Some of other benchmarking exercise or studies that were carried out are as listed in

Table 3. Electric Utilities’ Benchmarking Exercise.

Objective	Benchmarking Period	Author, Year [Reference]
To analyze efficiency of Indian coal fired power plants	2005–2014	Jindal & Nilakantan, 2021, [35]
To analyze efficiency of Brazilian electric distribution utilities	2003–2016	Cardoso de Mendonça et al., 2020, [61]
To benchmark electricity distribution companies in Spain	2016	Núñez et al., 2020, [62]
To carry out total factor productivity (TFP) analysis of vertically integrated electric utility in Malaysia using a Törnqvist index method.	1975–2005	See & Coelli, 2014, [63]
To benchmark transmission electric utilities in United States of America using latent class approach	2001–2009	Llorca et al., 2014, [64]
To assess the performance of electricity distribution utilities in Turkey using FAHP/TOPSIS/DEA methodology and incorporating quality of service.	2002–2009	Çelen & Yalçın, 2012, [65]
To benchmark 72 distribution electric utilities in Turkey for regulatory purposes	2004	Odyakmaz & Scarsi, 2007, [66]

below.

3. Methodology

The data analysis that was employed in this study is illustrated in Figure 3 below. The data analysis begins with data collection from secondary sources that are available online and free to access without any restrictions.

After that, the financial information was converted to local currency and adjusted to the value at year 2018. Next, the data was normalized and DMUs with extreme outliers were removed from the dataset.

Afterwards, the number of input and output variables were reduced using variable selection by loads method. The efficiency of all DMUs were then calculated using DEA, VF-DEA, 3S-DEA and 3S-VF-DEA methods and the efficiency scores were used to determine the discriminatory power. Next, a Monte Carlo simulation was performed to determine the mean absolute deviation (MAD) of the methods and the method with the highest discriminatory factor and consistent MAD score was selected as the optimum method.

Subsequently, a window analysis was performed to calculate average efficiency scores of the electric utilities. The electric utilities were then benchmarked or ranked using the average score of the optimum method to determine their efficiency compared to their peers.

3.1. Data Collection

The data for this study was collected from various publicly available secondary sources such annual reports, statistical databases, stock exchange publications and regulatory publications as shown in

Table 4. Secondary Data Sources.

Description	Source	Website
Database for electric utilities in the United States of America	US Department of Energy	www.energy.gov (accessed on 26 January 2021)
Malaysian electricity sector statistics	Suruhanjaya Tenaga, Malaysia	www.st.gov.my (accessed on 26 January 2021)
Annual report including financial report and performance.	Electric utilities	www.pugetenergy.com (accessed on 26 January 2021); www.fpl.com (accessed on 26 January 2021); www.duke-energy.com (accessed on 26 January 2021); www.aep.com (accessed on 26 January 2021); www.sarawakenergy.com (accessed on 26 January 2021); www.egat.co.th (accessed on 26 January 2021); www.eon.com (accessed on 26 January 2021); www.iplpower.com (accessed on 26 January 2021); www.centerpointenergy.com (accessed on 26 January 2021); www.nmgco.com (accessed on 26 January 2021); www.epelectric.com (accessed on 26 January 2021); www.myavista.com (accessed on 26 January 2021); www.xcelenergy.com (accessed on 26 January 2021); www.entergy.com (accessed on 26 January 2021); www.oncor.com (accessed on 26 January 2021); www.tnb.com.my (accessed on 26 January 2021); www.pacificorp.com (accessed on 26 January 2021); www.midamericanenergy.com (accessed on 26 January 2021)
Security exchange annual report	United States Securities and Exchange Commission	www.sec.gov (accessed on 26 January 2021)
Indiana’s electric utility reliability report	Indiana Utility Regulatory Commission	www.in.gov (accessed on 26 January 2021)
Reliability improvement initiatives	Quanta Technology LLC	www.all4energy.org (accessed on 26 January 2021)
Ireland’s energy generation capacity	Eirgrid Group	www.eirgridgroup.com (accessed on 26 January 2021)
Florida’s electricity service reliability report	Florida Public Service Commission	www.psc.state.fl.us (accessed on 26 January 2021)
Performance of electric utilities in Florida, USA	Florida Public Service Commission	www.publicpower.com (accessed on 26 January 2021)
Performance of electric utilities in Oklahoma, USA	Oklahoma Corporation Commission	oklahoma.gov (accessed on 26 January 2021)
Ireland’s energy statistics	Sustainable Energy Authority of Ireland	www.seai.ie (accessed on 26 January 2021)
Indiana statewide analysis of peak demand reduction initiatives	Indiana Advance Energy Economy	www.aee.net (accessed on 26 January 2021)
Iowa’s electricity performance report	Mid-American Energy Company	iub.iowa.gov (accessed on 26 January 2021)
Mid-American and IPL Reliability Indices	Iowa Utilities Board	iub.iowa.gov (accessed on 26 January 2021)
PLN’s electricity statistics	PLN Indonesia	web.pln.co.id (accessed on 26 January 2021)
Avista’s electric service reliability report	Avista Corp	www.utc.wa.gov (accessed on 26 January 2021)
AEP reliability performance report	American Electric Power Company	www.aepsustainability.com (accessed on 26 January 2021)
Germany electricity market monitoring report	Bundesnetzagentur	www.bundesnetzagentur.de (accessed on 26 January 2021)
Distribution Performance Report	ESB Network Ltd.	www.esbnetworks.ie (accessed on 26 January 2021)
Indiana’s electricity power forecast	State Utility Forecasting Group, Indiana	www.psc.state.fl.us (accessed on 26 January 2021)
NIPSCO Demand Side Management Market Potential Study	Applied Energy Group	www.nipsco.com (accessed on 26 January 2021)
California’s electricity reliability report	Pacific Corp	www.cpuc.ca.gov (accessed on 26 January 2021)
Texas’s electricity reliability report	Public Utility Commission of Texas	www.puc.texas.gov (accessed on 26 January 2021)
Data on California electric utilities	City of Los Angeles	ens.lacity.org (accessed on 26 January 2021)
Oregon’s electricity reliability report	Oregon Public Utility Commission	www.oregon.gov (accessed on 26 January 2021)
Indiana’s electricity statistics	NiSource Inc	www.eei.org (accessed on 26 January 2021)
NiSource’s statistical summary	NiSource	www.nisource.com (accessed on 26 January 2021)

below.

Secondary data from year 2000 to 2018 for 21 electric utilities from countries such as United States of America, Ireland, Germany, Indonesia, Thailand, and Malaysia were collected for this study. The data were collected from the secondary sources above and each of the electric utility were identified as Utility-1 to Utility-21.

Secondary data from 19 other utilities from the Philippines, Australia, Singapore, Thailand, China, United Kingdom, Austria, Portugal, United States of America, Vietnam, Cambodia, Laos, European Union, France, Spain, and Korea we collected during this study. However, those data were excluded from the analysis due data quality issues such as missing data, incomplete data, and language issues.

3.2. Variable Reduction

Initially 10 input variables and 15 output variables as listed in

Table 5. Potential Input and Output Variables.

Variable Description	Type	Variable Code
Operation Cost over Manpower	Input	A1
Operations Cost over Customer	Input	A2
Operations Cost over Total Asset	Input	A3
Operations Cost over Revenue	Input	A4
Operation Cost over Service Area	Input	A5
Inventory over Operation Cost	Input	A6
Utilization of net capacity over Total Assets	Input	A7
Service Area over Total Manpower	Input	A8
Total Assets over No of Customers	Input	A9
Inventory over Total assets	Input	A10
Revenue Over Total Outage Duration	Output	B1
Profit After Tax Over Total Outage Duration	Output	B2
Property, Plant & Equipment (PPE) Over Total Outage Duration	Output	B3
Revenue Over Total no of Interruptions	Output	B4
Profit After Tax Over Total Number of Interruptions	Output	B5
Total Assets over Total Number of Interruptions	Output	B6
PPE over Total Number of Interruptions	Output	B7
Earnings per Share	Output	B8
Profit After Tax over Number of Customer	Output	B9
Profit After Tax Over Total Manpower	Output	B10
Profit After Tax Over Total Assets	Output	B11
Revenue over Number of Customers	Output	B12
Revenue over Total Manpower	Output	B13
Revenue over Total Assets	Output	B14
Earnings Before Interest, Taxes, Depreciation, and Amortization (EBITDA) over Revenue	Output	B15

were investigated for this study. However, researchers such as Nataraja and Johnson have stated that higher number of variables will diminish DEA method’s accuracy and discriminatory power [27]. Other researchers also stated that the total number of input and output variables should be less than half of the number of DMUs [67].

Therefore, the variables in Table 5 will be reduced using variable selection by loads method. This method reduces the number of variables by following these steps:

(1)

Calculate efficiencies of DMUs by using input oriented DMU and with all potential variables,

(2)

Calculate the loads of the variables,

(3)

Eliminate the variable with the smallest load if the load is below the specified threshold,

(4)

Repeat step (1) until all variables’ load are above threshold.

An online application to calculate the above steps are available at http://knuth.uca.es/DEA (accessed on 8 June 2021) and this application was used in this study to reduce the number of variables.

3.3. Benchmarking

In this step, the efficiency of the electric utilities was calculated using the optimum method with variables that have been reduced by using variable selection by loads method. The average efficiency of the electric utilities was calculated using window analysis method for the period of 2000–2018. After that, an electric utility was selected and benchmarked against its peers for the same period.

4. Findings

For this study, a total of 399 observations with 10 input variables and 15 output variables were collected from 21 electric utilities. The electric utilities are from various countries such as United States of America, Malaysia, Thailand, Indonesia, Germany, and Ireland.

Secondary data from other electric utilities from other countries were explored too but were excluded from this study due to inconsistent or incomplete data. The collected secondary data also revealed that emission related data such as SO₂ and CO₂ emission are not available in older datasets.

The secondary data was collected over a period of 19 years from 2000 to 2018 because investments in electric utilities are generally long-term investments and majority of electric utilities’ assets have long lifespans. For example, coal power plants can operate up to 50 years [68], whereas distribution transformers have a lifespan of 75 years [69].

A dataset that with a short time frame may not reveal the actual effectiveness of the managerial decisions related to the investments and may produce inaccurate efficiency scores. Therefore, a dataset with a longer time frame may reveal the effectiveness of the managerial decision and may be more accurate.

The extreme outliers were identified and removed from this dataset using boxplot via IBM SPSS Statistics version 27. A total of 46 outliers were removed from the original dataset and the remaining 353 observations were used for in the variable reduction step.

4.1. Variable Reduction

The descriptive analysis for the variables from Table 5 are shown in

Table 6. Descriptive Analysis of the Variables.

Variable Code	Minimum	Maximum	Mean	Std. Deviation
A1	0.0311	0.4606	0.2189	0.0891
A2	0.0158	0.7110	0.2293	0.1312
A3	0.0144	0.2809	0.0998	0.0450
A4	0.0422	0.3485	0.1513	0.0593
A5	0.0000	0.5647	0.2250	0.1122
A6	0.0016	0.5566	0.1543	0.0951
A7	0.0000	1.0000	0.3084	0.2005
A8	0.0000	1.0000	0.1930	0.1918
A9	0.0000	1.0000	0.3119	0.1825
A10	0.0016	0.4720	0.1755	0.0868
B1	0.0003	0.3401	0.0889	0.0622
B2	0.3789	0.6197	0.4821	0.0361
B3	0.0010	0.8501	0.2517	0.1597
B4	0.0005	0.6456	0.2146	0.1312
B5	0.2953	0.5378	0.4037	0.0328
B6	0.0000	0.1888	0.0619	0.0344
B7	0.0000	0.7566	0.2777	0.1703
B8	0.5401	0.8613	0.7075	0.0452
B9	0.5761	0.8831	0.7463	0.0417
B10	0.4592	0.7945	0.6015	0.0465
B11	0.8274	0.9755	0.9034	0.0231
B12	0.0074	0.6742	0.2354	0.1295
B13	0.0074	0.6742	0.2354	0.1295
B14	0.0318	0.2851	0.1042	0.0461
B15	0.0991	0.5747	0.3220	0.0741

below.

The variables from Table 5 were reduced using variable selection by load method. Looking at the results in

Table 7. Variable Selection by Loads Method.

Step No	Total Variables	Loads	Input Variables	Output Variables
1	25	0.2808	A1, A2, A3, A4, A5, A6, A7, A8, A9, A10	B1, B2, B3, B4, B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15
2	24	0.3084	A1, A2, A3, A4, A5, A6, A7, A8, A9, A10	B2, B3, B4, B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15
3	23	0.3127	A1, A3, A4, A5, A6, A7, A8, A9, A10	B2, B3, B4, B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15
4	21	0.3242	A1, A3, A4, A5, A6, A7, A8, A9	B3, B4, B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15
5	20	0.4085	A1, A3, A4, A5, A6, A7, A8, A9	B3, B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15
6	19	0.4404	A1, A3, A4, A5, A6, A7, A8, A9	B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15
7	18	0.4472	A3, A4, A5, A6, A7, A8, A9	B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15
8	17	0.4741	A3, A4, A6, A7, A8, A9	B5, B6, B7, B8, B9, B10, B11, B12, B13, B14, B15
9	13	0.5007	A3, A4, A6, A7, A8, A9	B7, B8, B10, B11, B13, B14, B15
10	12	0.5282	A3, A4, A6, A7, A8, A9	B8, B10, B11, B13, B14, B15
11	6	0.6411	A3, A4	B10, B11, B13, B15
12	5	0.6546	A3, A4	B10, B11, B15
13	3	0.6729	A4	B10, B15
14	2	1	A4	B15

, there is a significant variable reduction between step number 8 and 9. The load at step is 0.5007 which is better than the recommended value of 0.6.

Therefore, the selected variables are as listed in step number 9 and shown in

Table 8. Selected Input and Output Variables.

Variable Description	Type	Variable Code
Operations Cost over Total Asset	Input	A3
Operations Cost over Revenue	Input	A4
Inventory over Operation Cost	Input	A6
Utilization of net capacity over Total Assets	Input	A7
Service Area over Total Manpower	Input	A8
Total Assets over No of Customers	Input	A9
PPE over Total Number of Interruptions	Output	B7
Earnings per Share	Output	B8
Profit After Tax Over Total Manpower	Output	B10
Profit After Tax Over Total Assets	Output	B11
Revenue over Total Manpower	Output	B13
Revenue over Total Assets	Output	B14
EBITDA over Revenue	Output	B15

below. The input and output variables include various dimensions such as financial, asset, service area, total manpower, electricity interruptions and share earnings.

4.2. Method Selection

Using the selected variables, the efficiency of electric utilities was calculated using DEA, VF-DEA, 3S-DEA, and 3S-VF-DEA methods. The discriminatory power of the methods or the ability of the methods to differentiate the efficiency scores of the DMUs is shown in the

Table 9. Discriminatory Power.

Method	DEA	VF-DEA	3S-DEA	3S-VF-DEA
Percentage Efficient DMUs	65.43%	38.76%	65.48%	26.35%

below.

Based on the Table 9, DEA was not able to differentiate the efficiency scores of 65.43% of the DMUs. Likewise, VF-DEA, 3S-DEA and 3S-VF-DEA were not able differentiate the efficiency score of 38.76%, 65.48% and 26.35% of the DMUs, respectively.

Therefore, 3S-VF-DEA method with variable selection by loads method has the highest discriminatory power because it can differentiate the efficiency score of 73.65% of the DMUs.

A Monte Carlo simulation was also carried out to investigate the performance of the methods under different conditions. The Monte Carlo simulations were developed and simulated using R language and R Studio version 1.3.1093. The Monte Carlo simulation is shown in the Figure 4 below and the simulation is repeated for 50, 100, 150, 200, 250, 350 and 400 DMU using random datasets that has a mean of 0.5 and standard deviation of 1.96.

MAD scores for the Monte Carlo simulation is shown in

Table 10. Monte Carlo Simulation’s MAD Score.

No of DMUs	DEA	VF-DEA	3S-DEA	3S-VF-DEA
50	0.00178	0.00178	0.00196	0.00214
100	0.00180	0.00227	0.00204	0.00238
150	0.00225	0.00234	0.00226	0.00225
250	0.00245	0.00210	0.00251	0.00257
300	0.00271	0.00215	0.00261	0.00234
350	0.00268	0.00236	0.00257	0.00233
400	0.00255	0.00235	0.00286	0.00254

below. The MAD scores indicate that DEA, VF-DEA, 3S-DEA, and 3S-VF-DEA methods were consistent without any significant deviations.

The 3S-VF-DEA has the highest discriminatory power followed by VF-DEA. However, 3S-VF-DEA was able to eliminate statistical noise and therefore was able to produce a more accurate efficiency scores compared to VF-DEA. Based on the findings above, the 3S-VF-DEA method was selected as the optimum method for this study.

4.3. Benchmarking

The average score for 21 electric utilities from year 2000–2018 was calculated using window analysis method. A total of 10 windows was analyzed for this study as shown in

Table 11. Example of Window Analysis for Utility-4.

Utility	Window	DEA	VF-DEA	3S-DEA	3S-VF-DEA
Utility-4	2000–2009	0.999322	0.949356	0.999321	0.949341
	2001–2010	0.999285	0.949321	0.999285	0.949316
	2002–2011	0.999095	0.949141	0.999095	0.949134
	2003–2012	0.998667	0.948580	0.998667	0.948574
	2004–2013	0.999430	0.949406	0.999430	0.949402
	2005–2014	0.999656	0.949593	0.999656	0.949589
	2006–2015	0.999404	0.949290	0.999404	0.949284
	2007–2016	0.999501	0.949486	0.999501	0.949476
	2008–2017	0.999884	0.949795	0.999884	0.949788
	2009–2018	1.000000	0.950000	1.000000	0.949993
	Average	0.999424	0.949397	0.999424	0.949390

. This first window is from 2000 to 2012, whereas the last window is from 2009–2018.

The difference of average score between DEA and 3S-DEA is not significant because the statistical noise is small. Similarly, the difference of average score between VF-DEA and 3S-VF-DEA is not significant for the same reason.

Referring to

Table 12. Average Efficiency Score Using Window Analysis.

Utility	DEA	VF-DEA	3S-DEA	3S-VF-DEA
Utility-1	1.000000	0.949907	1.000000	0.949905
Utility-2	1.000000	0.949971	1.000000	0.949968
Utility-3	0.997890	0.947358	0.997890	0.947355
Utility-4	0.999424	0.949397	0.999424	0.949390
Utility-5	0.995613	0.942379	0.995612	0.942374
Utility-6	1.000000	0.950000	1.000000	0.949994
Utility-7	0.994875	0.939257	0.994875	0.939252
Utility-8	0.964164	0.896576	0.964161	0.896568
Utility-9	0.997409	0.930511	0.997409	0.930505
Utility-10	0.980539	0.922352	0.980538	0.922346
Utility-11	0.998033	0.946898	0.998033	0.946894
Utility-12	0.969406	0.913471	0.969404	0.913465
Utility-13	0.955205	0.887640	0.955203	0.887634
Utility-14	0.989981	0.936039	0.989980	0.936034
Utility-15	0.998075	0.947762	0.998074	0.947757
Utility-16	0.921527	0.855464	0.921522	0.855454
Utility-17	0.926803	0.857814	0.926798	0.857804
Utility-18	0.967181	0.895631	0.967178	0.895624
Utility-19	0.999251	0.944828	0.999250	0.944823
Utility-20	0.989688	0.930621	0.989688	0.930616
Utility-21	0.998636	0.947567	0.998636	0.947559
Average	0.983033	0.925783	0.983032	0.925777

and 3S-VF-DEA method, Utility-6 was the most efficient utility with a score of 0.949994 and the least efficient electric utility was Utility-16 with an efficiency score of 0.855454.

For example, the efficiency of Utility-4′s efficiency compared to its peers is shown in Figure 5 below. This benchmarking example indicates that the efficiency of Utility-4 remained higher than the average score of its peers from year 2000 to 2018.

5. Conclusions

This study was able to determine the variables that significantly influence the efficiency of electric utilities, establish the optimum method to measure the efficiency of electric utilities and benchmark the electric utilities data using window analysis method.

The study indicates that there are 13 variables that significantly affect the efficiency score of electric utilities such as listed below. These variables encompass various dimensions such as financial, asset, service area, total manpower, electricity interruptions and share earnings:

• Operations cost over total asset
• Operations cost over revenue
• Inventory over operation cost
• Utilization of net capacity over total assets
• Service area over total manpower
• Total assets over No. of customers
• PPE over total number of interruptions
• Earnings per share
• Profit after tax over total manpower
• Profit after tax over total assets
• Revenue over total manpower
• Revenue over total assets
• EBITDA over revenue

The study also concludes that 3S-VF-DEA is the optimum method to measure the efficiency of the electric utilities because it has the highest discriminatory power, consistent MAD scores and was able to eliminate statistical noise.

Lastly, the study also concludes that Utility-6 was the most efficient electric utility for the period of 2010–2018 and Utility-16 was the least efficient electric utility for the same period.

5.1. Methodological and Practical Contributions

The main objective of this study was to measure the efficiency of electric utilities and benchmark the electric utilities against their peers. Since benchmarking of electric utilities is under-researched, this study was able to improve our understanding of this subject by identifying the significant variables that affect the efficiency of electric utilities and determining the optimum method to measure the efficiency of those utilities.

This study also improves the current electric utilities’ efficiency measurement and benchmarking methods by incorporating the variable selection by loads method that was able to eliminate bias in the selection of variables.

Apart from that, this study also enables regulators and policy makers to use the method to predict or simulate electric utilities’ efficiency by adjusting the expected or predicted input and output variables. Such simulation exercises will allow the regulators and policy makers to focus their efforts on variables that will significantly impact the efficiency of the electric utilities.

Similarly, the electric utilities can use the method to measure and predict their own efficiency by tweaking the input and output variables. The outcome of the simulations can be used by the electric utilities to realign their initiatives and activities to achieve better efficiency scores.

5.2. Limitations of the Study and Future Research

The time taken to gather secondary data for this study was longer than expected because the data was cross-checked with other sources to ensure its accuracy. Operational data for variables such as maximum demand (MW), generation capacity (MW), distribution network length (km) and number of distribution substations were excluded from the analysis due data quality issues such as inconsistent data, missing data, incomplete data, and language issues. Other than that, some data related to CO₂ and SO₂ emission was not available in datasets older than year 2005. Therefore, such variables were excluded from this study.