Peruvian Electrical Distribution Firms’ Efficiency Revisited: A Two-Stage Data Envelopment Analysis
Round 1
Reviewer 1 Report
Perez-Reyes and B. Tovar are named as Authors, but the Authors' contribution at the end of the article mentions other nine initials of the Authors, which confuses the reviewer. This point should be clarified for ethical reasons.
The article is generally of a good scientific standard, although it should be improved in some places. Some weakness is the reference only to the Peruvian electricity market and very strong references to an earlier article on this subject and the focus on econometric issues, but there is room for improvement in this regard. It is clear that the revision and addition of alternative models is the most important goal of the article, but due to the need to conform to the journal's profile, a broader thematic context is needed. There is no analysis (especially based on the literature review) regarding the reform of electricity distribution companies and indication of its impact on efficiency. There are several articles on this subject that are not just about the situation in Peru. The more so as modeling proves the relationship between reforms and the efficiency of electricity distribution companies. The issue of factors influencing the efficiency and the selection of inputs and outputs from the model was treated too briefly. When justifying the choice of the DEA method for the analysis of efficiency, insufficient literature was provided (excessive citation of Ramos-Real and Tovar, and yet theoretical grounding can be found in many other authors). There is also no reference to the importance of the issue of benchmarks when examining the efficiency of these specific entities operating in the electricity sector. It is not entirely clear how the companies were classified as reformed and what impact it actually had on efficiency. The impact of re-nationalization remains unclear. Does it matter whether the company remained privatized or returned to state ownership? In part 5, there are many repetitions from the introduction. It is redundant. It is better to deepen the layer of conclusions and possible policy implications and directions for future research, which were announced by the Authors in the introduction, and which are currently indicated to a very small extent.
There are also technical shortcomings in the article. The most important examples of these are presented below. In line 118 there is a sentence: "The efficiency of the evaluated firm is represented by." The appropriate expression is clearly missing. On line 128, only one factor is listed (via the symbol "(iii)") - it is not understood what this "(iii)" is meant to mean in this context. Furthermore, under Tab. 1 there is a superscript "1" of RMyE, which is unknown to what it refers to. In both figures (Fig. 1 and Fig. 2) the same mistake was made and the years were recorded using the wrong notation (e.g., 1,996 instead of 1996). The name Simar is misspelled on line 181. The bibliography at the end of the article has been duplicated.
Author Response
Replies to Reviewer 1
We are grateful to the reviewer for the points the reviewer has raised and by addressing them, we believe, the paper has gained in clarity. All suggestions, comments and corrections provided by Reviewer 1 have been addressed. In what follows we deal with the questions raised. For ease of exposition, the Referee’s comments are in italics.
Perez-Reyes and B. Tovar are named as Authors, but the Authors' contribution at the end of the article mentions other nine initials of the Authors, which confuses the reviewer. This point should be clarified for ethical reasons.
The version of the manuscript you received for revision it was not the one that we uploaded in the system but it was created from it. Unfortunately, this paragraph was created authomatically by the system. The Authors' contribution included did not correspond to our paper, for this reason you find nine initials. This has been corrected in the revised versión.
The article is generally of a good scientific standard, although it should be improved in some places. Some weakness is the reference only to the Peruvian electricity market and very strong references to an earlier article on this subject and the focus on econometric issues, but there is room for improvement in this regard. It is clear that the revision and addition of alternative models is the most important goal of the article, but due to the need to conform to the journal's profile, a broader thematic context is needed.
We thank the reviewer for his/her valuable comments. In the revised version of our paper we have included a new paragraph to better clarify the aims of the paper (see lines 85-93):
“Consequently, it is worth revisiting…... This paper does it covering the same firms but a longer period (1996-2014). Summarizing the aim of the present paper is threefold: first, to discuss all approaches offered in the two-stage DEA literature; second, to compare and test the results gotten when applied all of them to our dataset, and third, to compare our results with those obtained by [3] to check whether their conclusions regarding the efficiency drivers could also be upheld by using the new models proposed by [10].”
We have also included a new paragraph to better clarify the contributions of the paper (see lines 553-563):
“The present article covers a very important field in empirical literature related to efficiency issues. Its contributions are the following. First, this paper sheds light on the choice of a regression model for the second stage by presenting, discussing, and summarizing the arguments proposed by the different authors regarding all options available in the literature. Second, a survey of the empirical literature using a second stage DEA in the distribution sector has been performed which allows us to state that the present paper is the first to apply the different fractional models and also, it could be useful to other authors by present systematically the characteristics of the previous studies. Third, the estimation of all the second stage models identified in the empirical literature let us answer to the re-search question, i.e., the identification of the efficiency drivers in the Peruvian distribution sector after the reforms whatever of the model chosen in the second stage.”
Finally, and in order to highlight the importance of the measures obtained we have included the following paragraph to close the conclusion section (see lines 587-592):
“Last but not least, knowledge and accurate estimates of the distribution firms’ efficiency drivers are more and more crucial due to the potential utility of these measurements as support tools to regulators and governments, specially taking into account that the efficiency of the energy sector is becoming an important issue among growing concerns about global warming.”
There is no analysis (especially based on the literature review) regarding the reform of electricity distribution companies and indication of its impact on efficiency. There are several articles on this subject that are not just about the situation in Peru. The more so as modeling proves the relationship between reforms and the efficiency of electricity distribution companies.
As pointed out by Zhang, Y., Parker, D., Kirkpatrick, C (2009) and Jamasb and Pollit (2001), among others, the relationship between reforms in the electricity industry and efficiency is clear. Recent studies using the two-stage DEA approach in the electricty distribution are Bobde and Tanaka (2018), Leme et. al. (2014), and, Veronese da Silva et. al. (2019). The last two papers are related to the distribution of electricity in Brazil and its regulatory processes.
We have made clear the previous point in the revised versión. Please, kindly see lines 112-114 in section 2:
“As pointed out by [2] and [11], among others, the relationship between reforms in the electricity industry and efficiency is clear (some examples of recent studies related to the distribution of electricity and its regulatory processes could be found in section 3.2).”
and lines 352-356 in section 3.2:
“Finally, the relationship between reforms in the electricity industry and efficiency is clear as pointed out by [2] and [11] and could be seen in some of the studies reviewed which are related to the distribution of electricity and the regulatory processes such as, for example, [27] and [28] in the case of Brazil and [3] and the present paper in the case of Peru, to name but two.”
The issue of factors influencing the efficiency and the selection of inputs and outputs from the model was treated too briefly.
We thank you for your valuable comments. We explained in the paper (see line 135-140) that:
“Before conducting efficiency analyses, a variable selection process must be conducted with the appropriate care. To do so, several criteria are to be considered. Again, and in order to do the comparison sensible, we use a database which contains the same inputs and outputs variables than [3] but covers a longer time period (1996-2014). For the sake of brevity we only do a brief description of those variables and the criteria followed to select them5”.
Due to the fact that we use the same inputs and outputs variables than Perez-Reyes & Tovar (2009) we decide to sumarize the discussion in the footnote 5. Unfortunately, and as I told you before, the version of the manuscript you received for revision it was not the one that we uploaded in the system but it was created from it and all the footnotes included in the original paper were erased (although you can see all them in the new version of the paper as erased with track changes).
In the revised version all footnote are now incorporated in the main text of the manuscript to keep consistent with the format requirements of the Journal. Therefore, in the revised version of our paper, the explanation regarding the selection of input and output could be found in lines 141-154.
“The choice of variables was based on the availability of data, and on a review of the extant literature which showed that most papers follow [18] and consider four outputs: sales in MWh and number of customers, both variables for medium and Low voltage. Due to data restrictions, and following [19], [20] and [15], two outputs were considered: sales in MWh and the numbers of customers. Regarding inputs, a distribution company requires work and a network infrastructure. Labour is related to the number of employees and the network infrastructure was measured through the net fixed assets due to other physical measures, such as the extension of the existing electricity grid, were not available for all years included in the dataset. Finally, the only undesirable factor incorporated in are the power losses in MWh because other variables, such as statistical information on interrup-tions and quality of the distribution networks has been available only since 2004. Table 1 shows the statistics for the variables selected which follow the general consensus found in the current literature (see [11] for a survey of the use of frontier studies in regulation of electricity distribution and section 3.2)”
Moreover, in the revised version of the paper we have included a review of the empirical literature applying two-stage DEA to the distribution sector (please, kindly see new Section 3.2) where the issue of factors influencing the efficiency and the selection of inputs and outputs have been treated deeply.
When justifying the choice of the DEA method for the analysis of efficiency, insufficient literature was provided (excessive citation of Ramos-Real and Tovar, and yet theoretical grounding can be found in many other authors).
Following the reviewer recomendation we have included other authors to justify the choice of the DEA method (see lines 106-114). Moreover, in the revised version of the paper several references related which were erased (old footnote 4 and 5) have been included in the main text.
It is not entirely clear how the companies were classified as reformed and what impact it actually had on efficiency. The impact of renationalization remains unclear. Does it matter whether the company remained privatized or returned to state ownership?
Six electricity distribution companies were privatized since 1996. In 2002, 4 of them were returned to the property of the Peruvian State because the owner did not consider it profitable to invest in these companies.
How the companies were classified as reformed is explained in section 2 (see line 165-166):
“….. in the case of the reformed firms which are those that at some point in the analysis period were managed by a private company.”
And more deeply in Section 4 (see, lines 408-422):
“Finally, a qualitative variable to evaluate the reform of the industry was considered. When the reform process began the intention was to privatize all the firms but due to re-sistance from citizens in some places, certain distribution companies have remained state owned: Seal, Electro Puno, Electro Sur, Electro Sur Este, Electro Ucayali and Electro Oriente. Moreover, some companies were renationalized due to the concessionaires’ non compliance with the investment commitments: Hidrandina, Electro Norte, Electro Centro and Electro Noroeste. The reformed businesses, those that were privatized, take a value of one and those that were never privatized take a value of zero.
With regard to this research, a reformed company is one which at some time during the period of analysis was managed by a private company. For example, Electro Norte, Electro Centro, Electro Noroeste and Hidrandina are considered to be reformed as they were privatized for nearly two and a half years despite later returning to state ownership; during this period important changes in management were made.
And regarding what impact it actually had on efficiency. As we stated in Section 4 (see lines 423-427):
Our main hypothesis is that the relationship between the reform variable and efficiency is positive, since the process of reforms allowed for significant improvements in the management of the former state companies. Previously, due to the political nature of managerial appointments within the Peruvian framework, these companies did not have the goal of profit maximization.”
This was confirmed by our results (see section 4.2, lines 514-542):
“Tables 6 and 7 show that with regard to the “reform” variable the estimated parameters have a positive sign for all the considered specifications, and in almost all the estimated models it is considered as a relevant variable. This shows that the distribution companies demonstrated greater technical efficiency after having been reformed, which is evidence of a positive correlation between reforms and the efficiency in Peruvian electricity distribution companies.
….This favourable empirical evidence for the reforms is not an evaluation of the positive aspects of the property regime itself or of some chance relationship, in the same sense as [2]. This is related to the differentiated characteristics in the institution-al environment that private and public firms operate in Peru. Due to the framework of corporative government, private firms have the obligation to report their management activities to their minority shareholders, as suggested by [37] for Ukraine electricty distribution firms; Nonetheless, they do not face investment restrictions in con-tracting, consultancy or acquisition of services. However, the state distribution companies have to tolerate more and more administrative restrictions for investment objectives, complicated processes for acquiring inputs, subsequent audit processes from governmental offices and objectives related to profit maximization.”
In part 5, there are many repetitions from the introduction. It is redundant. It is better to deepen the layer of conclusions and possible policy implications and directions for future research, which were announced by the Authors in the introduction, and which are currently indicated to a very small extent.
Following the reviewer comments Section 5 has been redrafted to avoid repetitions and to hightlight the contributions, policy implication, limitations and future research.
There are also technical shortcomings in the article. The most important examples of these are presented below.
In line 118 there is a sentence: "The efficiency of the evaluated firm is represented by." The appropriate expression is clearly missing.
The complete sentence is (see line 131-132): “The efficiency of the evaluated firm is represented by q.” Being q the efficiency score calculated using the following DEA model (see lines 122-128):
s.t.:
On line 128, only one factor is listed (via the symbol "(iii)") - it is not understood what this "(iii)" is meant to mean in this context.
In the revised versión of the paper this paragraph has been erased
Furthermore, under Tab. 1 there is a superscript "1" of RMyE, which is unknown to what it refers to.
In the revised versión of the paper this paragraph has been erased
In both figures (Fig. 1 and Fig. 2) the same mistake was made and the years were recorded using the wrong notation (e.g., 1,996 instead of 1996).
In the revised versión of the paper both Figures have been corrected
The name Simar is misspelled on line 181.
In the revised versión of the paper this has been corrected
The bibliography at the end of the article has been duplicated.
The version of the manuscript you received for revision it was not the one that we uploaded in the system but it was created from it. Unfortunately, the references not only were duplicated but also do not correspond to our paper. This has been corrected in the revised version.
Author Response File: 
Author Response.docx
Reviewer 2 Report
Dear authors,
The manuscript is an extension, or revisit, of an article published in Energy Policy by the same authors in 2009. To be published as a new paper, the manuscript has failed to offer the readers/reviewers some important things as follows.
1. The main aims (and contribution) of the manuscript, compared to Perez-Reyes & Tovar (2009), are (i) the use of fractional regression model (FRM) and (ii) a longer period. However, it is not clear to me as how these two can affect the results (e.g. significant changes in the Peruvian electrical industry after 2006 or specific advantages of FRM over Tobit). Particularly, the FRM was not discussed in details - Section 3 only provides the (general) equations of FRM with a Graph 1 (no discussion on this graph as well).
2. The discussions and uses of bootstrap DEA (Simar & Wilson, 2007) in the manuscript were incorrect.
First, please note that Simar & Wilson (2007) criticized Tobit regression and proposed to use truncated regression instead. In the manuscript, the authors first argued that Banker & Natarajan (2008)'s criticism on Tobit is 'in contrast with the issues raised by Simar and Wilson (2007)' (line 199, page 8). Later, in the last column of Table 6, the authors presented the results of Tobit (Simar-Wilson) regression, suggesting that they misunderstood the Simar & Wilson's idea.
Second, it is more important to note that both bootstrap algorithms (#1 and #2) of Simar & Wilson (2007) is totally different from bootstrapping OLS/Tobit (which can be easily done in Stata using a command likes 'bootstrap, iter(1000): regress DEAefficiency KN Mountain Jungle ...'). In particular, Simar & Wilson's bootstrap requires resampling and re-estimating of DEA efficiency in each iteration and thus, comparing those results as in Table 6 is incorrect.
Third, due to the complication of Simar & Wilson's bootstrap as above, to the best of our knowledge, there is no current statistical software or (Stata) package that can deal with bootstrap FRM in two-stage DEA considering explanatory variables. Please note that Wilson himself, as the creator of package FEAR in R (not in Stata), have stated that the command boot.sw98 of the package can only bootstrap the DEA efficiency scores (as in Wilson, 2008, FEAR: A software package for frontier efficiency analysis with R, Socio-Economic Planning Sciences, 42(4): 247-254) following Simar & Wilson (1998), Sensitivity analysis of efficiency scores: How to bootstrap in nonparametric frontier models, Management Science, 44 (1998): 49-61.
All in all, I doubt that the Simar & Wilson (2007)'s bootstrapping technique has been properly used for the 2-stage DEA-FRM estimation in this manuscript.
3. Could the authors please double check the reference list at least twice before submitting? The current list does not make any sense.
4. The authors argued that their aim is to compare with previous results of Reyes & Tovar (2009). However, no comparison (e.g. data, efficiency or regression differences) was made. Do you expect the readers to go back and forth between the two papers and make the comparison themselves?
Given these issues, I think the current manuscript is very weak to be published in Sustainability.
Kind regards.
Author Response
Replies to Reviewer 2
We are grateful to the reviewer for the points the reviewer has raised and by addressing them, we believe, the paper has gained in clarity. All suggestions, comments and corrections provided by Reviewer 2 have been addressed. In what follows we deal with the questions raised. For ease of exposition, the Referee’s comments are in italics.
The manuscript is an extension, or revisit, of an article published in Energy Policy by the same authors in 2009. To be published as a new paper, the manuscript has failed to offer the readers/reviewers some important things as follows.
- The main aims (and contribution) of the manuscript, compared to Perez-Reyes & Tovar (2009), are (i) the use of fractional regression model (FRM) and (ii) a longer period.
We thank the reviewer for his/her valuable comments. In the revised version of our paper we have included a new paragraph to better clarify the aims of the paper (see lines 83-93):
“… [10] have argued that the use of Tobit regression (censored regression) is inappropriate in the second stage of DEA, and have suggested using other recently developed options which perform better. Consequently, it is worth revisiting [3] and adding those other alternative models to check whether their conclusions remain not only on the relationship between efficiency and reforms but also regarding the other efficiency drivers. This paper does it covering the same firms but a longer period (1996-2014). Summarizing the aim of the present paper is threefold: first, to discuss all approaches offered in the two-stage DEA literature; second, to compare and test the results gotten when applied all of them to our dataset, and third, to compare our results with those obtained by [3] to check whether their conclusions regarding the efficiency drivers could also be upheld by using the new models proposed by [10].”
However, it is not clear to me as how these two can affect the results (e.g. significant changes in the Peruvian electrical industry after 2006 or specific advantages of FRM over Tobit).
The selection of a regression model for the second stage is not a minor econometric problem as we show in the introduction section (see lines 67-79):
“[5] and [6] have echoed the copious number of studies carried out following a two-stage Data Envelopment Analysis (studies using the two-stage DEA approach in the electricity distribution are reviewed in section 3.2. The same methodology has also been used in other industries recently, such as [7] in banks and [8] in ports, to name but two). However, the proliferation of such studies has not been accompanied by a theoretical framework that permits us to establish an appropiate estimate of the “true model”; this is due to the limitations of the strict empirical focus of the applied econometrics employed. In this sense, in the last fifteen years there has been an important conceptual development in the second stage methodology associated with measuring the technical efficiency using DEA. This conceptual development has been in the context of the academic debate between [9], [5] and [6]. This explains the significant increase in the use of bootstrap techniques when carrying out interference on estimated models, be they linear models, Tobit or Fractional regression models (FRM).”
This is the reason we have devoted Section 3.1 to review the second-stage-regression model proposed in the literature from a methodological point of view. Afterwards, we explained why we agree with the arguments proposed by Ramalho et al., (2010) to choose their second-stage-regression model at the end of the methodology section (see lines 289-295):
“The nature of the measurement of parametric efficiency and of the DEA estimators make it clear that an unrestricted OLS model at the positive unitary circle is not conceptually consistent. Conversely, nor is it possible to establish that the measurements of efficiency and the DEA estimators reflect a process of data censorship, as the Tobit models set out. This being so, as [21] points out, fractional regression models (FRM) are considered to represent a better approach to the data generating process of the non-parametric measurements of efficiency, and they should be used to estimate second stage models.”
Particularly, the FRM was not discussed in details - Section 3 only provides the (general) equations of FRM with a Graph 1 (no discussion on this graph as well).
Since one of the most important discussions is related to the functional form of the second-stage equation, fractional models allow proposing a non-linear functional form that is reasonable as a way of representing the data generating process, so the extent that the true values of the DEA coefficient are limited by 0 and 1, and they are also concentrated around 1 rather than zero.
Moreover, the use of fractional models together with alternative models does not pretend to establish which model is better. It seeks to contrast the impact of the reforms and other efficiency explanatory variables, beyond the model used and the general equations provided are enough to fullfiled our aims. The results indicate that regardless of the model used, and as we stated in lines 568-573:
“The results of the nine alternative second stage models considered show that the results of [3] could be upheld and that the efficiencies of the analysed companies are directly related to the reforms of the sector that were initiated in the 1990s; also, they demonstrate that the proportion of low voltage versus medium voltage sales is inversely related to the investment per customer. The variables associated with the geographical difficulties do not appear to be relevant when explaining the inefficiencies in the distribution companies.”
Finally, the interpretation of Graph 1 is the standar one for any Function of Acumulated Density. Therefore, the graph shows the different functions of accumulated density of each FRM model, as could be read in the legend located in the right corner of the graph. For the previous reason we do not consider neccesary to include it.
- The discussions and uses of bootstrap DEA (Simar & Wilson, 2007) in the manuscript were incorrect.
We thank the reviewer for his/her valuable comments. In the following we clarify some aspects that could have been misunderstood and lead the reviewer to believe that the discussion and the uses of boostrap DEA were incorrect.
First, please note that Simar & Wilson (2007) criticized Tobit regression and proposed to use truncated regression instead. In the manuscript, the authors first argued that Banker & Natarajan (2008)'s criticism on Tobit is 'in contrast with the issues raised by Simar and Wilson (2007)' (line 199, page 8).
Our comment alludes to the fact that one of the criticism is based on the inconsistency of the model's predictions while the other criticism questions the efficiency and bias of the Tobit estimate.
Later, in the last column of Table 6, the authors presented the results of Tobit (Simar-Wilson) regression, suggesting that they misunderstood the Simar & Wilson's idea.
Please, see our answer to your following comment below.
Second, it is more important to note that both bootstrap algorithms (#1 and #2) of Simar & Wilson (2007) is totally different from bootstrapping OLS/Tobit (which can be easily done in Stata using a command likes 'bootstrap, iter(1000): regress DEAefficiency KN Mountain Jungle ...'). In particular, Simar & Wilson's bootstrap requires resampling and re-estimating of DEA efficiency in each iteration and thus, comparing those results as in Table 6 is incorrect.
We agree with the reviewer. The table (old table 6) header erroneously said Tobit corrected by Simar and Wilson, when it should have said “truncated regression corrected by Simar and Wilson”. This has been corrected in the revised versin of our paper (Please, kindly see new Table 7 (old table 6), line 479)
Third, due to the complication of Simar & Wilson's bootstrap as above, to the best of our knowledge, there is no current statistical software or (Stata) package that can deal with bootstrap FRM in two-stage DEA considering explanatory variables. Please note that Wilson himself, as the creator of package FEAR in R (not in Stata), have stated that the command boot.sw98 of the package can only bootstrap the DEA efficiency scores (as in Wilson, 2008, FEAR: A software package for frontier efficiency analysis with R, Socio-Economic Planning Sciences, 42(4): 247-254) following Simar & Wilson (1998), Sensitivity analysis of efficiency scores: How to bootstrap in nonparametric frontier models, Management Science, 44 (1998): 49-61.
What we did in the old versión of our paper was implement algorithm 1 in Stata. Unfortunately, as we explained you before the table (old table 6) header erroneously said Tobit corrected by Simar and Wilson, when it should have said “truncated regression corrected by Simar and Wilson”
All in all, I doubt that the Simar & Wilson (2007)'s bootstrapping technique has been properly used for the 2-stage DEA-FRM estimation in this manuscript.
As we state in our response to the previous comment, algorithm 1 of the truncated regression model proposed by Simar and Wilson was used in the previous version of this manuscript. The reviewer's comment, regarding the iteration in the estimation of theta, is applicable to algorithm 2 and not to algorithm 1.
However, in the new versión of our paper we have incorporated algorithm 2 in the estimation of the truncated model, as a better estimate of the truncated model. For this we use the teradial and simarwilson command in Stata as used and developed by Badunenko and Tauchmann (2018) (Please, kindly see the new estimations in Table 7 (old table 6), line 479-480)
- Could the authors please double check the reference list at least twice before submitting? The current list does not make any sense.
The version of the manuscript you received for revision it was not the one that we uploaded in the system but it was created from it. Unfortunately, the references not only were duplicated but also do not correspond to our paper. This has been corrected in the revised version.
- The authors argued that their aim is to compare with previous results of Reyes & Tovar (2009).
In the revised version of our paper we have included a new paragraph to better clarify the aims of the paper (see lines 85-93):
“Consequently, it is worth revisiting…... This paper does it covering the same firms but a longer period (1996-2014). Summarizing the aim of the present paper is threefold: first to discuss all approaches offered in the two-stage DEA literature; second, to compare and test the results gotten when applied all of them to our dataset, and third, to compare our results with those obtained by [3] to check whether their conclusions regarding the efficiency drivers could also be upheld by using the new models proposed by [10].”
However, no comparison (e.g. data, efficiency or regression differences) was made. Do you expect the readers to go back and forth between the two papers and make the comparison themselves?
As we state in section 2 (see lines 119-120):
“In order to make the comparison sensible we use the same input oriented DEA model than [3]”
And see also lines 136-138:
“….in order to do the comparison sensible, we use a database which contains the same inputs and outputs variables than [3] but covers a longer time period (1996-2014).”
Therefore the efficiency scores are the same for those years which have been included in both studies (1996-2006). Anycase, all efficinecy scores by each firm and each year are represented in Figure 1 and 2, therefore the comparison is inmediate.
Due to the fact that we are interested in the second stage, we use a longer time period in this paper in order to improve the estimation of the regresion models in the second stage. Therefore, the comparison between the results of both papers, which has to do with the efficiency drivers, is done and could be found in section 4.
We have also included a new paragraph to better clarify the contributions of the paper (see lines 553-563):
“The present article covers a very important field in empirical literature related to efficiency issues. Its contributions are the following. First, this paper sheds light on the choice of a regression model for the second stage by presenting, discussing, and summarizing the arguments proposed by the different authors regarding all options available in the literature. Second, a survey of the empirical literature using a second stage DEA in the distribution sector has been performed which allows us to state that the present paper is the first to apply the different fractional models and also, it could be useful to other authors by present systematically the characteristics of the previous studies. Third, the estimation of all the second stage models identified in the empirical literature let us answer to the research question, i.e., the identification of the efficiency drivers in the Peruvian distribution sector after the reforms whatever of the model chosen in the second stage.”
Reviewer 3 Report
The article should be improved in the following points- work contributions must be developed
Author Response
Replies to Reviewer 3
We are grateful to the reviewer for the points the reviewer has raised and by addressing them, we believe, the paper has gained in clarity. All suggestions, comments and corrections provided by Reviewer 3 have been addressed. In what follows we deal with the questions raised. For ease of exposition, the Referee’s comments are in italics.
The article should be improved in the following points
- work contributions must be developed
We thank the reviewer for his/her valuable comments. In the revised version of our paper we have included a new paragraph to better clarify the aims of the paper (see lines 85-93):
“Consequently, it is worth revisiting…... This paper does it covering the same firms but a longer period (1996-2014). Summarizing the aim of the present paper is threefold: first to discuss all approaches offered in the two-stage DEA literature; second, to compare and test the results gotten when applied all of them to our dataset, and third, to compare our results with those obtained by [3] to check whether their conclusions regarding the efficiency drivers could also be upheld by using the new models proposed by [10].”
We have also included a new paragraph to better clarify the contributions of the paper (see lines 553-563):
“The present article covers a very important field in empirical literature related to effi-ciency issues. Its contributions are the following. First, this paper sheds light on the choice of a regression model for the second stage by presenting, discussing, and summarizing the arguments proposed by the different authors regarding all options available in the lit-erature. Second, a survey of the empirical literature using a second stage DEA in the dis-tribution sector has been performed which allows us to state that the present paper is the first to apply the different fractional models and also, it could be useful to other authors by present systematically the characteristics of the previous studies. Third, the estimation of all the second stage models identified in the empirical literature let us answer to the re-search question, i.e., the identification of the efficiency drivers in the Peruvian distribution sector after the reforms whatever of the model chosen in the second stage.
Finally, we have also included two new paragraph to highlight the limitation, future research and policy implications (see lines 577-592):
“…. It is also the case that the aforementioned state institutionalism does not permit autonomy in the management of enterprises run on behalf of the Peruvian state. Therefore, policy measures should aim to correct this anomaly.
A limitation of this study should be considered. It would be interesting to check whether our results might be affected by using a higher number and/or different potential efficiency drivers not available in our data set. Among them could be measures of quality of the distribution networks such as, for example, interruptions (duration, frequency, etc.) or weather factors, such as, temperature variables and other whether conditions (rainfall, thunder, etc.). Therefore, future research should be focus on including these efficiency drivers.
Last but not least, knowledge and accurate estimates of the distribution firms’ efficiency drivers are more and more crucial due to the potential utility of these measurements as support tools to regulators and governments, specially taking into account that the efficiency of the energy sector is becoming an important issue among growing concerns about global warming.”
Author Response File: Author Response.docx
Reviewer 4 Report
The authors present a relatively robust study dealing with an interesting (although very geographically limited) topic. Gradually, I will comment on all the relevant sections in which I see room for improvement.
Abstract:
The abstract in its current form seems quite non-standard, especially when it comes to the view of the reader, who did not have the opportunity to get acquainted with the previous works of the authors. I would prefer an abstract in a more structured form, where the authors would clearly clarify the idea but especially the social demand for the present study, then clearly define the objectives and point out the methods used. Subsequently, it is possible to proceed to a short evaluation of the results and to point out the benefits. I strongly advise the authors not to refer to specific works in the abstract, especially in the case of their past studies. The citations in the abstract are quite non-standard.
Introduction:
The first part of the introduction shows some signs of storytelling, for this type of study I would prefer a more detailed form, especially if I take into account that the authors work with quantifiable data whose sources they do not cite.
If the authors refer to their own works, it is in my opinion non-standard to refer to them from the point of view of a third party.
The introduction should be synthesized into a clearly identifiable research question, to which the authors then seek answers through the tools of scientific research.
Overview of the current state of the issue:
In the work of the authors, this element is almost completely absent, despite the relatively large number of sources listed (even twice..., more bout it in the "other" section of my review) in the reference section.
Methods and goals of the work:
This part of the work should follow the current state of the issue on the basis of reference literature. In the present study, this part is almost completely absent, I propose to add a chapter that clearly describes the main goal of the work, the processing methods used as well as the data sources on the basis of which the authors elaborate they study. Subsequently, it is possible to proceed to the presentation of the results, I deduce that in the present study, this part begins at line 94.
Discussion:
Conclusion should be preceded by a discussion, in which the (synthesized) results should be clearly presented in one place, as well as discussed (and compared with previous works of the authors), but mainly placed in the context of social demand, which would be demonstrated by comparing findings with other works of (relevant) authors.
Conclusion:
Conclusion is quantitatively and qualitatively insufficient due to the scope of the findings.
At the end of the conclusion I propose to the authors to complete the part describing the limitations of the research and also the part proposing the future direction of the research.
Other:
Authors do not cite sources in accordance with the rules of the journal, the combination of continuous citation in the text and numbering of references in their list makes reading obscure.
I also don't understand the duplication of the list of references, starting with line 325. As the reviewer I evaluates this very negative when it comes to submitting a work to a journal of this category.
Author Response
Replies to Reviewer 4
We are grateful to the reviewer for the points the reviewer has raised and by addressing them, we believe, the paper has gained in clarity. All suggestions, comments and corrections provided by Reviewer 4 have been addressed. In what follows we deal with the questions raised. For ease of exposition, the Referee’s comments are in italics.
The authors present a relatively robust study dealing with an interesting (although very geographically limited) topic. Gradually, I will comment on all the relevant sections in which I see room for improvement.
Abstract:
The abstract in its current form seems quite non-standard, especially when it comes to the view of the reader, who did not have the opportunity to get acquainted with the previous works of the authors. I would prefer an abstract in a more structured form, where the authors would clearly clarify the idea but especially the social demand for the present study, then clearly define the objectives and point out the methods used. Subsequently, it is possible to proceed to a short evaluation of the results and to point out the benefits. I strongly advise the authors not to refer to specific works in the abstract, especially in the case of their past studies. The citations in the abstract are quite non-standard.
We thank the reviewer for his/her valuable comments. In the revised version of our paper we have redrafted the abstract and the citation has been eliminated
Introduction:
The first part of the introduction shows some signs of storytelling, for this type of study I would prefer a more detailed form, especially if I take into account that the authors work with quantifiable data whose sources they do not cite.
We thank the reviewer for his/her valuable comments. In the revised version of our paper we have redrafted the introduction gaining in readability and clearness. Please, kindly see also our answer to your comment regarding the introduction below.
On the other hand, the quantificable data sources were cited when presenting the statistics of the variables used in the model below table 1 (see lines 159-160) where it could be read.
"Source: Energy Investment Supervisory Body of Peru, OSINERGMIN. Own: elaborated from statistical yearbooks.”
If the authors refer to their own works, it is in my opinion non-standard to refer to them from the point of view of a third party.
We thank the reviewer for his/her valuable comments. We respect your opinion although but as you recognize is something debatable. We did it to ease the blinded review process. Anycase, and due to the reference format requirements of the Journal, in our opinion, this issue has little relevance. For this reason we preffer to keep the original style.
The introduction should be synthesized into a clearly identifiable research question, to which the authors then seek answers through the tools of scientific research.
We thank the reviewer for his/her valuable comments. In the revised version of our paper we have redrafted the introduction in order to synthesized it and clearly identify the research question.
We have included a new paragraph to better clarify the aims of the paper (see lines 85-93):
“Consequently, it is worth revisiting…... This paper does it covering the same firms but a longer period (1996-2014). Summarizing the aim of the present paper is threefold: first to discuss all approaches offered in the two-stage DEA literature; second, to compare and test the results gotten when applied all of them to our dataset, and third, to compare our results with those obtained by [3] to check whether their conclusions regarding the efficiency drivers could also be upheld by using the new models proposed by [10].”
We have also included a new paragraph to better clarify the contributions of the paper (see lines 553-563):
“The present article covers a very important field in empirical literature related to efficiency issues. Its contributions are the following. First, this paper sheds light on the choice of a regression model for the second stage by presenting, discussing, and summarizing the arguments proposed by the different authors regarding all options available in the literature. Second, a survey of the empirical literature using a second stage DEA in the distribution sector has been performed which allows us to state that the present paper is the first to apply the different fractional models and also, it could be useful to other authors by present systematically the characteristics of the previous studies. Third, the estimation of all the second stage models identified in the empirical literature let us answer to the research question, i.e., the identification of the efficiency drivers in the Peruvian distribution sector after the reforms whatever of the model chosen in the second stage.
Overview of the current state of the issue:
In the work of the authors, this element is almost completely absent, despite the relatively large number of sources listed (even twice..., more bout it in the "other" section of my review) in the reference section.
The version of the manuscript you received for revision it was not the one that we uploaded in the system but it was created from it. Unfortunately, the references not only were duplicated but also do not correspond to our paper. This has been corrected in the revised version.
Moreover, other parts of the paper (footnotes) related with this issue were not included. This information has been included in the main text in the revised version of the paper (although you can see all of them in the new version of the paper as erased with track changes).
Finally and regarding the current state of the issue, we have analysed it deeply not only from the methodological point of view in our Section 2 devoted to DEA first stage and Section 3.1 devoted to review two-stage DEA literatura, but also in the new Section 3.2 devoted to review the empirical literature applying two-stage DEA to the electric distribution sector.
Methods and goals of the work:
This part of the work should follow the current state of the issue on the basis of reference literature. In the present study, this part is almost completely absent, I propose to add a chapter that clearly describes the main goal of the work, the processing methods used as well as the data sources on the basis of which the authors elaborate they study. Subsequently, it is possible to proceed to the presentation of the results, I deduce that in the present study, this part begins at line 94.
In the revised version of our paper we have included a new paragraph to better clarify the aims of the paper (see lines 85-93):
“Consequently, it is worth revisiting…... This paper does it covering the same firms but a longer period (1996-2014). Summarizing the aim of the present paper is threefold: first to discuss all approaches offered in the two-stage DEA literature; second, to compare and test the results gotten when applied all of them to our dataset, and third, to compare our results with those obtained by [3] to check whether their conclusions regarding the efficiency drivers could also be upheld by using the new models proposed by [10].”
The revised version of our paper is organized as follows (see lines 94-104)
“After an introduction, where we briefly summarize the reforms in Peru’s electricity distribution sector and present the aims of this paper, the second section briefly describes the data and presents the results obtained in the first stage. Afterwards, in section three, we undertake a brief and critical review of the literature on two-stage DEA from a methodological point of view, to identify what are the advantages and drawbacks of the different models employed. We also include a review of the empirical literature applying two-stage DEA to the distribution sector. Section four, presents, by firm, the specific explanatory variables which have been postulated as efficiency drivers in the second stage analysis, and the empirical results obtained from nine alternative models. Finally, section five presents the most relevant conclusions, possible policy implications, limitations and and directions for future research.”
Therefore, the paper has the chapter you are asking for
Discussion:
Conclusion should be preceded by a discussion, in which the (synthesized) results should be clearly presented in one place, as well as discussed (and compared with previous works of the authors), but mainly placed in the context of social demand, which would be demonstrated by comparing findings with other works of (relevant) authors.
We agree with the reviewer in that the conclusions should be preceded by a discussion in which the (synthesized) results should be clearly presented in one place, as well as discussed (and compared with previous works of the authors). This is included in the following last paragraphs of section 4 (see lines 520-542) placed just before the conclusion section:
“To summarize, after reviewing the two-stage Data Envelopment Analysis (DEA) literature in section 3.1, we decided to estimate a total of nine models: four Fractional Regression Models (logit, probit, loglog y cloglog) and five alternatives (OLS, fixed and random effects of the logarithmic model using a bootstrap suggested by [9], Tobit model with a bootstrap and a truncated model with a bootstrap developed by [5]) and, as Tables 6 and 7 have shown, all of them allow us to confirm that the incentives generated by the reform process led to the firms becoming more efficient. Moreover, when comparing our results with those in [3], with the exception of the Mountain variable, which has not been significant in any of the estimated models, the other variables maintained the same sign that was obtained by them. A lack of relevance is maintained for the Jungle and Mountain variables, but their ratio for sales of low and medium voltage and capital per customer improves slightly. The reform variable maintains the sign and the individual relevance.
This favourable empirical evidence for the reforms is not an evaluation of the positive aspects of the property regime itself or of some chance relationship, in the same sense as [2]. This is related to the differentiated characteristics in the institutional environment that private and public firms operate in Peru. Due to the framework of corporative government, private firms have the obligation to report their management activities to their minority shareholders, as suggested by [37] for Ukraine electricty distribution firms; Nonetheless, they do not face investment restrictions in contracting, consultancy or acquisition of services. However, the state distribution companies have to tolerate more and more administrative restrictions for investment objectives, complicated processes for acquiring inputs, subsequent audit processes from governmental offices and objectives related to profit maximization.”
Conclusion:
Conclusion is quantitatively and qualitatively insufficient due to the scope of the findings.
At the end of the conclusion I propose to the authors to complete the part describing the limitations of the research and also the part proposing the future direction of the research.
Following other reviewer comments Section 5 has been reddradted to avoid repetitions and to hightlight the contributions, policy implication, limitations and also future reseach.
Other:
Authors do not cite sources in accordance with the rules of the journal, the combination of continuous citation in the text and numbering of references in their list makes reading obscure.
The version of the manuscript you received for revision it was not the one that we uploaded in the system but it was created from it. Unfortunately, the references not only were duplicated but also do not correspond to our paper. This has been corrected in the revised version.
I also don't understand the duplication of the list of references, starting with line 325. As the reviewer I evaluates this very negative when it comes to submitting a work to a journal of this category.
Please, see our answer to your previous comment
Round 2
Reviewer 2 Report
Dear Authors,
Thank you for your replies as well as your efforts to revise the manuscript. The manuscript is now much clearer; however, I still have a few recommendations as follows.
- I suggest that the authors should treat the manuscript not as an extension/revisit of Perez-Reyes and Tovar (2009) but as a new and independent study. You can compare your results with those of Perez-Reyes and Tovar (2009), but only to show that yours are consistent with previous findings.
- I suggest that the authors should make it clearer, in the Introduction as well as throughout the manuscript, that the bootstrap DEA is used in this study as a robustness tool. For instance, the argument that 'This explains the significant increase in the use of bootstrap techniques when carrying out interference on estimated models, be they linear models, Tobit or Fractional regression models (FRM)' in lines 77-79 (page 2) creates a misunderstanding that the bootstrap technique can be applied for FRM (and that this study will do so). I also would like to ask if the correct term for the above argument is 'inference' or 'interference'.
- The authors choose FRM as the main model of their study, but they also show that their results are consistent and robust among nine alternative models. One may ask 'why FRM and not bootstrap, as results are similar but bootstrap is more advance?' - even the authors acknowledged that bootstrap (with its inference characteristics) is a trend in DEA studies. In this sense, a limitation of this study is that it cannot provide information on the confidence intervals for the estimated DEA efficiency scores. The authors may need to look at Ngo and Tsui (2021) as a potential approach for future study to extend their research.
Ngo, T., & Tsui, K. W. H. (2021). Estimating the confidence intervals for DEA efficiency scores of Asia-Pacific airlines. Operational Research. https://doi.org/10.1007/s12351-021-00667-w
Author Response
REVIEWER 2
Dear Authors,
Thank you for your replies as well as your efforts to revise the manuscript. The manuscript is now much clearer; however, I still have a few recommendations as follows.
- I suggest that the authors should treat the manuscript not as an extension/revisit of Perez-Reyes and Tovar (2009) but as a new and independent study. You can compare your results with those of Perez-Reyes and Tovar (2009), but only to show that yours are consistent with previous findings.
We thank the reviewer for his/her valuable comments. We agree with the reviewer in the appreciation of our paper is more than an extension/revisit of Perez-Reyes and Tovar (2009). However, we consider that this is clear throughout the paper and specially in the introduction section where we included a paragraph to better clarify the aims of the paper (see lines 85-93):
“Consequently, it is worth revisiting…... This paper does it covering the same firms but a longer period (1996-2014). Summarizing the aim of the present paper is threefold: first, to discuss all approaches offered in the two-stage DEA literature; second, to compare and test the results gotten when applied all of them to our dataset, and third, to compare our results with those obtained by [3] to check whether their conclusions regarding the efficiency drivers could also be upheld by using the new models proposed by [10].”
and in the conclusion section where another paragraph clarifies the contributions of the paper (see lines 553-563):
“The present article covers a very important field in empirical literature related to efficiency issues. Its contributions are the following. First, this paper sheds light on the choice of a regression model for the second stage by presenting, discussing, and summarizing the arguments proposed by the different authors regarding all options available in the literature. Second, a survey of the empirical literature using a second stage DEA in the distribution sector has been performed which allows us to state that the present paper is the first to apply the different fractional models and also, it could be useful to other authors by present systematically the characteristics of the previous studies. Third, the estimation of all the second stage models identified in the empirical literature let us answer to the re-search question, i.e., the identification of the efficiency drivers in the Peruvian distribution sector after the reforms whatever of the model chosen in the second stage.”
- I suggest that the authors should make it clearer, in the Introduction as well as throughout the manuscript, that the bootstrap DEA is used in this study as a robustness tool. For instance, the argument that 'This explains the significant increase in the use of bootstrap techniques when carrying out interference on estimated models, be they linear models, Tobit or Fractional regression models (FRM)' in lines 77-79 (page 2) creates a misunderstanding that the bootstrap technique can be applied for FRM (and that this study will do so). I also would like to ask if the correct term for the above argument is 'inference' or 'interference'.
We have modified the text, in order to clarify what the reviewer commented, as follows (see lines 76-79):
“This explains the significant increase in the use of bootstrap techniques when carrying out inference on estimated models, be they linear models or Tobit, and also in the use of models that question the functional form associated with linear or Tobit models, as is the case of Fractional regression models (FRM)”
The error associated with the term interference was corrected, the correct term, as suggested by the reviewer, is inference.
- The authors choose FRM as the main model of their study, but they also show that their results are consistent and robust among nine alternative models. One may ask 'why FRM and not bootstrap, as results are similar but bootstrap is more advance?' - even the authors acknowledged that bootstrap (with its inference characteristics) is a trend in DEA studies.
We have devoted Section 3.1 to review the second-stage-regression model proposed in the literature from a methodological point of view. Afterwards, we explained why we agree with the arguments proposed by Ramalho et al., (2010) to choose their second-stage-regression model at the end of the methodology section (see lines 289-295):
“The nature of the measurement of parametric efficiency and of the DEA estimators make it clear that an unrestricted OLS model at the positive unitary circle is not conceptually consistent. Conversely, nor is it possible to establish that the measurements of efficiency and the DEA estimators reflect a process of data censorship, as the Tobit models set out. This being so, as [21] points out, fractional regression models (FRM) are considered to represent a better approach to the data generating process of the non-parametric measurements of efficiency, and they should be used to estimate second stage models.”
Our preference for FRM models is related to the use of a functional form that allows capturing DEA estimates in a better way, without the need to impose lower or upper limits to obtain reasonable predictions. As Ramalho et. to (2010): “We have shown that, instead, using FRMs is the most natural way of modelling bounded, proportional response variables such as DEA scores.”, (page 254).
- In this sense, a limitation of this study is that it cannot provide information on the confidence intervals for the estimated DEA efficiency scores. The authors may need to look at Ngo and Tsui (2021) as a potential approach for future study to extend their research.
Ngo, T., & Tsui, K. W. H. (2021). Estimating the confidence intervals for DEA efficiency scores of Asia-Pacific airlines. Operational Research. https://doi.org/10.1007/s12351-021-00667-w
Indeed, as suggested by the reviewer, we have included this suggestion in our paper (see lines 585-587):
“Moreover, another future extension of this work is to consider the use of the DEAS approach, as suggested by [38], to make inference of the estimated DEA score and their implications in the second stage models.”
Reviewer 4 Report
The recommendations have been incorporated to an acceptable extent.
Minor findings: I do not recommend giving the name of the study in capital letters
Author Response
The recommendations have been incorporated to an acceptable extent.
Minor findings: I do not recommend giving the name of the study in capital letters
We thank the reviewer for his/her valuable comments. Following the reviewer comment, in the revised version of our paper, the title has been adjusted
