A Dominance-Based Network Method for Ranking Efficient Decision-Making Units in Data Envelopment Analysis
Round 1
Reviewer 1 Report
Here are my comments:
1) The authors state in page 1 (the abstract): “Through these two examples, we confirmed that the proposed ranking method facilitates more precise evaluation and network representation of data envelopment analysis leading to insight on the structural information in the data". ……. Through these two examples, we confirmed that the proposed ranking method facilitates more precise evaluation ……. Page 3 – “It is confirmed that the proposed method facilitates more precise evaluation and network visualization of data envelopment analysis providing useful structural information about the data”.
1.1) The authors should prove that the proposed ranking method facilitates more precise evaluation as they stated. The two examples are not a proof. The authors must explain how the decide that one ranking is more precise than other ranking.
1.2) There are several ranking methods via DEA. For a survey of ranking method see:
"Hadad, Y & Hanani, ZM 2011. Combining the AHP and DEA methodologies for selecting the best alternative. International Journal of Logistics Systems and Management, vol. 9, no. 3, pp. 251- 267. "
· This paper should be added to the references.
· A comparison should be made between the proposed ranking method and other ranking methods. The authors should explain why the results according to their proposed ranking method are indeed "better" than the other ranking methods.
· The authors used in their examples one type of input and several types of outputs (two outputs in the first example and three outputs in the second example). In my opinion, the authors should show only one example with several inputs (at least two) and several of outputs (at least two). When there is only one type of input, a multiple linear regression can be uses and the DEA or ranking method is not appropriate in such case.
2) In my opinion, the background is not well organized. It would be better if the authors first explain the motivation for their study, then discuss the problem statement, and finally review the relevant studies.
3) The authors should present in details the steps or the stages that should to be taken to determine the proposed ranking method.
4) The paper not includes any citation from the journal (sustainability). In my opinion, the authors should find some papers from the journal and quote them in order to show that the paper is matter to the journal audience.
5) The technical quality of all the equations and the symbols in the paper should be improved.
6) Page 1 line 16 – add once the term “DMUs” (decision making units (DMUs)) instead the full term “decision making units” and use DMUs all over the paper.
7) " Page 1 – add once the term DEA (Data Envelopment Analysis) instead the full term and use DEA all over the paper.
8) For consistency, Unit (units) must be changed to DMU (DMUs).
9) Model 1 (page-3) is a BCC model. The authors should note this.
Author Response
1) The authors state in page 1 (the abstract): “Through these two examples, we confirmed that the proposed ranking method facilitates more precise evaluation and network representation of data envelopment analysis leading to insight on the structural information in the data". ……. Through these two examples, we confirmed that the proposed ranking method facilitates more precise evaluation ……. Page 3 – “It is confirmed that the proposed method facilitates more precise evaluation and network visualization of data envelopment analysis providing useful structural information about the data”.
1.1) The authors should prove that the proposed ranking method facilitates more precise evaluation as they stated. The two examples are not a proof. The authors must explain how the decide that one ranking is more precise than other ranking.
ü Ans: There were some misrepresentations in the original text. The proposed method provides another new meaningful ranking result that can show different implications from the existing ranking methods. Accordingly, we corrected the wrong expressions (Page 1, line 24-27; page 2, line 86-89; page 18, line 552-554).
1.2) There are several ranking methods via DEA. For a survey of ranking method see:
"Hadad, Y & Hanani, ZM 2011. Combining the AHP and DEA methodologies for selecting the best alternative. International Journal of Logistics Systems and Management, vol. 9, no. 3, pp. 251- 267. "
This paper should be added to the references.
ü Ans: Including the recommended literature [19], we revised the literature review section by adding a new subsection 2.1 for the review of related DEA-based ranking methods. Also, we reviewed 27 more references and added a table (Page 2, line 61-64; Page 3-5, line 98-176 and Table 1).
ü Added literatures [ref no.] are as follows.
– DEA applications
(Page 1, line 41-44) Energy [3], innovation [4,5], environment [6], agriculture [7], banking [8], and review paper (Emrouznejad and Yang [9] and Liu et al. [10]).
– Usage of DEA
(Page 1-2, line 44-46) Benchmarking target selection [13,14], technological forecasting [15], and clustering [16]
– Ranking methods
(Page 2, line 61-64) Ranking methods with additional information [17-20]
(Page 3, line 110-114) Super efficiency-based methods [22-24]
(Page 3, line 127-128) Alternative frontier-based method [30]
(Page 3, line 129-138) Efficiency change of inefficient DMU-based methods [31-33]
(Page 4, line 146-152) Virtual DMU-based methods [35-38]
- A comparison should be made between the proposed ranking method and other ranking methods. The authors should explain why the results according to their proposed ranking method are indeed "better" than the other ranking methods.
ü Ans: Similar to the answer of 1.1, we corrected the wrong expressions.
- The authors used in their examples one type of input and several types of outputs (two outputs in the first example and three outputs in the second example). In my opinion, the authors should show only one example with several inputs (at least two) and several of outputs (at least two). When there is only one type of input, a multiple linear regression can be uses and the DEA or ranking method is not appropriate in such case.
ü Ans: We tested the first example (2.1 R&D efficiency of SMEs) with two inputs as shown in following table. The corresponding descriptions of the example (Page 9, line 352-356) and results are also modified (Page 9-12, line 372-419).
Before | After | ||
Input (1) | R&D expenditures | Inputs (2) | R&D employees R&D expenditures |
Outputs (2) | R&D sales Intellectual property rights (IPs) | Outputs (2) | R&D sales Intellectual property rights (IPs) |
2) In my opinion, the background is not well organized. It would be better if the authors first explain the motivation for their study, then discuss the problem statement, and finally review the relevant studies.
ü Ans: Section 2 was amended as a whole. In Section 2.1, we review the ranking methods related to this study. In Section 2.2, we discuss network representation for DEA. The existing positional feature described in the background section was modified to describe the method in Section 3 before describing it.
3) The authors should present in details the steps or the stages that should to be taken to determine the proposed ranking method.
ü Ans: We added Figure 3 to illustrate the overall procedure of the proposed method.
4) The paper not includes any citation from the journal (sustainability). In my opinion, the authors should find some papers from the journal and quote them in order to show that the paper is matter to the journal audience.
ü Ans: Following references published in Sustainability are added.
[3] Lin, W.; Chen, B.; Xie, L.; Pan, H. Estimating energy consumption of transport modes in China using DEA. Sustain. 2015, 7, 4225–4239.
[7] Pang, J.; Chen, X.; Zhang, Z.; Li, H. Measuring eco-efficiency of agriculture in China. Sustain. 2016, 8.
[13] Park, J.; Sung, S. Il Integrated approach to construction of benchmarking network in DEA-based stepwise benchmark target selection. Sustain. 2016, 8, 1–15.
6) Page 1 line 16 – add once the term “DMUs” (decision making units (DMUs)) instead the full term “decision making units” and use DMUs all over the paper.
ü Ans: Revised accordingly to the comment.
7) " Page 1 – add once the term DEA (Data Envelopment Analysis) instead the full term and use DEA all over the paper.
ü Ans: Revised accordingly to the comment.
8) For consistency, Unit (units) must be changed to DMU (DMUs).
ü Ans: Revised accordingly to the comment.
9) Model 1 (page-3) is a BCC model. The authors should note this.
ü Ans: Revised accordingly to the comment.
Reviewer 2 Report
The paper has potential and it can be a good contribution to the literature. The authors propose a new ranking method using network analysis for efficient decision making units to improve the discriminating power of data envelopment analysis. Nevertheless, some aspects need to be changed or better explained, for example:
· More results can be included in the abstract;
· In the keywords, I would recommend to avoid the use of abbreviations. Moreover, I would change “ranking efficient DMUs” for “ranking”;
· All the abbreviations must be presented in the text;
· The introduction can be improved, for example, including more advantages and weak points of the DEA (De Witte and Marques, 2010);
· There are several spaces missing in the text before brackets;
· DEA is a methodology widely used in the literature (Simoes and Marques, 2012);
· All the equations must be numbered;
· Figure 1 can be better explained;
· More information about the case study can be included (for example about the region, GDP, …);
· The methodology must be justified, for example regarding the model, the orientation, …);
· Include a table with the statistical analysis for the data used;
· Justify the evolutions in figure 2;
· Some recommendations can be included in the conclusions;
· The references must be homogenized and in line with the author guidelines (for example, some issues are missing).
References:
DE WITTE, K.; MARQUES, R. (2010). Influential observations in frontier models, a robust non-oriented approach to the water sector. Annals of Operations Research. ISSN: 0254-5530. Vol. 181, no. 1, pp. 377-392.
SIMÕES, P.; MARQUES, R. (2012). On the economic performance of the waste sector. A literature review. Journal of Environmental Management. Elsevier. ISSN: 0301-4797. Vol. 106, no. 15, pp. 40–47.
Author Response
1. More results can be included in the abstract;
ü Ans: More results are added and abstract is revised accordingly (Page 1, line 24-27).
2. In the keywords, I would recommend to avoid the use of abbreviations. Moreover, I would change “ranking efficient DMUs” for “ranking”;
ü Ans: We changed “ranking efficient DMUs” for “ranking” (Page 1, line 28).
3. All the abbreviations must be presented in the text;
ü Ans: Revised accordingly to the comment.
4. The introduction can be improved, for example, including more advantages and weak points of the DEA (De Witte and Marques, 2010);
ü Ans: Introduction part is reorganized and supplemented. Modified structure and corresponding texts are as follows:
DEA introduction, features, advantages and weak points (Page 1, line 31-40)
DEA applications and purposes for use (Page 1-2, line 41-46)
Importance of ranking in DEA (Page 2, line 47-54)
Limitations of conventional DEA-based ranking (Page 2, line 55-58)
Brief review of existing DEA-based ranking methods (Page 2, line 59-73)
5. There are several spaces missing in the text before brackets;
ü Ans: We revised them accordingly to the comment (Figure 2(a), Figure 2(b) à Figure 2 (a), Figure 2 (b)).
6. DEA is a methodology widely used in the literature (Simoes and Marques, 2012);
ü Ans: Introduction part is supplemented with literatures on DEA applications (Page 1, line 41-44).
Energy [3], innovation [4,5], environment [6], agriculture [7], and banking [8].
Review paper: Emrouznejad and Yang [9] and Liu et al. [10].
7. All the equations must be numbered;
ü Ans: We revised them.
8. Figure 1 can be better explained;
ü Ans: Figure 1 was modified to remove the confusing part (a) and add a description with respect to Equation (1) (Page 5, line 192-193).
9. More information about the case study can be included (for example about the region, GDP, …);
ü Ans: An explanation about Korean SMEs is provided in Page 9, line 348-351.
10. The methodology must be justified, for example regarding the model, the orientation, …);
ü Ans: Justification of the method is added according to the comment (Page 8, line 323-326).
11. Include a table with the statistical analysis for the data used;
ü Ans: Descriptive statistics of the data used are added as Table 2 and Table 4.
12. Justify the evolutions in figure 2;
ü Ans: We added Section 2.1 (Ranking methods using positional features) and reviewed 27 more references, and we added explanations about Figure 2 in relation to previous methods (Page 6, line 228-230, 236-238).
13. Some recommendations can be included in the conclusions;
ü Ans: We revised conclusion part according to the comment (Page 18, line 566-582).
14. The references must be homogenized and in line with the author guidelines (for example, some issues are missing).
Ans: We revised them according to the author guidelines.
Reviewer 3 Report
Since many years there have appeared many modifications of the DEA method, which prove its popularity. Various problems are solved, such as for instance the problem of the selection of variables, high sensitivity to atypical data characterizing effective units, etc. One of the problems is also creating a ranking for efficient DMUs.
This article takes up important issues, and the considered efficiency method fits well into the subject matter of the journal (Sustainability).
However, the literature review (quantitative and qualitative) in the scope of methods for creating the DMU ranking is modest. A lot of research has not been taken into account, though this would increase the value of the publication. For example, Literature review in "Ranking efficient DMUs using cooperative game theory" (https://doi.org/10.1016/j.eswa.2017.03.004). A tabular table with references could be an interesting solution.
A thorough literature review is necessary to explain the contribution of the research.
The conclusions should emphasize the scientific added value of the article in the light of the analyzed literature. The potential application of the test results (areas) as well as any possible limitations of the method should be indicated.
Mathematical formulas should be numbered.
Author Response
A thorough literature review is necessary to explain the contribution of the research.
ü Ans: Including the recommended literature [42], we revised the literature review section by adding a new subsection 2.1 exclusively for the review of related DEA-based ranking methods. Also, we reviewed 27 more references and added a table (Page 2, line 61-64; Page 3-5, line 98-176 and Table 1).
ü Added literatures [ref no.] are as follows.
– DEA applications
(Page 1, line 41-44) Energy [3], innovation [4,5], environment [6], agriculture [7], banking [8], and review paper (Emrouznejad and Yang [9] and Liu et al. [10]).
– Usage of DEA
(Page 1-2, line 44-46) Benchmarking target selection [13,14], technological forecasting [15], and clustering [16]
– Ranking methods
(Page 2, line 61-64) Ranking methods with additional information [17-20]
(Page 3, line 110-114) Super efficiency-based methods [22-24]
(Page 3, line 127-128) Alternative frontier-based method [30]
(Page 3, line 129-138) Efficiency change of inefficient DMU-based methods [31-33]
(Page 4, line 146-152) Virtual DMU-based methods [35-38]
The conclusions should emphasize the scientific added value of the article in the light of the analyzed literature. The potential application of the test results (areas) as well as any possible limitations of the method should be indicated.
ü Ans: We revised the conclusion part as indicated (Page 18-19, line 573-582 (limitation), page 18 line 566-572 (application)).
Mathematical formulas should be numbered.
ü Ans: We revised them.
Round 2
Reviewer 1 Report
No
