Sustainability Indicators for Industrial Organizations: Systematic Review of Literature
, Dusan Schreiber 2, Claus Haetinger 3, Virgílio José Strasburg 4 and Claudia Luisa Barkert 5Round 1
Reviewer 1 Report
This paper is based on a SLR methodology using also snowballing for the identification of relevant scientific works in literature. The focus of the study is to identify sustainability indicators for industrial organizations where mining was used as method.
The theoretical background part is very well organized and structured and gives a clear overview of sustainability, sustainability in industry and related indicators available in literature
The methodological part is also well done, but in one point not very credible. It contains that the research team was reading nearly 4.000 titles and abstracts doing a coding based on inclusion criteria. This seems a horrendous job and you should spend some sentences to explain how you did this job (e.g. distributing the work to 20 students?), also because the search has been done in July 2018 and there were not so many months for content analysis and later framework construction.
Usually having some many papers a criteria could also be to focus only on journal paper and to not consider conference paper or bock chapters etc. You should also spend some words if you did such a sorting or not.
What is the meaning of the y-axis of this graph??? I would change this with a graph showing how many publications there are for every year 1998-2018.
What means "mentions" i Table 2? The amount of citations???
I would change Table 4 as there is no linkage to the analyzed papers. You need to add to each of the single statements in the boxes also the related references e.g. [1, 2, 6, 9]. It would suggest to use a row for each of the single statements "business attitude", "lack of information",...
Overall evaluation of the paper: The papers uses a structured SLR method to identify existing indicators for sustainability in industrial organizations. The study leads to a number of >30 indicators which are useful as well for academics as also for practitioners. Considering that the recommended changes will be implemented (means major revisions) I would be in favor of publishing this article.
Minor comments:
References should be numbered according to author guidelines e.g. [1] - please change this in the whole document
regarding sustainability I would add also some words about the importance of sustainability in emerging countries where legal restrictions are often not so strong. I could suggest "Sustainable production in emerging markets through Distributed Manufacturing Systems (DMS)" as a source
Table 1: should be adapted to the standards you can find in the author guidelines or in the template of "Sustainability" articles - up to now it is not immediately clear that it is a Table and also the size of the text seems not adequate. Also the reference in the table has to be adapted to the journal standards
"As Presented In Table 1" change to "as presented in Table 1"
there is no need to write "Source prepared by the authors at graphs and tables
do not refer to a table in the text with upper letters (e.g. TABLE 2)
write YALE UNIVERSITY not all in upper letter
there should be a space between figure/table and text - see author guidelines
Kg => kg
Table 3: Add a first column with the reference number e.g. [1]
Table 5: I would change * with a bullet point
Author Response
Reviewer 1
This paper is based on a SLR methodology using also snowballing for the identification of relevant scientific works in literature. The focus of the study is to identify sustainability indicators for industrial organizations where mining was used as method.
The theoretical background part is very well organized and structured and gives a clear overview of sustainability, sustainability in industry and related indicators available in literature
The methodological part is also well done, but in one point not very credible. It contains that the research team was reading nearly 4.000 titles and abstracts doing a coding based on inclusion criteria. This seems a horrendous job and you should spend some sentences to explain how you did this job (e.g. distributing the work to 20 students?), also because the search has been done in July 2018 and there were not so many months for content analysis and later framework construction. Usually having some many papers a criteria could also be to focus only on journal paper and to not consider conference paper or bock chapters etc. You should also spend some words if you did such a sorting or not.
Answer: The methodology was substantially improved in order to make clear points highlighted by reviewer, as to the question of the initial screening, that is, the reading of the title and the summary of almost four thousand titles. In the inclusion criterion "d" it was not clear the topic that the titles should be the result of scientific articles with double review. In addition, the articles were distributed to the five authors of this article to carry out the initial screening of the title and the abstract to fit the title to the predefined criteria. This initial screening stage span from July to October 2018.
What is the meaning of the y-axis of this graph??? I would change this with a graph showing how many publications there are for every year 1998-2018.
Answer: The chart has changed as suggested by the reviewer.
What means "mentions" i Table 2? The amount of citations???
Answer: The “mentions” expression has been replaced by the expression Number of citations.
I would change Table 4 as there is no linkage to the analyzed papers. You need to add to each of the single statements in the boxes also the related references e.g. [1, 2, 6, 9]. It would suggest to use a row for each of the single statements "business attitude", "lack of information",...
Answer: The information in table 4 was changed according to the request of the reviewer, including the numbering of the citations of each of the declarations.
Overall evaluation of the paper: The papers uses a structured SLR method to identify existing indicators for sustainability in industrial organizations. The study leads to a number of >30 indicators which are useful as well for academics as also for practitioners. Considering that the recommended changes will be implemented (means major revisions) I would be in favor of publishing this article.
Minor comments:
References should be numbered according to author guidelines e.g. [1] - please change this in the whole document.
Answer: Citations from references have been adjusted according to journal guidelines.
regarding sustainability I would add also some words about the importance of sustainability in emerging countries where legal restrictions are often not so strong. I could suggest "Sustainable production in emerging markets through Distributed Manufacturing Systems (DMS)" as a source.
Answer: The presentation of sustainability and sustainable development was not intended to be a deep review, but only a conceptual differentiation, similarities and analogies. In the title of the section was added the expression "brief review on ...", to show that the focus of this study is sustainability rather than sustainable development. In view of this, we did not have the idea to provide the state of the art on these subjects, but to clarify their congruences and analogies. We understand that the main focus of the study is the set of sustainability indicators, therefore, we explore a state of the art sustainability indicators, that, for this paper we think as more appropriate.
Table 1: should be adapted to the standards you can find in the author guidelines or in the template of "Sustainability" articles - up to now it is not immediately clear that it is a Table and also the size of the text seems not adequate. Also the reference in the table has to be adapted to the journal standards
Answer: The formatting of the table was revised and the organization of the text was adjusted to clarify the information. In addition, the authors were included following the description of the information.
"As Presented In Table 1" change to "as presented in Table 1".
Answer: The text has been adjusted as suggested by the reviewer.
there is no need to write "Source prepared by the authors at graphs and tables.
Answer: The sources were removed at the suggestion of the reviewer.
do not refer to a table in the text with upper letters (e.g. TABLE 2)
Answer: The writing format was adjusted as suggested by the reviewer.
write YALE UNIVERSITY not all in upper letter
Answer: The citation was adjusted based on the reviewer's suggestion.
there should be a space between figure/table and text - see author guidelines.
Answer: The space between the text and the figures / tables has been entered.
Kg => kg
Answer: adjusted.
Table 3: Add a first column with the reference number e.g. [1].
Response: References were adjusted based on the authors' guidelines.
Table 5: I would change * with a bullet point.
Answer.: Adjusted
Reviewer 2 Report
The paper deals with an interesting and relevant topic: how to measure the sustainability performance of organisations. The presented review has been carried out in a systematical and methodologically appropriate manner. However there are also a number of flaws that should be improved.
Section 2 is dealing with the topics sustainability and sustainable development in a too simplistic way. Both terms should be explained and defined in more detail and with a more accurate use of the English language. It should be made transparent where to items listed in Table 1 originate from: taken from a specific source or own ideas of the authors? Section 2 should also provide a summarizing critical reflection of the state of the art.
Sec. 3 is well structured but leaves open a number of quite important details:
-Why only the key term "indicator" and no other synonyms (such as indizes)?
-Why "Indicators of sustainable" and no other variations such as "Sustainability indicators" or "Indicators of sustainab"
-how were the chosen terms logically connected: AND or OR?
-how did you select specifically these key terms?
-were the key terms applied to a) the title, b) title and abstract or c) title, abstract and full text?
Sec. 4 is supposed to be the key part of this paper, but leaves out some of the most interesting results. Sec 4.4 should be enriched significantly with a lot of findings of your analysis (complete list of indicators incl. their respective frequency, lessons learned for selected important indicators - how were they perceived/discussed in the analysed literature). This could be the main takeaway for the future readers of this paper. You also need to explain where the content of Table 5 originates from and how it was derived. Please also pay more attention to the appropriate use of the English language to make sure your readers can learn as much as possible from your results. There are also some minor issues:
-Table2: mentions supposed to mean citations?
-line 251: "focused on industry in general" what does that mean?
-line 273 is a contradiction to the previous sec2.3 where it said 30 max
-line 274f. cannot be understood
-line 311f: what does the n stand for
Sec. 5 could benefit from a more controversial discussion of your insights gained during your analysis.
Author Response
Reviewer 2
he paper deals with an interesting and relevant topic: how to measure the sustainability performance of organisations. The presented review has been carried out in a systematical and methodologically appropriate manner. However there are also a number of flaws that should be improved.
Section 2 is dealing with the topics sustainability and sustainable development in a too simplistic way. Both terms should be explained and defined in more detail and with a more accurate use of the English language. It should be made transparent where to items listed in Table 1 originate from: taken from a specific source or own ideas of the authors? Section 2 should also provide a summarizing critical reflection of the state of the art.
Answer: The presentation of sustainability and sustainable development was not intended to be a deep review, but only a conceptual differentiation, similarities and analogies. In the title of the section was added the expression "brief review on ...", to show that the focus of this study is sustainability rather than sustainable development. In view of this, we did not have the idea to provide the state of the art on these subjects, but to clarify their congruences and analogies. We understand that the main focus of the study is the set of sustainability indicators, therefore, we explore a state of the art sustainability indicators, that, for this paper we think as more appropriate.
Both terms should be explained and defined in more detail and with a more accurate use of the English language.
Answer: The spelling and grammar of writing was reviewed and improved, as suggested by the reviewers.
Sec. 3 is well structured but leaves open a number of quite important details:
Answer: The authors were distributed according to the information presented in Table 1.
-Why only the key term "indicator" and no other synonyms (such as indizes)?
Answer: We did not included any key word distinct from the indicator, because studies dealing with index or synonym of indicators are centered on a set of indicators. In this sense, it is understood that the key word indicator is standardized
-Why "Indicators of sustainable" and no other variations such as "Sustainability indicators" or "Indicators of sustainab"
Answer: Prior to the definition of the expression tests were performed in the databases and the expression what resulted in great range of return of titles of "indicators of sustainability", being detailed in the methodology in order to make clear why we decided use this definition of the key words.
-how were the chosen terms logically connected: AND or OR?
Answer: in the text of the methodology it was detailed that we decided to use "and", because inserting the key works with double quotes, as presented in the study, the database consulted the complete texts of the materials in order to identify if all the expressions appear in only one material.
-how did you select specifically these key terms? -were the key terms applied to a) the title, b) title and abstract or c) title, abstract and full text?
Answer: In the methodology was explained how the terms were defined through inclusion of the text: "These keywords were defined as a keyword for research, since they were those that presented the most scientific titles in the databases consulted, that is, other pre-queries were also performed with keywords synonymous to these, but did not return with a quantity of titles as defined officially. "
Sec. 4 is supposed to be the key part of this paper, but leaves out some of the most interesting results. Sec 4.4 should be enriched significantly with a lot of findings of your analysis (complete list of indicators incl. their respective frequency, lessons learned for selected important indicators - how were they perceived/discussed in the analysed literature). This could be the main takeaway for the future readers of this paper.
Answer: Section 4.4 was rewritten following reviewer's suggestions and presenting the interesting results, such as the lessons learned, the differentials of the set of indicators suggested in this study, their weaknesses and advantages. We highlight as the contributions of the study to the managerial practice in the industries the use of indicators to evaluate the performance of sustainability.
You also need to explain where the content of Table 5 originates from and how it was derived.
Answer: In the methodology was described the detail on the generation of the information described in Table 5, as suggested by the reviewer.
Please also pay more attention to the appropriate use of the English language to make sure your readers can learn as much as possible from your results.
Answer: The spelling and grammar of writing was reviewed and improved as suggested by reviewers.
There are also some minor issues:
-Table2: mentions supposed to mean citations?
Answer: The term has been changed to citations.
-line 251: "focused on industry in general" what does that mean?
Answer: The word was changed to make it clearer that it refers to the industrial sector, not to mention any specific activity.
-line 273 is a contradiction to the previous sec2.3 where it said 30 max
Answer: The information was adjusted as suggested by the reviewer
-line 274f. cannot be understood
Answer: The text was adjusted in order to make clear information.
-line 311f: what does the n stand for
Answer: A footnote has been inserted explaining the abbreviation of n "number of indicators"
Sec. 5 could benefit from a more controversial discussion of your insights gained during your analysis.
Answer: Text of the section 5 was reviewed and improved as suggested by the reviewer.
Round 2
Reviewer 1 Report
All recommendations were considered in the revised manuscript by authors.
Minor comment. Please check upper case and lower case letters in the subtitle were you added "short review"
