Validating Circular Performance Indicators: The Interface between Circular Economy and Stakeholders

Round 1

Reviewer 1 Report

The paper provides an interesting overview of the topic in hand - Circular Economy.

Akthough it is very well written and provides the readers with all the necessary information, the paper needs to be improved in terms of Conclusion - it is too short for a paper of this lenght.

Additionally, Discussion segment could be added, in order to discuss the obtained results and present the possibilities of future research.

Moreover, in terms of practical part, the paper requires some checking - some of the references are out of place (errors appear).

Author Response

The authors would like to thank reviewer 1 for the valuable comments. The replies can be found in the attached file.

Author Response File: Author Response.pdf

Reviewer 2 Report

The article is touching upon a very important topic that is in the focus of both the academic community, with a vast body of work published in the past few years, as well as of policy-makers on various levels, from local to European and even global. The authors have surveyed this work as a preparation for this article. However, if I am to recommend the article for publication, several important revisions or clarifications need to be included.

1. Although the authors have inspected relatively important body of work, reading it gives the perspective that this is the first piece of scholarly work, that takes onto account a number of different indicators and includes a specific type od modeling technique. The authors should elaborate at least briefly on other attempts. You may consult the following work:

- MILEVA BOSHKOSKA, Biljana, RONČEVIĆ, Borut, DŽAJIĆ URŠIČ, Erika. Modeling and evaluation of the possibilities of forming a regional industrial symbiosis networks. Social sciences, ISSN 2076-0760, 2018, vol. 7, iss. 1. 
- DŽAJIĆ URŠIČ, Erika. Morphogenesis of Industrial Symbiotic Networks. Berlin: Peter Lang, cop. 2020.

Other approaches can be presented and discussed as well.

2. The indicators are ranked by surveying stakeholders and academic experts. The policy relevance of including stakeholders is clear. However, we need more meaningful elaboration on why this is important from scientific perspective and how this approach it impacts the scientific validity of the analysis and its conclusions.

3. The authors state that To a large extent, CE indicators are expected to be used by the non-scientific community, such as corporations and businesses, regional authorities, and others. Is this assumption plausible? The practice and anecdotal evidence oftentimes speak to the contrary and the authors do not provide evidence on that in the article.

4. The authors state that the aim of this research is to close the gap between academia and industry in terms of selecting acceptable CE indicators. The gap is not closed with indicators. This is somewhat naive assumption. Other approaches are used to close the gap and it is a hard-to-achieve goal. The authors should consult the literature on regional development to elaborate on that, or rephrase the aim of the article.

5. The indicators database was developed within the MSWCA framework. More information is required, so that the average reader can receive the basic information without having to read additional work to understand this article. How and why were these indicators collected? What is the theoretical background and conceptualisation?

6. On page 7 the authors state: "Evaluation and validation of the CPIs was performed by interviewing." However, what they describe, are not interviews, but surveys. This is also indicated by the data collection tool in the appendix. If the data collection tool was in fact used as a part of a broader interviews, then it should be clearly stated and described.

7. The data was collected via an online questionnaire for the public and private stakeholders and via an online work shop for the academic experts. These are two vastly different approaches, with important implications for the data and analysis. We need more details on how data collection was in fact implemented? I am especially keen to read about the online workshop? In a workshop, does then single survey filled in by a stakeholder present the opinion of a single stakeholder, or was the opinion shaped through discussion? If there was in fact a discussion, followed by the surveying, we may have serious reservation regarding the methodology.

8. How were stakeholders in the sample selected? How was representativeness of the sample ensured? If representativeness was not the goal, what other consideration was taken into account in selecting the sample? Furthermore, sample size of 40 is rather small, so how does this impact the validity and generalisability of results? This point is extremely important, especially in view of ambitious conclusions of the article, emphasising their importance in "Sustainable Development Goals" of Agenda 2030 and the "New Green Deal". So, the foundations need to be solid and carefully explained. Sample is in fact mentioned as a limitation in the conclusion, but very scarcely and not discussed at any length.

Author Response

The authors would like to thank reviewer 2 for the valuable comments. The answers to the comments can be found in the attached file.

Author Response File: Author Response.pdf

Reviewer 3 Report

Paper is well written and has potential. My main question is - why Water? How is this relevant to this Journal? Why not some journal that is interested in economic development, perhaps Sustainability? 

My other concern is lack of any proof of concept or example where the indicators are used and tested. Without that the whole framework looks very theoretical and untested. Why should we trust it? What is it's added value? What can we learn from applying it? How is it supposed to be used? How do we know that it is adequate?

It also remains unclear how to operationalise the framework and the results that we get from it. How various indicators can be improved to increase the circular performance? It also seems that 20 might be a bit too many to be practical to account for. The indicators have been basically taken from a previous publication by the authors, Nika et al., 2020. Are all the indicators needed? Would be good to see more justification for that and perhaps some more critical analysis of why all of them and exactly them are important in all cases. 

Author Response

The authors would like to thank reviewer 3 for the valuable comments. The replies can be found in the attached file.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

I do not have further comments.

Author Response

We would like to thank the reviewer.

Reviewer 3 Report

You have tried to explain the relevance of the paper to "Water" in the response letter to me, but not in the text of the article. The explanation given hardly changes my opinion about the relevance. Yes, water is important and everything is important for water, but it does not mean that the Journal should cover all the aspects. In fact, I believe that this will be a disservice to the authors, because the paper will be read by an audience that will hardly appreciate the topic and understand the importance, unless you do a very good job explaining it.

Also mentioning how the frame could be used does not really prove that it is useful. A bit more about the application and the gains and lessons learned would be helpful.

Author Response

We would like to thank the reviewer for the valuable comments. The answers to the comments can be found in the attached document.

Author Response File: Author Response.pdf