Methodology for Calculating the European Innovation Scoreboard—Proposition for Modification
Round 1
Reviewer 1 Report
Authors suggest a method to calculate the EIS index. The suggested method uses a small number of indicators. Although, the study provides useful insights and the method is clearly described, I have some concerns that authors should address. These concerns are as follow.
Line 197: what is the criterion used to identify groups? Please, emphasize this issue.
Line 206: “…statistical and mathematical methods….” is bullet point missing here?
Line 425: “…In the first step, the missing data was estimated by means of the mice package, implemented in the R environment. Than Stepwise regression…” Check English…data is plural….then and not than.
Lines 424 - : As I understand, authors adopt a static perspective because they use data relative to a single year. Results of their study are consequently affected by year. Please, discuss such issue. It is important to understand if determinants affecting the EIS measurement.
Line 435, Table 3: please, explain what variable SII is.
Line 452: indicator C.7 - PCT patent applications has been excluded from the analysis. That is odd because generally filing patent applications lead to innovation. Please, discuss such issue.
Author Response
Please see the attachment
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The article is very good and I support it to be published!
Author Response
We would like to thank you for a thorough review of our article, as well as for your high mark.
The Authors would like to inform, that publication will be sent to language editing by MDPI.
Reviewer 3 Report
Any change. Conglatulations for the work done it
Author Response
We would like to thank you for a thorough review of our article, as well as for your high mark.
The Authors would like to inform, that publication will be sent to language editing by MDPI.
Reviewer 4 Report
The article, entitled "Methodology for Calculating the European Innovation Scoreboard EIS - Proposition of Modification", is a theoretical and methodical study that discusses issues related to the EIS (The European Innovation Scoreboard) methodology and its simplification in an innovative way.
The aim of the study is to show that the creation of the EIS ranking, measuring and comparing the level of innovation in EU countries, as well as other European and neighboring countries, can be simpler, more transparent and more effective than before. In my opinion, he aim of the work has been achieved.
The structure of the study includes three main parts.
In the first, the authors describe the concept of innovation as an unwavering subject of interest for researchers from around the world, conducive to achieving sustainable development. They pay attention to the interdisciplinary, multidimensional, and systemic approach of the analysis of this problem and point to the importance of using innovation rankings in shaping an innovative environment.
The second part deals with the methodological aspects of the discussed issues. The general characteristics of the methods of researching the innovativeness of the sector, enterprise and the European Innovation Scoreboard are presented. Researchers believe that EIS, like any other method, should meet the methodological requirements; therefore, the methodological aspects of ranking building with the theoretical principles being presented.
The third and last part presents the results of the quantitative methods used for the statistical evaluation and verification of the adopted goal. The research procedure includes the use of the stepwise regression method, which allowed for an assessment of the impact of 27 indicators on the ranking and identification of 22 determinants that affect the innovation of individual countries in the EIS.
Subsequently, the linear ordering method was applied, and a new ranking was built using the determinants identified in the first stage. The use of the Kendall concordance coefficient, the test examining its significance, and the Rand index allowed to confirm the high agreement of both rankings, thus confirming the legitimacy of the procedure. In the last step, using the mean and standard deviation, individual countries were divided according to the degree of innovation; moreover, the consistency of the structure of these groups with the groups created on the basis of the European Innovation Scoreboard was examined.
Undoubtedly, the advantage of the study is the proposed new approach to creating the EIS ranking. It indicates the possibility of reducing the number of indicators used to create it from 27 to 22 and the proposal to use the linear ordering method. This approach simplifies the current ranking methodology, which is important both for the units that develop them and for all its users. In the study, the authors also noted that the methods and indicators used to measure innovation are imperfect, complicated, labor-intensive, and require thorough reconstruction and further research.
Suggested changes:
- Please consider changing the title from: “Methodology for Calculating the European Innovation Scoreboard EIS - Proposition of Modification” into “Methodology for Calculating the European Innovation Scoreboard (EIS) - Proposition of Modification”.
- Rephrase and clarifythe abstract section, and make it more prominent. You state that “The article is of theoretical and methodological nature with the use of a variety of research methods”. If your paper uses secondary data (European Innovation Scoreboard 2020) and applies statistical methods, then it is an empirical paper as well, not just theoretical. Be clear about the nature of the paper. State clearly what is the role of the conceptual section (2) and what is the aim of the calculations; what all these methods are used for. Put it simply, what is the idea behind calculating in the paper. In the abstract do not enumerate all methods used, rather make the reader curious about your idea, the procedure applied and the main findings.
- Please specify clearly, what is the goal of the paper, since it is not consistent throughout the paper.
In the abstract you write: “The aim of the study is to discuss issues related to the simplification of the methodology 10 of the EIS ranking”;
On page 3 you write: “The purpose of this publication is to show that the creation of the EIS ranking can be more straightforward, more transparent and more effective than before”
On page 9 you state that: “The aim of the publication is to identify determinants affecting the Summary Innovation Index result and, consequently, the country's position in the European Innovation Scoreboard (EIS), and then, based on the identified determinants, to attempt to rank countries using the linear ordering method”.
In Conclusions: “The aim of this article was (to) identify determinants affecting the Summary Innovation Index”
I assume, that with the third one, you understand what is the aim of the calculations (Identification of determinants influencing the position in the European Innovation Scoreboard?)but that needs to be made clear (what is the aim of the paper, what is the aim of each section), starting with the info in the abstract and keeping it homogeneous in the whole text.
- In the theoretical part it could be worthwhile to mention the organizational innovation measurement tools found in scientific publications, not just consulting tools or EU statistics. You could refer specifically to Miller (1983) – innovation as number of new products introduced in relation to competition; Lumpkin & Dess (1996) – innovation as part of entrepreneurial orientation; Antoncic&Hisrich (2010) – financial performance as a measure of innovativeness.
- Point 2: “Literature review” – I suggest changing for “Innovation and its measurement - Conceptual background”.
- Point 3: “Materials and methods” – I suggest changing for “Determinants affecting the Summary Innovation Index – research results”
- Avoid repetitions when starting paragraphs (lines 98, 107. Instead, use “Hence,”, “Consequently,”, “For this reason,”, etc.).
- In the “Conclusions” section more emphasis could be put onto actual findings and their interpretations, not just the technics behind statistical reasoning.
- Questioning the existing scoreboard is a challenging thing to do. Please indicate potential limitations of your proposition.
Overall, the paper raises an interesting topic on innovation measurement, and simplifying the EIS methodology. It is well-written, interesting and sheds some conceptual light on the methods of innovation measurement. I recommend the paper for publication after minor changes.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Authors have addressed my concerns. There are only some minor concerns as follows:
Line 200: it is micro and not micor
Maybe using the term “organizational innovation” may be misleading as it has different meanings, either meaning the change of the organizational characteristics (innovation of the organization) or the product of the innovation process in the organization. Please, consider such am ambiguity.
Lines 286, 300, 310, 324: why using capital letters?
Table 3 and Table 4: I suggest edit these tables according to the journal style (see Table 5 and Table 6)
Author Response
The Autors would like to thank you for a thorough review of article. The constructive remarks contained in the review served to supplement the content of the work and indicated the proper direction of its refinement. Referring in detail to the remarks, we would like to inform you that the following changes have been made:
- "Line 200: it is micro and not micor" - Corrections were made.
- "Maybe using the term “organizational innovation” may be misleading as it has different meanings, either meaning the change of the organizational characteristics (innovation of the organization) or the product of the innovation process in the organization. Please, consider such am ambiguity." – “the organizational innovation measurement tools” has been added based on another Reviewer's comment.
- "Lines 286, 300, 310, 324: why using capital letters?" - Corrections were made.
- "Table 3 and Table 4: I suggest edit these tables according to the journal style (see Table 5 and Table 6)" - Changes were made as the article has been sent to language corection by MDPI.
