Assessing and Ranking EU Cities Based on the Development Phase of the Smart City Concept
Round 1
Reviewer 1 Report (Previous Reviewer 1)
The authors assessed and ranked the smart city maturity of EU cities based on the number of existing projects on the literature organized in the smart city category through a quantitative analysis.My suggestions are as follows:
1.In the introduction, it is clear that the authors do not have enough information on the existing studies and suggest that the authors must look into more studies on air quality and emissions done by others. See for example “VAR-tree model based spatio-temporal characterization and prediction of O3 concentration in China” , "Haze Risk Assessment Based on Improved PCA-MEE and ISPO-LightGBM Model"and"Prediction of fluctuation loads based on GARCH family-CatBoost-CNNLSTM".
2.Authors need to describe the data in detail, not list it.
3.The author needs to go more in depth and elaborate on A1-A27, if it's just a list this is pointless.
4.We have given many comments and the authors have always ignored them, if the next time there is no change according to the comments, I will reject the paper directly!
Some minor and detailed grammatical errors.
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted in the blue colour. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Reviewer 1
Comment 1: In the introduction, it is clear that the authors do not have enough information on the existing studies and suggest that the authors must look into more studies on air quality and emissions done by others. See for example “VAR-tree model based spatio-temporal characterization and prediction of O3 concentration in China” , "Haze Risk Assessment Based on Improved PCA-MEE and ISPO-LightGBM Model"and"Prediction of fluctuation loads based on GARCH family-CatBoost-CNNLSTM".
Answer 1:
Thank you for your comment. We have decided to include the mentioned references and perform a final literature review (with the Keywords: “Smart City”; “Area”; “Vertical”; and “Category”) to enunciate other research papers that could be missing and try to address this specific subject.
Comment 2: Authors need to describe the data in detail, not list it.
Answer 2:
Thank you very much for your comment. Several amendments were performed to standardize the terms used to explain each step of the methodology (namely the weights vs loadings). Furthermore, the structure was also adapted and several texts added to improve the methodology and results explanation. a summary table was added contemplating the factor results of each step of the methodology for Dublin, and the steps were standardized throughout the document. The structure of the document suffered several changes to respect the 4 steps described in the methodology.
Comment 3: The author needs to go more in depth and elaborate on A1-A27, if it's just a list this is pointless.
Answer 3:
Thank you. Additional text was added before every table in Appendix A to describe the data of the tables.
Reviewer 2 Report (Previous Reviewer 4)
I appreciate your work to improve the submitted paper. In my view, the improvements made during the exhaustive review process made this version of your paper clear. This can be seen in the significant improvements in explaining the methodology and the mathematical relationships that support each step proposed. The complete and concise example that was added allows readers to follow each step performed and reproduce the results presented if necessary. Some errors (typographical and numerical) that the document contained were also corrected.
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted in the blue colour. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Reviewer 2
Comment 1:
I appreciate your work to improve the submitted paper. In my view, the improvements made during the exhaustive review process made this version of your paper clear. This can be seen in the significant improvements in explaining the methodology and the mathematical relationships that support each step proposed. The complete and concise example that was added allows readers to follow each step performed and reproduce the results presented if necessary. Some errors (typographical and numerical) that the document contained were also corrected.
Answer 1:
We would like to thank you for your careful revision, encouragement, and suggestions for improving the paper.
Reviewer 3 Report (Previous Reviewer 5)
The revision is done well.
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted in the blue colour. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Reviewer 3
Comment 1:
The revision is done well.
Answer 1:
We would like to thank you for the time spent reviewing the paper, and your constructive feedback.
Round 2
Reviewer 1 Report (Previous Reviewer 1)
Recommendation accepted in its current form.
Minor grammatical errors.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
The authors assessed and ranked the smart city maturity of EU cities based on the number of existing projects on the literature organized in the smart city category through a quantitative analysis.My suggestions are as follows:
1.The novelty of the paper should be described in the introduction.
2. In the introduction, it is clear that the authors do not have enough information on the existing studies and suggest that the authors should look into more studies on air quality and emissions done by others.See for example “VAR-tree model based spatio-temporal characterization and prediction of O3 concentration in China” , "Haze Risk Assessment Based on Improved PCA-MEE and ISPO-LightGBM Model"and"Prediction of fluctuation loads based on GARCH family-CatBoost-CNNLSTM".
3.Were the authors' rating levels determined by principal component analysis? The authors should elaborate on how the weights of each factor are calculated?
4.The title of the form should not start with . as the end.
5.The author has too little in the exposition part, instead of listing a large number of tables A1-A27, the author should choose representative ones for discussion and description, instead of listing tables, otherwise it feels like an appendix to me. Otherwise, it feels like an appendix to me and ignores the content of the text.
6.The author's discussion section is instead very much like a review and should be combined with others' literature to compare your study to further illustrate your findings.
Some minor and detailed grammatical errors.
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted as “track changes”. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Comment 1:
The novelty of the paper should be described in the introduction.
Answer 1:
Thank you for your attention. Several paragraphs were added to the introduction section to elaborate more on the novelty of the paper.
Comment 2:
In the introduction, it is clear that the authors do not have enough information on the existing studies and suggest that the authors should look into more studies on air quality and emissions done by others. See for example “VAR-tree model based spatio-temporal characterization and prediction of O3 concentration in China” , "Haze Risk Assessment Based on Improved PCA-MEE and ISPO-LightGBM Model"and"Prediction of fluctuation loads based on GARCH family-CatBoost-CNNLSTM".
Answer 2:
Thank you for your comment. The authors’ intention in the introduction was to reflect the fact that policymakers are challenged to assess the current Smart City maturity level and understand its meaning. Moreover, this study aimed to quantify and categorize the initiatives found in the literature. The goal was not to describe or discuss each Smart City initiative.
Comment 3:
Were the authors' rating levels determined by principal component analysis? The authors should elaborate on how the weights of each factor are calculated?
Answer 3:
Thank you. We have decided to elaborate more on the manuscript. Thus, we included several new paragraphs to explain how the weights were calculated. In addition, please see Table 2 (it was also added)
Comment 4:
The title of the form should not start with . as the end.
Answer 4:
Thank you for your attention. The typo was amended.
Comment 5:
The author has too little in the exposition part, instead of listing a large number of tables A1-A27, the author should choose representative ones for discussion and description, instead of listing tables, otherwise it feels like an appendix to me. Otherwise, it feels like an appendix to me and ignores the content of the text.
Answer 5:
Thank you for your comment. In fact, tables A1-A27 are part of Appendix A. Nevertheless, it was added new text to describe more what these tables represent, giving specific examples in some parts of the manuscript (e.g. in the methodology section).
Comment 6:
The author's discussion section is instead very much like a review and should be combined with others' literature to compare your study to further illustrate your findings.
Answer 6:
Thank you. The discussion section was extended and improved based on your comment.
Reviewer 2 Report
Dear Editor,
I hope this email finds you well. I have completed my review of the manuscript titled "[Manuscript Title]," and I would like to provide my feedback for your consideration.
Overall, I find that the manuscript falls short in several aspects, particularly in the clarity of the presented method for ranking and the demonstration of innovation and creativity. In my opinion, these issues significantly hinder the manuscript's potential contribution to the field.
Firstly, the method for ranking described in the manuscript is unclear and lacks sufficient detail. Without a clear understanding of the methodology employed, it becomes challenging to evaluate the reliability and validity of the results. I would suggest that the authors revise and expand upon the methodological section, providing additional explanations and potentially including illustrative examples to improve clarity.
Furthermore, I find the manuscript to be lacking in terms of innovation and creativity. It is crucial for scientific publications to introduce novel ideas or approaches that advance the current state of knowledge. Unfortunately, this manuscript does not sufficiently demonstrate such qualities. I would encourage the authors to reevaluate their work, considering potential avenues for originality and innovative thinking, which would enhance the manuscript's overall impact and significance.
Taking these factors into account, I recommend that the manuscript be rejected in its current form. However, I do believe that with substantial revisions and improvements in addressing the aforementioned concerns, the manuscript could have the potential for resubmission and consideration for publication in the future.
Thank you for considering my feedback. Please let me know if you require any further information or clarification. I am available to provide additional assistance if needed.
Kind regards,
Dear Editor,
I hope this email finds you well. I have completed my review of the manuscript titled "[Manuscript Title]," and I would like to provide my feedback for your consideration.
Overall, I find that the manuscript falls short in several aspects, particularly in the clarity of the presented method for ranking and the demonstration of innovation and creativity. In my opinion, these issues significantly hinder the manuscript's potential contribution to the field.
Firstly, the method for ranking described in the manuscript is unclear and lacks sufficient detail. Without a clear understanding of the methodology employed, it becomes challenging to evaluate the reliability and validity of the results. I would suggest that the authors revise and expand upon the methodological section, providing additional explanations and potentially including illustrative examples to improve clarity.
Furthermore, I find the manuscript to be lacking in terms of innovation and creativity. It is crucial for scientific publications to introduce novel ideas or approaches that advance the current state of knowledge. Unfortunately, this manuscript does not sufficiently demonstrate such qualities. I would encourage the authors to reevaluate their work, considering potential avenues for originality and innovative thinking, which would enhance the manuscript's overall impact and significance.
Taking these factors into account, I recommend that the manuscript be rejected in its current form. However, I do believe that with substantial revisions and improvements in addressing the aforementioned concerns, the manuscript could have the potential for resubmission and consideration for publication in the future.
Thank you for considering my feedback. Please let me know if you require any further information or clarification. I am available to provide additional assistance if needed.
Kind regards,
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted as “track changes”. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Comment 1:
Firstly, the method for ranking described in the manuscript is unclear and lacks sufficient detail. Without a clear understanding of the methodology employed, it becomes challenging to evaluate the reliability and validity of the results. I would suggest that the authors revise and expand upon the methodological section, providing additional explanations and potentially including illustrative examples to improve clarity.
Answer 1:
Thank you very much for your feedback. The methodology section was improved based on your comment. Several paragraphs were added, including additional explanations and illustrative examples to improve clarity as stated.
Comment 2:
Furthermore, I find the manuscript to be lacking in terms of innovation and creativity. It is crucial for scientific publications to introduce novel ideas or approaches that advance the current state of knowledge. Unfortunately, this manuscript does not sufficiently demonstrate such qualities. I would encourage the authors to reevaluate their work, considering potential avenues for originality and innovative thinking, which would enhance the manuscript's overall impact and significance.
Answer 2:
Thank you. We appreciate your feedback. If any policymaker or researcher sought for data about existing Smart City projects they would be challenged to find structured and categorized information by city and vertical. Thus, we humbly think we are filling an existing gap. In addition, our approach allows smaller cities to have a voice, since they are always forgotten in existing rankings and Smart City discussions. The future work that can arise from this research is significant. Several working groups can be joint by having the cities that are working on the same projects sharing best practices and learnings, or even a city that is being considering a new vertical, can see which are the cities that have already experience on a specific field.
Reviewer 3 Report
Overall, the topic is up to date. Nevertheless, there are some points regarding the structure of the paper and the logic of the arguments that should be considered before publication. The comments and suggestions are as follows:
1. The authors cite themselves with more than 20% of the total references; only Correia is cited with 9 out of 44 referenced articles. This raises the question of an unbiased perspective.
2. The authors state in the abstract that they compare existing initiatives in the selected cities based on the number of projects funded and the socio-demographic variables of the cities. In reality, they use only one variable as a proxy for very complex sociodemographic phenomena, population density, and this should be stated accordingly.
3. There is no theoretical background on the smart city concept or smart city maturity models in the paper, and therefore the hypothesis is not clearly derived from the existing literature gap on smart city maturity models. In addition, the authors do not provide a definition or describe what the term "smart city" encompasses for the purposes of their paper.
4. The maturity models in the literature are far too complex to be narrowed down to only the number of projects funded at a given point in time; in fact, they do not account for funds invested in projects before or after the case studies in SCOPUS articles in the cities listed.
5. In addition, "maturity models' assume stages of development, e.g., capacity building, establishment of processes or structures, etc. The proposed smart city maturity model assumes that the acquisition of resources for health, education, or sports projects depends on previous investments in infrastructure and transportation (i.e., it assumes that the infrastructure and transportation in these cities are already in order and therefore projects in the sports sector represent a higher level of maturity for a city). The paper does not provide evidence in the literature to support these claims or assumptions, nor do the authors prove these assumptions in their research. Since the authors only analyze case studies from the SCOPUS database, it is not possible to draw conclusions about city development based on evolution over time, i.e., evolution should be indicative of a process; this also applies to the assumed relationship between smart city maturity and the number of projects in a given category (or causality). In summary, the Smart City Maturity Development Model should imply that development occurs in stages. Please provide theoretical and/or empirical evidence that could link the project category to the proposed stage of development.
6. Data question: data were taken from the database SCOPUS. The authors did not provide information on the data range of the articles analyzed. However, if we assume that the articles analyzed include all the articles listed in the database over the last 20 years and describe projects developed and implemented in these cities during the time span analyzed, it is not logical to rank these cities at a single point in time (e.g., in 2023). It would also be beneficial to indicate the years in which the projects were active. Can you indicate the duration, scope, and success of the projects? Can you provide more detail on why we should infer the "smartness" of a city based solely on the number of projects scattered over time? Can you provide a logical substitute for ranking these cities?
7. The authors do not explicitly state that the selected cities were analyzed throughout their activity. Therefore, it is not clear whether the papers analyzed describe all projects that were active in a given city at a given date (or preferably up to that date). Rather, the SCOPUS articles analyzed could represent a single project of interest or a group/category of projects (e.g., rural and agricultural projects) in a particular area in which the authors of those articles were interested. Since SCOPUS is the source for the data and SCOPUS is a tool characteristic of an academic audience, the number of projects presented for a city could also depend on the presence of a college in a particular area, as well as the inclination of smart city researchers in a particular region. Presenting the overall activity of a city based on a case study article could lead to a wrong conclusion.
8. The number of projects can sometimes serve as a good proxy for the activity of a city or region; in the case of maturity models, it would be wiser to also use additional data (size of projects, project scope, financial data, project impact, number of people involved, etc.). The number of projects is likely to depend on the timing of the analysis and the available funding (project calls) in the (previous) period (e.g., open calls for EU funding or the economic situation in the city/region); in addition, it may depend on the specialization of the staff in the field (e.g., urban development), the research area of the researchers describing the projects (e.g., rural and agriculture), and societal trends (e.g., toward waste reduction in recent years). Instead of an analysis based on the number of projects, a content analysis should be conducted to assess the maturity of the cities.
9. In addition, the hypothesis tacitly states that more community projects mean higher smart city maturity level. Infrastructure and transportation projects tend to be much more complex and financially challenging than community projects. Community projects are easier to propose, acquire, and implement, and are usually additionally supported with external human capital (people from associations). Therefore, their bare numbers cannot easily be an indicator of a city's highest level of maturity. Please support any hypotheses with theoretical foundations or empirical evidence.
10. Please provide a theoretical explanation for why the most important urban development projects (infrastructure and transportation) are weighted with a disproportionately low index (2.85% for infrastructure/transportation versus 43.86% for community projects).
11. The cities listed 165th through 172nd are all ranked 74th based on the same score (points are earned with one project per city in the mobility and transportation category and the same population density ponder of 1). The authors ranked these 8 cities the same regardless of any other characteristics of these cities or dimensions and elements of the smart city concept. The question arises, what does this mean for the maturity level of the cities? Is it also the same? An the same as the maturity level of the 173rd city, which is also ranked 74th because it has a different population density (5 instead of 1), but 2 projects in the same mobility and transport category?
12. What does "density = 1" mean in the case of Thessaloniki, Greece?
13. Table A5 - the totals are calculated incorrectly.
14. A change of the paper title is suggested. Considering all these comments, there are serious doubts that the model presented in this paper can reliably describe the maturity levels.
Overal, English is good. There are a few tipfelers in the text.
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted as “track changes”. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Comment 1:
The authors cite themselves with more than 20% of the total references; only Correia is cited with 9 out of 44 referenced articles. This raises the question of an unbiased perspective.
Answer 1:
Thank you for your attention. This work is continuous research from a PhD Thesis, thus the authors felt that was important to cite some of those references because they give also context on several aspects. Nevertheless, new references were added to the manuscript, and author references were reduced based on your feedback.
Comment 2:
The authors state in the abstract that they compare existing initiatives in the selected cities based on the number of projects funded and the socio-demographic variables of the cities. In reality, they use only one variable as a proxy for very complex sociodemographic phenomena, population density, and this should be stated accordingly.
Answer 2:
Thank you very much. Several amendments were performed not just to the abstract but also in the manuscript where it was clarified that cities’ population density was used as proxy of socio-demographic characteristics.
Comment 3:
There is no theoretical background on the smart city concept or smart city maturity models in the paper, and therefore the hypothesis is not clearly derived from the existing literature gap on smart city maturity models. In addition, the authors do not provide a definition or describe what the term "smart city" encompasses for the purposes of their paper.
Answer 3:
Thank you for your comment. Based on your comment, several paragraphs were added on the introduction section to describe the existing literature and understanding of the Smart City concept. Regarding the Maturity Model literature, after internal discussion and reflection, the authors shift from the Maturity Model understanding to an Assessment and Ranking only, as stated in the new title.
Comment 4:
The maturity models in the literature are far too complex to be narrowed down to only the number of projects funded at a given point in time; in fact, they do not account for funds invested in projects before or after the case studies in SCOPUS articles in the cities listed.
Answer 4:
We understand your comment. This work aimed to analyze all the papers present in the literature. They were not distinguished by the receiving or applied funds. By doing this, the goal was to have an overview of the existing projects. This was a first step that can be enhance in the future by considering other assumptions. The goal was to focus on accuracy instead of precision. Several limitations were added on the conclusion section.
Comment 5:
In addition, "maturity models' assume stages of development, e.g., capacity building, establishment of processes or structures, etc. The proposed smart city maturity model assumes that the acquisition of resources for health, education, or sports projects depends on previous investments in infrastructure and transportation (i.e., it assumes that the infrastructure and transportation in these cities are already in order and therefore projects in the sports sector represent a higher level of maturity for a city). The paper does not provide evidence in the literature to support these claims or assumptions, nor do the authors prove these assumptions in their research. Since the authors only analyze case studies from the SCOPUS database, it is not possible to draw conclusions about city development based on evolution over time, i.e., evolution should be indicative of a process; this also applies to the assumed relationship between smart city maturity and the number of projects in a given category (or causality). In summary, the Smart City Maturity Development Model should imply that development occurs in stages. Please provide theoretical and/or empirical evidence that could link the project category to the proposed stage of development.
Answer 5:
Thank you for your comment. Since it is explained in the introduction and also on the point 3.1.1. Smart City Categories, this research is an enhanced work of a previous scientific paper where was performed a content analysis to each paper’s title and abstract, and the categories were found after a Principal Component Analysis was performed and matched to the respective Smart City phase. Furthermore, based on your comment, several sources were added to the section 3.1.1. providing empirical evidence links the project category to the proposed Smart City development stage.
Comment 6:
Data question: data were taken from the database SCOPUS. The authors did not provide information on the data range of the articles analyzed. However, if we assume that the articles analyzed include all the articles listed in the database over the last 20 years and describe projects developed and implemented in these cities during the time span analyzed, it is not logical to rank these cities at a single point in time (e.g., in 2023). It would also be beneficial to indicate the years in which the projects were active. Can you indicate the duration, scope, and success of the projects? Can you provide more detail on why we should infer the "smartness" of a city based solely on the number of projects scattered over time? Can you provide a logical substitute for ranking these cities?
Answer 6:
We understand your comment. The papers considered are from 2010 onwards. This information was added to the manuscript. Some of the facts mentioned were considered by us as limitations and future work, namely the content analysis of each paper. Taking into consideration that we are matching the number of projects with the respective Smart City concept development phase, and that it has different weights, the authors think that the temporal aspect is already considered. In addition, the literature about the success of the projects, as well as standard KPIs that are used and contemplated that could facilitate the analysis. If we considered that, the results would be biased by the small amount of data available.
Comment 7:
The authors do not explicitly state that the selected cities were analyzed throughout their activity. Therefore, it is not clear whether the papers analyzed describe all projects that were active in a given city at a given date (or preferably up to that date). Rather, the SCOPUS articles analyzed could represent a single project of interest or a group/category of projects (e.g., rural and agricultural projects) in a particular area in which the authors of those articles were interested. Since SCOPUS is the source for the data and SCOPUS is a tool characteristic of an academic audience, the number of projects presented for a city could also depend on the presence of a college in a particular area, as well as the inclination of smart city researchers in a particular region. Presenting the overall activity of a city based on a case study article could lead to a wrong conclusion.
Answer 7:
Thank you for your feedback. The following paragraph was added on the conclusion section “The first limitation of this paper may be the fact that the only source considered was Scopus. However, since the goal was to study every city present in the literature, this recognized scientific database may the most suitable, given its reliability, coverage and integrity to use as a proxy. In the future, the study can be enhanced to include other sources. Furthermore, this study took the existing articles in the literature as a proxy for existing initiatives in each territory and considered only the analysis of titles and abstracts. In future work, the exhaustive analysis of each paper can be performed to understand the success of the case study and what impact it had on the city's strategy. Thus, additional data such as the project size, scope, funding, and people involved can be considered.”
Comment 8:
The number of projects can sometimes serve as a good proxy for the activity of a city or region; in the case of maturity models, it would be wiser to also use additional data (size of projects, project scope, financial data, project impact, number of people involved, etc.). The number of projects is likely to depend on the timing of the analysis and the available funding (project calls) in the (previous) period (e.g., open calls for EU funding or the economic situation in the city/region); in addition, it may depend on the specialization of the staff in the field (e.g., urban development), the research area of the researchers describing the projects (e.g., rural and agriculture), and societal trends (e.g., toward waste reduction in recent years). Instead of an analysis based on the number of projects, a content analysis should be conducted to assess the maturity of the cities.
Answer 8:
Thank you very much for your comment. Since we analyzed thousands of papers, we would also need other resources to be capable of having all that information, since as stated on out comment 6, it is not present in the literature. It is still very scarce about projects results. We had included the content analysis of the papers as future work, which the authors agree would bring more insightful information and accurate conclusion.
Comment 9:
In addition, the hypothesis tacitly states that more community projects mean higher smart city maturity level. Infrastructure and transportation projects tend to be much more complex and financially challenging than community projects. Community projects are easier to propose, acquire, and implement, and are usually additionally supported with external human capital (people from associations). Therefore, their bare numbers cannot easily be an indicator of a city's highest level of maturity. Please support any hypotheses with theoretical foundations or empirical evidence.
Answer 9:
Thank you for your comment. Our hypothesis considers and is supported by the evolution of the Smart City concept that was added to the introduction section. If community projects were easier, the overall number should be higher among cities, what it is not the case. That can be explained by several reasons, namely cities are still funding and focusing their projects on infrastructure.
Comment 10:
Please provide a theoretical explanation for why the most important urban development projects (infrastructure and transportation) are weighted with a disproportionately low index (2.85% for infrastructure/transportation versus 43.86% for community projects).
Answer 10:
Thank you for your comment. As stated in other comments, the weighs match the contribution of each factor that is connected to the Smart City concept development phase. A new paragraph was also added to the “Materials and Methods” section, to clarify the understanding and usage of the inverse values on this methodology.
Comment 11:
The cities listed 165th through 172nd are all ranked 74th based on the same score (points are earned with one project per city in the mobility and transportation category and the same population density ponder of 1). The authors ranked these 8 cities the same regardless of any other characteristics of these cities or dimensions and elements of the smart city concept. The question arises, what does this mean for the maturity level of the cities? Is it also the same? An the same as the maturity level of the 173rd city, which is also ranked 74th because it has a different population density (5 instead of 1), but 2 projects in the same mobility and transport category?
Answer 11:
Thank you for your attention. In those cases the development stage would be the same. That was exactly the intention. Although the city has 2 projects, since the population density class is 5, it ranks lower than the other ones that just have 1 project but their population density class is lower. The significancy is higher.
Comment 12:
What does "density = 1" mean in the case of Thessaloniki, Greece?
Answer 12:
Thank you. The density when is equal to 1 means that the population is equal to the area. In the case of Thessaloniki in Greece, the population is 325182 people, and the area is 315196 Km2, which gives a population density of 1.03 (when rounding to the units it is equal to 1).
Comment 13:
Table A5 - the totals are calculated incorrectly.
Answer 13:
Thank you for your attention. The totals were correctly, the error was on the transcription number of projects in each category, but it was solved.
Comment 14:
A change of the paper title is suggested. Considering all these comments, there are serious doubts that the model presented in this paper can reliably describe the maturity levels.
Answer 14:
We understand your comment. Based on your comments, we decided to remove the focus on the maturity model/level, and change the paper title “Assessing and ranking EU cities about their Smart City concept development phase”.
Reviewer 4 Report
The central part that refers to the proposal and the step-by-step explanation of how the results are obtained must be improved.
Firstly, you took your work as a reference and a brief but detailed explanation must be added. Also, how is obtained the analysis factor must be explained straightforwardly and concisely. In this way, the understanding of your work is self-contained in the document that is presented. Then, the working basis of the proposal can be clearly understood.
I suggest that mathematical relationships must be added in the form of equations. Also, in the results section, give an example taking into account the tables in the appendices.
It is necessary to explain Figures 1 and 4 in greater detail. These graphic elements make up the logic of the work. In the results section, an example should be given for a category of the procedure performed to obtain the factor analysis and the contribution that each factor has. Explain how these factors are recalculated and show it in a case as an example.
Justify why to use the inverse value instead of some other form of weighting. In terms of the final contribution of each factor, explain what it would imply towards the degree of maturity of a smart city.
As I commented, I think that sections 2 and 3 should be readjusted to provide a section with the proposed methodology and the results section providing an example of the procedure used referring to the tables in the annexes. They can show the procedure for obtaining the values of the city of Dublin, ranked first.
Add the units of population density, and area.
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted as “track changes”. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Comment 1:
The central part that refers to the proposal and the step-by-step explanation of how the results are obtained must be improved.
Answer 1:
Thank you very much for your feedback. The methodology section was improved based on your comment. Several paragraphs were added, including additional explanations and illustrative examples to improve clarity.
Comment 2:
Firstly, you took your work as a reference and a brief but detailed explanation must be added. Also, how is obtained the analysis factor must be explained straightforwardly and concisely. In this way, the understanding of your work is self-contained in the document that is presented. Then, the working basis of the proposal can be clearly understood.
Answer 2:
We truly understand your comment, and understand that although this is a following work, it should be self-explanatory. Therefore, we included new text to improve the description of the Principal Component Analysis, and to clearly explain how the Smart City categories were obtained.
Comment 3:
I suggest that mathematical relationships must be added in the form of equations. Also, in the results section, give an example taking into account the tables in the appendices.
Answer 3:
Thank you. Based on your comment, several mathematical relationships in the form of equations were added (please see sub-sections 3.1.2 and 3.2.1), and examples were included in the results and discussion sections.
Comment 4:
It is necessary to explain Figures 1 and 4 in greater detail. These graphic elements make up the logic of the work. In the results section, an example should be given for a category of the procedure performed to obtain the factor analysis and the contribution that each factor has. Explain how these factors are recalculated and show it in a case as an example.
Answer 4:
Thank you very much for your feedback. The methodology section was improved based on your comment.
Comment 5:
Justify why to use the inverse value instead of some other form of weighting. In terms of the final contribution of each factor, explain what it would imply towards the degree of maturity of a smart city.
Answer 5:
Thank you for your attention. The explanation of the usage of the inverse value was improved. Afterall, the reason why the authors considered the inverse values was because when matching with the Smart City phase, the Factor 1 corresponds to the lowest stage (Smart City 1.0) and the Factor 4corresponds to the highest stage (Smart City 3.0). Since the understanding is that cities are ranked higher on their Smart City concept development stage if they have more Smart City 3.0 areas’ projects, this should contribute more to their final ranking. It is normal, that initially Factor 1 has a greater contribution because, generally, cities have more projects on the first stage of the concept. Thus, considering the inverse values allows to give greater preponderance to the cities that have projects on the third stage of the concept.
Comment 6:
As I commented, I think that sections 2 and 3 should be readjusted to provide a section with the proposed methodology and the results section providing an example of the procedure used referring to the tables in the annexes. They can show the procedure for obtaining the values of the city of Dublin, ranked first.
Answer 6:
Thank you for your comment. As suggested, the authors added the sub-section 3.3 with the example of Dublin to explain all the results and sub-results obtained.
Comment 7:
Add the units of population density, and area.
Answer 7:
Thank you, the units were added.
Reviewer 5 Report
Definitions of smart city and maturity level need to be addressed in the review of literature review section.
Authors need to present the research purpose more clearly.
Which factor analysis and rotation were used? Also, is there no goodness of fit results? Authors need to present clearly in the method section.
Discussion is to shallow. What is the meaning of the raking and its criteria?
Authors also has very weak theoretical contribution of this research. Elaborate it.
Too many appendix, what are the main points? It needs to be addressed in the manuscript rather than appendix.
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted as “track changes”. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Comment 1:
Definitions of smart city and maturity level need to be addressed in the review of literature review section.
Answer 1:
Thank you for your comment. Based on your comment, several paragraphs were added on the introduction section to describe the existing literature and understanding of the Smart City concept. Regarding the Maturity Model literature, after internal discussion and reflection, the authors shift from the Maturity Model understanding to an Assessment and Ranking only, as stated in the new title.
Comment 2:
Authors need to present the research purpose more clearly.
Answer 2:
Thank you. We understand your comment. A dedicated paragraph was added to the introduction section to clarify the research question and purpose.
Comment 3:
Which factor analysis and rotation were used? Also, is there no goodness of fit results? Authors need to present clearly in the method section.
Answer 3:
We appreciate your feedback and the opportunity to address your concerns.
1) Regarding the type of rotation used, we considered a Kaiser-Varimax orthogonal rotation to achieve a more interpretable and meaningful factor structure. Varimax rotation typically leads to high factor loadings for a smaller number of variables and low factor loadings for the remaining variables.
2) Regarding the goodness of fit of the results, in principal component analysis (PCA), there isn't a specific test designed to assess the overall analysis's goodness of fit. However, several measures are commonly used to evaluate the quality of a PCA model and ensure its goodness of fit. These measures include:
i) Kaiser criterion and eigenvalues: It suggests retaining principal components with eigenvalues greater than 1. ii) Scree plot: It helps identify the number of principal components that significantly contribute to explaining the variance in the data. A steep drop-off in the plot suggests that the retained principal components capture most of the variability. iii) Proportion of variance explained: A value greater than 70% is generally considered acceptable.
In response to your suggestion, we have made the necessary revisions to clearly present the results regarding the goodness of fit.
Comment 4:
Discussion is to shallow. What is the meaning of the raking and its criteria?
Answer 4:
Thank you. The discussion section was extended and improved based on your comment.
Comment 5:
Authors also has very weak theoretical contribution of this research. Elaborate it.
Answer 5:
We appreciate your feedback. Several paragraphs were added to the introduction section to elaborate more the theoretical foundation and contribution of this research.
Comment 6:
Too many appendix, what are the main points? It needs to be addressed in the manuscript rather than appendix.
Answer 6:
Thank you for your comment. We decided to keep the Tables of Appendix A because in our understanding they show valuable data that helps to understand the premises and foundations of this study. The way it is structured and organized also allows future studies to use the data. Nevertheless, based on your comment we enhance several parts of the manuscript to highlight the information that is reflected on the appendixes.
Reviewer 6 Report
In this manuscript, the authors reported a study on Smart City Ranking, a composite indicator was developed for EU cities ranking. It is an interesting work. However, I do not think this paper meets the publication standard of the journal. The manuscript still contains many points and needs to be revised. Here below are some specific questions or suggestions:
1. The novelty and motivation of the work are not well justified. Also, the experimental study is shallow. How does your effort compare with the state-of-the-art?
2. In general, there is a lack of explanation of the methods used in the study. Try to set the problem discussed in this paper more clearly.
3. The experiment is the major problem in the paper. The description and the discussion are very rough. It is impossible to replicate the experiment and verify the claim of the author. Furthermore, almost no discussion of the experimental result is given. E.g. why the author would obtain this result? Which component is the most important? Any further improvement?
4. The English of your manuscript must be improved before resubmission. We strongly suggest that you obtain assistance from a colleague who is well-versed in English or whose native language is English.
5. Before submitting a revision be sure that your material is properly prepared and formatted.
Moderate editing of the English language is required.
Author Response
We are pleased to resubmit for examination the revised version of our Manuscript. We would like to thank the editor and the reviewers for the time spent reviewing the paper, their constructive feedback, encouragement, and suggestions for improving the paper. A major revision was applied to the structure and content of the paper. We have followed all suggestions closely and addressed these entirely in our revision - the changes in the manuscript are highlighted as “track changes”. Additional explanations for each of the comments are presented below. We hope this revision meets the expectations of the reviewers and the editor
Comment 1:
The novelty and motivation of the work are not well justified. Also, the experimental study is shallow. How does your effort compare with the state-of-the-art?
Answer 1:
Thank you for your careful review. Several paragraphs were added to the introduction section to elaborate more the theoretical foundation and contribution of this research. This way the novelty and motivation are clarified.
Comment 2:
In general, there is a lack of explanation of the methods used in the study. Try to set the problem discussed in this paper more clearly.
Answer 2:
Thank you for your attention. The explanation of the methods was improved. Furthermore, a dedicated paragraph was added to the introduction section to clarify the research question and the problem discussed.
Comment 3:
The experiment is the major problem in the paper. The description and the discussion are very rough. It is impossible to replicate the experiment and verify the claim of the author. Furthermore, almost no discussion of the experimental result is given. E.g. why the author would obtain this result? Which component is the most important? Any further improvement?
Answer 3:
Thank you. Although this study may present a methodology that can has some replicability issues, provides a fair ranking based on the necessary assumptions to improve the recognition of the work done by cities with less population and reduce biased results. The discussion section was improved to explain these facts. In terms of the components, their meaning and the respective usage of their inverse values is explained in section 2.
Comment 4:
The English of your manuscript must be improved before resubmission. We strongly suggest that you obtain assistance from a colleague who is well-versed in English or whose native language is English.
Answer 4:
We understand your comment. The document was revised by an English-native speaker professional. The changes were highlighted in the manuscript.
Comment 5:
Before submitting a revision be sure that your material is properly prepared and formatted.
Answer 5:
Thank you for your attention. We took your comment into consideration when revising and submitting.
Round 2
Reviewer 1 Report
The author has not changed his opinion as I gave it, in the last comment I have clearly pointed out the lack of appropriate references and in the second one I gave the solution they sat completely ignored, I strongly demand that the author must change his opinion as I gave it, this is the last chance, also other comments must be re-responded to!
Many grammatical errors need to be changed.
Reviewer 2 Report
I read the manuscript, I think the weighting process in each step according to Figure 3 is not properly described. This weighting process requires a matrix of pairwise comparisons by experts, which is not considered in this manuscript.
I consider the results invalid and my proposal is to reject the manuscript.
I read the manuscript, I think the weighting process in each step according to Figure 3 is not properly described. This weighting process requires a matrix of pairwise comparisons by experts, which is not considered in this manuscript.
I consider the results invalid and my proposal is to reject the manuscript.
Reviewer 4 Report
The work has improved significantly, however, there are still details to work on. In particular, statements are made about the procedure without being clear about how it is performed. An important aspect to correct is standardizing the terms used in the proposed methodology. Mainly consider the terms used in the method and procedure to make the calculations (shown in Figure 1 and Figure 3) and the way they are presented in the document.
1.- On line (210) it is mentioned that the 4 factors are recalculated (I understand that the result is presented in Table B1). However, it seems to me that you have to review the procedure in lines 213 to 223. Check variable X with index j (line 216 goes to 22 and line 222 goes to 4).
2.- in Line (211-213) mentions that in Table 1 there is a correlation of each category with each factor. Could you mention the procedure to obtain it? I assume the calculation involves the sum of all projects by category and by factor. Could you give more details?
Continuing with the step-by-step of Figure 3.
3.- First, multiply the number of projects by a factor. in line 215 it is mentioned as "Loadings" and in figure 3 as "weight". Again, I assume the results of the first step are in Table B1.
4.- The second step is to divide the factors by city according to the population density by class. As an example, for Dublin online (339) it is mentioned that it is 3.68 people /km^2 and in Table 3 it is 4756. How is this population density factor by class obtained? Is it, not the class number or the proportion (figure 4)?
5.-third step according to figure 3, involves multiplying the values obtained by "[Factor Variation Explanation]-1". What is that value? In line 343 it refers to the inverse value, but also to Figure 2 (values in percentages). 6.- Finally, a normalization is made from 0 to 1000. What rule or value is taken as a reference to normalize?
7.- Section 3.3 presents a summarized example. I think that some rows or a table could be added where the calculations that are made could be presented. In this way, explicitly express the calculations that derive from the results shown.
8.- Finally, clearly define whether a "composite indicator" or a ranking is obtained. Table 3 is presented as a ranking. column 1 indicates a position and in the last one a ranking. According to the procedure of figure 3, at the end, a "Smart City composite indicator (SCCI)" is obtained. Dublin ranks 1 with a SCCI of 907? Can you explain or correct the document to properly clarify this point?
