Using Machine Learning Tools to Classify Sustainability Levels in the Development of Urban Ecosystems

Round 1

Reviewer 1 Report

There are a few things that could be improved. In line 80, it states that natural ecosystems are needed to ensure the quality of life. I would change that to state "human survival." Natural ecosystems are basic to human life.

Also, I liked the use of  machine learning tools and the concept of a machine learning model. But I notice that the results are similiar to traditional statistical analysis. So, we can say that the machine learning model reinforces results from traditional methods.

The paper was well-written. It was easy to follow and definitions were given so that the reader could understand the argument of the researchers along the way. But, I wanted to find out more about the results from the workshops. How many people attended; how were they selected; and how was this information included in the machine learning model. We are taking the authors word that their data backs up ther results.

My biggest concern with this paper is that it only studies one micro area in Bogota, - Kennedy. I would have liked to see a comparative analysis with another micro area in Bogota, perhaps one which was a high income mciro area. In this way we could assess whether lower income areas are proceeding better with the SDG's than higher income areas. 

Also, the researchers used a variation of the Dephi method which they did not source.

And finally, the outcome of the research showed that the micro area moved from low to medium sustainability, the same as Bogota. Again, I would have liked the comparison between micro areas in Bogota to indicate how different socio-economic groups are moving toward sutainability. Hence, the results of this paper are not useful for policy makers in my opinion. Also, I have done work utilizing social media for SDG acceptance. I believe this approach revealed more usable information for policy makers than the approach taken in the paper. 

Author Response

Dear reviewer of Sustainability Journal.

Thank you for your kind and quick response about the review procedure of our sent document "Using machine learning tools to classify sustainability levels in the development of urban ecosystems (Manuscript ID: sustainability-775446)".

You can find the new version of the attached document, considering all the reviewers' comments, point by point. The adjustments are highlighted on the document in red color. Additionally, the final version of the document includes all the corrections.

A statement, explaining you the response to the specific suggestions and comments to facilitate the re-review of the manuscript, is also provided. Each comment and its corresponding response are presented below:

Point 1:  There are a few things that could be improved. In line 80, it states that natural ecosystems are needed to ensure the quality of life. I would change that to state "human survival." Natural ecosystems are basic to human life.

Response: Thanks to the reviewer’s suggestion, the text was modified and is included in the final version of the paper on line 73-74: “Natural ecosystems are basic to human life”

Point 2:  Also, I liked the use of machine learning tools and the concept of a machine learning model. But I notice that the results are similar to traditional statistical analysis. So, we can say that the machine learning model reinforces results from traditional methods.

Response: Thanks to the reviewer’s suggestion, it was included a paragraph whit this information in the conclusion section of the manuscript:

“Concerning the results of the statistical analysis and the important variables through the Gini index in machine learning models, it is important to note that the later reinforces results from traditional methods”.

The adjustments are included in the final version of the paper on lines 654-656.

Point 3:  The paper was well-written. It was easy to follow and definitions were given so that the reader could understand the argument of the researchers along the way. But, I wanted to find out more about the results from the workshops. How many people attended; how were they selected; and how was this information included in the machine learning model. We are taking the authors word that their data backs up their results.

Response: Thanks to the reviewer’s comments, it was complemented the description about the workshops:

“It is important to note that considering the current participation spaces promoted by the local administration (i.e., local environment commission and economic observatory) 32 representatives from district entities, community leaders, and delegates from universities within the territory attended the workshops. This structured work developed so that different participants, with knowledgeable of the territory's priorities, could establish the importance of the indicators in the study area, making it possible to determine the set of indicators to evaluate the sustainability level of the urban zone”.

The adjustments are included in the final version of the paper on lines 186-192.

Furthermore, regarding the question “how was this information included in the machine learning model. We are taking the authors word that their data backs up their results” it was included a paragraph pointing out the function of the importance of the indicators gave by people attended the workshops:

“It is noteworthy that the indicators’ level of importance, which was stated by the people with extensive knowledge of the territory, was one of the characteristics included in the AHP assessment.”

This information enabled, as part of the SDI calculation, to establish the labels for the supervised classification with machine learning tools as is described on the lines 250-257 of the final version of the paper.

The adjustments are included in the final version of the paper on lines 242-244.

Point 4: My biggest concern with this paper is that it only studies one micro area in Bogota, - Kennedy. I would have liked to see a comparative analysis with another micro area in Bogota, perhaps one which was a high income micro area. In this way we could assess whether lower income areas are proceeding better with the SDG's than higher income areas. 

Response: This is a very important point. However, regarding the limitations of the research related to the availability of the information for the evaluation of the micro-territory sustainability, it was included as an important aspect for further research. These aspects were included in the document on lines 657 to 669:

“This study found limitations on information availability for indicators that describe the behavior of sustainability dimensions in the micro territory. It is necessary to have a significant amount of information either for an appropriate characterization of each sustainability dimension, or to feed the machine learning models. Therefore, the information gathering phase required the most time and resources of this study.

Further research studies will be able to apply the methodology developed herein, in conjunction with machine learning models for each micro-territory in Bogota. The studies contemplate an analysis of micro-territories and how sustainable dimensions and their interactions are influenced by socio-economic aspects. This will enable a comparative analysis of the behavior of micro-territories, taking into account indicators on the environmental, social, and economic dimensions, as useful tools for decision-making related to resource prioritization and allocation. Additionally, conducting research that considers spatialized information will identify the behavior of habitability interactions and the viability of sustainable development in different territories”.

Additionally, considering the reviewer's suggestions, and taking in account results of prior studies, a comparison with a higher income micro-territory was included:

“Furthermore, a comparison of the influence of a micro-territory with better socio-economic behavior than Kennedy found that the results obtained through the SDI evaluation for Kennedy in this study are consistent with results from prior studies [8]. Teusaquillo is another micro-territory in Bogota, which unlike Kennedy is characterized by having greater purchasing power, more employed people, as well as having better educational, financial, cultural, and recreational services. In this vein, according to Carrillo & Toca (2013) [8], Teusaquillo achieved a high sustainable level in the evaluation. These are aspects which, despite the difference in methodologies, influence territories’ progress towards sustainability.”

The adjustments are included in the final version of the paper on lines 557-564.

Point 5:  Also, the researchers used a variation of the Dephi method which they did not source.

Response: Thanks to the reviewer’s comments, the text in the document was complement and it is included in the final version of the paper on lines 172-174:

“In addition to analyzing the characteristics, a variation of the Delphi method was conducted [47], in which technical experts and people with an extensive knowledge of the territory evaluated the established importance of the indicators”

Point 6: And finally, the outcome of the research showed that the micro area moved from low to medium sustainability, the same as Bogota. Again, I would have liked the comparison between micro areas in Bogota to indicate how different socio-economic groups are moving toward sustainability. Hence, the results of this paper are not useful for policy makers in my opinion. Also, I have done work utilizing social media for SDG acceptance. I believe this approach revealed more usable information for policy makers than the approach taken in the paper. 

Response: A comparison between different territories is desirable, considering not only socio-economic but also environmental conditions. Nevertheless, there are limitations regarding availability information for micro-territories in amount and quality. Thanks to the reviewer’s comments, we applied the following adjustments:

1. Taking into account results from prior studies, a comparison with a higher income micro-territory was included:

“Furthermore, a comparison of the influence of a micro-territory with better socio-economic behavior than Kennedy found that the results obtained through the SDI evaluation for Kennedy in this study are consistent with results from prior studies [8]. Teusaquillo is another micro-territory in Bogota, which unlike Kennedy is characterized by having greater purchasing power, more employed people, as well as having better educational, financial, cultural, and recreational services. In this vein, according to Carrillo & Toca (2013) [8], Teusaquillo achieved a high sustainable level in the evaluation. These are aspects which, despite the difference in methodologies, influence territories’ progress towards sustainability”.

The adjustments are included in the final version of the paper on lines 557-564.

2. Thanks to the limitations regarding the availability of the information, comparison between micro territories in order to identify how it moves toward sustainability was included as an important aspect for further research:

“This study found limitations on information availability for indicators that describe the behavior of sustainability dimensions in the micro territory. It is necessary to have a significant amount of information either for an appropriate characterization of each sustainability dimension, or to feed the machine learning models. Therefore, the information gathering phase required the most time and resources of this study.

Further research studies will be able to apply the methodology developed herein, in conjunction with machine learning models for each micro-territory in Bogota. The studies contemplate an analysis of micro-territories and how sustainable dimensions and their interactions are influenced by socio-economic aspects. This will enable a comparative analysis of the behavior of micro-territories, taking into account indicators on the environmental, social, and economic dimensions, as useful tools for decision-making related to resource prioritization and allocation. Additionally, conducting research that considers spatialized information will identify the behavior of habitability interactions and the viability of sustainable development in different territories”.

The adjustments are included in the final version of the paper on lines 657-669.

3. One of the outcomes of this research corresponds to the applied methodology. A methodological scheme was included, which summarized the principal steps for future research lines regarding the application for other micro-territories. The scheme is included in the final version of the paper in Figure 1:

"Please see the attachment."

Figure 1 Methodological framework for the sustainability levels classified trough machine learning tools

There is a lack of machine learning models to forecast the sustainability behavior of urban territories, starting at the micro territorial level. Additionally, it is important to note that countries of Latin America and the Caribbean have deficiencies in the standardized collecting of information. In this vein, conducted a research study, considering the limitations of information of a micro territory like Kennedy, is an important contribution for policymakers. Nations have established guidelines to achieve the SDGs. However, the reality of micro-territories like Kennedy is not considered enough by those guidelines. Latin American and Caribbean countries have different micro territories with differences among others, their population, culture, resources, and income. That is why a research study that defines different phases to identify the most important challenges either by the application of statistical tools or through the application of machine learning tools is for the interest of policymakers.

Additional results of this study are the level of sustainability for the study area, the characterization of the micro-territory regarding sustainability indicators, and a forecast model based on machine learning tools. The later enable forecasting sustainability levels and the identification of important variables to be considered for policy-makers. This study is innovative in the sense that it recognizes the importance of the synergy of micro-territories to contribute from the bottom up for SDGs achievement. Moreover, the inclusion of community needs through the analyses of community complaints filed with the public sector, and the identification of relevant indicators included in the SDI evaluation are important contributions of the community concerning their specific needs.

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear all,

the manuscript is very interesting and, in my opinion, shows relevant tools that should be more often consider to assess territorial sustainability.

Nevertheless, some issues should be solved in the paper before the acceptance:

• the manuscript does not follow MDPI template - i.e. the reference style in the line 176; or the spacing between paragraphs (revise these issues in all document);
  - the use of a methodological scheme should be considered - to help the reader to easily understand and follow the used steps;
• consider the complexity of the methods used, sub-sections should be considered in the methodological part ;
• there are no study limitations and further research lines - this section should be added
  
  

Author Response

Dear reviewer of Sustainability Journal.

Thank you for your kind and quick response about the review procedure of our sent document "Using machine learning tools to classify sustainability levels in the development of urban ecosystems (Manuscript ID: sustainability-775446)".

You can find the new version of the attached document, considering all the reviewers' comments, point by point. The adjustments are highlighted on the document in red color. Additionally, the final version of the document includes all the corrections.

A statement, explaining the response to the specific suggestions and comments to facilitate the re-review of the manuscript, is also provided. Each comment and its corresponding response are presented below:

Point 1: The manuscript does not follow MDPI template - i.e. the reference style in the line 176; or the spacing between paragraphs (revise these issues in all document);

Response: Thank you for the remarks; the entire document was reviewed, it was made the adjustments of reference style and spacing between paragraphs.

Point 2: The use of a methodological scheme should be considered - to help the reader to easily understand and follow the used steps;

Response: Thanks to the reviewer’s suggestion, it was included a methodological scheme:

"Please see the attachment."

Figure 1 Methodological framework for the sustainability levels classified trough machine learning tools

The adjustments are included in the final version of the paper on line 137 and Figure 1. Furthermore, considering the inclusion of the methodological scheme on the paper, it was updated the number of the subsequent Figures.

Point 3: Consider the complexity of the methods used, sub-sections should be considered in the methodological part;

Response: Thanks to the reviewer’s comments. In order to enable a better understanding of the methodology it was included the following sub-sections’ headings in the Materials and Methods’ section:

 “Defining the set of sustainability indicators”

“Collecting data on each indicator and information analysis”

“Information required to feed the models”

“Performance evaluation of the machine learning models”

The adjustments are included in the final version of the paper on lines 150, 194, 289, and 321, accordingly.

Point 4: There are no study limitations and further research lines - this section should be added

Response: Thanks to the reviewer’s comments, it was included the study limitations and further research lines in the conclusions section:

“This study found limitations on information availability for indicators that describe the behavior of sustainability dimensions in the micro territory. It is necessary to have a significant amount of information either for an appropriate characterization of each sustainability dimension, or to feed the machine learning models. Therefore, the information gathering phase required the most time and resources of this study.

Further research studies will be able to apply the methodology developed herein, in conjunction with machine learning models for each micro-territory in Bogota. The studies contemplate an analysis of micro-territories and how sustainable dimensions and their interactions are influenced by socio-economic aspects. This will enable a comparative analysis of the behavior of micro-territories, taking into account indicators on the environmental, social, and economic dimensions, as useful tools for decision-making related to resource prioritization and allocation. Additionally, conducting research that considers spatialized information will identify the behavior of habitability interactions and the viability of sustainable development in different territories”.

The adjustments are included in the final version of the paper on lines 657-669.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report