Spatial Decision Support Systems with Automated Machine Learning: A Review
Round 1
Reviewer 1 Report
The paper examines a very interesting topic that bridges the literature gap between SDSS and AutoML. It is an extensive review that nicely summarizes a large number of works in the intersection of these two fields. The overview of relevant works is well-organized and presented through comprehensive tables and figures. Overall, The paper is well-written, well-structured and easy-to-follow.
Yet, there is plenty of room for improvements. More specifically, the authors should address the following points:
1) there is a missing reference (i.e., [?]) in lines 36, 54, 157, 166 and 435
2) in line 51, please provide a reference (url + publication) to Summon 2.0
3) in line 58, please provide a very brief description of the snowball search strategy
4) at the beginning of Section 3, please explain why the distinction between primary and supplementary publications is important
5) in line 85, please explain how the number of citations per article was measured (e.g., did you use Google Scholar?). Please also specify when this information was gathered.
6) in Figures 4 and 5, please replace the DOI in the vertical axis with the number of every publication in the list of references
7) in line 147, "started to increased" should be "started to increase"
8) please describe Figures 6 and 7 in more detail
9) please add references to Tables 1 and 2
10) in line 183, please explain that the topics covered in Table 1 are examined below in more detail
11) in line 199, "help" should be "helps to"
12) in line 479, "useability" should be "usability"
13) in line 498, "select" should be "selected"
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Overall
1. The paper presents a comprehensive systematic review of recent literature about spatial decision support systems and Automated ML. The results are very interesting and topical, however, there are some concerns that need to be addressed before publication, as explained in the more detailed comments that follow.
2. The paper reviews ALL spatial decision support systems, in all domains and application fields. While this results in a comprehensive review, I feel some of the value gets lost, because the requirements, implementation and user adoption for Automated ML for environmental impact SDSS are very different to those in, e.g., public health, - the users are different, the data is different and the questions to be answered by the SDSS are also different. I suggest that this approach be acknowledged and justified in the introduction and conclusion. There could also be future work, e.g., a more focused review and/or a comparison between different fields.
Abstract
3. “This paper argues that integrating spatial decision support systems with automated machine learning can not only encourage user adoption…”. This should probably be the other way around, i.e., as expressed in the question in the second paragraph of the introduction: “This paper argues that integrating automated machine learning into spatial decision support systems can not only encourage user adoption…”
Introduction
4. According to the first paragraph, user adoption is a challenge. This makes one think that the review will provide insight into user adoption. However, the three research questions in the subsequent paragraph are not clearly linked to user adoption.
5. There is a question mark in quite a few citations, e.g., Page 1, line 36: [4,12?], Page 2, line 54: [12? - 15], etc.
6. I suggest the introduction be concluded with an overview of the paper, so that the reader knows what to expect.
Methods
7. What is meant by “literature review related articles only”? Most articles have a literature review section. Would they be included? Or only articles that are literature reviews only? Please clarify.
8. Based on 2.1, the title and abstract of 382 articles from 2019-2022 were manually filtered, however, later it is stated that only “articles within two years were used”. Does this imply that the articles of 2019 and 2020 were not manually filtered after all?
9. When using the snowball search strategy, were only articles from the last two years added as supplementary references? Please clarify.
10. Please clarify what is meant by “Similar problems solvable by AutoML and SDSS”. Based on which criteria was similarity determined, e.g., based on application field, based on data type, based on spatial question (such as adjacency or cluster or temporal trend)?
Results
11. What is meant by “earlier than 2022”? Only articles “within two years” (as stated in 2.1) or also earlier?
12. Figure 1 provides a nice overview of the search process, however, 312 (AutoML) + 115 (SDSS) does not add up to 382 (database result)?
13. Figure 2: the symbolization for AutoML and SDSS is very similar – from the legend it is not possible to see which one is represented by a dotted or solid line.
14. Why are the number of citations for articles presented in the bulleted lists on pages 2 to 4? Please clarify / justify.
Literature review
15. The heading of this section is odd – ‘Literature review’ is typically used to present related work, however, the content of this section is part of the results for the research presented in this paper.
16. The subsections commence without an explanation of how they fit into the paper / research. Either here, or at the end of the introduction (see comment above), the paper should explain what is presented in this section and why.
17. At the end of the first paragraph, why is it “knowledge” and not “spatial knowledge”?
18. Similar to the comment above, in section 4.3, the paper should explain how the spatial problems were identified and grouped.
Discussion
19. Similar to the comment above, subsections commence without an explanation of how they fit into the paper / research. Either here, or at the end of the introduction (see comment above), the paper should explain what is presented in this section and why.
20. Figure 8 needs more explanation. There are several questions that arise:
- Where is the knowledge from Figure 6?
- Where is the feature optimization and model optimization from Figure 7?
- Where are the inputs, algorithms and outputs from Figure 7?
21. How does this framework based on three considerations (spatial problem, metric and potential solution) relate to (literature about) geographic problem solving generally? The framework seems to describe a process and considerations that one has to follow whenever one wants to solve a geographic problem? Not only in SDSS. I suggest that this link to geographic / spatial problem solving generally be made explicit in the paper, as it widens the applicability and usefulness of the paper.
Conclusion
22. The first sentence can be challenged – the paper answers the three research questions, but these are not related to accessibility and user adoption. See also comment above. The contribution of this paper lies in the framework and the description of current implementation challenges and research opportunities.
23. The conclusion should summarize how the three research questions were answered and what ‘the answers’ were.
Language
24. Page 2, line 79: The number of articles per year are steady ïƒ The number of articles per year is steady
25. Page 3, line 106: topics had much less citations ïƒ topics had far fewer citations
26. The word ‘notable’ is used frequently in the paper, but it is not always clear why something is notable, e.g., notable because of high frequency or notable because it is a new / different? This should be clarified.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments for author File: Comments.pdf
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The authors have significantly improved their work and have successfully addressed my initial comments.
Reviewer 3 Report
The authors have made the changes according to my suggestions. The review article can now be published in IJGI
