StreetScouting: A Deep Learning Platform for Automatic Detection and Geotagging of Urban Features from Street-Level Images

Round 1

Reviewer 1 Report

This paper proposes StreetScouting as a platform to automatically extract the features such as tree, trash bin, street lamp, retail shop, and others in urban regions.

• Scope of the proposed platform is not focused and thus needs to be clearly specified.
• Is the proposed platform running locally on the client device or all the data stored into a server? What if some identical objects are detected more than once in the same or different devices? Please, explain how the platform deals with that.
• The features or object classes (mentioned in section 1 paragraph 1) should be determined and provide justification why these classes are chosen.
• Is the proposed platform working online and always updating detected objects including their locations in real-time? I think, detecting moving objects is not essential in describing features of particular locations since they will change every time. Please, explain how the platform deals with that.
• In addition to accuracy-based metric, the StreetScouting's performance should also measured and analyzed in terms of inference/execution time or latency.
• Regarding position estimation task, why don't you just use the information from geolocation or GPS? The purpose of performing depth estimation task is vague.
• There is no section for conclusion.
• Figure 5 needs to be enhanced to improve its readability.
• Why did you put "urban regions" in the title? Is the urban region becoming just the case or the limitation in this study? Is the proposed platform able to be extended to rural or other kinds of regions?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

For the purpose of optimizing the efficiency of urban street visual feature extraction, this paper integrates multiple computer visual analyzing methods to create StreetScouting, a platform that can automatically extract geotagged features of urban regions, and demonstrates the feasibility and potential value of this platform in commercial and public urban space design. The research topic of this paper has strong innovation and academic significance, the research method is scientific and effective, and the research findings have promising practical value. However, there are also some problems in the manuscript.

1. The review of related works is not extensive enough, the illustration of literature lacks depth, and the correlation between these works and the theme of this study can be further improved. The development of StreetScouting platform has been inspired and helped by many related algorithms and academic results in the automatic recognition of urban planning and design by computer vision. The significant findings of these studies and the corresponding academic value need to be reviewed and discussed. Currently, the review section just mainly evaluates the problems existing in those literature, but how these existing problems related to the invention of StreetScouting is not well explained. In addition, the textual description of the review needs to be revised and strengthened. Most of the research work reviews are too brief to know the actual content of them, and some of the descriptions are not complete (such as reference [15]). The authors are suggested to solve the above problems.

2. The advantages of the data collecting method on this platform are not obvious compared with the existing research works. When introducing the traditional computer aided visual detection algorithm based on Google Maps, the authors believe that the flaw of this kind of data collecting method is that the update is not timely enough, which will affect the effectiveness and credibility of the platform. Although StreetScouting uses the cameras on the vehicle to capture visual objects to obtain an urban image data set, but there are still problems that need to be updated, especially for users who actually use this platform to collect data in the future, if they need to maintain the timeliness of urban street data, they must constantly repeat the data sampling link, and the time and economic cost caused by this should not be underestimated. Therefore, the way of data collecting on StreetScouting may not have absolute advantages over traditional methods.

3. The object detection algorithms involved in the data analysis work of StreetScouting are various and have good problem-solving ability. However, when describing the features of these deep learning and machine learning algorithms, the authors did not objectively compare the advantages of these features over other DL and ML algorithms. It seems that the object recognition algorithm on StreetScouting has a natural superiority. And there is not much introduction to how to integrate the indicators' values calculated by different algorithms, such as whether street width and sidewalk have an impact on the accuracy of urban landscape spatial location? How is the spatial landscape data collected by the cameras integrated with the two-dimensional map data? The authors can make additional explanations in this section.

4. Illustrations of the research findings need to be more objective. It is too arbitrary and absolute for the authors to consider that there is no need to evaluate models’ performance further. The verification of the computational performance of algorithmic models should be based on quantitative computational comparisons. In addition, the authors claim that the street scene data will be analyzed incorrectly due to the excessive speed of the vehicle during the collection process, they need to give a scheme for automatic correction of error data for such problems.

5. The depth of discussion of this manuscript needs to be strengthened. Currently, the contents of the discussion section are still limited to the summary and review of StreetScouting platform functions. There is no mention of practical questions, such as who can and how to apply StreetScouting in the future? What size of city or urban region is more efficient to use this platform? How much is the actual economic and administrative cost estimation of this platform operation? How can this platform be improved in the future considering there are still minor problems? The practical guiding significance of this manuscript can be further improved through authors’ revisions on the discussion section.

6. The writing of this paper still needs to be optimized. Some paragraphs and sentences are too scattered and short, which makes the readers unable to understand the author’s intention and meaning well.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The research content is interesting. The following aspects need to be further improved.

1) In L8, the author said they utilized many state-of-the-art computer vision approaches. Would you please list the concrete appoaches in the abstract or in the conclusion part?

2) For the key words, web application only appeared once. There are no introduction in the paper.

3) Have the authors designed new models for object dectection, object tracking, road width estimation or sidewalk segmentation, or just employed the existing models? No network structures are not listed.

4) Would you please list your key contributions? For example, manually annotate particular objects, as listed in Table 1.

5) The results need to be analysized clearly.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

This paper introduces StreetScouting as a platform that aims to automate the process of detecting urban features of a particular region using various state-of-the-art deep learning models.

This paper has been improved since the previous submission. The authors respond the reviewer's notes very well in the answer letter. Things that were ambiguous are clearly explained in the current version.

Nevertheless, few things can still be improved:

1. I do not see much improvement on the Figure 5; Texts on this image are still blurry when zoomed in. It needs higher resolution.
2. Figure 4 needs more discussion. The formulae converting the distance from pixels to meters could be written based on the regression model shown in Figure 4.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

In the result part , the authors listed the experimental results of their own method. Please compare with the state-of-arts which had been referenced in the "related works" part.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf