Next Article in Journal
Predicting the Spatial Distribution and Severity of Soil Erosion in the Global Tropics using Satellite Remote Sensing
Previous Article in Journal
Extraction of Power Line Pylons and Wires Using Airborne LiDAR Data at Different Height Levels
Previous Article in Special Issue
Fusing High-Spatial-Resolution Remotely Sensed Imagery and OpenStreetMap Data for Land Cover Classification Over Urban Areas
Open AccessArticle
Peer-Review Record

Mapping Human Settlements with Higher Accuracy and Less Volunteer Efforts by Combining Crowdsourcing and Deep Learning

Remote Sens. 2019, 11(15), 1799; https://doi.org/10.3390/rs11151799
Reviewer 1: Anonymous
Reviewer 2: Anonymous
Remote Sens. 2019, 11(15), 1799; https://doi.org/10.3390/rs11151799
Received: 26 June 2019 / Revised: 22 July 2019 / Accepted: 26 July 2019 / Published: 31 July 2019
(This article belongs to the Special Issue Citizen Science and Earth Observation II)

Round 1

Reviewer 1 Report

The research aims to assess whether mapping human settlements can be improved (efficiency and accuracy) by combining crowdsourcing and deep learning.

The paper is well written. The hypotheses are clear and the methods used to validate them are well described. The research seems scientifically sound. The authors have made a comprehensive and critical review of the relevant literature. Results are well presented and critically examined. The figures (particularly 6 and 7) demonstrate at a glance the authors’ methods and claims about their results. The Discussion section proposes relevant links with other results found in the literature. The conclusion is comprehensive and pragmatic. Only a few references are missing in order to complete a very good paper.

Minor comments

Lines      Comment

3             Replace “e.g., …” by “as defined by the United Nations.”

37-38      Some references to suggest?

47           Replace “existing data sets” by “existing one.”

50           DeepVGI… A reference to suggest?

51           MapSwipe… A reference to suggest?

67-68      “Previous research shows … data set.” Which research? (a reference)

223-224  Regarding the resolution, it is not clear whether it is the resolution of the displayed image at zoom 18 or the actual resolution of the image obtained from the satellite.

421         Same comment.

458-459  Regarding the reduction of volunteer efforts. It is an important statement but I did not find in the text where it comes from. How in your case did you evaluate this figure?


Author Response

Dear reviewer,

thanks for your valuable feedback. We addressed you points as outlined below:


line 3, comment: Replace “e.g., …” by “as defined by the United Nations.”

replaced as suggested


lines 37-38, comments: Some references to suggest?

we added 3 references which show the potential of crowdsourcing to collect human settlements


line 47, comment: Replace “existing data sets” by “existing one.”

replaced as suggested


line 50, comment: DeepVGI… A reference to suggest?

added the following reference:

Chen, J., & Zipf, A. (2017). DeepVGI: Deep Learning with Volunteered Geographic Information. WWW ’17 Companion: Proceedings of the 26th International Conference Companion on World Wide Web, (1), 771–772. https://doi.org/10.1145/3041021.3054250


line 51, comment: MapSwipe… A reference to suggest?

added the following reference:

Herfort, B. (2017). Understanding MapSwipe: Analysing Data Quality of Crowdsourced Classifications on Human Settlements. Heidelberg University. https://doi.org/10.11588/heidok.00024257


lines 67-68, comment: “Previous research shows … data set.” Which research? (a reference)

added the following reference:

Klotz, M., Kemper, T., Geiß, C., Esch, T., & Taubenböck, H. (2016). How good is the map? A multi-scale cross-comparison framework for global settlement layers: Evidence from Central Europe. Remote Sensing of Environment, 178, 191–212. https://doi.org/10.1016/j.rse.2016.03.001

Esch, T., Heldens, W., Hirner, A., Keil, M., Marconcini, M., Roth, A., … Strano, E. (2017). Breaking new ground in mapping human settlements from space – The Global Urban Footprint. ISPRS Journal of Photogrammetry and Remote Sensing, 134, 30–42. https://doi.org/10.1016/j.isprsjprs.2017.10.012


lines 223-224, comment:  Regarding the resolution, it is not clear whether it is the resolution of the displayed image at zoom 18 or the actual resolution of the image obtained from the satellite.

adjusted the sentence to: "This corresponded to a spatial resolution of the displayed image of roughly 0.6 meters per pixel, as measured at the equator."


line 421, comment: Same comment.

added: For the regions analysed in this study, satellite imagery tiles with a higher image resolution at zoom level 19 were not available from Bing Maps. Nevertheless, new earth observation satellites such as WorldView3 would potentially provide sufficient imagery data for this zoom level


lines 458-459, comment: Regarding the reduction of volunteer efforts. It is an important statement but I did not find in the text where it comes from. How in your case did you evaluate this figure?

added a sentence in line 344 to state this more explicit:

For all study sites allocating 10% to 20% of the tiles to MapSwipe (raising the crowd proportion from 0.0 to around 0.1 - 0.2) resulted in an overall performance increase in respect to accuracy ACC and Matthew’s correlation coefficient MCC. Reducing the volunteer efforts to one fifths (labor reduction of 80 percentage points) resulted in a performance gain of 3-5 percentage points measured as MCC in all regions.

Reviewer 2 Report

This article presents a method for combining crowdsourcing and deep learning for mapping human settlements, which is of great interest to the community. The research design is sound and evaluations and discussions very comprehensive. I do not have comments for further improving the manuscript and thus suggest acceptance in its current form.

Author Response

Dear reviewer,

thanks for this very positive feedback.

Back to TopTop