Satellite-Based Human Settlement Datasets Inadequately Detect Refugee Settlements: A Critical Assessment at Thirty Refugee Settlements in Uganda

Round 1

Reviewer 1 Report

This is a valuable article that outlines some of the relationships between population datasets and ground trotting.

 

I believe the paper will be strengthened with an improved definition of the term: Accuracy. While it is alluded to, I don’t see where accuracy or ‘sufficient’ accuracy is clearly stated. This definition should also be justified using academic literature - and if there isn’t a definition, perhaps the authors here can either 1. Propose one or 2. Outline why they believe this hasn’t been done yet.

 

 

— This is stated as the overall goal of the study:

“The goal of this study is to assess how well broad-scale satellite image-based human 86 settlement products capture individual refugee settlements.”

However, while the article includes content that speaks to that goal, there are various elements that could and should be refined to create greater clarity and allow for the reader to infer the key outputs of this work. This is elaborated in my comments.

 

I am struggling with how to react to the identification of 2016 as a critical year. Clearly, as stated by the authors, there are reasons to identify 2016 as a critical year due to the satellite imagery element. However, I believe the biases that may be introduced from the refugee camp characteristics element should be acknowledged. Perhaps considering questions such as, ‘to what extent were there or weren’t there changes in camp design, management, materials, etc in 2016 (or any year within the scope of the study) that may introduce bias in the study?

 

Further, how do you classify camps that are established before 2016, then become decommissioned, and then re established again after 2016. Are there any of these cases? Or even if a reestablishment does not occur after 2016, are there examples of established, decommission and re establishment? If yes, how might this impact your study? I feel like a discussion on this can complement the discussion, around lines 501-510.

 

 

More specific comments, broken down by section.

 

1. Introduction

- line 44 - what does ‘foundational awareness’ mean?

- line 46 - awkward sentence. It is important here to be clear about what measures of urbanization. It is also a long sentence. Perhaps make a new sentence that highlights the importance to focus on Africa, where this study takes place. What does this gap allow to occur? Or perpetuate? For example, without this type of analysis, small-scale settlements in Africa are more prone to x,y,z….

- Line 52 - the choice of using only UNHCR settlements is not justified. Just because they make up 1/3 of all settlements its not a reason to isolate them. What makes them more ideal than using the other 2/3? Why sacrifice a greater sample size? 

-line 57 - just for UNHCR settlements? Or all?

- line 70 - Can you expand on what is meant by ‘record’ in ‘record dwellings’? Given the intention of this article, it think it will strengthen the article to be more specific about what has been done in terms of recording, in order for the reader to see the gaps and position this paper within that context.

line 74 - should this read, ‘… with several characteristics…?’ I see this sentence awkward in its current state.

lines 75-85 - This is valuable information that I think if highlighted further would improve the structure of the article. Consider creating a new paragraph to highlight these 3 characteristics.

line 81- Please cite re ‘spectral separability’. Perhaps:

 Herold, M., Gardner, M.E. and Roberts, D.A., 2003. Spectral resolution requirements for mapping urban areas. IEEE Transactions on Geoscience and remote sensing, 41(9), pp.1907-1919.

line 99- This is a long sentence and the intention is unclear. Please rephrase. Consider ending the first sentence after ‘…settlement mapping’. I think the second part is valuable to clarify as it speaks to the ‘real world’ benefits.

 

2. Study Area

Figure 1 - please change the y-axis title to match the figure description. It should be, ‘Total Refugee Population in Uganda’. The current title can be misinterpreted as refugees that re Ugandan.

Figure 1 - Should the date of website access be included here? Or elsewhere?

Table 1 - What is the purpose of including ‘primary country of origin’? I don’t see how this is relevant for this study and it could be interpreting as a type of ‘othering’ if not relevant.

 

3. Materials and data

Figure 3 - It is good to see the background image cited as google, however given the scope of the article and journal, it would be beneficial to add additional information, such as sensor - especially as it seems there is a difference across the three. This comment should be consider for other figures that include the google maps background imagery.

lines 196-198 - Can this structure for disaggregation be cited? Or, if something the authors developed, please explain why.

Line 205-  what types of preliminary analyses? I feel that without further elaboration of what type, then the argument to not include them is weak.

Sub Section 3.3 - This section is well done. Clear and provides good context for the reader.

 

4. Methods

Line 253 - in reading the objectives, I think it would be useful to consider the initial statement of the article goal of: The goal of this study is to assess how well broad-scale satellite image-based human 86 settlement products capture individual refugee settlements. While I see how the 3 goals noted here align with that initially stated goal, I think the authors should consider how to position the sub goals within the overall goal. I believe doing so will strengthen the conclusions and allow for the reader to get a better sense of the process, motivation and impact of this work. Maybe figure 5 can be revised?

Line 315- This sentence has a few issues. First, I don’t think its best to begin a sentence with, ‘so’ especially as in this case it is the responsibility of the authors to justify why they believe this approach should be taken. It is clear (in lines 312-315) that an approach is needed to address value detection accuracy, however given the scope of the paper, lines 315 must be more clear, with justification.

Subsections 4.1 and 4.2 are well done. Very clear

 

5. Results

Figure 6. The reader may have issues interpreting the shading (such as the horizontal line segments) of the objects in the plot. 

Line 351 - very interesting. Why do the authors think this is the case?

Figure 9 - Powerful figure. Very interesting and surprising. I believe the readers will learn from this figure.

Figure 10 - Please improve the x axis title. I don’t think ‘#’ is acceptable here. Same with Table 4.

 

6. Discussion

Overall, this section is long and I found myself having to read it 3-4 times to capture the important pieces. I have included some points below, however I am not confidence I fully grasp the ‘story’ within the discussion. Maybe a first step would be to review and be more clear about how this content is related to the 3 (or 4?) goals/objectives of the paper.

 

Line 430 - Im not sure I agree with the perception of the settlements ‘evading’ detection. Maybe more accurately, ‘are not detected by…’

Line 523 - I think this is an important element to elaborate on. I fear that if left at a single sentence, readers may not see the importance in evaluating if and to what extent these methods could and/or should be applied elsewhere.

Lines 538-540 - It would be great to expand here… Yes researchers can do as noted, however what might this allow practitioners/policymakers to do? What can they improve in terms of standards?

Lines 587-599 - this seems redundant and should be considering for deletion.

 

7. Conclusion

After the long discussion, I feel like the conclusion does not add much. I feel like each sections 5,6,7 are making the same points, with 6 providing the most context. I suggest reducing the conclusion to 3-4 sentences in one paragraph, recapping the goals and objectives, and note next steps.

Author Response

Thanks much for your very helpful comments. We include our response in the attached Word doc.

Author Response File: Author Response.pdf

Reviewer 2 Report

Please see attachment.

Comments for author File: Comments.pdf

Author Response

Thanks much for your very helpful comments. We include our response in the attached Word doc.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Dear All,

I am glad to see that my review was useful and appreciated. I would like to extend a thank you to the authors who carefully responded to all suggestions and comments, as well as being professional and polite in their responses to the review.

While I still think that the content could be streamlined, especially regarding methods and discussion, I now believe the paper is ready and I suggest it is accepted.

Congratulations to the author for this work, as it will be useful to advance this important field.

Best regards,

 

Reviewer 2 Report

The edits made to the text have transformed the paper. Reframing the paper as 'detection/comparison' rather than 'validation/accuracy' completely changes the tone, and allows the paper to act as a valuable datasets guide and evidence-based commentary, which I believe was its original intention. The additional discussion of satellite imagery and of the settlement extents vs building footprints have been incorporated seamlessly. 

In light of the shift from validation to detection, I completely agree that additional comparisons using a newly derived buffered OSM-MS building footprints dataset (as suggested in my initial review) is not suitable. 

This is now a fantastic piece of work in both analysis and discussion. 

One very minor comment - the replacement of GRID3 with GRID3-SE seems to have been done from the methods section onwards. I suggest doing the same for the abstract and intro too.

This has been a really interesting and exciting paper to review!