Validity and Reliability of the Facility List Coder, a New Tool to Evaluate Community Food Environments
Round 1
Reviewer 1 Report
I enjoyed reading this paper and found the work to be of important value. Food environments have been identified as critical factors influencing public health. Research has reported that food deserts contribute to poor dietary quality and health outcomes such as obesity. Often occurring in areas of poor socio-economic standing. A tool such as presented in this study that could be adapted to multiple communities would be of great value to the research community as well as governmental programs looking to support their constituents.
The paper describes a strong methodological assessment of the facility list coder (FLC) to evaluate food environments using GIS technology with big data available for analysis accessible to many people. The approach of the study and the information provided indicate that the FLC is a considerable tool for use.
Some elements need attention before publication. In general, the introduction would benefit from revision for clarity and reading. The methods section is well organized but would benefit from headings to delineate what work has been completed for reliability vs validity. This comment also follows for the results section. The recommended headings would make it easier for the reader to consider how the results provide evidence of the reliability and validity of the FLC.
Major comments:
The concepts of validity and reliability are interconnected but are not the same. The purpose of the study states that ‘introduction and validity assessment of the FLC’ are under investigation. However, throughout the manuscript concepts of reliability are inserted and in the primary conclusion of the study, reliability is claimed. The concepts of reliability and validity in this study should be organized more clearly and included in the purpose.
Minor comments:
Line 2-3: Consider adding the term validity and reliability to the title to provide more information for potential readers to understand the paper content
Line 41 – 42: sentence unclear and verb tense mismatched with the following sentence
Line 49-50: sentence unclear and verb tense mismatched with previous
Line 50: should “They find” be “They identified”
Line 69: consider removing the word ‘nonetheless’
Line 77: consider removing the word ‘thus’
Line 82: consider describing who the ‘scholar population’ is in Mataró or just using the name of the city. Could you mean the ‘student population’?
Line 86-87: The purpose stated on line 86-87 [purpose to introduce and validate a new….] does not match the conclusion statement on Line 341 “To conclude, the FLC is a valid and reliable tool for evaluating community food environments” Why is the term reliability not part of the purpose?
Line 91-171: consider subheadings of reliability and validity
Line 94: (Spain) after Mataró not necessary as the remainder of the sentence explains that it is in Spain and approx. where
Line 95: consider removing the word ‘important’
Line 99-100: How have the mixture of population and culture increased the risk of health-related problems such as child overweight and obesity? Or do you mean are associated with?
Line 118: consider removing the word ‘thereby’
Line 132: consider removing the word ‘furthermore’
Line 193 – 242: Explain to the reader specifically what data supports (i) the reliability of the FLC (ii) the validity of the FLC
Line 201-208 & Table 4: Please provide an explanation of the ‘color’ legend or numerical legend. As the number increases to 10 (dark purple) what does this express? Are false positive purple (0-10) and false negatives pink (?-0)
Line 204: ‘tolerance rate of just one facility’ – can you explain this category for the reader?
Line 231: seems to be missing something at the end of the line …significa
Line 301: The sentence begins with ‘First,’ which leads the reader to anticipate second, third, or a sequence of points. The following paragraph begins with ‘another concern…’ which could be the continuation of this concept. It is not clear or connected easily. Consider editing to create a coherent context of intended messages.
Line 341: Validity and reliability is iterative and culturally specific. A case study of Mataró, Spain was used for this analysis. Would the indices of validity be the same in a medium-sized city of the United States or Lebanon? I would recommend using the terminology “building evidence of validity”.
Author Response
We would like to thank the reviewer for their detailed and thoughtful comments and suggestions for our manuscript. We are convinced that our manuscript is now a more consistent and clearer piece of research thanks to their feedback. Below, we provide a point by point description of how each those comments were addressed in our manuscript. Original reviewer comments in boldface, responses in regular typeface.
|
Figure 1: I assume the second symbol that appears is Open street Map...I do not think that it is clear to everyone. Consider using legends. |
We added the labels in the graph |
|
Figures 2, 3: It can be smaller in the text.
|
We reduced their size. |
|
Lines 165-171. This classification could be introduced before when they are also mentioned.
|
We have included a clear introduction of our classification method in the section where we introduced the FLC (ie. Secondary data: Introducing the Facility List Coder ) |
|
Lines 198-199: The meaning of "only 571 grids had at least one facility" is not clear as a result of the study.
|
It was indeed a typo. We have deleted to avoid miss-understanding. |
|
Lines 204-205: Likewise, it is not clear: tolerance rate of just one facility
|
We have introduced a comment to clarify this point. |
|
Line 207: Why to present the value of the t-test?. The mean difference value between the pairs and the p-value would be more meaningful together.
|
We agreed on this. However, it is standard to start by this test although it is not completely informative in this context. We provided a further statistical evidence on this comparation. |
|
Line 221-222: please describe these sentences better. It is not totally clear what is meant.
|
We corrected this sentence, now it says: “we found that these differences were mainly due to how local food markets were counted: whereas the field work treated market as a single facility, the FLC coded all the facilities that were located within the markets individually.” |
|
Please clarify how both searches were combined. What was the coverage based on Google Maps and on Open Street Maps. Does it mean that when using the tool, the search needs to be made in both search-engines? Please clarify it.
|
We include as a supplementary resource a public repository with all the codes from the FLC (https://github.com/jcmunozmora/facilitylistcoder.git). In this repository, we include all the technicalities (e.g. packages) and explanation for the implementation. Moreover, we are preparing a more detailed manuscript where we present in detail the type of data that is obtained using the FLC. |
|
Tables 1 and 2 - I find unnecessary to present F values, df1, df2 I think that the reproducibility of the tool to other regions in the globe cannot be warranted without further investigation and this should be noted.
|
Although we agree about the lack of external reliability of our findings, we still need to give strong evidence on the internal reliability. In this sense, we think that keeping the F values are very informative. |
|
When the conclusion says that the method is reliable, you should also restrict to the population/region that was tested. |
We agreed on this comment, now we say that: “To conclude, the FLC is a valid and reliable tool for evaluating community food environments in our case of study. This result is building evidence of validity of using GIS-based solution as FLC to evaluate food environment” |
|
There is some repetitive use of the words "in order to". They are not always needed. |
We have reviewed the manuscript changing “in order to” for “to” in most cases. |
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Overall, I very much like this paper. The methodology is very sound, and will be of interest to many. The scholarship is good and the writing and construction of the paper is generally sound.
However, some revisions would much improve it:
I think the many readers would like more detailed information the process by which Google Maps and Open Street map obtain their data. And, more on FLC and how it "abstracts" this information. There are other researchers and practitioners who would like to try this approach, but there is insufficient information here for them. Terminology: I think defining some terminology here and there would be helpful. For example, "buffer zone" is referred to later in the paper as synonymous with your sampling grids (if I understand correctly). Although much of the terminology (meta-data) will be familiar to GIS savy readers, for others the meaning will be "fuzzy." English: the manuscript, although very well written, would benefit from some editing by a native English speaker. There are places whether the technical meaning is lost due to the English. Figure 1 is not at all clear. There needs to be much more descriptive text in the caption. Sample: The first couple of sentences concerning the simple random sample could be clearer. I understood that the 95% CI was to adequatly represent Mataro, but its a bit opaque here. Figure 23: "audit zones" is a new term not used in the text (I think). Figure 4: The precise meaning of the shading should be made clear in the caption. I assume that this is the number of disagreements, but...Author Response
We would like to thank the reviewer for their detailed and thoughtful comments and suggestions for our manuscript. We are convinced that our manuscript is now a more consistent and clearer piece of research thanks to their feedback. Below, we provide a point by point description of how each those comments were addressed in our manuscript. Original reviewer comments in boldface, responses in regular typeface.
Line number refers to the word file with the document changes active.
|
I think the many readers would like more detailed information the process by which Google Maps and Open Street map obtain their data. And, more on FLC and how it "abstracts" this information. There are other researchers and practitioners who would like to try this approach, but there is insufficient information here for them. |
We include as a supplementary resource a public repository with all the codes from the FLC (https://github.com/jcmunozmora/facilitylistcoder.git). In this repository, we include all the technicalities (e.g. packages) and explanation for the implementation. Moreover, we are preparing a more detailed manuscript where we present in detail the type of data that is obtained using the FLC. |
|
Terminology: I think defining some terminology here and there would be helpful. For example, "buffer zone" is referred to later in the paper as synonymous with your sampling grids (if I understand correctly). Although much of the terminology (meta-data) will be familiar to GIS savy readers, for others the meaning will be "fuzzy."
|
We add extra comments to clarify un-common terms as well as we homogenize terms. In particular, we changed the term ‘buffer zone’ for ‘grid’ to be clearer. |
|
English: the manuscript, although very well written, would benefit from some editing by a native English speaker. There are places whether the technical meaning is lost due to the English.
|
We have reviewed the paper entirely to guarantee its consistency and clarity. |
|
Figure 1 is not at all clear. There needs to be much more descriptive text in the caption.
|
We include extra explanation in the caption on this. |
|
Sample: The first couple of sentences concerning the simple random sample could be clearer. I understood that the 95% CI was to adequatly represent Mataro, but its a bit opaque here.
|
This a standard assumption for the random sampling strategy, we include a comment to clarify this. |
|
Figure 23: "audit zones" is a new term not used in the text (I think).
|
We changed this term for street audits to avoid miss-understandings. |
|
Figure 4: The precise meaning of the shading should be made clear in the caption. I assume that this is the number of disagreements, but...
|
We included a description of each figure. |
Author Response File: Author Response.pdf
Reviewer 3 Report
I read with interest the manuscript entitled "he Facility List Coder: A tool to evaluate community food environments". It is a very nice piece of work described and presented well.
I have some suggestions for improvement of the manuscript:
Figure 1: I assume the second symbol that appears is Open street Map...I do not think that it is clear to everyone. Consider using legends. Figures 2, 3: It can be smaller in the text. Lines 165-171. This classification could be introduced before when they are also mentioned. Lines 198-199: The meaning of "only 571 grids had at least one facility" is not clear as a result of the study. Lines 204-205: Likewise, it is not clear: tolerance rate of just one facility Line 207: Why to present the value of the t-test?. The mean difference value between the pairs and the p-value would be more meaningful together. Line 221-222: please describe these sentences better. It is not totally clear what is meant. Please clarify how both searches were combined. What was the coverage based on Google Maps and on Open Street Maps. Does it mean that when using the tool, the search needs to be made in both search-engines? Please clarify it. Tables 1 and 2 - I find unnecessary to present F values, df1, df2 I think that the reproducibility of the tool to other regions in the globe cannot be warranted without further investigation and this should be noted. When the conclusion says that the method is reliable, you should also restrict to the population/region that was tested. There is some repetitive use of the words "in order to". They are not always needed.Author Response
We would like to thank the reviewer for their detailed and thoughtful comments and suggestions for our manuscript. We are convinced that our manuscript is now a more consistent and clearer piece of research thanks to their feedback. Below, we provide a point by point description of how each those comments were addressed in our manuscript. Original reviewer comments in boldface, responses in regular typeface.
Line number refers to the word file with the document changes active.
|
Figure 1: I assume the second symbol that appears is Open street Map...I do not think that it is clear to everyone. Consider using legends. |
We added the labels in the graph |
|
Figures 2, 3: It can be smaller in the text.
|
We reduced their size. |
|
Lines 165-171. This classification could be introduced before when they are also mentioned.
|
We have included a clear introduction of our classification method in the section where we introduced the FLC (ie. Secondary data: Introducing the Facility List Coder ) |
|
Lines 198-199: The meaning of "only 571 grids had at least one facility" is not clear as a result of the study.
|
It was indeed a typo. We have deleted to avoid miss-understanding. |
|
Lines 204-205: Likewise, it is not clear: tolerance rate of just one facility
|
We have introduced a comment to clarify this point. |
|
Line 207: Why to present the value of the t-test?. The mean difference value between the pairs and the p-value would be more meaningful together.
|
We agreed on this. However, it is standard to start by this test although it is not completely informative in this context. We provided a further statistical evidence on this comparation. |
|
Line 221-222: please describe these sentences better. It is not totally clear what is meant.
|
We corrected this sentence, now it says: “we found that these differences were mainly due to how local food markets were counted: whereas the field work treated market as a single facility, the FLC coded all the facilities that were located within the markets individually.” |
|
Please clarify how both searches were combined. What was the coverage based on Google Maps and on Open Street Maps. Does it mean that when using the tool, the search needs to be made in both search-engines? Please clarify it.
|
We include as a supplementary resource a public repository with all the codes from the FLC (https://github.com/jcmunozmora/facilitylistcoder.git). In this repository, we include all the technicalities (e.g. packages) and explanation for the implementation. Moreover, we are preparing a more detailed manuscript where we present in detail the type of data that is obtained using the FLC. |
|
Tables 1 and 2 - I find unnecessary to present F values, df1, df2 I think that the reproducibility of the tool to other regions in the globe cannot be warranted without further investigation and this should be noted.
|
Although we agree about the lack of external reliability of our findings, we still need to give strong evidence on the internal reliability. In this sense, we think that keeping the F values are very informative. |
|
When the conclusion says that the method is reliable, you should also restrict to the population/region that was tested. |
We agreed on this comment, now we say that: “To conclude, the FLC is a valid and reliable tool for evaluating community food environments in our case of study. This result is building evidence of validity of using GIS-based solution as FLC to evaluate food environment” |
|
There is some repetitive use of the words "in order to". They are not always needed. |
We have reviewed the manuscript changing “in order to” for “to” in most cases. |
Author Response File: Author Response.pdf
