Review Reports - Automated Measurement of Grid Cell Firing Characteristics

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This protocol article provides a detailed description of GridMet, an automated tool for measuring grid cell firing characteristics. The authors have developed MATLAB code that is compatible with both Linux and Windows systems, allowing users to customize the source code according to their specific research needs. I believe this article will be highly beneficial for researchers analyzing grid cell firing characteristics.

However, I suggest that the authors enhance the clarity of their methodology by presenting a step-by-step guide in the form of a mind map. This visual representation would help users grasp the key concepts more effectively and facilitate the application of GridMet to their own experiments.

Additionally, I recommend that the authors compare GridMet with gridCAT (as described in the paper "The gridCAT: A toolbox for automated analysis of human grid cell codes in fMRI"), which also offers automated analysis options for studying grid cell firing patterns. A comparative discussion of the strengths and limitations of both tools would provide valuable context for potential users.

Overall, this protocol article has significant publication value, and I believe it will be ready for publication once these suggestions are addressed.

Author Response

Dear Reviewer 1,

Thank you very much for your insightful and constructive feedback on this article. Please find below our responses to your feedback.

Comment 1: This protocol article provides a detailed description of GridMet, an automated tool for measuring grid cell firing characteristics. The authors have developed MATLAB code that is compatible with both Linux and Windows systems, allowing users to customize the source code according to their specific research needs. I believe this article will be highly beneficial for researchers analyzing grid cell firing characteristics.

Response 1: Thank you for your kind compliment on our work. We appreciate your encouraging remarks about the utility of this software in reporting grid field properties.

Comment 2: However, I suggest that the authors enhance the clarity of their methodology by presenting a step-by-step guide in the form of a mind map. This visual representation would help users grasp the key concepts more effectively and facilitate the application of GridMet to their own experiments.

Response 2: The new figure 1 we have added provides a mind map in the form of a flowchart that depicts a step-by-step flow of operations performed in the software to detect grid fields. We also included a section in the figure on data discarded during the process to help readers understand how data filtering occurs. We color-coded this figure to show readers how each part of the data processing relates to overall concepts in the field detection process. We believe that this figure can aid readers to visually follow each stage of the field detection operations while providing an overview of the methodology used in our work. We also added materials to describe the radar scan step of the flow chart in more detail. This is included as Algorithm 1 and 2 and the fourth paragraph in the “Design of Algorithms” section.

Your comment also mentions describing to readers how GridMet can be applied to their experiments. We have added a new second-to-last paragraph in the “Design of Algorithms” sections that describes potential applications of GridMet. Also, our new “Software Performance” section also helps indicate how GridMet can be applied to researcher’s experiments by describing strengths and limitations of the software.

Comment 3: Additionally, I recommend that the authors compare GridMet with gridCAT (as described in the paper "The gridCAT: A toolbox for automated analysis of human grid cell codes in fMRI"), which also offers automated analysis options for studying grid cell firing patterns. A comparative discussion of the strengths and limitations of both tools would provide valuable context for potential users.

Response 3: Thank you for providing this helpful reference. We have added a paragraph in the Introduction section that describes GridCAT and compares it with GridMet. We also include a comparative discussion about the complementary advantages and disadvantages of these two software programs.

Comment 4: Overall, this protocol article has significant publication value, and I believe it will be ready for publication once these suggestions are addressed.

Response 4: Thank you for your positive remarks about the publication value of our article. We greatly appreciate your ideas and analysis of our work, and we hope that we have fulfilled your suggestions.

A general note is that the results from software version 1.0.1 have been used to update all figures and results reporting. More details on the included code patch and its effects (e.g., minor possible performance reduction) are in the online release notes. In some cases, this reduced field detection size relative to the field filtering threshold (default threshold remains the same).

Thank you,

The authors

Reviewer 2 Report

Comments and Suggestions for Authors

The article presents an open-source software designed for the automatic measurement of specific parameters of grid cells in the medial entorhinal cortex of the brain. While the article is well-written and provides a detailed description of the software, its scientific contribution is limited to the presentation of the software itself, thereby restricting its scientific value. This approach is particularly problematic in the context of a journal that focuses on algorithms. In such a scenario, it would be more appropriate to delve into and discuss the algorithms underlying the software rather than focusing solely on its operational and procedural aspects, which could only be achieved through a major review of the article.

Author Response

Dear Reviewer 2,

Thank you very much for your insightful and constructive feedback on this article. Please find below our responses to your feedback.

Comment 1: The article presents an open-source software designed for the automatic measurement of specific parameters of grid cells in the medial entorhinal cortex of the brain. While the article is well written and provides a detailed description of the software, its scientific contribution is limited to the presentation of the software itself, thereby restricting its scientific value. This approach is particularly problematic in the context of a journal that focuses on algorithms. In such a scenario, it would be more appropriate to delve into and discuss the algorithms underlying the software rather than focusing solely on its operational and procedural aspects, which could only be achieved through a major review of the article.

Response 1: Thank you for your kind compliments about our article being well written and our software description being detailed. We appreciate your advice on content that can enhance our article. We added a section describing the methodology, including algorithms, used in this work. This includes a flowchart (new figure 1) that provides information on each part of data processing involved in field detection.

We also added two algorithm descriptions (Algorithm 1 and 2) to the “Design of Algorithms” section. We chose to present the radar scan algorithm because it plays a key role in improving field detection performance. That algorithm has been inspired by radar scanning methods in other fields, but is an original code design by us.

We added a paragraph (starting with “The radar scan algorithm first…”) that delves into the mechanisms used within the algorithms to produce a boundary estimate of fields. This can be used in conjunction with the algorithms presented to inform the reader of each part of the process used to generate the results. We find that this covers the most intricate algorithm in the field detection process aside from the DBScan clustering method. The DBScan method is not an original algorithm by us, and readers can find details on that through MATLAB resources or in prior works.

Other new content has also been added to the article as a result of responding to the other reviewers. We added a new second-to-last paragraph in the “Introduction” section that compare our work to that of software named GridCAT. A new second-to-last paragraph in the “Design of Algorithms” section describes potential application areas of our software. Also, a new “Software Performance” section has been added.

Thank you,

The authors

Reviewer 3 Report

Comments and Suggestions for Authors

1. Please mention novelty, contribution, and currently available tools and comparisons in the abstract.
2. Required some additional examples related to GridMet applications so that those doing research in other areas like neuroscience society will also use your study.
3. Comparison study required for similar tools.
4. The advantages and disadvantages of the previous study were required.
5. Performance metrics of the previous study must be included in your work.
6. Explain some of the potential limits that affect your study performance.
7. To create more impact on your study flowchart was required. Please mention the algorithm in pictorial form.
8. Explain how Python, user-friendly support for GridMet potential strategies.
9. Performance metrics must be included in your study.
10. Please adda future work section to tour study to improve the number of readers.

Author Response

Dear Reviewer 3,

Thank you very much for your insightful and constructive feedback on this article. Please find below our responses to your feedback.

Comment 1: Please mention novelty, contribution, and currently available tools and comparisons in the abstract.

Response 1: Thank you for your suggestions on materials that can improve our article. We appreciate your ideas on the different areas in which our article can be enhanced. We inserted the following sentence into the abstract: “Novel contributions of this work include an effective approach for automated field size estimation and an original method to estimate field spacing that can overcome challenges encountered in other software.” This addition highlights the novelty and contribution of this work while comparing results to currently state of the art.

Comment 2: Required some additional examples related to GridMet applications so that those doing research in other areas like neuroscience society will also use your study.

Response 2: We added additional examples of potential application areas of our work. In the “Design of Algorithms” section, we inserted a new second-to-last paragraph. The added text describes the possibility of comparing simulated fields to animal recordings and assessing the ratio of spacing to size across different scales of those statistics. We also added a description in the Introduction section about how GridMet can be applied to comparing results from non-invasive human brain imaging obtained with another software named GridCAT. This can help create translational research connecting rodent observations to studies performed of human grid codes. We believe that these examples illustrate how the presented work can benefit the neuroscience community.

Comment 3: Comparison study required for similar tools.

Response 3: We added a new section titled “Software Performance.” Within this section, we compare GridMet to the existing tool Opexebo when analyzing the data from all mouse-recorded cells included in a prior article of ours, and report details of the results.

Comment 4: The advantages and disadvantages of the previous study were required.

Response 4: We used the results provided by the comparison between the field spacing estimates of GridMet and Opexebo added in the “Software Performance” section to assess the relative advantages and disadvantages between the two software programs. In general, the estimates are similar, which provides a corroborating inter-validation between the two distinct methods. Approaching the statistical estimation in different ways, and generating similar findings, indicate agreement of the estimates. We also report is that in four cells Opexebo noticeably underperforms detection compared to GridMet.

Comment 5: Performance metrics of the previous study must be included in your work.

Response 5: We provide new reporting on results of the previous work’s Opexebo tool compared to GridMet in both the article (third paragraph of the new “Software Performance” section) and online documentation as an extended description. Readers that want additional details on the measurement metrics of both software programs can read the statistics and plots reported for each in the new online documentation “GridMet Performance Tests” page [1]. The page specifically reports when the performance of the two software tools diverges noticeably. In particular, this occurs with four cells, and GridMet reports more accurate statistics in these cases. This exemplifies the performance of the previous study, Opexebo, compared to GridMet.

[1] hco-dev-docs.readthedocs.io/en/latest/gridcell_metrics/performance_tests.html

Comment 6: Explain some of the potential limits that affect your study performance.

Response 6: We added a description of the limited applicability of GridMet to human recordings in the Introduction section (in the second-to-last paragraph), when comparing our work to GridCAT. We also added a discussion of limitations in the second-to-last paragraph of the “Software Performance” section. This goes into detail about hypothesized reasons for minor detection issues found in three cells in GridMet’s field detection performance testing.

Comment 7: To create more impact on your study flowchart was required. Please mention the algorithm in pictorial form.

Response 7: We added a new figure 1 which provides a step-by-step flowchart for the generation of field detection. The newly added Algorithms 1 and 2 provide more details about the radar scan step in the flowchart. We also cover the specific details about how the algorithms in that step work in the new paragraph starting with “The radar scan algorithm first…” This offers a more in-depth explanation of how each component of those algorithms are designed. We chose to add content about the radar scan algorithm because, aside from DBScan, it is the most intricate algorithm in the field detection process. Readers who would like more information about DBScan can refer to MATLAB documentation or prior work. The radar scan algorithm has been inspired by such methods used in other fields, but the code design of the algorithm is original to our work. DBScan’s code has been explained in other works.

Comment 8: Explain how Python, user-friendly support for GridMet potential strategies.

Response 8: We added a specific reference to Python (“...languages including Python...”; last paragraph of “Design of Algorithms” section) in our description of how batch scripting of GridMet is facilitated by its runtime version. The runtime version of GridMet makes the process of scripting automated experiment workflows with Python a more user-friendly process. More details about how GridMet is designed to be user-friendly are included in the last paragraph of the “Design of Algorithms” section. For example, a description is provided of how users can employ the runtime version without programming knowledge because only the configuration file needs to be edited for its use. This makes GridMet more user-friendly for researchers not experienced in programming.

Comment 9: Performance metrics must be included in your study.

Response 9: The new section on “Software Performance” explains with more detail the performance of the software. We added a new evaluation of the software and found that it had a 90% correct field detection performance on 30 cells, and minor issues with 3 cells that caused only small effects on results. We also report on comparing performance between GridMet and Opexebo. We now report as well on benchmarking the speed of operation of GridMet, along with the computer specifications used for this purpose. An analysis is provided of the types of errors that were observed in the three cells that had field detection issues.

Comment 10: Please add a future work section to your study to improve the number of readers.

Response 10: We added new descriptions about future work as the last paragraph in the “Software Performance” section and as the last sentence (starting with “future work could include…”) in the second-to-last paragraph of the “Introduction” section. We also added a sentence (starting with “future work can include…”) to describe how further investigations of the ratio between spacing and size of grid fields could be performed. A sentence (starting with “Further research could investigate…”) explains possible future work using GridMet to compare simulated grid fields to animal recorded fields.

Thank you,

The authors

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The revision of the article has made it more suitable for publication in the journal Algorithms. The reorganization of the content and the inclusion of the procedural flow of the algorithms implemented by the designed software provide the article with the necessary context for its publication.