LiDAR-Generated Images Derived Keypoints Assisted Point Cloud Registration Scheme in Odometry Estimation

Round 1

Reviewer 1 Report

1-The experimental results are mentioned briefly, but more details are needed for a thorough evaluation.

2-It would be helpful to comprehensively analyse the experimental results, including performance metrics, statistical analysis, and comparison with other state-of-the-art algorithms. This would strengthen the proposed algorithm's validity and superiority over other approaches.

3-More evaluation metrics must be added in Section 4.

Clarify the research problem: The introduction should provide a clear and concise problem statement that motivates the research. The authors should clearly state the research question or hypothesis that the study aims to answer.

Improve the writing: The paper needs to be thoroughly proofread and edited for grammar, spelling, and punctuation errors. 

Author Response

****************** Responses to Reviewer 1 <START> *******************

1.1 Comment:

The experimental results are mentioned briefly, but more details are needed for a thorough evaluation.

1.1 Response:

Many thanks for your feedback, we understand that the evaluation of different performance metrics in section 4.2 was not clear enough in the manuscript. We have augmented the context to elucidate the results of various metrics, elaborated on the internal relationships of different metrics, also emphasized primary metrics, and rephrased their sequence for a logical presentation.

1.1 Action

In the analysis sections for various metrics, additional content has been incorporated. For instance, within the 'Robustness of Detector' metric, we have added: “Another thing we may notice is that ORB detector really doesn’t show a good robust performance for Lidar-based images, especially considering that ORB-SLAM is one of the most famous algorithms in the last decades...” Also, we have added more analysis about different metrics, for example, we presented the varying levels of importance for different metrics:” It’s important to note that for this metric, a high number of key points could still contain numerous false detections. Therefore, this metric should be considered in conjunction with other accuracy indicators, especially Match Score, Match Ratio, and Distinctiveness, to be more convincing”, etc.

---------------------------------------------------------------------------------------

1.2 Comment:

It would be helpful to comprehensively analyze the experimental results, including performance metrics, statistical analysis, and comparison with other state-of-the-art algorithms. This would strengthen the proposed algorithm's validity and superiority over other approaches.

1.2 Response:

Thanks for your valuable feedback. We acknowledge that it would be significantly helpful to have a comprehensive analysis of other state-of-the-art algorithms. However, the extra experimental work is substantial. Additionally, the KISS-ICP and NDT we utilized in the experiment are the representatives of the state-of-the-art.  Therefore, we thought that it was enough to show the validity and superiority of the proposed approach.

1.2 Action:

We thank the reviewer for his comments. However, as we thought the current two state-of-the-art approaches utilized in the experiment are enough to show the validity and superiority of the proposed approach. Additionally, the extra work for more approaches will bring will exceed the minor revision required.

--------------------------------------------------------------------------------------

1.3 Comment:

More evaluation metrics must be added in Section 4.

1.3 Response:

We thank you for pointing this out, we have added more metrics to Section 4.

1.3 Action:

In Figure 7, we incorporated metrics from three dimensions to analyze the robustness of the detector, encompassing rotation, scaling, and noise interference. Also, in the section on different metrics, we have expanded the context to illuminate the outcomes of assorted metrics, delving into the interrelationships among them, while highlighting primary metrics and presenting them in a logical arrangement.

--------------------------------------------------------------------------------------

1.4 Comment:

Clarify the research problem: The introduction should provide a clear and concise problem statement that motivates the research. The authors should clearly state the research question or hypothesis that the study aims to answer.

1.4 Response:

Thank you for your valuable feedback. We added an extra paragraph in the introduction part to clarify the research problem that drove the research.

1.4 Action:

 In the introduction part, we added one paragraph to summarize the targeted research problem to clarify the research motivation and contribution. Specifically, “In summary, the extant LiDAR-based point cloud registration paradigm confronts notable challenges, principally arising from the presence of drifts or misalignments engendered by the inherent density of the point cloud, coupled with the substantial computational overhead it imposes. Given the escalating ubiquity of LiDAR-derived imagery in contemporary contexts, there exists a propitious potential to leverage these data modalities for the amelioration of these prevailing challenges.”

---------------------------------------------------------------------------------------

1.5 Comment:

Improve the writing: The paper needs to be thoroughly proofread and edited for grammar, spelling, and punctuation errors.

1.5 Response:

Thank you for pointing out the necessity for improvements in our manuscript's writing. We recognize the importance of clear and error-free writing for effective communication of our research. We deeply appreciate your insights and have taken your feedback to heart.

1.5 Action:

We have rigorously proofread the entire manuscript, focusing on correcting grammar, spelling, and punctuation mistakes. We believe that these actions have substantially enhanced the quality of our paper, making it more comprehensible and aligning it with academic writing standards.

--------------------------------------------------------------------------------------

****************** Responses to Reviewer 1 <END> *******************

Reviewer 2 Report

Some strengths related to both scientific content and form are:

1. The paper begins by clearly stating the problem in LiDAR-based odometry estimation, emphasizing the importance of precise point cloud registration. This sets the stage for the research conducted in the paper.

2. The paper provides a thorough literature review of keypoint detectors, descriptors, and related techniques in the context of LiDAR-generated images and point cloud registration.

3. The authors describe their original contribution related to the topic in discussion.

4. The paper outlines a comprehensive methodology, including dataset description, preprocessing steps, keypoint detection, descriptor extraction, and point cloud downsampling. The step-by-step explanation enhances the reproducibility of the research.

5. The paper presents detailed experimental results, including evaluation metrics, to assess the performance of various keypoint detectors and descriptors. It also compares the proposed approach with existing methods in terms of translation and rotation error, computational efficiency, and other factors. The results are supported by figures and tables, making them easy to understand.

5. The paper discusses the findings in-depth, highlighting the advantages and limitations of different keypoint detectors and descriptors. It also emphasizes the performance of the proposed approach in scenarios where traditional methods fail.

6. The references are in accordance with the topic of the paper.

Some weaknesses/suggestions are:

1. The paper, while informative, can be challenging to follow due to its extensive content. It might benefit from improved organization and clearer transitions between sections to aid readers in navigating the research.

2. While the paper mentions the use of an open-source dataset, it does not provide information or references for accessing the dataset. Including a link to the dataset source would improve the paper's reproducibility.

3. The paper briefly mentions future research directions but could expand on this aspect. Providing more concrete suggestions for potential follow-up studies would enhance the paper's impact.

4. While the paper includes some figures and tables, additional visual aids, such as diagrams or flowcharts illustrating the proposed approach, could help clarify complex concepts. For e.g.: a high-level diagram illustrating the overall system architecture, including the integration of LiDAR sensors, LiDAR-generated image processing, keypoint detection, point cloud registration, and odometry estimation modules; a visual representation of the point cloud downsampling process, showing how keypoint information is used to select and downsample points from the raw point cloud; a table or chart that summarizes the pros and cons of different keypoint detectors and descriptors, etc.

5. The figures usually should appear after they are mentioned in the text.  Also, Figure 1 is placed in Introduction, but it refers to Results.

6. The use of English language should be revised.

Author Response

****************** Responses to Reviewer 2 <START> *******************

2.1 Comment:

Some strengths related to both scientific content and form are:

1. The paper begins by clearly stating the problem in LiDAR-based odometry estimation, emphasizing the importance of precise point cloud registration. This sets the stage for the research conducted in the paper.
2. The paper provides a thorough literature review of keypoint detectors, descriptors, and related techniques in the context of LiDAR-generated images and point cloud registration.
3. The authors describe their original contribution related to the topic in discussion.
4. The paper outlines a comprehensive methodology, including dataset description, preprocessing steps, keypoint detection, descriptor extraction, and point cloud downsampling. The step-by-step explanation enhances the reproducibility of the research.
5. The paper presents detailed experimental results, including evaluation metrics, to assess the performance of various keypoint detectors and descriptors. It also compares the proposed approach with existing methods in terms of translation and rotation error, computational efficiency, and other factors. The results are supported by figures and tables, making them easy to understand.
6. The paper discusses the findings in-depth, highlighting the advantages and limitations of different keypoint detectors and descriptors. It also emphasizes the performance of the proposed approach in scenarios where traditional methods fail.
7. The references are in accordance with the topic of the paper.

2.1 Response:

Thank you for your positive words and summary of the strength of our paper.

--------------------------------------------------------------------------------------

2.2 Comment:

Some weaknesses/suggestions are:

The paper, while informative, can be challenging to follow due to its extensive content. It might benefit from improved organization and clearer transitions between sections to aid readers in navigating the research.

2.2 Response:

Many thanks for your feedback, we have added and modified more content in each section to make a clearer transition and organization.

2.2 Action:

we have added more content in the transition of each.  Specifically, at the beginning of each section, we modified and added more content for a clearer transition. Notably, at the end of the section 1 introduction part, we explained in more detail the organization of the paper.

--------------------------------------------------------------------------------------

2.3 Comment:

While the paper mentions the use of an open-source dataset, it does not provide information or references for accessing the dataset. Including a link to the dataset source would improve the paper's reproducibility.

2.3 Response:

Many thanks for your feedback; to improve the paper's reproducibility, we included the link to the dataset in subsection 3.1 for accessibility. In the description of the dataset, we specifically introduced the data sequence utilized in the experiment of the paper.

2.3 Action:

We have added the sentence “The dataset is available and accessible via the GitHub repos^1,2” in the "3.1 dataset" under the "3. methodology". And we put the links to the dataset as footnotes in the manuscript.

--------------------------------------------------------------------------------------

2.4 Comment:

The paper briefly mentions future research directions but could expand on this aspect. Providing more concrete suggestions for potential follow-up studies would enhance the paper's impact.

2.4 Response:

Thank you for pointing out the opportunity to elaborate on future research directions. We revised the section to include more detailed and concrete suggestions for subsequent studies, thereby amplifying the paper's potential impact.

2.4 Action:

We have refined the section on future work, specifically within lines 456-467. In the revision, we elucidate the forthcoming steps, emphasizing the refinement of the SLAM systems' synergy with keypoints discerned via LiDAR. We advocate for a sturdier system architecture that contemplates the incorporation of supplementary sensors, including IMUs. Furthermore, we propose the integration of deep learning to enhance precision and flexibility significantly. These modifications aim to furnish a more lucid and intricately detailed blueprint for prospective research, drawing upon our present findings.

----------------------------------------------------------------------------------

2.5 Comment:

While the paper includes some figures and tables, additional visual aids, such as diagrams or flowcharts illustrating the proposed approach, could help clarify complex concepts. For e.g.: a high-level diagram illustrating the overall system architecture, including the integration of LiDAR sensors, LiDAR-generated image processing, keypoint detection, point cloud registration, and odometry estimation modules; a visual representation of the point cloud downsampling process, showing how keypoint information is used to select and downsample points from the raw point cloud; a table or chart that summarizes the pros and cons of different keypoint detectors and descriptors, etc.

2.5 Response:

Thank you for your comments. The diagram in Figure 3 in the manuscript shows the main workflow of the proposed LiDAR-generated images keypoint-assisted point cloud registration. Alongside the diagram, we have a comprehensive step-by-step explanation in subsection 3.4.3 including the key components, i.e., the LiDAR-generated image preprocessing, keypoint detection, and the integration of the keypoint for the raw point cloud downsampling purpose.

2.5 Action:

We add more explanation based on the diagram in Figure 3 of the overall workflow of our proposed approach. Additionally, we added another Table 2 to include more about the explanation of the pros and cons of the keypoint detectors and descriptors as the review suggested. We hope the above modifications provide a clear understanding of the approach.

-------------------------------------------------------------------------------------

2.6 Comment:

The figures usually should appear after they are mentioned in the text. Also, Figure 1 is placed in the Introduction, but it refers to Results.

2.6 Response:

Thank you for your comment. We understand that the figures should appear after they are mentioned in the text. However, Figure 1 is placed in introduction because we referred it in introduction as well for the explanation of the problems or challenges the current point cloud registration approaches have and the advantages of our proposed approach.

2.6 Action:

               We have gone throughout the paper trying to avoid similar problems which are not only for figures but also for tables and other referred contents, raised by the reviewers.

--------------------------------------------------------------------------------------

2.7 Comment:

The use of the English language should be revised.

2.7 Response:

Thank you for your comment. We have carefully noted your comment regarding the needed revision of the use of the English language. We acknowledge the importance of adhering to the required writing standards and have taken your suggestion seriously in our revision process.

2.7 Action:

To ensure compliance with MDPI standards, we have thoroughly reviewed our manuscript, paying particular attention to clarity, formality, and organization. Additionally, we tried to avoid all the possible and noticeable mistakes that happened in the English writing. We sincerely hope that our revised version meets the required standard and fulfills your expectations.

--------------------------------------------------------------------------------------

****************** Responses to Reviewer 2 <END> *******************

Reviewer 3 Report

The degree of novelty of the research is adequate, there are concerns on the subject.

The article is adequately written, the issues presented are clearly analysed, there are relevant comparisons. For this reason it can be judged that the content is well-founded.

The quality of the results presentation, however, could have been more attractive and suggestive on the subject.

The scientific level is very good, the quality of the references studied attests to a good knowledge of the subject of the research.

Author Response

****************** Responses to Reviewer 3 <START> *******************

3.1 Comments:

The degree of novelty of the research is adequate, but there are concerns on the subject.

3.1 Response:

We would like to sincerely thank you for your thoughtful review and positive evaluation.

--------------------------------------------------------------------------------------

3.2 Comments:

The article is adequately written, the issues presented are clearly analysed, and there are relevant comparisons. For this reason, it can be judged that the content is well-founded.

3.2 Response:

Thank you for your positive feedback on the clarity, analysis, and relevancy of our content.

-------------------------------------------------------------------------------------

3.3 Comment:

The quality of the presentation of the results, however, could have been more attractive and suggestive of the subject.

3.3 Response:

We acknowledge the need to enhance the attractiveness and suggestiveness of the presentation of our results and have made the necessary improvements promptly for better readability and corrected basic mistakes that happened in the previous manuscript.

3.3 Action:

The changes made include several aspects. We corrected the mistakes that happened in the last manuscript. And we changed the color of the boxplot to a color of a color-blind-friendly palette so that the readability of the results is better. Furthermore, we bold more content of the tables in order to stress the best results in the experiment.

----------------------------------------------------------------------------------------

3.4 Comment:

The scientific level is very good, and the quality of the references studied attests to a good knowledge of the subject of the research.

3.4 Response:

Thank you for your positive words and summary of our paper.

---------------------------------------------------------------------------------------

****************** Responses to Reviewer 3 <END> *******************

Reviewer 4 Report

The experiments in this paper are comprehensive, and the results are convincing. The authors have done a decent job of accomplishing their stated aim.

I would suggest that the authors check the formatting in detail. There are a few details that could be improved. For example, a space should be left before brackets in references in the text (e.g., [7] and [25]).

Author Response

****************** Responses to Reviewer 4 <START> *******************

4.1 Comments:

The experiments in this paper are comprehensive, and the results are convincing. The authors have done a decent job of accomplishing their stated aim.

4.1 Response:

Thank you for your positive words and summary of our paper.

--------------------------------------------------------------------------------------

4.2 Comment:

I would suggest that the authors check the formatting in detail. There are a few details that could be improved. For example, a space should be left before brackets in references in the text (e.g., [7] and [25]).

4.2 Response:

Thanks for pointing out the formatting inconsistencies, particularly regarding the spacing before brackets in text references. We have carefully reviewed and corrected these details in the manuscript.

4.2 Action:

We have gone through the paper to correct the similar issue raised by the reviewer. Specifically, We corrected the spacing near references including [7], [19],[25], [45], and [46].

---------------------------------------------------------------------------------------

****************** Responses to Reviewer 4 <END> *******************