Enhancing Railway Detection by Priming Neural Networks with Project Exaptations

Round 1

Reviewer 1 Report

This study proposed a method that uses synthetic data generated from 3D CAD model for learning-based point cloud semantic segmentation. The topic is relevant and the manuscript is easy to follow. Some comments are as follows:

 

General comments

1. What are the contributions & objectives of this paper? In my opinion, the contribution of this paper is to take advantage of existing CAD models to generate synthetic data for point cloud semantic segmentation. This is not clear in the introduction.

Based on the experiments, you used full synthetic datasets to train a baseline model and used small real-world datasets for fine-tuning. Again, what is the key point you wanted to highlight here? Synthetic data can help improve model performance? This is a good contribution but in this case, I would not call it “boosting”. In other words, the contribution is to develop a framework for generating synthetic data and utilize it in the learning-based model; it is not using small real data to boost the performance. Also to avoid the confusion between “the proposed boosting methodology” and “boosting” in machine learning.

2. Having an overview section (Section 3) after the introduction & background is weird. If this is an overview of the methodology, combine it with Section4.

3. In the real-world datasets, there are typically artifacts caused by LiDAR scanning mechanisms or moving objects (e.g., vehicles & passengers). Do such artifacts exist in your datasets? If so, how do you handle them? Do they impact the model performance? If not, why?

4. It would be good to include a brief introduction regarding the KpConv network architecture. This will help the readers to understand the workflow.

5. Any quantitative results (similar to Fig 6) for Section 6.3?

 

Other minor comments

Fig 5. What are the symbols (circle & triangle) representing?

Fig 8 & 9. Please add label (for the colors).

 

Fig 11. Please enlarge the font size.

Author Response

Dear reviewer,

thank you for your helpful response. We made multiple improvements to the paper but only highlighted the changes directly related to the reviewer's comments. We hope this supports the reviewing process.

 

General comments:

R1: What are the contributions & objectives of this paper? In my opinion, the contribution of this paper is to take advantage of existing CAD models to generate synthetic data for point cloud semantic segmentation. This is not clear in the introduction.

 

A1: Thank you for this comment. We totally agree that the goal of our contribution was not clearly stated. We updated the abstract and rewrote parts of the introduction. 

 

R2: Based on the experiments, you used full synthetic datasets to train a baseline model and used small real-world datasets for fine-tuning. Again, what is the key point you wanted to highlight here? Synthetic data can help improve model performance? This is a good contribution but in this case, I would not call it "boosting". In other words, the contribution is to develop a framework for generating synthetic data and utilize it in the learning-based model; it is not using small real data to boost the performance. Also to avoid the confusion between "the proposed boosting methodology" and "boosting" in machine learning.

 

A2: Thank you again for this helpful comment. We used the word boosting, as it describes our motivation as well as our goal, but we also see the possibility of confusion. We, therefore, relabelled the figures (and text) to use "priming", which we think help to message our intent. We also rewrote parts of the methodology to better convey our key goal and motivation. 

 

R3: 2. Having an overview section (Section 3) after the introduction & background is weird. If this is an overview of the methodology, combine it with Section4.

 

A3: We agree that the overview section does not work well in the context of this journal. Since we want to keep the methodology clean from the problem statement, we combined the overview section with the background chapter. We hope the reviewer agrees with this decision.

 

R4: In the real-world datasets, there are typically artifacts caused by LiDAR scanning mechanisms or moving objects (e.g., vehicles & passengers). Do such artifacts exist in your datasets? If so, how do you handle them? Do they impact the model performance? If not, why?

 

A4: A paragraph explaining the missing artefacts was added to the "data set" section (line 414++). We hope the reviewer finds this information sufficient. The recording is geometrically very clean, and most artefacts are actually in the colour assignment. However, as we do not use colours in our approach, we tried to keep the explanation compact.

 

R5: It would be good to include a brief introduction regarding the KpConv network architecture. This will help the readers to understand the workflow.

 

A5: Thank you for this recommendation. We added more information about KPConv in the background (Line 214++), modified the KPConv setup paragraph (451++) and added a vizualization of the used architectures in the appendix (Figure B1). We hope we hit a good balance between supporting the reader and retelling.

 

R6: Any quantitative results (similar to Fig 6) for Section 6.3?

 

A6: Gladly! We added a table showing the object count (A1), an image of the 20%-runs testing (Figure A2 & A3 ) and the mixed validations (Figures A4 – A7) to the appendix. 

 

Other minor comments:

R7: Fig 5. What are the symbols (circle & triangle) representing?

A7: We are very sorry; we extended the legend with the symbols.

 

R8: Fig 8 & 9. Please add label (for the colors).

A8: Colours have been added beneath every image.

 

R9: Fig 11. Please enlarge the font size.

A9: We enlarged font sizes for Figure. 4 and Figure.11 

 

 

 

Reviewer 2 Report

The content of the manuscript is novel, but the content needs to be improved before it can be further considered for publication. Overall, this manuscript does not clearly describe the methods and experiments.

1. Abstract: There is too much background information in this part, so authors should pay more attention to the methods and results.

2. The contribution of this study should be explicitly mentioned.

3. The description of the method part is very weak and should be strengthened so that readers can better understand the method of this article. For example, network structure, training details, etc.

4. A clear method flow chart needs to be provided.

5. The experimental results were not clearly described.

6. Comparison methods are needed to highlight the effectiveness of this method.

7. Conclusion and Future Work: It should be reorganized, and most of the content belongs to the discussion part.

Author Response

Dear reviewer,
thank you for your helpful response. We made multiple improvements to the paper but only highlighted the changes directly related to the reviewer's comments. We hope this supports the reviewing process. 
We did our best to incorporate your comments and hope we understood your comments correctly.

R0:    The content of the manuscript is novel, but the content needs to be improved before it can be further considered for publication. Overall, this manuscript does not clearly describe the methods and experiments.

R1:    Abstract: There is too much background information in this part, so authors should pay more attention to the methods and results.

A1:    Thank you for this recommendation. We shortened and merged the general background in the abstract and extended the description of the methods.

R2: The contribution of this study should be explicitly mentioned.

A2:    We improved the introduction. Aside from minor changes, we better explained the reasoning behind the motivation (line 56++). More importantly, we rewrote the end of the introduction and are now explicitly stating the contribution (line 84++).

R3:    The description of the method part is very weak and should be strengthened so that readers can better understand the method of this article. For example, network structure, training details, etc.
 
A3:    We were a little confused by this comment. Our approach is independent of the network structure and the actual training parameters. 
To state this clearer and improve the readers' understanding, we rewrote the overview of our methodology. 
As we used a neural network for showcasing our concept, we reinforced the content by adding more information about the KPconv used in the background chapter (line 214++). Furthermore, more information about the experimental setup was added to the subsection <<5.1 KPConv Setup>>. Additionally, we created visualizations of the used network structures, which we provide in the appendix (Figure B.1 and B.2).

As each configuration file spans multiple pages, it was impossible to attach multiple configurations. As we improved the description of 5.1 to explain the most important parameter,  we would refer to the source code. All software prototypes used in this paper are/will be published in the GITs linked in the paper. This is now explicitly mentioned in the paper. The source code contains all configurations used and also the modified parsers. 

R4:     A clear method flow chart needs to be provided.

A4:    The respective overview flow chart is provided in Figure 1. For a better outline, we updated the Figure, moved it to the start of the methodlogy section and reformulated the caption. 
A detailed view of the Virtualizer is provided separately in Figure 2. Additionally, we updated subsection 3.2. This subsection now includes explicitly naming the current step. 

R5:    The experimental results were not clearly described.

We updated (uncoloured) multiple parts in the result description. We especially relate closer to the experimental setup mentioned in sections 4 & 5. We also revised the Figures' captions and section headings. We hope these support the reader in understanding the reasoning behind the discussion.

R6. Comparison methods are needed to highlight the effectiveness of this method.

A6:    We are very sorry, but we were unsure how to incorporate this comment. Out of all possible benchmarks, we selected the two most descriptive experiments.
1.) The first experiments highlight the difference between synthetic and real-world data. The experiment is deliberately designed to maximize the C1 error to show validity in this worst-case scenario.

2.) The second experiment evaluates the performance gain when our approach is incorporated for an entire project training cycle. For this, we compare the method to two other training setups, which are superior to traditional training. We want to point out that the focus is on the effectiveness of priming the network and not on the actual detection quality. 
If the reviewer disagrees with the selection of our experiments, we gladly add additional information, results, or experiments. If so, we kindly ask for a more detailed description.

R7. Conclusion and Future Work: It should be reorganized, and most of the content belongs to the discussion part.

A7: We thank the reviewer for this comment. We rewrote parts of the conclusion only to contain the summary and the broader picture. 

Round 2

Reviewer 2 Report

Thank you very much for considering all my suggestions. Although some modifications deviate from my suggestions to some extent, I think the problems in the manuscript have been well solved. I recommend that this manuscript be published.