SSGAM-Net: A Hybrid Semi-Supervised and Supervised Network for Robust Semantic Segmentation Based on Drone LiDAR Data

Round 1

Reviewer 1 Report

 The authors proposed a hybrid learning framework, called SSGAM-Net, which directly segments objects from drone-based noisy point clouds without any pre-processing or post-processing. To the best of authors knowledge, they are the first to build a true-color industrial point cloud dataset, which is obtained by drones and covers 90000m2. Second, the adjacency relationships between objects can be regarded as global contextual relationships, they represent the possibilities of some objects coexisting in a local space, so they proposed a plug-and-play module, named Global Adjacency Matrix (GAM), which utilizes only few labeled data to generate the pseudo labels and guide the network to learn global contextual relationships within noisy point clouds in semi-supervised settings. The module improves the ability of the network to focus on global contextual relationships during training. Finally, they built a hybrid learning-based network, SSGAM-Net, which combines semi-supervised and supervised learning for semantic segmentation of airborne noisy point clouds. To evaluate the performance of the proposed method, they conducted experiments to compare SSGAM-Net with existing advanced methods on the collected expert-labeled dataset. The experimental results show that the SSGAM-Net outperforms the current advanced methods, reaching 85.3% in mIoU, which ranges from 4.2% to 58.0% higher than other methods, achieving a competitive level.

This is a good paper, with a useful application. However, this paper needs major revision before being considered for publication, as follows:

- First, try to improve the presentation of the introduction section, as well as the related work section. They are long without deep discussion. Try to add a table to summarize related works to be more useful for the readers.  Also, avoid block citations, such as [9-11], [12-18].

- Evaluation could be enhanced by adding advanced deep-learning models for further comparisons.

- The authors mentioned that “To the best of our knowledge, we are the first to build a true-color industrial point cloud dataset.” thus, Sharing this dataset is necessary to give this paper more value. The source codes of the proposed SSGAM-Net approach also can be shared.

- re-check the quality of the figures. Some have big sizes with low quality, and they can be reduced with better quality and resolution.

-The parameter settings of all compared methods must be given.

- The complexity of the SSGAM-Net model as well as compared methods, can be given. 

- English editing is needed.

- It would be helpful to integrate some of the key findings from the discussion into the conclusion to provide a concise summary of the paper's contributions and implications.

moderate changes are required. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper studies the problem of semantic segmentation of drone LiDAR data, builds an airborne true-color point cloud dataset and proposed a hybrid learning-based network, SSGAM-Net, which combines semi-supervised and supervised learning for semantic segmentation of airborne noisy point clouds.

The idea is interesting and the proposed methodology is sound and novel. The paper is well-organized and presented. The experimental results seem fairly promising and competitive, which consistently outperforms comparable schemes.

Few minor concerns are as follows:

1.        The paper's title could be more informative and accurately reflect the content. The current title is too general and does not indicate the specific contribution of the proposed method.

2.        In the introduction, the authors should provide a more detailed literature review of existing methods for semantic segmentation of drone LiDAR data. Besides, the introduction of related work could be strengthened by providing more details on the limitations of existing methods when applied to noisy point clouds. This will help readers understand the motivation and significance of the proposed method.

3.        The authors should clarify the definition of "noisy true-color point clouds" and explain how they differ from other types of point clouds.

4.        The authors should also provide more information about the Encoder-Decoder module, including its architecture and how it is trained.

5.        As for the dataset, it would be beneficial to clarify the scanner/sensor specifications used to collect the point clouds. Besides, statistical analysis of the proposed airborne true-color point cloud dataset is suggested.

6.        Lines 240-241: The explanation of how the adjacency matrix captures relationships could be more intuitive, perhaps with a simple example.

7.        Lines 294-295: Briefly discussion on why spatial pyramid pooling is beneficial for combining multi-scale features are suggested.

1.       Abstract: The abstract provides a brief overview of the motivation, methodology, and results of the study. However, the language could be more concise and some terms could be better explained. For example, terms such as "Global Adjacency Matrix" and "hybrid learning framework" may not be familiar to all readers, and a brief explanation would be helpful.

2.       In the abstract, the sentence "It promotes the development of digital twins for BIM models." could be revised to "It promotes the development of digital twins for BIM models, which is an important application in intelligent industrial operation and maintenance."

3.       Consider rephrasing "boundary point effect" for clarity since its meaning is not immediately clear from the terms used.

4.       Line 9-13: this is a very long sentence with grammar error. Rephrasing in two short sentences would be better, or a conjunction “since” could be added in the very beginning of the sentence.

5.       In the Methods section, the sentence "We build an airborne true-color point cloud dataset with precise annotations for 3D semantic learning." could be revised to "We built an airborne true-color point cloud dataset with precise annotations for 3D semantic learning."

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Focus on the inefficiency in directly processing airborne true-color point clouds that contain geometrical and color noise, the manuscript proposed a novel hybrid learning framework, named SSGAM-Net, which directly segments objects from drone-based noisy point clouds without any pre-processing or post-processing, and claiming that they are the first to build a true-color industrial point cloud dataset. And the given experimental results indicate that the proposed method outperforms the current advanced methods, reaching 85.3% in mIoU, which ranges from 4.2% to 58.0% higher than other methods, achieving a competitive level.

Around the issue of processing airborne true-color point clouds that contain geometrical and color noise, authors has done a lot of useful work and the intention and ideas about this manuscript are good, but there are shortcomings in the expression and organization of this manuscript, as follows.

1, The section about Abstract should briefly highlight the contribution of this article, especially in a structured manner, in order to facilitate potential readers' quick access to the key points of the article. Suggestion: Reorganize the presentation of the abstract section to highlight the theme and make it easier to understand.

2, Regarding the design of images in a paper, the author should not only consider the accuracy of the information, but also consider aesthetics, compactness, coordination and readability.

Suggestion: Some images should be redesigned and typeset.

3,In the section of experimental verification, few comparative algorithms were used, resulting in insufficient persuasiveness.
        Suggestion: Adding comparative algorithms and corresponding experiments.

 The section about Abstract should briefly highlight the contribution of this article, especially in a structured manner, in order to facilitate potential readers' quick access to the key points of the article. Suggestion: Reorganize the presentation of the abstract section to highlight the theme and make it easier to understand.

Please refer to any native speaker or English-Service.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

A novel hybrid learning framework, named SSGAM-Net, which directly segments objects from drone-based noisy point clouds without any pre-processing or post-processing is proposed in this paper. To evaluate the performance of the proposed method, authors conduct experiments to compare the SSGAM-Net with existing advanced methods on the expert-labeled dataset. The experimental results show that SSGAM-Net outperforms the current advanced methods, reaching 85.3% in mIoU, which ranges from 4.2% to 58.0% higher than other methods, achieving a competitive level. Overall, the results in the paper are promising and the paper is generally well-written. I have some minor comments to improve the paper:

-Please provide a holistic framework of SSGAM-Net to allow the readers to better understand the framework before detailing the algorithms.

- Please make the conclusions more concise to highlight the key conclusions.

- Some relevant deep-learning approaches might be helpful to be considered in the work such as: hydro-3D: hybrid object detection and tracking for cooperative perception using 3D LiDAR; Detecting tassels in RGB UAV imagery with improved YOLOv5 based on transfer learning; an automated driving systems data acquisition and analytics platform.

- Please refine the abstract and make it concise. There is unnecessary description context.  

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 5 Report

The objective of this study is to develop a hybrid learning-based semantic segmentation network capable of accurately segmenting industrial objects from airborne noisy true-color point clouds.

The paper is interesting and corresponds to the alomst state-of-the-art level, however, I reccomend authors to compare their results with some SOTA (transformer based), like could be found at https://paperswithcode.com/sota/3d-semantic-segmentation-on-semantickitti 

the discussion section with concideration of the proposed architecture addressed to the limitation and novelity in the applications could also be added 

Also some explanation or intuition under all new proposed blocks in the architecture could be added, 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

No more comments. 

Reviewer 3 Report

The author responded point-to-point to the concerns in the previous review carefully and made corresponding modifications, it is worth acknowledging and commendable.

I suggest publishing this manuscript in present form.