A Self-Adaptive Mean Shift Tree-Segmentation Method Using UAV LiDAR Data

Round 1

Reviewer 1 Report

General comments:

The paper proposes a new method for individual tree crown delineation based on a self-adaptive mean shift algorithm. The novelty here would be the self-calibration of the mean shift parameter which authors hypothesize would solve the problem of crown size heterogeneity, present in fixed-bandwidth algorithms. Overall the paper is well-written, introduction covers the state-of-the-art, and I have no comment regarding that. However, it lacks at least two main critical analyses, the tree crown segmentation analyses, and the variability of accuracy with crown size and shape. Without these components, the results are not enough convincing that the method solves the problem of canopy heterogeneity and move forward with ITC delineation. See below in detail some major and specific comments:

1) Methodology is mixed up with results. Please describe the experiments in methodology and only report the results/discussion in the proper sections.

2) Only the tree detection was analyzed, but not the crown segmentation, i.e. how well the automatic delineated tree crown matches the observed tree crown. This is a crucial assessment for tree crown segmentation. The best metric, that I know of, to do this would be the Intersection over Union (IoU) metric, where automatic delineated crowns are compared versus reference crowns, e.g. calculate intersection, calculate union, calculate the ratio between intersection and union (IoU). You can assess the distribution of IoU, calculate some summary statistics, and potentially set a threshold to consider crowns as a match, i.e. IoU > 0.8, to get an overall accuracy of correctly delineated crowns.

3) Maybe I missed something here but, if I understood it correctly, the step 1 from “Self-Adaptive Kernel Bandwidth” is only performed once for the tallest tree in the forest? If that is correct, wouldn’t you expect that the method will work best for trees similar to that and if the forest is heterogeneous the method will often fail? Therefore, not really solving the problem of heterogeneous crown sizes and shapes (?). Moreover, in the introduction when you argue about the self-calibration from Ferraz et al. 2016, you say “The method required complicated pre-processing and the established allometric model might not be suitable for all tree species”, I wonder if this wouldn’t be a similar issue in your method?

4) With the given results I am not convinced that the paper solves the mentioned problem of heterogeneous crown sizes and shapes delineation. To test this hypothesis, besides properly assessing crown segmentation (point 2), you should show how the variability of segmentation accuracy (e.g. IoU of each crown) varies as a function of tree crown area and shape (e.g. perimeter-area ratio or the GSCI index - Koukoulas & Blackburn 2004), and also calculate some statistics to support the argument. Obs: the GSCI index was originally built to analyze canopy gaps but it basically analyses the shape of polygons so it does not matter if it is a crown or a gap, and it should be better than the perimeter-area ratio.

Koukoulas, S.; Blackburn, G. Quantifying the spatial properties of forest canopy gaps using LiDAR imagery and GIS. Int. J. Remote Sens. 2004, 25, 3049–3072.

5) Following last comments, tropical forests are environments where ITC methods often fail because of the great forest heterogeneity. However, the mentioned method from Ferraz et al. 2016 was tested in such environments and should work for that. Do you expect your method to work for tropical forests? Maybe you could add a comment about this as discussion. I would suggest for next studies to get access to some LiDAR point cloud dataset of tropical forests to integrate on your analyses – this would truly enrich your work with broader analyses and audience of interest.

6) Another key determining factor affecting tree crown delineations with LiDAR is point density. This will vary depending on platform (UAV, airplane), sensor and acquisition parameters. If you expect the results to be transferable between systems and acquisition, the paper should provide such experiment with different point density thinning.

 

Specific comments:

- L100, I would suggest splitting the paragraph, placing the objective on a new paragraph. This would make easier to spot the objective of the work.

- Figure 3, it would be nice if you show an example of each of the two major forest types; place it at results not in methods; add a scale so the reader can see the crown size

L209, describe a bit more in detail what you consider as simple or complex forest structure, this should be important if other people will use the algorithm and would like to know if it produces X or Y result for their forest type

L285, “segmentation accuracies” this is tree detection accuracy not segmentation

L342, as a suggestion, it would be nice if you make code available in github or some other repository so anyone can have access, apply your method and cite your paper.

Author Response

Please see the attachment.

 

 

Author Response File: Author Response.docx

Reviewer 2 Report

REVIEW MANUSCRIPT # Remote Sensing 685077

 

General comments

The authors in their article attempt to introduce a new method to improve tree segmentation accuracy using UAV LiDAR data which is further tested against independent data from a total sample of 7 plots. The paper is well written and I think that it can be published in RS journal after some improvements, which in my opinion will increase its scientific merit. My general comments are focused on two separate points:   

The authors acknowledge that forest density and structure may create problems in individual tree segmentation using LiDAR data (Lines 51-53) which is evident in multi-layered forest stands. I suggest the authors include descriptive data that are related to forest density (stems per hectare or basal area) to present the boundaries (upper and lower limits) where their method is applicable. The authors use data from 7 plots to validate their proposed methodology and they compare it with other relevant approaches. I believe that the outcome of the comparison between the methods cannot lead to clear conclusions before any statistical analysis and in my opinion, it is the biggest flaw in this study. For example, according to the authors (Lines 306-308), Yan’s method is less accurate when compared to their method in terms of computational efficiency which may be true for the specific samples but what about the entire population? At this point some statistical tests are essential.   

Specific comments (Suggestions for improvement)

Line 53. I suggest the authors use the term “multi-layered forest stands”. Line 202. Please refer to the specific species with their scientific name (Binomial nomenclature). This is important since each of those has a unique capability to create multilayered stand structures due to light demands. Line 204. How the DBH was estimated?

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

The introduction is clear and concise. The problem statement is well articulated. The improvements are well illustrated and make sense.

I have objections to the evaluation procedure and the overall rigor of the performance evaluation. The authors are vague on L215 “reference tree locations is within a certain range” What range is that? Why a range at all? Why not use the convex hull of the predicted tree? If the reference tree does not fall within that polygon, it isn’t accurate enough for use? 

I think it is far better to be honest about the ability of these algorithms than to produce an artificially inflated score. My sense is that there have been dozens of unsupervised LiDAR algorithms and the community is saturated with examples of proposed innovations but weak evaluation criteria.

I’m not convinced that the evaluation procedure reflects the desired use case. Practitioners want an algorithm that can be trained on one set of data and then applied to a broad area. Instead the authors use their test data to “train” their algorithm but setting the parameters that receive the best score.

“According to literature [30], h 228 r was set to 5 m. Search radius, R controls a neighboring size to include 229 the largest tree in the study area. According to our test data, R was set to 5 m.”

This inflates the perceived accuracy of the proposed approach. Ideally, the authors should have strict training and test splits. One set for finding the appropriate parameter values, the other for testing the generalization of the approach. Without this, it is impossible to know the level of overfitting and reduces the reliability of purported results. The authors make a first step towards this by first performing sensitivity analysis on plot 2 (L237), but they don’t state whether plot 2 was withheld from evaluation scores. From the writing, I do not have confidence that parameter values were blind to the evaluation dataset. For example, was “R” determined by iterating through tests until the optimal performance was found? What if you parameterized the complex plots to predict the simple and vice versa?

            Overall I’d like to see many more visual examples of predictions alongside the reference trees. We get some sense that some plots are “simple” and “complex”, but not much more than that. Since future users will want to apply these kinds of tools to their own forests, it is critical they have a sense for the strenuousness of the performance tests. Especially if no code or data are released, it is very hard for the reader to evaluate the authors contributions.

I’d like to stress that all of these comments can be addressed, and in no way do I mean to suggest the contributions are not valid. It is just the evaluation is not convincing and matching the touted accuracy.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

See comments in attachment.

Comments for author File: Comments.pdf

Author Response

Please see the attachment.

Author Response File: Author Response.pdf