Interoperability Study of Data Preprocessing for Deep Learning and High-Resolution Aerial Photographs for Forest and Vegetation Type Identification

Round 1

Reviewer 1 Report

The process of interpretation types of vegetation cover is a complex process, since the result is influenced both by the errors of the survey tools themselves and by many other factors - the similarity of color, shape, texture and spatial pattern of vegetation, shadows, crown density, etc. To solve this problem, the authors have proposed algorithms on the basis of which the automatic classification of images is performed. The researchers carried out a fairly in-depth analysis to review, analyze and compare three algorithms - VGG19, ResNet50 and SegNet.

The conclusions of the authors based on the results of the study are of sufficient importance and can serve be useful in further studies for the interpretation of vegetation cover in other regions of the world. The study is presented quite well and can be recommended for publication.

Comments for author File: Comments.pdf

Author Response

Revision Summary

Manuscript Number: remotesensing-1315358

Paper Title: Interoperability research of data augmentation preprocessing on deep learning and high-resolution aerial photographs for forest and vegetation type identification

Dear Editors and Reviewers,

We would like to express our greatest gratitude for your valuable comments on this paper. We have addressed them all in the revised manuscript and listed the modifications as follows.

Yung-Chung Chuang, Ph.D.

Associate Professor, Department of Urban Planning and Spatial Information

Feng Chia University, Taichung, Taiwan

(E-mail: [email protected])

(Tel: 886-4-24517250 ext. 3371)

______________________________________________________________

Response to Reviewer 1’s Comments

Comment 1

The process of interpretation types of vegetation cover is a complex process, since the result is influenced both by the errors of the survey tools themselves and by many other factors - the similarity of color, shape, texture and spatial pattern of vegetation, shadows, crown density, etc. To solve this problem, the authors have proposed algorithms on the basis of which the automatic classification of images is performed. The researchers carried out a fairly in-depth analysis to review, analyze and compare three algorithms - VGG19, ResNet50 and SegNet.

The conclusions of the authors based on the results of the study are of sufficient importance and can serve be useful in further studies for the interpretation of vegetation cover in other regions of the world. The study is presented quite well and can be recommended for publication.

Author’s Response

Thank you for carefully reviewing our research articles, and thank you for your affirmation and approval of the research results. We have revised the article, which will make the article more readable.

Sincerely,

YC Chuang (2021/08/30)

Author Response File: Author Response.docx

Reviewer 2 Report

There is confusion in the experimental method of the manuscript.

1. Because PCA is a data-dependent method, data consistency cannot be guaranteed during generating training data. Therefore, I do not agree that the deep learning model generated using the training data used in the experiment can be applied to the test data not used in the training.
2. VGG, SegNet, and Resnet are known as semantic segmentation techniques. However, in this manuscript, it was applied as a classification technique. Therefore, a more description of the deep learning model, which is applied in the manuscript, should be needed.

Author Response

Revision Summary

Manuscript Number: remotesensing-1315358

Paper Title: Interoperability research of data augmentation preprocessing on deep learning and high-resolution aerial photographs for forest and vegetation type identification

Dear Reviewer,

We would like to express our greatest gratitude for your valuable comments on this paper. We have addressed them all in the revised manuscript and listed the modifications as follows.

Yung-Chung Chuang, Ph.D.

Associate Professor, Department of Urban Planning and Spatial Information

Feng Chia University, Taichung, Taiwan

(E-mail: [email protected])

(Tel: 886-4-24517250 ext. 3371)

______________________________________________________________

Response to Reviewer 2’s Comments

Comment 1

Because PCA is a data-dependent method, data consistency cannot be guaranteed during generating training data. Therefore, I do not agree that the deep learning model generated using the training data used in the experiment can be applied to the test data not used in the training.

Author’s Response

Thank you very much for your valuable comments and suggestions. The text of our original manuscript is not very clear. We have made extensive improvements in this part. A lot of detailed explanations and supplements have been added to the revised manuscript.

Indeed, I agree with your idea, so this is also the value of this paper. Too many field workers just want to input images into different types of CNN models, but they don't know how the characteristics of input image will affect the analysis results, and they also don't know the connectivity between the input data and the output data. For this reason, we want to observe whether preprocessed images can improve the better results under the same deep learning framework. In the remote sensing literature, PCA is defined as a method to improve data interpretation, being widely used for the detection of vegetation as well as for changes in land use. Dutra et al., (2020) used PCA method in comparative analysis of methods applied in vegetation cover delimitation. In comparison with the original data, the results provided by application of PCA technique allow a better reading of the image features. Objects present on a land surface are better identified by applying the PCA if compared to the original spectral bands of the image.

In actual forestland survey operations, field operators must complete aerial photography of the entire area and perform image stitching before proceeding with interpretation. Therefore, image pre-processing and conversion are also processed at once for the entire area. During the analysis process, we performed CS-based and PCA-based image conversion for the graphics of the entire study area, and then uniformly extracted the training group and testing group images of various different vegetation types from the entire area (All belong to the forest land of Taipei City). Therefore, every image for training and testing had undergone the same and consistent conversion process. In other words, there will not be a problem of the essential difference between the training sample and the test data.

New Reference

Dutra, D. J.; Elmiro, M. A. T.; Garcia, R. A. Comparative analysis of methods applied in vegetation cover delimitation using Landsat 8 images. Sociedade & Natureza, v. 32, n. July, p. 699–710, 9 out. 2020. https://doi.org/10.14393/SN-v32-2020-56139

Comment 2

VGG, SegNet, and Resnet are known as semantic segmentation techniques. However, in this manuscript, it was applied as a classification technique. Therefore, a more description of the deep learning model, which is applied in the manuscript, should be needed.

Author’s Response

Regular image multi-classification would mean classifying the entire image into one of the classes, semantic segmentation works on a more granular level, classifying all of the pixels within the image into a class, again, this depends on the data. In our research, we classified the entire image into one of the classes from given small piece of images. Therefore, for VGG, SegNet, and Resnet, the final activation layer was a softmax function that outputs the probability of each class. We then applied an argmax function to obtain the class label for the class with the highest probability. We add more descriptions in the following chapters:

2.4.1. VGG19

The single image classification was performed at the end of layer with a softmax acti-vation. This softmax activation mapped the learned features (original, CS-based, and PCA-based photographs of 7 or 14 forest and vegetation types) to the final class prob-abilities. The maximum class probability of a pixel represented the final class of the respective pixel. We then applied an argmax function to obtain the class label for the class with the highest probability.

2.4.2. ResNet50

Then we added one fully connected layers after average pooling at the end of ResNet50 according to different classification situations (7 or 14 classifications). This allowed us to make them more appropriate for our data and finish final classifier. The same ideas but different structure from Resnet-based tree species classification using UAV Images by Natesan et al., []

2.4.3. SegNet

In the network structure of SegNet, the encoder alternately uses convolution and pooling, and the decoder alternately uses deconvolution+upsampling. Finally, we uses softmax for pixel-based classification to obtain the final class label for the class with the highest probability.

New Reference

Natesan, S.; Armenakis, C.; Vepakomma, U. Resnet-based tree species classification using uav images. In Proceedings of the 2019 International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Enschede, The Netherlands, 10–14 June 2019.

Thank you again for your valuable comments. We have also made substantial corrections and supplements to the content of the article. I hope this meet your expectations.

Sincerely,

YC Chuang (2021/08/30)

Author Response File: Author Response.docx

Reviewer 3 Report

This paper has a poor structure, and is hard to read. I strongly suggest the authors re-write this paper. Lots of repetition, and redundant and vague expressions can be found in the paper, while important information fails to be highlighted. The proposed method for vegetated type classification refers to standard practice for image classification using CNNs. The data augmentation only refers to a PCA transformation. I would say the novelty is too weak. What is new? Another concern is that the selected 14 vegetated types are quite similar in visual features. I am not convinced that these types can be classified with so high accuracies as listed in Tables 4 and 5. I was wondering if the collected testing datasets were highly correlated with training datasets. Moreover, VGG19, ResNet50, and SegNet models are widely known to people using CNN. It is not necessary to describe so many details, please shorten the description. The same problems can be found in the description of PCA and evaluation measures. Please move the description of methods from the results to methods sections (lines 462-510), and delete repeated contents.

Author Response

Revision Summary

Manuscript Number: remotesensing-1315358

Paper Title: Interoperability research of data augmentation preprocessing on deep learning and high-resolution aerial photographs for forest and vegetation type identification

Dear Reviewer,

We would like to express our greatest gratitude for your valuable comments on this paper. We have addressed them all in the revised manuscript and listed the modifications as the attached file.

Yung-Chung Chuang, Ph.D.

Associate Professor, Department of Urban Planning and Spatial Information

Feng Chia University, Taichung, Taiwan

(E-mail: [email protected])

(Tel: 886-4-24517250 ext. 3371)

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

The manuscript was well revised. According to the authors' cover letter, I recommend this manuscript for publication.

Author Response

Dear Editor,

We would like to express our greatest gratitude for your valuable comments on this paper. 

Yung-Chung Chuang, Ph.D.

Associate Professor, Department of Urban Planning and Spatial Information

Feng Chia University, Taichung, Taiwan

(E-mail: [email protected])

(Tel: 886-4-24517250 ext. 3371)