Combining Object-Oriented and Deep Learning Methods to Estimate Photosynthetic and Non-Photosynthetic Vegetation Cover in the Desert from Unmanned Aerial Vehicle Images with Consideration of Shadows

Round 1

Reviewer 1 Report

L 158-167, Is ground orthographic correction required for UAV images?

L206-207. Is the classification easier by using the vegetation image with no shadow on cloudy day?

Author Response

Response to Reviewer 1 Comments

Dear Reviewer

Thank you for your letter and comments concerning our manuscript entitled “Combining Object-oriented and Deep Learning Methods to Extract Photosynthetic and Non-photosynthetic Vegetation Cover from Unmanned Aerial Vehicle Images with Consideration of Shadows” (No. remotesensing-2055256). We were encouraged by the positive comments regarding the usefulness of our research. The revised manuscript now includes appropriate modifications and/or improvements which have been made possible from your comments. In addition, our manuscript has undergone extensive English revisions. Revised portions are marked in blue in the revised manuscript. The main corrections and responds to comments are as flows:

Comment 1: Line 158-167, Is ground orthographic correction required for UAV images?

Response: Thanks very much for your valuable comments.

Indeed, UAV images need to be ground orthorectified. In this study, the images were orthorectified and stitched using the Pix4Dmapper UAV image stitching software. First, the filtered UAV images were imported, and the geographic location information was added. Second, the eponymous feature points were automatically encrypted by an aerial triangulation model. The geometric correction of the images was then obtained by combining the above eponymous feature points with ground control point data. At last the orthophoto and digital surface model were finally generated.

This has been added in the revised draft, Please see Line 160 to 166 in section 2.2 of the revised version.

Comment 2: L206-207. Is the classification easier by using the vegetation image with no shadow on cloudy day?

Response: Thanks very much for your valuable comments.

It is approved by some references that it is really easier to use vegetation images without shadows for classification on cloudy days. Due to lighting and atmospheric scattering (amongst other reasons), large numbers of shadows are often present in UAV images [35]. Particularly for our study area, which is located in the sandy land, and the weather is generally sunny. It is necessary to investigate the optimal method when estimating PV-NPV coverage under shadow light exists.

There are few UAV-based studies that discuss the effects of shadows on their classification results. De Sá et al. [44] found that shadows greatly reduced the model accuracy of detecting Acacia species under sunny conditions, while images collected under diffuse light conditions caused by clouds showed a significant improvement in classification accuracy Because of the reduction of cast shadows. Moreover, Zhang et al.[45] found that cloudy conditions also reduce the separability of spectrally similar classes. However, Ishida et al. [46] proposed that incorporating shadows into the training samples improved classification performance. It is clear that no consistent conclusion was obtained about the effect of shadow to classification when using UAV images [12, 36].

Please see the revision from Line 560 to 572 in section 2.2 of the revised version.

Thank you for your consideration. I look forward to hearing from you.

Yours sincerely,

Jie He and all authors

Author Response File: Author Response.docx

Reviewer 2 Report

The proposed method combining object-oriented and deep learning methods to extract photosynthetic and non-photosynthetic vegetation for UAV images, which is sounds reasonable and efficient. However, the experiments' images contain similar ground objects and texture. Therefore, we cannot verify the generalization capability of proposed method. If we change the experiment area which is not in desert, we need to reselect the training samples and retrain the deep learning model. It seems not such efficient. From this point of view, I suggest the authors modify the title to match the content. 

There are some suggest for revise the manuscript:

1. There are some miswriting in Fig2 and Fig5.

2. There are some terms are mis expressed. For example, 'sensory field' in line 288.

3. Experiments in section 4.2 are suggested to display the label mask in figures.

Author Response

Response to Reviewer 2 Comments

Dear Reviewer

Thank you for your letter and comments concerning our manuscript entitled “Combining Object-oriented and Deep Learning Methods to Extract Photosynthetic and Non-photosynthetic Vegetation Cover from Unmanned Aerial Vehicle Images with Consideration of Shadows” (No. remotesensing-2055256). We were encouraged by the positive comments regarding the usefulness of our research. The revised manuscript now includes appropriate modifications and/or improvements which have been made possible from your comments. In addition, our manuscript has undergone extensive English revisions. Revised portions are marked in blue in the revised manuscript. The main corrections and responds to comments are as flows:

Comment 1: The proposed method combining object-oriented and deep learning methods to extract photosynthetic and non-photosynthetic vegetation for UAV images, which is sounds reasonable and efficient. However, the experiments' images contain similar ground objects and texture. Therefore, we cannot verify the generalization capability of proposed method. If we change the experiment area which is not in desert, we need to reselect the training samples and retrain the deep learning model. It seems not such efficient. From this point of view, I suggest the authors modify the title to match the content.

Response: Thanks very much for your valuable comments.

After careful consideration, the title was changed to ‘Combining Object-oriented and Deep Learning Methods to Estimate Photosynthetic and Non-photosynthetic Vegetation Cover from Unmanned Aerial Vehicle Images with Consideration of Shadows in the Desert’.

Please see Line 2 to 5 in the title of the revised version.

Comment 2: There are some miswriting in Fig2 and Fig5.

Response: Thanks very much for your valuable comments.

We have verified and corrected Figure 2 and Figure 5.

In Figure 2, 'DeepLaV3+' has been corrected to 'DeepLabV3+';

In Figure 5, 'Eecoder' has been corrected to 'Encoder'.

Please see Line 193 in section 3.1 and Line 310 in section 3.3.1 of the revised version.

Comment 3: There are some terms are mis expressed. For example, 'sensory field' in line 288.

Response: Thanks very much for your remindng.

We have verified and corrected the 'sensory field' to 'receptive field'.

Please see Line 294 in section 3.3.1 of the revised version.

Comment 4: Experiments in section 4.2 are suggested to display the label mask in figures.

Response: Thanks very much for your valuable comments.

We have displayed the label mask in figure 11.

Please see Line 499 in section 4.2 of the revised version.

Thank you for your consideration. I look forward to hearing from you.

Yours sincerely,

Jie He and all authors

Author Response File: Author Response.docx

Reviewer 3 Report

This study took the Maowusu Desert as an example to explore the feasibility and efficiency of shadow-sensitive photosynthetic vegetation and non-photosynthetic vegetation classification using three deep learning semantic segmentation models. Although the article is interesting, there are a number of important elements that must be explained and corrected:

First, if the goal is to analyze the feasibility and efficiency of shadow-sensitive photosynthetic vegetation and non-photosynthetic vegetation classification using three deep learning semantic segmentation models, more emphasis should be placed on shadow results. Let me explain, lines 377 to 381 do not detail what percentages of pixels have been sampled, it is necessary that they be at least 2-3% to be representative. Lines 403 to 405 state "All three models achieved lower accuracies for PV2, NPV2, and BS2 extractions under shadow conditions; this may be attributable to insufficient training samples (owing to the small number of these types of features in the study area ) Shouldn't a greater sampling effort be made?

Secondly, the discussion is not enough, there are only 5 bibliographical references and only one refers to a similar work.

Other minor details to keep in mind:

- In the title, is "extract"the most appropriate word?

- Keywords: The words of the title are repeated, you have to put other different words

- Line 135, the species descriptor is not in italics

- In table 1 it is mentioned that the speed is 1.2 m/s, but in the text (line 150) it is said that it is 6 m/s.

- Point 3.2.4 better explain how the manual correction was made

- Table 2: Indicate the meaning of the abbreviations, the table must be understandable by itself.

- Lines 591-598 are results, not discussion.

- Line 599 further develop the work carried out by Ayhan et al.

- Conclusions 1, 2, 3 and 4 are results, not conclusions.

I hope that the comments serve to improve the quality of the work

Author Response

Response to Reviewer 3 Comments

Dear Reviewer

Thank you for your letter and comments concerning our manuscript entitled “Combining Object-oriented and Deep Learning Methods to Extract Photosynthetic and Non-photosynthetic Vegetation Cover from Unmanned Aerial Vehicle Images with Consideration of Shadows” (No. remotesensing-2055256). We were encouraged by the positive comments regarding the usefulness of our research. The revised manuscript now includes appropriate modifications and/or improvements which have been made possible from your comments. In addition, our manuscript has undergone extensive English revisions. Revised portions are marked in blue in the revised manuscript. The main corrections and responds to comments are as flows:

Comment 1: This study took the Maowusu Desert as an example to explore the feasibility and efficiency of shadow-sensitive photosynthetic vegetation and non-photosynthetic vegetation classification using three deep learning semantic segmentation models. Although the article is interesting, there are a number of important elements that must be explained and corrected:

Response: Thanks very much for your positive evaluation.

We read them carefully and revised the manuscript profoundly. The revised portions are marked in blue in the revised manuscript.

Comment 2: First, if the goal is to analyze the feasibility and efficiency of shadow-sensitive photosynthetic vegetation and non-photosynthetic vegetation classification using three deep learning semantic segmentation models, more emphasis should be placed on shadow results. Let me explain, lines 377 to 381 do not detail what percentages of pixels have been sampled, it is necessary that they be at least 2-3% to be representative.

Response: Thanks very much for your valuable comments.

Within 1000 sample points, the percentages of feature types known by visual identification were as follows: PV1(25%); PV2(4%); NPV1(9%); NPV2(4%); BS1(50%); BS2(8%), which could be representative to the area.

Please see Line 388 to 390 in section 3.4 of the revised version.

Comment 3: Lines 403 to 405 state “All three models achieved lower accuracies for PV2, NPV2, and BS2 extractions under shadow conditions; this may be attributable to insufficient training samples (owing to the small number of these types of features in the study area ) Shouldn't a greater sampling effort be made?

Response: Thanks very much for your valuable comments.

We fully agree with what you mentioned that sample data should be enlarged, especially PV2, NPV2 and BS2. In this study, considering the computer performance, 728 image labeldata training models generated by object-oriented combined with manual correction are investigated, and the results basically reach the requirement of classification accuracy in the area. In the later stage, we will increase the number of labels to check if the accuracy of ground object estimation can be improved or not.

Please see Line 650 to 654 in section 5.2 of the revised version.

Comment 4: Secondly, the discussion is not enough, there are only 5 bibliographical references and only one refers to a similar work.

Response: Thanks very much for your valuable comments.

Following your suggestions, we reorganized and improved the section of DISCUSSION, stated the similarities and differences with references, and highlighted the main findings of this study. The corresponding references are also added. See below.

1. We improved the discussion of shaded features. The modified discussion is as follows:

“Due to lighting and atmospheric scattering (amongst other reasons), large numbers of shadows are often present in UAV images [35]. Particularly for our study area, which is located in the sandy land, and the weather is generally sunny. It is necessary to investigate the optimal method when estimating PV-NPV coverage under shadow light exists.

There are few UAV-based studies that discuss the effects of shadows on their classification results. De Sá et al. [44] found that shadows greatly reduced the model accuracy of detecting Acacia species under sunny conditions, while images collected under diffuse light conditions caused by clouds showed a significant improvement in classification accuracy Because of the reduction of cast shadows. Moreover, Zhang et al.[45] found that cloudy conditions also reduce the separability of spectrally similar classes. However, Ishida et al. [46] proposed that incorporating shadows into the training samples improved classification performance. It is clear that no consistent conclusion was obtained about the effect of shadow to classification when using UAV images [12, 36].”

Please see Line 560 to 572 in section 5.1 of the revised version.

2. We improved the discussion on deep learning feature classification.

“Wang et al. [48] propose a new method to classify woody vegetation and herbaceous vegetation and calculate their FVC based on the high-resolution orthomosaic generated from UAV images by the machine learning algorithm of classification and regression tree (CART). Yue et al. [49] evaluate the use of broadband remote sensing, the triangular space method, and the random forest (RF) technique to estimate and map the FVC, CRC, and BS of cropland in which SM-CRM changes dramatically. DeepLabV3+, PSPNet, and U-Net, the deep learning semantic segmentation algorithms adopted in this study, apply relatively mature computer vision technologies and rules. Whether deep learning methods may improve accuracy when applied to the estimation of photosynthetic and non-photosynthetic vegetation in desert areas requires further research to verify. ”

Please see Line 598 to 607 in section 5.2 of the revised version.

3. We also improved the discussion of similar studies on deep learning feature classification.

“Our results further developed the work carried out by Ayhan et al. [45]. In recent years, a series of studies showed that deep learning methods of convolutional neural networks (CNNs) can help to efficiently capture spatial patterns of the area, enabling the extraction of a wide range of vegetation features from remotely sensed images. Yang et al. [53] proposed an efficient convolutional neural network (CNN) architecture to learn important features related to rice yield from images remotely detected at low altitude. Egli et al. [54] proposed a novel CNN based on the lowcost UAV-RGB image using a tree species classification method, achieving 92% validation results based on spatially and temporally independent data.”

Please see Line 627 to 635 in section 5.2 of the revised version.

4. We refined the discussion of sample data labels The modified discussion is as follows:

“In this study, considering the computer performance, 728 image labeldata training models generated by object-oriented combined with manual correction are investigated, and the results basically reach the requirement of classification accuracy in the area. In the later stage, we will increase the number of labels to check if the accuracy of ground object estimation can be improved or not.”

Please see Line 650 to 654 in section 5.2 of the revised version.

Comment 5: In the title, is “extract”the most appropriate word?

Response: Thanks very much for your valuable comments.

After careful consideration, the title was purposefully changed to “Combining Object-oriented and Deep Learning Methods to Estimate Photosynthetic and Non-photosynthetic Vegetation Cover from Unmanned Aerial Vehicle Images with Consideration of Shadows in the Desert”. In the title, “extract” was changed to “estimate”.

Please see Line 2 to 5 in the title of the revised version.

Comment 6: Keywords: The words of the title are repeated, you have to put other different words

Response: Thanks very much for your valuable comments.

We have modified the keywords, and the modified keywords are “Unmanned aerial vehicle (UAV) images; PV/NPV classification; the shaded features; deep learning semantic segmentation; object-oriented technique; desert”

Please see Line 42 to 43 in the keywords of the revised version.

Comment 7: Line 135, the species descriptor is not in italics

Response: Thanks very much for your valuable comments.

We have verified and corrected the species descriptor.

Please see Line 136 in section 2.1 of the revised version.

Comment 8: In table 1 it is mentioned that the speed is 1.2 m/s, but in the text (line 150) it is said that it is 6 m/s.

Response: Thanks very much for your valuable comments.

We changed the word as “1.2 m/s”.

Please see Line 151 in section 2.2 of the revised version.

Comment 9: Point 3.2.4 better explain how the manual correction was made

Response: Thanks very much for your valuable comments.

Manual correction requires direct assignment of attributes to polygon units or change of attributes based on the results of the manual visual interpretation.

Please see Line 263 to 265 in section 3.2.4 of the revised version.

Comment 10: Table 2: Indicate the meaning of the abbreviations, the table must be understandable by itself.

Response: Thanks very much for your valuable comments.

We have completed the abbreviations in the all tables.

Please see Table 2 in section 4.1, line 423, Table 3 in section 4.2, line 478, Table 4 in section,line 526, and Table 5 in section 4.3, line 552 of the revised version.

Comment 11: Lines 591-598 are results, not discussion.

Response: Thanks very much for your valuable comments.

But we think we can keep it here for only a brief explanation of the results. The corresponding discussion is enriched in terms of the reasons. We also improved the discussion of similar studies on deep learning feature classification.

“Our results further developed the work carried out by Ayhan et al. [45]. In recent years, a series of studies showed that deep learning methods of convolutional neural networks (CNNs) can help to efficiently capture spatial patterns of the area, enabling the extraction of a wide range of vegetation features from remotely sensed images. Yang et al. [53] proposed an efficient convolutional neural network (CNN) architecture to learn important features related to rice yield from images remotely detected at low altitude. Egli et al. [54] proposed a novel CNN based on the lowcost UAV-RGB image using a tree species classification method, achieving 92% validation results based on spatially and temporally independent data.”

Please see Line 627 to 635 of the revised version.

Comment 12: Line 599 further develop the work carried out by Ayhan et al.

Response: Thanks very much for your valuable comments.

We have amended the sentence as your suggestion for ” The results further develop the work carried out by Ayhan et al.”.

Please see Line 627 to 628 in section 5.2 of the revised version.

Comment 13: Conclusions 1, 2, 3 and 4 are results, not conclusions.

Response: Thanks very much for your valuable comments.

In order to better answer your questions, we have reorganized and improved the results section. The modified conclusions is as follows:

“(1) The application of deep learning semantic segmentation models combined with object-oriented techniques simplifies the PV-NPV estimation process of UAV visible images without reducing the classification accuracy. (2) The accuracy of DeepLapV3+ model is higher than that of U-net and PSPNet models. (3) The estimation experiments for different time periods of the same ground class confirm that this method has better generalization ability. (4) Compared with three typical machine learning methods, RF, SVM and KNN, DeepLabV3+ method can achieve accurate and fast automatic PV-NPV estimation.”

Please see Line 667 to 673 in section 6 of the revised version.

Thank you for your consideration. I look forward to hearing from you.

Yours sincerely,

Jie He and all authors

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

The authors have revised the manuscript significantlly.

Author Response

Response to Reviewer 2 Comments

Dear Reviewer

Thank you for your letter and comments concerning our manuscript entitled “Combining Object-oriented and Deep Learning Methods to Estimate Photosynthetic and Non-photosynthetic Vegetation Cover in the Desert from Unmanned Aerial Vehicle Images with Consideration of Shadows” (No. remotesensing-2055256). We were encouraged by the positive comments regarding the usefulness of our research. The revised manuscript now includes appropriate modifications and improvements which have been made possible from your comments. In addition, our manuscript has undergone extensive English revisions Revised portions are marked in red in the revised manuscript. We retained the last revision in blue for your review. The main corrections and responds to comments are as flows:

Comment 1: The authors have revised the manuscript significantly.

Response: Thanks very much for your positive evaluation.

The manuscript has been rechecked and the necessary modifications have been made in accordance with the reviewers’ suggestions. And we have checked all the references and have made sure that they are relevant to the contents of the manuscript. In addition, our manuscript has been extensively revised in English, and reviewed by native English-speaking professors, and edited in English through Editage (www.editage.com). The revised parts are marked in red in the revised manuscript. We retained the last revision in blue for your review.

Thank you for your consideration. I look forward to hearing from you.

Yours sincerely,

Jie He and all authors

Author Response File: Author Response.docx

Reviewer 3 Report

The suggested corrections have been made and the article, from my point of view, is ready to be published.

A single detail, regarding comment 12, I did not mean to place verbatim "The results further develop the work carried out by Ayhan et al.". What I was referring to is developing in the text the research carried out by Ayhan et al.

Once this small modification is made, the article can be published.

Author Response

Response to Reviewer 3 Comments

Dear Reviewer

Thank you for your letter and comments concerning our manuscript entitled “Combining Object-oriented and Deep Learning Methods to Estimate Photosynthetic and Non-photosynthetic Vegetation Cover in the Desert from Unmanned Aerial Vehicle Images with Consideration of Shadows” (No. remotesensing-2055256). We were encouraged by the positive comments regarding the usefulness of our research. The revised manuscript now includes appropriate modifications and/or improvements which have been made possible from your comments. In addition, our manuscript has undergone extensive English revisions Revised portions are marked in red in the revised manuscript. We retained the last revision in blue for your review. The main corrections and responds to comments are as flows:

Comment 1: The suggested corrections have been made and the article, from my point of view, is ready to be published.

A single detail, regarding comment 12, I did not mean to place verbatim "The results further develop the work carried out by Ayhan et al.". What I was referring to is developing in the text the research carried out by Ayhan et al.

Once this small modification is made, the article can be published.

Response: Thanks very much for your positive evaluation.

We have amended the sentence as your suggestion for ” Ayhan et al. also confirmed that DeepLabV3+ had a significantly higher accuracy than other methods to classify three vegetation land covers, which were tree, shrub, and grass using only three band color (RGB) images [50]”.

Please see Line 627 to 630 in section 5.2 of the revised version.

The manuscript has been rechecked and the necessary modifications have been made in accordance with the reviewers’ suggestions. And we have checked all the references and have made sure that they are relevant to the contents of the manuscript. In addition, our manuscript has been extensively revised in English, and reviewed by native English-speaking professors, and edited in English through Editage (www.editage.com). The revised parts are marked in red in the revised manuscript. We retained the last revision in blue for your review.

Thank you for your consideration. I look forward to hearing from you.

Yours sincerely,

Jie He and all authors