Fusion of Sentinel-1 with Official Topographic and Cadastral Geodata for Crop-Type Enriched LULC Mapping Using FOSS and Open Data

Round 1

Reviewer 1 Report

The manuscript presented a case study of crop classification relying on open data and open software. The method in this paper has application significance, but it is lack of innovation. Therefore, I think it cannot be published at its present status but the resubmission after major revisions is encouraged. And, the major comments and suggestions are as follows:

(1)Main concern refers to the methods presented in this paper.

First, the whole article is lack of new ideas, more like a technical report

Second, in the line of 80-81 “…We designed the whole workflow using FOSS to follow the demands of TOP.”, it is suggested to add a flow chart to describe the workflow designed in this paper manifestly and illustratively.

Third, in the line of 189-194, I think more details are needed on how to “deploying the ALKIS real estate cadastre for delineation of the cropland enabled all non-crop pixels to be removed”.

Finally, in the line of 223-229, the authors need to justify more the advantages of the software environments selected in this paper.

(2)The title of this paper, “FOSS for Regional Crop Type Mapping with Open Data: A Case Study Combining Multitemporal Sentinel-1 Microwave Satellite Images with Topographic and Cadastral data from Open.NRW”, should be modified because it is too long and the “Regional” didn’t seem to be reflected in this paper.

(3)In the line of 44, “For our AOI, …”, the abbreviation “AOI” should be defined.

(4)In the line of 127-128, the authors mentioned that “The consequent division into independent training and validation fields was obtained by sorting the fields by crop type and field size.”, why?

(5)In the line of 129-131, the authors mentioned that “…the tallest field per crop was always chosen for validation.”. Actually, a statistically rigorous accuracy assessment requires choosing an appropriate sample size and sampling scheme. In order to assure a statistically rigorous evaluation of entire population, it is essential that randomization is incorporated into the selection of validation samples.

(6)Different crops showed different backscattering characteristics during period of growth. So, it is suggested to add more analysis about crops in data selection, method design and result discussion.

Author Response

Dear Reviewer1 thank you so so much for your precious comments. Please see the point by point response in the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear authors

 

I found the article “FOSS for Regional Crop Type Mapping with Open Data: A Case Study Combining Multitemporal Sentinel-1 Microwave Satellite Images with Topographic and Cadastral data from Open.NRW” quite interesting and with some approaches off the grid.

In my modest opinion, the structure is ok, the text is written in a comprehensive way and references are extensive, appropriate, and up-to-date. I clearly like the article and enjoy reading it. Nevertheless, I have some concerns, doubts and suggestions to put to authors consideration. So if you do not mind I will focus first on the big issues and then in the picky stuff.

My major concern goes to methods. In pre-processing I did not quite understood if the buffer is applied to all the agricultural parcels as one or for each one individually, but if the goal is to have only the parcels that completely fall inside the AOI why is the buffer needed? Also, if one have a vector map with the agriculture uses for each parcel and used for creating a binary mask of agricultural/non-agricultural why do we need the classification on the first place? I think some discussion is needed here.

Sentinel-1A SAR data has a spatial resolution of 5 m × 20 m and the DEM was resampled to 5 meters, Then why performing the classification using 10 meters resolution. And why first creating a DEM with 5 meters and then resampling it through bilinear interpolation to 10 meters?

Second, the article was submitted to the International Journal of Geo-information and the core of it stands on image processing technics, more precisely on the Random Forest (RF) algorithm. As such, little information is provided about it. RF algorithms require the tuning of several parameters to optimize classification (e.g. https://www.researchgate.net/publication/324438530_Hyperparameters_and_Tuning_Strategies_for_Random_Forest).

For instance, what was the number of trees in the forest? The higher this variable is the better the algorithm learns the data, but too many trees can delay the training process. Then we have the depth of each tree in the forest. The deepness implies more splits and thus extracts more knowledge from the data. In addition, one need to set the minimum number of samples required to split an internal node. This can vary between one single sample and all of the samples at each node. This is a constraining parameter as its increasing implies considering more samples at each node. Similarly, we do not know the settings for the minimum number of samples required to be at a leaf node and the maximum number of features to account for finding the best split.

In validation, we are not informed how the training and validation samples are selected. They are random by classe? by parcel? How it was defined the number of samples or we are using the total amount of pixels within each parcel? Then, the option for giving preference to bigger areas in validation sample gathering can biases the results since these areas have a higher probability of  achieving high accuracies.

I also would prefer to see the F1 score - the harmonic mean of user’s accuracy and producer’s accuracy. F1 score is usually more useful than accuracy, especially if the class distribution is uneven. Are there systematic biases in all the classes evaluated or all the errors are random? I suggest authors compute error tolerance for each class (e.g. https://www.researchgate.net/publication/3204190_Automatic_Spectral_Rule-Based_Preliminary_Mapping_of_Calibrated_Landsat_TM_and_ETM_Images)

As minor things, I recommend that authors define ESA also in abstract or simply remove it since it is not crucial information at that moment. In the opposite, AOI – area of interest is never defined in the text.

In keywords, I think that crop type classification and Land Use Land Cover are redundant. I would remove Land Use Land Cover especially because avoids the discussion if your classes are land use classes or land cover classes. You can use other keywords such as machine learning algorithms or ancillary information.

In line 30 when saying that “This data gap causes issues not only for modeling agroecosystems but also for decision-makers.” One should be specific and give some examples of the issues.

Figure 1. Appears without being mentioned in the text and out of context since we are in introduction and not yet speaking of the study area.

In line 38 “…and algorithms [10,15]. One recent approach classifies the crops without needing annual training data. Unsupervised algorithms are not new and they have been tested through time in image classification but without achieving the accuracy of supervised algorithms.

In line 40 “However, optical approaches are unreliable for acquiring data of a distinct phenological stage, as clouds during image acquisition hamper successful analysis [10,17].” Nowadays there are several available methods to deal with clouds. You can even found multitemporal classification algorithms that deal with it without the need of any pre-processing (e.g. https://www.mdpi.com/2072-4292/11/9/1104/htm)

In line 47 the authors refer the observation period (January-September 2017) of the study. Why just until September? For some readers can be clear that this period encompasses the phenological stages of all the considered cultures but for others this could not be clear.

Figure 2. Legend please replace “Legend” by Land Use Land Cover or simply LULC. Moreover, there are some blank pixels that are think are unclassified, probably due to roads. In my opinion that class should be contemplated and discussed.

Author also need to justify why they choose to use a Gamma Map Speckle Filter (e.g. https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLI-B7/269/2016/isprs-archives-XLI-B7-269-2016.pdf). The 3x3 window was used with a 4 or 8 neighbourhood?

In line 117 – table 1, please define vertically transmitted vertically received (VV) and vertically transmitted horizontally received (VH). Dual polarisation data is useful for land cover classification. The C-SAR instrument supports operation in dual polarisation (HH+HV, VV+VH). obtained, and for each pixel we took a VH intensity value and a VV intensity value. Further, sometimes we can improve classification results using an intensity ratio (VH/VV) band and add it into SAR data to be used as an additional variable for time-series analysis. The 70 images are VV or VH, or author have both for all and then we have 140 images?

In line 146 “…only parcels that have "agricultural land" as their primary usage were selected and used as a crop mask [48].” This means that parcels can be not 100% agriculture? We can have houses in the middle? If yes, classifying those pixels as culture is a classification error and we should have a class for those non-agricultural uses.

 

In line 170 when stating “For a better data handling, conversion of the raster values from linear to dB was applied.”, I would prefer “For a better data handling, conversion of the raster values from linear to a decibel (dB) scale backscatter coefficient was applied.”

Line 194 – “The filtering was conducted twice: the first one with a ball radius of three pixels, the second one with two pixels.” We can have Moore neighbourhood (square) or Von Neumann neighbourhood (circular), or if you prefer a more mathematical approach, rook case or kings case (also known as queen case). Please avoid designations as “ball radius”. The dimension and shape of the neighbourhood, and also the number of times we apply the filter changes the outcome, so these options have to be justified (please see:

https://www.sciencedirect.com/science/article/pii/S1364815215300724

https://www.tandfonline.com/doi/full/10.1080/13658816.2016.1219035)

Line 199 please replace GDAL by Geospatial Data Abstraction Library (GDAL)

In please remove Pea, Carrot and Oat since we are unable to compare them.

 

In conclusions, I would not focus in the multi-data-approach (MDA). This technique is not new, the only innovation here could be the datasets used. There are several examples of the use of ancillary data even for classification (e.g. with Bayesian classifiers). In this case, this data is just used for pre-processing the data.

Please do not see my concerns and doubts as criticisms but only as a group of suggestions intending to contribute to enhance the great article you already have. Continue the good work and I hope you get it published.

Best regards

Other perhaps interesting reading:

Long-short-term-memory-based crop classification using high-resolution optical images and multi-temporal SAR data

Combining Deep Learning and Prior Knowledge for Crop Mapping in Tropical Regions from Multitemporal SAR Image Sequences

Author Response

Dear Reviewer number two, thank you so very much for your precious comment on our manuscript. You gave us quite some work, but we learned a lot from your comments and they initiated good discussions. We gave our very best to make the most out of every comment. Please see the attached revision document where we responded to every comment also of the other Reviewer.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

In this manuscript, a transparent LULT workflow is proposed. Based on the publicly available open data from Open.NRW, the Sentinel-1 time series and the FOSS, the proposed workflow is reproducible with high accuracy. The experimental analysis is very detailed. Additionally, after the revision, the content of the article is obviously improved. The manuscript is acceptable. The corresponding comments are as follows:
1. The main contribution of the manuscript should be reorganized. In my opinion, what really matters is the experimental data, not the workflow. The AOI in the manuscript, the corresponding SAR data and the auxiliary cadastral data can be published as the benchmark data. But the workflow is lack of innovation. The classic benchmark datasets for reference include the AIRSAR image of Flavoland and San Francisco.
2. The workflow is reproducible but require very much about the quality of the input data. The deficiency of the auxiliary data or the time series may lead to decrease in accuracy in another AOI. Therefore, the method is reproducible but with poor robustness.
3. Each abbreviation should be defined when appeared for the first time. 

Author Response

We thank the reviewer for his review of our manuscript, and we are grateful for his recommendation to accept our manuscript. We also believe the comments of the reviewer helped to improve the paper, and we are very thankful for the time spent on our manuscript.

The main contribution of the manuscript should be reorganized. In my opinion, what really matters is the experimental data, not the workflow. The AOI in the manuscript, the corresponding SAR data and the auxiliary cadastral data can be published as the benchmark data. But the workflow is lack of innovation. The classic benchmark datasets for reference include the AIRSAR image of Flevoland and San Francisco.

We thank the reviewer for his valuable comment. We also think that publishing the data alongside the manuscript could be used for future benchmark studies. New benchmark data is even more critical as the currently used benchmark images were acquired 30 years ago. As already discussed in the comments of our revision, we evaluate the other argument differently.  

The workflow is reproducible but require very much about the quality of the input data. The deficiency of the auxiliary data or the time series may lead to decrease in accuracy in another AOI. Therefore, the method is reproducible but with poor robustness.

We again thank the reviewer for his comment. However, the time series of the study stems from Sentinel-1, which is a satellite constellation currently consisting of two satellites (a + b) acquiring data systematically. The next two satellites (c + d) planned and are now under construction. One of the key mission principles is to ensure data availability and continuity. We, therefore, think that the time series should be available with a similar density over the global landmasses (monitoring area of Sentinel-1)

https://sentinel.esa.int/web/sentinel/missions/sentinel-1/satellite-description/geographical-coverage

The auxiliary data used in the stems from the German cadastral data. As in many countries tax their citizens depending on the size of their properties, cadastres are usually of very high accuracy (spatially and thematically). Therefore, at least all developed countries have a cadastre that meets the highest possible accuracy. Besides official cadastre data, there is also Open Street Map (OSM), which has been used by many studies for the delineation of the arable land. It could also be used for further characterization of the area outside the arable land.

Each abbreviation should be defined when appeared for the first time.

We carefully checked each abbreviation of the manuscript. We added “(TR32)” in the introduction, “(LULC)” in the abstract, "SNAP" on its first usage, and "ALKIS".

 

Reviewer 2 Report

Dear authors

First of all sorry for taking more time than expected in the second review. However, this is because I have payed careful attention to your extensive changes in the text and complete and clear answers. It looks like I was the annoying reviewer: sorry about that.  As you stated you had a lot of work and the least you could ask for was I to respect your work and give it extreme attention.

I have to congratulate you for the extremely careful job and for the great amount of changes. I would take the chance to called improvements, because they in my opinion are and if you agree to make such changes is because you somehow agree. This to say that is not usual for me to pass from a major revision directly to an accepted article. However, in this case in conscience I have to do it.

I hope you do not find my comments to much meticulous. Instead, I like you to regard them as a way to give some suggestions and raise some questions intending to improve the article. It was a pleasure to review you work and to interact with you. Continue the good work and I hope you would come to publish many more papers.

Best regards

P.s. responding to comment 4 you touch precisely the point that I wanted: the Nyquist’s Theorem.

Author Response

Dear Reviewer,

Thank you so much for your review and kind and motivational words on our work. The pleasure of working with you was entirely on our side and stimulated fruitful discussion among the authors. Furthermore, regarding a possible delay, after more than two years working on the research and this manuscript, a couple of days do not matter that much. Thank you so much for your time. We believe your comments gave the paper the final improvement of quality it well deserves. Please do not stop being the "the annoying reviewer". Hopefully, many more manuscripts can benefit from your sharp thoughts.

Kind regards, Guido Waldhoff, Georg Bareth and Christoph Hütt