Application of Three Deep Machine-Learning Algorithms in a Construction Assessment Model of Farmland Quality at the County Scale: Case Study of Xiangzhou, Hubei Province, China

Round 1

Reviewer 1 Report

- It is well written and throughout article about rising machine learning algorithms. In particular, comparing RF to Neural Network and EW. However, the data that the authors implemented weren’t dealt with sufficiently. In terms of data collection, variables, sample, and so on the paper needs to reveal the detail of data used. Quality evaluation of agricultural fields need to prioritise what variables were the most effective. Plus how meaningful each degree of variables is. These tends to affect the evaluation of agricultural field. Normally neural network algorithms within machine learning would identify the weights and of course, it depends on input data.

- The Keywords could be chosen with more relevant ones.

Author Response

Response to Reviewer 1 Comments

Dear reviewer,

       Thank you for taking your time offer me so many valuable suggestion.

       I have amend my manuscript followed by your suggestion.

       My short reply for your comment is in the "author-coverletter-9820012.v1.docx". And all  the specific changes please read the new version of my manuscript.

       Best regards.

Point 1: The data that the authors implemented weren’t dealt with sufficiently. In terms of data collection, variables, sample, and so on the paper needs to reveal the detail of data used.

Response 1: (1) Based on your suggestions, data collection was analyzed in detail. That is revised “the multisource environmental data involved in this study mainly included 3 databases (The Land Use Change Survey Database of Xiangzhou in 2018 (1:10,000), The Survey and Assessment of the Farmland Quality Grade of Xiangzhou in 2018 (1:10,000), and The Farmland Soil Environmental Quality Category Classification Database of Xiangzhou in 2018 (1:10,000) ), soil sampling data (containing 225 soil fertility sample points and 326 soil heavy metal sample points), and socioeconomic statistics.” Revisions of the data collection details can be found in lines 160 - 171 in 2.2.1. Section.

• Based on your suggestions, data processing was analyzed in detail. That is revised “soil index data processing and socioeconomic index data processing were involved in this study. 255 soil sampling points were selected at the center of representative plots in the study. More than 30 items of information related to the sampling point and its surrounding environment were recorded, such as topographic site, surface soil texture, texture configuration, tillage layer thickness, barrier factors, biodiversity, reticulation of agricultural land and forestry, and other soil properties, among which tillage layer thickness was obtained by field measurement and biodiversity was obtained by calculating the percentage of earthworms at the sample point. Soil characteristics such as pH, moisture, available phosphorus, available potassium, and organic matter were obtained by soil sample assay analysis. Besides, there were 326 heavy metal sampling points in the farmland of the study area, and the contents of 5 soil polluting elements, Pb, Cd, Cr, Hg, and As, were obtained by sample assay analysis. Using ArcGIS 10.2 software, socioeconomic indicators of farmland quality, such as farming distance, ease of farming, traffic accessibility, drainage capacity, and irrigation capacity were obtained, using neighborhood analysis tools.” Revisions of the data processing can be found in lines 172 - 205 in 2.2.2. section.

Point 2: Quality evaluation of agricultural fields need to prioritise what variables were the most effective. Plus how meaningful each degree of variables is. These tends to affect the evaluation of agricultural field.

Response 2: (1) Quality evaluation of agricultural fields need to prioritise what variables were the most effective. Based on your suggestions, the framework of a farmland quality assessment index system was added based on element–function–regulation by analyzing the connotation of farmland quality in this study. That is revised “In this framework, the term “element” refers to influencing factors. Soil conditions are the basis of high agricultural yield and are the most important factors affecting farmland quality, giving priority to soil pH, available phosphorus, available potassium, organic matter, and other indicators reflecting the fertility. The term “function” refers to the functional role of farmland quality in agricultural production, ecological landscape, and environmental protection, with an emphasis on indicators reflecting the sustainable use of farmland, such as biodiversity and soil environmental quality. The term “regulation” refers to the implementation of appropriate regulation measures to enhance farmland quality. Topography, field drainage and irrigation facilities, and transportation locations are obstacle factors that limit the improvement of farmland quality.” Revisions of variables selection can be found in lines 212 - 245 in 2.3.1. section.

(2)How meaningful each degree of variables is. Based on your suggestions, in this study, each degree of variables determined based on three models was analyzed and compared quantitatively (Figure 1.). That is revised “In the RFW, the drainage capacity, irrigation capacity, soil available phosphorus, topographic site, and soil organic matter were identified as the five most important indicators affecting farmland quality, while slope, traffic accessibility, and ease of farming were the three indicators that had the least impact on farmland quality. The BPNNW identification was more balanced, with drainage capacity and irrigation capacity being more important, and slope and cleanliness the least important indicators. EWW identified soil available phosphorus as the most important indicator and ease of farming as the least important. Thus, the difference between RFW and BPNNW was small in terms of indicator weight values, while the difference between EWW and BPNNW was significant.” Revisions of each degree of variables can be found in lines 387 - 415 in 3.1. section.

Figure 1. Indicator weights for the three assessment methods.

Point 3: Normally neural network algorithms within machine learning would identify the weights and of course, it depends on input data. 

Response 3: Based on your suggestions, for BPNN, the neuralnet package in R was used to identify the weights using backpropagation depends on index data in this study (Figure1.). That is revised “In the main parameter settings, the hidden layer was 5, the number of random seeds was 2000, the initial weight range was 0–1, the weight adjustment rate was 0.1, the minimum target error was 0.01, the learning rate was 0.05, the maximum number of iterations was 1000, the number of training repetitions was 1, the impulse coefficient was 1.2, the error type was sse, the activation function was tanh, and the output form was linear. Revisions of determination methods of weights in BPNN can be found in lines 337 - 342 in 2.3.1. section.” Revisions of index weights in BPNN can be found in lines 387 - 415 in 3.1. section.

Point 4: The Keywords could be chosen with more relevant ones.

Author Response File: Author Response.docx

Reviewer 2 Report

agriculture-1025721

The manuscript “Application of Random Forest in Construction Assessment Model of Farmland Quality at The County Scale: A Case Study of Xiangzhou, Hubei Province, China” addresses an interesting and up-to-date subject, which adhere to Agriculture journal policies. The manuscript has publication potential, but it needs further improvement and revisions.

In this research there was evaluated three different deep learning methods in order to identify favourable quality assessment of farmland. The manuscript contains original and interesting results, it does not have high novelty, but in my opinion, it well presents a good case study to justify. Below, I have inserted just some recommendations through which the readability and fairness of the manuscript could be improved.

• The ‘Abstract’ must be revised and improved, in order to be better correlated with the content of the paper and objectives. The current abstract is more of a list of conclusions
• In my opinion, the title should be changed to include the fact that the manuscript contains 3 different deep learning methods, not only RF
• Figure 2 should be improved; I recommend to add relief data to the right side of the figure (Xiangzhou) and not the smaller regions that serve no purpose. That way, the readers can have a better understanding of the study area, the altitudes and configuration of the land. Also add gridlines
• Moderate English corrections and changes are required, please carefully check the text, there are many instances of bad grammar (ex. Slop instead of Slope in 3-4 places)
• R255, further explained in a paragraph the expert scoring method used in Tab 2
• In subchapter 3.3.3. “20 percent as a test sample, and 10 percent as a test sample” please correct, one of the percentages has to be for validation
• Figure 5 is in my opinion the most important result/figure. Improve the quality of the resolution and enlarge the 3 figures for a better visualisation.
• In Table 5, the difference of percentage for first grade area between RF and EW is quite large (double); further address in Discussions your opinion regarding this difference

Author Response

Dear reviewer,

        Thank you for taking your time offer me so many valuable suggestion.

        I have amend my manuscript followed by your suggestion.

        My short reply for your comment is in the"author-coverletter-9849434.v1.docx ". And all the specific changes please read the new version of my manuscript.

        Best regards.

Point 1: The ‘Abstract’ must be revised and improved, in order to be better correlated with the content of the paper and objectives. The current abstract is more of a list of conclusions.

Response 1: (1)The content of the paper. Based on your suggestions, in the “Abstract”, a comprehensive assessment index system for farmland quality, representative samples selection in the study area, and construction of three models of entropy weight (EW), backpropagation neural network (BPNN), and random forest (RF) for training were added. In the results, quantity distribution and spatial distribution of the farmland quality grades in Xiangzhou were added. Revisions of the content of the paper can be found in lines 15 - 25 and 35-39 in the ‘Abstract’ section.

(2)The objectives of the paper. Based on your suggestions, the objectives of this study were added. The objectives were revised “This study enriches and improves the index system and methodological research of farmland quality assessment at the county scale while also serving as a reference for similar regions and countries.” Revisions of the objectives of the paper can be found in lines 41 - 44 in the “Abstract” section.

Point 2: In my opinion, the title should be changed to include the fact that the manuscript contains 3 different deep learning methods, not only RF.

Response 2: Based on your suggestions, in this study, 3 different deep learning methods were contained, thus, the title was revised “Application of Three Deep Machine Learning Algorithms in a Construction Assessment Model of Farmland Quality at the County Scale: Case Study of Xiangzhou, Hubei Province, China”. Revisions of the title can be found in lines 2 - 5.

Point 3: Figure 2 should be improved; I recommend to add relief data to the right side of the figure (Xiangzhou) and not the smaller regions that serve no purpose. That way, the readers can have a better understanding of the study area, the altitudes and configuration of the land. Also add gridlines.

Response 3: Based on your suggestions, in this study, relief data to the right side of Figure 2. (Xiangzhou) in the study area and gridlines were added as shown in the figure below. Revisions of Figure 2. can be found in lines 157 in 2.1. section.

Figure 2. Study area: (a) location in Hubei Province, China; (b) digital elevation map (DEM) of Xiangzhou.

Point 4: Moderate English corrections and changes are required, please carefully check the text, there are many instances of bad grammar (ex. Slop instead of Slope in 3-4 places).

Response 4: Based on your suggestions, we have carefully examined the manuscript, also submitted it to the English-editing service of MDPI, and have fully revised it in English in this study. Revisions of English of the manuscript could be found in tracked changes for the new manuscript.

Point 5: R255, further explained in a paragraph the expert scoring method used in Tab 2.

Response 5: Based on your suggestions, the expert scoring method used in Tab 2 was further explained. That is revised “the affiliation of the 7 qualitative indicators was determined based on the Farmland Quality Grade and National Arable Land Quality Grade Assessment Indicator System developed by the Ministry of Agriculture and Rural Development in China. In the Farmland Quality Grade, 16 well-known, experienced experts in relevant fields from the National Agricultural Technology Extension Service Center, Beijing Soil and Fertilizer Workstation, Shandong Soil and Fertilizer General Station, Jiangsu Farmland Quality and Agricultural Environmental Protection Station, Shanxi Soil and Fertilizer Workstation, and South China Agricultural University scored the values in the qualitative indicator affiliation table.”  Revisions of the expert scoring method used in Tab 2 could be found in lines 265-279 in 2.3.2. section.

Point 6: In subchapter 3.3.3. “20 percent as a test sample, and 10 percent as a test sample” please correct, one of the percentages has to be for validation

Response 6: Based on your suggestions, “20 percent as a test sample, and 10 percent as a test sample” was revised “20 percent as a test sample, and 10 percent as a validation sample.” Revisions could be found in line 317 in 2.3.3. section.

Point 7: Figure 5 is in my opinion the most important result/figure. Improve the quality of the resolution and enlarge the 3 figures for a better visualisation.

Response 7: Based on your suggestions, Figure 5 resolution has been revised to 600 pixels as shown in the figure below. Revisions could be found in line 445 in 3.2.1. section.

Figure 5. Spatial distribution of farmland quality grades: (a) RF-based, (b) BPNN-based, and (c) EW-based.

Point 8: In Table 5, the difference of percentage for first grade area between RF and EW is quite large (double); further address in Discussions your opinion regarding this difference.

Response 8: Based on your suggestions, the difference of percentage for first-grade area between RF and EW was analyzed from the spatial distribution of first-grade farmland in this study. That is revised “First -grade farmland based on RF was mainly distributed in Zhangwan, Guyi, and Longwang, while that based on EW was mainly distributed in Shuanggou, Chenghe, Guyi, Longwang, and Shiqiao (Figure 5.). Shuanggou and Chenghe are industrial towns in Xiangzhou with high urbanization and high intensity of farmland use, and existing studies have shown that the heavy metal element content of farmland soil around the towns is overall high, so it is unreasonable for the farmland to be identified as the first-grade land. The topography of Shiqiao is high and the soil water and fertilizer retention ability is poor so, theoretically, the quality grade of farmland should be lower, which is consistent with the existing research results.” Revisions could be found in line 453-463 in 3.2.2. section.

Author Response File: Author Response.docx

Reviewer 3 Report

Dear authors,

despite article has serious drawbacks in present form, it still could be considered for publishing after major revisions are implemented.

Major criticism:

1. Introduction too vague – the problem addressed in article should be more stressed and previous researches (not only from region of study) cited. Avoid general descriptions of well-known methods.
2. Restructure – the second section “RF model construction…” should be moved to methods and could be easily narrowed to workflow chart of model building.
3. Statements in study area description lack citations.
4. Data collecting part It is missing description of important parts of research design of field survey. It is hard to understand how variables were chosen. How ‘biodiversity’ was determined from orthophoto? Why it is important to mention that ArcGIS 10.2 is developed in USA?
5. Methods section has too much “textbook” style explanations – consolidate.
6. Avoid slogan-style statements of ruling party – especially in methods part.
7. In discussion section discuss the results, compare them with results of other studies, argue on dissimilarities. Completely avoid textbook explanations of methods used.
8. Conclusions should be numbered as “6”
9. In acknowledgments do not thank reviewers for “valuable suggestions”, this is added only after review is completed and paper accepted for publishing.

Author Response

Dear reviewer,

       Thank you for taking your time offer me so many valuable suggestion.

       I have amend my manuscript  followed by your suggestion.

       My short reply for your comment are below. And all  the specific changes please read the new version of my manuscript.

Point 1: Introduction too vague – the problem addressed in article should be more stressed and previous researches (not only from region of study) cited. Avoid general descriptions of well-known methods.

Response 1: (1) The problem addressed in article should be more stressed. Based on your suggestions, in this study, the problem addressed was stressed that “farmland is an important natural resource for people to carry out agricultural production and plays an important role in ensuring food security. However, in China, farmland quality has decreased overall, with only 27.3% corresponding to high-quality farmland. Therefore, actively carrying out studies on farmland quality assessment, effectively improving and protecting farmland quality, implementing a threefold production pattern of farmland quantity, quality, and ecology, and firmly guarding the red line of farmland are reliable ways to achieve coordinated and sustainable economic and social development and address food security.” Revisions of the problem addressed in the article can be found in lines 48 - 63 in 1. section.

• previous researches. Based on your suggestions, in this study, previous researches on farmland quality and farmland quality assessment were added. In terms of farmland quality, that is revised “scholars in other countries mainly study the temporal and spatial evolution and sustainable use of farmland, while those in China are more concerned with assessing farmland quality and its connection to food production capacity.” In terms of farmland quality assessment, what has been studied in traditional farmland quality evaluation and what has been studied in recent years were added. That is revised “Traditional farmland quality assessment mainly considers the natural production potential of land, selects natural factors to construct an index system, and evaluates the suitability of farmland. In recent years, environmental factors such as social development, economic level, and utilization patterns have received attention in farmland quality assessment, and the focus has shifted to considering many factors, such as natural, ecological, social, and economic factors.” Revisions of the previous researches can be found in lines 66 - 68, 74-86 in 1. section.
• Avoid general descriptions of well-known methods. Based on your suggestions, in this study, in order to discuss the feasibility of artificial intelligence methods in farmland evaluation, the domestic & international previous researches on the uses of artificial neural networks, support vector machines, and random forest in the farmland quality assessment were added. Revisions of the previous researches can be found in lines 96 - 100, 125-130 in 1. section.

Point 2: Restructure – the second section “RF model construction…” should be moved to methods and could be easily narrowed to workflow chart of model building.

Response 2: Based on your suggestions, the second section “RF model construction…” has been moved to methods and consolidated with the RF algorithm implementation. Revisions can be found in lines 298 - 311, 343-385 in 2.3.3. section.

Point 3: Statements in study area description lack citations.

Response 3: Based on your suggestions, in the study area , 2 citations have been added. Revisions can be found in lines 150,153 in 2.1. section.

Point 4: Data collecting part. It is missing description of important parts of research design of field survey. It is hard to understand how variables were chosen. How ‘biodiversity’ was determined from orthophoto? Why it is important to mention that ArcGIS 10.2 is developed in USA?

Response 4: (1)Data collecting part. It is missing description of important parts of research design of field survey. It is hard to understand how variables were chosen. Based on your suggestions, the data collection and processing section has been revised comprehensively to reflect the details of data use. In terms of data collection, that is revised “the multisource environmental data involved in this study mainly included 3database (The Land Use Change Survey Database of Xiangzhou in 2018 (1:10,000), The Survey and Assessment of the Farmland Quality Grade of Xiangzhou in 2018 (1:10,000), and The Farmland Soil Environmental Quality Category Classification Database of Xiangzhou in 2018 (1:10,000) ), soil sampling data (containing 225 soil fertility sample points and 326 soil heavy metal sample points), and socioeconomic statistics.” Revisions of the data collection details can be found in lines 160 - 171 in 2.2.1. Section.

In terms of data processing, that is revised “soil index data processing and socioeconomic index data processing were involved in this study. 255 soil sampling points were selected at the center of representative plots in the study. More than 30 items of information related to the sampling point and its surrounding environment were recorded, such as topographic site, surface soil texture, texture configuration, tillage layer thickness, barrier factors, biodiversity, reticulation of agricultural land and forestry, and other soil properties, among which tillage layer thickness was obtained by field measurement and biodiversity was obtained by calculating the percentage of earthworms at the sample point. Soil characteristics such as pH, moisture, available phosphorus, available potassium, and organic matter were obtained by soil sample assay analysis. In addition, there were 326 heavy metal sampling points in the farmland of the study area, and the contents of 5 soil polluting elements, Pb, Cd, Cr, Hg, and As, were obtained by sample assay analysis. Using ArcGIS 10.2 software, socioeconomic indicators of farmland quality, such as farming distance, ease of farming, traffic accessibility, drainage capacity, and irrigation capacity were obtained, using neighborhood analysis tools.” Revisions of the data processing can be found in lines 172 - 205 in 2.2.2. section.

• How ‘biodiversity’ was determined from orthophoto? Based on your suggestions, biodiversity is an important indicator of farmland ecological values, and is usually composed of genetic, species, and ecosystem diversity. Thus, biodiversity is difficult to access through remote sensing interpretation. Therefore, based on the Farmland Quality Grade and National Arable Land Quality Grade Assessment Indicator System developed by the Ministry of Agriculture and Rural Development in China, biodiversity was obtained from the percentage of earthworms at the sample point. However, due to the limitation of experimental conditions in the study area, it was not possible to analyze them quantitatively, and their levels were only analyzed qualitatively by the percentage of earthworms at the sample point, and further in-depth study is needed. Revisions of the acquisition of biodiversity indicators can be found in lines 183 -184 in 2.2.2. section.
• Why it is important to mention that ArcGIS 10.2 is developed in USA? Based on your suggestions, in this study, data were processed with the help of ArcGIS 10.2 software, which is not the focus of this study, and therefore the country where the software was developed - USA should not be emphasized. Thus, the country where ArcGIS was developed-USA has been removed in this study. Revisions can be found in lines 198, 207 in 2.2.2. section.

Point 5: Methods section has too much “textbook” style explanations – consolidate.

Response 5: Based on your suggestions, in this study, “textbook” style explanations of RF assessment model construction have been consolidated by integrating the second section “RF model construction…” and the RF algorithm implementation in the methods section. That is, combined with 1590 training samples selected from the study area, the process of RF assessment model construction was explained in detail in terms of training set generation, parameter optimization, weight calculation, and quality index calculation, especially quantitative analysis of training set generation and parameter optimization, thus, evading the textbook “textbook” style explanations.” Revisions can be found in lines 313-336, 343-365 in 2.3.3. section.

Point 6: Avoid slogan-style statements of ruling party – especially in methods part.

Response 6: Based on your suggestions, in terms of building a comprehensive index system for farmland quality assessment, by analyzing the connotation of arable land quality, the framework of a farmland quality assessment index system was established based on element–function–regulation in this study, which avoided slogan-style statements of ruling party. Revisions can be found in lines 212-245 in 2.3.1. section.

Point 7: In discussion section discuss the results, compare them with results of other studies, argue on dissimilarities. Completely avoid textbook explanations of methods used.

Response 7: Based on your suggestions, in the discussion section, in terms of the construction of a comprehensive assessment index system for farmland quality, some previous researches was cited to analyze how to select evaluation indicators. That is revised “Kong et al. established a pressure–state–effect–response assessment system based on changes in farmers’ land use objectives. Chen constructed an element–demand–regulation assessment system for slope farmland based on the minimum dataset.” In this study, based on previous studies, it is pointed out that the evaluation indicators of farmland quality are selected considering both natural ecological factors and socioeconomic factors. The basis for the selection of each evaluation index is also analyzed. At the same time, the limitations of access to biodiversity and cleanliness indicators were pointed out and need further exploration and in-depth study. Revisions can be found in lines 516-526, 555-563 in 4.1. section.

      In terms of the influence of research scale on indicator selection, some previous researches were cited to discuss spatial scale effects of farmland quality. That is revised “Kong et al.argued that farmland quality has significant scale variability, with some characteristics acting at the micro level and some at the macro level. At the same time, it is pointed out that as influenced by the scale effect of the assessment, there are problems such as the lack of comparability of different scales of farmland quality assessment index systems and results, and research on the construction of a multiscale farmland quality assessment index system and the establishment of an assessment model and scale conversion need to be further developed.” Revisions can be found in lines 565-576, 591-595 in 4.2. section.

     In terms of the construction of farmland quality assessment model, some previous researches were cited to analyze the feasibility of the model in farmland quality evaluation. In discussion, that is revised “To further verify the reliability and superiority of the RF model, typical samples of the study area were selected, and three deep machine learning models were selected for training, namely entropy weight (EW), BP neural network (BPNN), and random forest (RF). It is pointed out that the selection of training samples is also critical, and different samples have some influence on the final evaluation results, and this study only used the uniform method to select samples, so the next step can select samples in different ways and in-depth comparative analysis can be conducted.” Revisions can be found in lines 598-605, 609-613 in 4.3. section.

Point 8: Conclusions should be numbered as “6”.

Response 8: Based on your suggestions, because the second section “RF model construction…” has been moved to methods and consolidated with the RF algorithm implementation, the number of the conclusions has been revised as “5” which can be found in line 624 in 5. section.

Point 9: In acknowledgments do not thank reviewers for “valuable suggestions”, this is added only after review is completed and paper accepted for publishing.

Response 9: Based on your suggestions, in the acknowledgments, the "valuable suggestions" of the reviewers who were thanked have been deleted. Revisions can be found in line 660.

Round 2

Reviewer 2 Report

agriculture-1025721 second review

The revised manuscript demonstrates the author’s commitment in improving the overall paper, thus obtaining a more cohesive and interesting article.

• some additional text and corrections make the manuscript more legible
• figures were improved
• the error and shortcomings in the first review were addressed

In my opinion, this is a publishable research.

In the future, please use different colour for the revisions or track change. 

Reviewer 3 Report

One minor remark: consider to use different term for “socioeconomic index/indicators” throughout the manuscript. The described indicators are rather “geographic”.