An Arable Field for Benchmarking of Metaheuristic Algorithms for Capacitated Coverage Path Planning Problems

Round 1

Reviewer 1 Report

This article “An arable field for benchmarking of metaheuristic algorithms for capacitated coverage path planning problems” presents the problem of finding the most efficient route that covers a field.
The layout of the article and the content are presented in an understandable and appropriate way, enriched with many drawings that facilitate understanding of the subject matter.
The presented objective of the article is correctly (emphatically) formulated
It is interesting to use in the methodology of metaheuristic algorithms, simulated annealing algorithm (SAA) and ant colony optimization (ACO).
The analysis was conducted for four scenarios. Each algorithm was applied to each scenario.
The literature cited is relevant - a total of 32 items, including some of the most recent (4 items from 2020, 4 items from 2019).
The work is valuable and, in my opinion, after adding a few amendments, it should be published.
Comments on the article:
1. Information on the location of the research field is missing in the abstract
2. It is worth adding additional information to the sentence
” Graph modelling was applied to generate a field graph consist of all the types of edges (connections) in the field such as G2H, H, T, T2H, T2T, H2H” (186-187 lines)
This part is not fully understandable.
4. please delete the sentence
“A polygon is regular if all its sides have equal length and all interior angles are equal.”(165-166 lines )- self-evident information
5. Please improve the discussions . The discussion should not only include information about the results obtained but also refer to pre-existing results. This is missing from the article.

 

Author Response

Response to the 1st Reviewer’s Comments

Point 1: It is interesting to use in the methodology of metaheuristic algorithms, simulated annealing algorithm (SAA) and ant colony optimization (ACO).

Response 1:
The authors thank the reviewer for noting this point. The expressions: “simulated annealing algorithm (SAA) and ant colony optimization (ACO)” are added to the text in the methodology part (Line 96).

Point 2: Information on the location of the research field is missing in the abstract

Response 2:
The authors appreciate the reviewer for pointing this issue. The location of the field added to the abstract (Line 13).

Point 3: It is worth adding additional information to the sentence “Graph modelling was applied to generate a field graph consist of all the types of edges (connections) in the field such as G2H, H, T, T2H, T2T, H2H” (186-187 lines) .This part is not fully understandable.

Response 3:
The authors thank the reviewer for noting this point. Extra information about the type of generated edges added to the text in the lines (197-204) to make it clearer.

Point 4: Please delete the sentence “A polygon is regular if all its sides have equal length and all interior angles are equal.”(165-166 lines)- Self-evident information

Response 4:
The authors appreciate the reviewer for pointing this issue. The mentioned sentence removed from the text.

Point 5: Please improve the discussions. The discussion should not only include information about the results obtained but also refer to pre-existing results. This is missing from the article.

Response 5:
The authors appreciate the reviewer for pointing this out. Eextra discussion added to the text in lines (357-370).

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper present the performance comparison of two meherestic methods for capacitated coverage path planning problem. For the testing, small field was used due to possibility of exact solution. Some notes:

• Table 2 - The precision of the coordinates are unreasonably high. Can be lower.
• Table 8 and Table 7 - How the parameters were choose? Why this values? It will be suitable to add some chart that visualizate the performance of the algorithms dependent on parameters (e.g., evaporation rate) or justify the choice.
• I think that two algorithms for benchmarking is not enough. There are much more algorithm for this problem (TABU search). Why this two were chosen? It would be good to add some more or say why this two.
• The formal (mathematical) definition of the problem would be useful (but it is not necessary).

 

Author Response

Response to the 2nd Reviewer’s Comments

Point 1: Table 2 - The precision of the coordinates are unreasonably high. Can be lower.

Response 1:
The authors thank the reviewer’s point of view. The reason behind that is, we want to make it possible for other people to precisely replicate the same condition as presented in this study.

Point 2: Table 8 and Table 7 - How the parameters were choose? Why this values? It will be suitable to add some chart that visualize the performance of the algorithms dependent on parameters (e.g., evaporation rate) or justify the choice.

Response 2:
The authors appreciate the reviewer for pointing this out. The simulated annealing algorithm (SSA) and ant colony optimization (ACO) are governed by a set of parameters which mentioned in other studies ([17, 18, and 21]). The values of these parameters were defined based on the previous empirical tests.

Point 3: I think that two algorithms for benchmarking is not enough. There are much more algorithm for this problem (TABU search). Why this two were chosen? It would be good to add some more or say why this two.

Response 3:
The authors appreciate the reviewer’s opinion. These two algorithm were chosen because the results that gained by other algorithm weren’t comparative with ACO and SSA. Moreover, the main objective of this study was providing a benchmark field with global optimum solution and we believe that comparing these two algorithm with the benchmark as an example is quite enough according to the scope of this study.

Point 4: The formal (mathematical) definition of the problem would be useful (but it is not necessary).

Response 4:
The authors appreciate the reviewer’s opinion. We believe that the mathematical definition of the problem can help readers to better understand the concept of this study.

Author Response File: Author Response.pdf

Reviewer 3 Report

Manuscript title: An arable field for benchmarking of metaheuristic algorithms for capacitated coverage path planning  problems
Manuscript number: agronomy-935560

In this paper, authors solved the challenging problem of metaheuristic methods in finding the most efficient route that covers a field. The proposed study specifies an agricultural field as the basis for benchmarking of meta-heuristic algorithms. Even though the selected benchmarking field requires only eight tracks, the solution space consists of more than 1.3 billion solutions. The optimal solution for the capacitated coverage path planning problem was determined by calculating the non-working distance of the entire solution space and determining the solution with the shortest non-working distance.

Authors verified the proposed method feasibility through simulations with four scenarios consisting of low/high bin capacity and short/long distance between field and storage depot. In each scenario, the optimal solution and its associated cost value (minimum non-working distance) were compared to the solutions of two metaheuristic algorithms; Simulated Annealing Algorithm (SAA) and Ant Colony Optimization (ACO). The final results demonstrate  the feasibility in terms of obtaining most efficient route in CPP.   In my opinion, overall approach is contributing in the body of the literature, I encourage the authors to consider the following suggestions/comments to further improve the Manuscript in case they want to submit their work to this or any other journal in the future.

Major Comments/concerns/suggestions

• The abstract in present form contains basic details, the key things are in the very last, and not adequately described. I would suggest the author to re-write the abstract in following way concisely.
  • What methods to use
  • What problems to solve
  • Experimental results analysis and discussion
  • Evaluation of the proposed method
• The introduction section doesn’t include latest papers. Majority of the reported papers are significantly old (e.g., published before 2011). I would suggest the authors to include at least 10 recent papers and explain them. Also, describe how proposed method is different from them in detail. Authors can possibly refer to
  • Majeed, A., & Lee, S. (2019). A new coverage flight path planning algorithm based on footprint sweep fitting for unmanned aerial vehicle navigation in urban environments. Applied Sciences, 9(7), 1470.
  • Torres, Marina, David A. Pelta, José L. Verdegay, and Juan C. Torres. "Coverage path planning with unmanned aerial vehicles for 3D terrain reconstruction." Expert Systems with Applications55 (2016): 441-451.
• All equations need to be described in a proper fonts. In current form all equations are somewhere bold. Also the notations can be described in a separate table for better understanding.
• The authors didn’t specify the utility of the proposed approach in terms of real-world aerial applications. I would suggest the authors to include some practical examples of the proposed approach that can verify both theoretical and practical novelty of the proposal.
• The proposed method is chosen for an arable field. Although, it’s a very unique concept but recently the UAV’s applications are shifting more and more towards urban environments where substantial obstacles of varying geometries are present. How proposal will incorporate this change? Also, it will be interesting to consider the case when the endpoints are not on the  target area boundary and randomly distributed in alternate directions.
• Although proposed approach has significant improvements. The authors need to state the limitations of their study in the revised work.
• Conclusion should be brief and summarize the key findings. Also, the authors can state few promising future directions.

Minor Comments/concerns/suggestions

1. Figure 4 needs improvements in terms of fonts. The authors can enlarge the fonts of both axis.
2. Authors can include few more keywords related to the subject matter and contents presented.
3. Equations can be numbered throughout the paper.
4. Table 1 and figure are describing roughly the same thing. It would be ideal to include the contents of Table 1 in Figure 2 with proper colours to improve the contents readability.
5. The caption of figures and tables can be placed alongside each other on the similar page.

………………………………………………………………………………………………

Author Response

Response to the 3rd Reviewer’s Comments

Point 1: The abstract in present form contains basic details, the key things are in the very last, and not adequately described. I would suggest the author to re-write the abstract in following way concisely.
• What methods to use
• What problems to solve
• Experimental results analysis and discussion
• Evaluation of the proposed method

Response 1:
The authors thank the reviewer for noting this issue. The abstract was changed according to the suggested format. (Lines 13-33)

Point 2: The introduction section doesn’t include latest papers. Majority of the reported papers are significantly old (e.g., published before 2011). I would suggest the authors to include at least 10 recent papers and explain them. Also, describe how proposed method is different from them in detail. Authors can possibly refer to:
• Majeed, A., & Lee, S. (2019). A new coverage flight path planning algorithm based on footprint sweep fitting for unmanned aerial vehicle navigation in urban environments. Applied Sciences, 9(7), 1470.
• Torres, Marina, David A. Pelta, José L. Verdegay, and Juan C. Torres. "Coverage path planning with unmanned aerial vehicles for 3D terrain reconstruction." Expert Systems with Applications55 (2016): 441-451.

Response 2:
The authors appreciate the reviewer for pointing this out. All the old papers were substituted with the recent studies. Moreover, one of the suggested papers by the reviewer was considered in the text. (12 papers: 2019-2020; 11 papers: after 2011)

Point 3: All equations need to be described in a proper fonts. In current form all equations are somewhere bold. Also the notations can be described in a separate table for better understanding.

Response 3:
The authors thank the reviewer for noting this issue. All the equations were edited based on the suggested format.

Point 4: The authors didn’t specify the utility of the proposed approach in terms of real-world aerial applications. I would suggest the authors to include some practical examples of the proposed approach that can verify both theoretical and practical novelty of the proposal.

Response 4:
The authors appreciate the reviewer’s opinion. The proposed global optimum coverage path for this field supposed to follow by an agricultural machine on the ground. However, the same path can be followed by a drone in a certain level (height) to cover a field. We should consider this point that the defined constraints in UAV’s application are different from the agricultural machines. For instance, in order to reach to the depot for refilling, the agricultural machines usually use the headland passes and they are not allowed to pass through the main cropping area. However, for drones it is different and they can reach to the depot in a direct path.
Point 5: The proposed method is chosen for an arable field. Although, it’s a very unique concept but recently the UAV’s applications are shifting more and more towards urban environments where substantial obstacles of varying geometries are present. How proposal will incorporate this change? Also, it will be interesting to consider the case when the endpoints are not on the target area boundary and randomly distributed in alternate directions.

Response 5:
The authors appreciate the reviewer’s point of view. If we consider the UAV’s application, their coverage path planning is somehow different with the one for agricultural machines. However, there is also 3D route planning in agricultural field that the elevation data of the field is considered to minimize the fuel consumption of the vehicle (to have less load in the up heal). We believe that the proposed approach can be applied in UAV’s application to find the absolute optimal coverage path by considering the optimal level for UAVs to fly.

The suggested case is a general traveling salesman problem (TSP), which there are less constraint in the mathematical formulation of the problem and there are several benchmark datasets for that case.

Point 6: Although proposed approach has significant improvements. The authors need to state the limitations of their study in the revised work.

Response 6:
The authors thank the reviewer for noting this issue. The limitations of this study added to the text (Lines 363-366).

Point 7: Conclusion should be brief and summarize the key findings. Also, the authors can state few promising future directions.

Response 7:
The authors thank the reviewer for noting this point. The suggested part added to the text (Lines 367-371).

Point 8: Figure 4 needs improvements in terms of fonts. The authors can enlarge the fonts of both axis.

Response 8:
The authors thank the reviewer for noting this issue. The suggested changes applied on the Figure 4. (Line 188)

Point 9: Authors can include few more keywords related to the subject matter and contents presented.

Response 9:
The authors appreciate the reviewer for pointing this out. More related keywords added to the list. (Line 35-36)

Point 10: Equations can be numbered throughout the paper.

Response 10:
The authors appreciate the reviewer for pointing this out. The suggested change applied on all the equations.

Point 11: Table 1 and figure are describing roughly the same thing. It would be ideal to include the contents of Table 1 in Figure 2 with proper colours to improve the contents readability.

Response 11:
The authors appreciate the reviewer for pointing this issue. Figure 2 edited based on the reviewer’s suggestion (Line 168).

Point 12: The caption of figures and tables can be placed alongside each other on the similar page.

Response 12:
The authors appreciate the reviewer for pointing this out. The captions formatted based on the reviewer’s suggestion.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

I think that more methods should be take in to account. Despite this I think that the paper should be accepted.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Dear Authors,

 

This is a good work as to providing researchers with a benchmark for their developed algorithms for the coverage problems in agriculture. Below are my comments:

 

The references should be consistent in the introduction, either in number format or author-year format.

 

Number all the equations in the manuscript.

 

Line 188: Authors may present the Djikstra’s algorithm and the way it is adopted for the presented field. Likewise, how the Dubins Curves is employed, particularly present obtaining the turning distances when the vehicle enters an adjacent track vs a non-adjacent track.

 

Line 276: As this is a coverage problem, explain how a single node solution, or each node in the solution, is interpreted.

 

Line 283: Please explain how the two solutions for the scenario #4 are different as their set of tracks in each tour are same. If the travel pattern is different, how they both are the absolute solution.

 

Line 293: Please illustrate one representative solution in the field.  

 

Line 317: Provide which programming language and software are used, and present the specification of the utilized computation.

 

Please add a paragraph in the discussion section regarding what are the limitations for utilizing such benchmark (as this work provides a benchmark for researchers); also indicate the future work for this study.

Reviewer 2 Report

In the reviewer's opinion, the paper should be rejected for the following reasons:

-the "total" number of possible routes is not relevant (the 1.3 billions you stress multiple times throughout the paper). This is since in agricultural settings the transition graph can always be significantly pruned: It makes no sense to first drive track 1-2 and then 14-13. In your paper no effort is undertaken to exploit and prune typical agricultural graph structure. For your paper you motivate the need for "metaheuristic algorithms" for coverage path planning in agriculture with "1.3 billion possible solutions for an 8-track field of size 2.97ha" (see Abstract and the last paragraph of Introduction). In the reviewer's opinion this motivation is not merited.

-a single and very tiny field (2.97ha) with only 8 tracks is considered for analysis. Despite this simplistic setup you report solutions (i) with solve times up to 120 seconds (see Table 10), which yet (ii) are not even optimal (See the end of p. 12: "Neither of the algorithms was able to find the absolute solution for scenarios 3 and 4, but both succeeded for scenarios 1 and 2 (Table 10). It may be a little surprising that for two scenarios both algorithms failed to find the optimum for the relatively simple field with only eight tracks in 1,000 attempts, ..."). What improvement does the method proposed in your paper really yield to the current working practice of a farmer? By non-negativity of path lengths there always exists a path length optimal coverage plan. This is not found and additionally solve times are very long (already for a 2.97ha-field).

-very weak literature review. There are recent papers on coverage path planning in the agricultural setting proposing deterministic optimal algorithms (in contrast to your probabilistic sub-optimal proposal) that are evaluated on much larger and more complex fields. Not a single of them is referenced. Your most recent reference is from 2016!