A Reinforcement Learning Model of Multiple UAVs for Transporting Emergency Relief Supplies

Round 1

Reviewer 1 Report

This article is engaging and addresses an important topic. But there are issues that need to be addressed.

1.       There is room for improvement in the presented contribution. It would be helpful to provide more detail.

2.       The literature review should examine studies that combine land and air transport. See:

·       Gharib, Zahra, et al. "Developing an integrated model for planning the delivery of construction materials to post-disaster reconstruction projects." Journal of Computational Design and Engineering 9.3 (2022): 1135-1156.

·       "Post-Disaster Temporary Shelters Distribution after a Large-Scale Disaster: An Integrated Model." Buildings 12.4 (2022): 414.

3.       Research gaps are unclear in the study.

4.       There have been many relevant studies that have been overlooked. see:

·       Chowdhury, Sudipta, et al. "Drone routing and optimization for post-disaster inspection." Computers & Industrial Engineering 159 (2021): 107495.

·       Shi, Yuhe, et al. "A bi-objective optimization model for the medical supplies' simultaneous pickup and delivery with drones." Computers & Industrial Engineering 171 (2022): 108389.

·       Adsanver, Birce, Elvin Coban, and Burcu Balcik. "Drone Routing for Post-disaster Damage Assessment." Dynamics of Disasters. Springer, Cham, 2021. 1-29.

Author Response

We sincerely acknowledge the reviewer for their dedication in the evaluation of our manuscript. We have addressed the reviewer comments and provided two sections for each comment: (i) our reply to the reviewer comments, and (ii) a report of the changes in our manuscript based on the reviewer comment. In this document, we refer to the manuscript the reviewer has revised as the original manuscript, and the manuscript with the modifications generated from the reviewer comments is referred to as the revised manuscript. Besides, note that all the modifications in the manuscript are colored yellow.

Please see the attachment for our responses.

Author Response File: Author Response.pdf

Reviewer 2 Report

Topic of the manuscript is a contemporary important topic. The authors are experts in diverse, complex methods listed in the text.

And that is about all that is valid in the manuscript. Seemingly, the manuscript develops methods for improved utilisation of drones in the post-disaster actions. Yet, despite the many words related to disasters and drones, the substantial part of the text does not have any relevance to post-disaster actions or to drones.

In particular, authors think that helicopters are not capable of providing help because - one cannot expect that there are helidroms in needed locations!!! I hope that the authors will read about the use of helicopters, and that they will learn that helicopters can hover above different surfaces, land or water, and that they can provide substantial help. Moreover, currently the helicopters are capable of providing help in many post-disaster situations in which existing drones cannot fly at all!!!

The authors does not treat the drones in any important detail - the drones can be of fixed wing configuration, drones can be rotorcraft like unmanned helicopters (!) or quadrotors, or octocopters, or airships ... that all have mutually considerably different flying characteristics and considerably different capabilities (flight endurance, ...), thus also considerably different time constants that are included in the formalism.

Furthermore, current post-disaster operations have definite protocols that, among other consequences, minimise the additional risks, optimise duration and other relevant factors - but the authors completely ignore these. If authors want that drones are included in the post-disaster treatment as they write, then the first step would be to align the drone use with the current operations that include human-operated aircrafts - but that is completely ignored in the text.

There are some unrealistic assumptions and some missing points in the formalism - but having in mind the aforementioned substantial deficiencies, I will not go into further details.

Overall, the presented idea could maybe serve as an exercise in some class for graduate students - but it is definitely unrealistic, partially inherently contradictory.

I suggest to reject the manuscript.

Author Response

We sincerely acknowledge the reviewer for their dedication in the evaluation of our manuscript. We have addressed the reviewer comments and provided two sections for each comment: (i) our reply to the reviewer comments, and (ii) a report of the changes in our manuscript based on the reviewer comment. In this document, we refer to the manuscript the reviewer has revised as the original manuscript, and the manuscript with the modifications generated from the reviewer comments is referred to as the revised manuscript. Besides, note that all the modifications in the manuscript are colored yellow.

Please see the attachment for our responses.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper proposes a method for transporting items using UAVs, using the reinforcement learning methods to construct a supply transportation model. The overall quality of the article is in line with the expectation and has some technical innovations. However, the authors may consider the following comments: 

Abstract

In the abstract, the paper presents the nature and scope of the problem investigated. It is well-known that using UAVs to transport supplies is very important. However, you should limit the question and state your aim.

1. Introduction

The paper provides some background information and context in the second paragraph; however, you should emphasise the problem you are addressing. Limit the scope of your discussion and clearly state your research question.

In addition, you should give a quick preview of the previous UAV approaches for transporting emergency relief supplies.

It should state the reasons for choosing the reinforcement learning model. 

2. Related works

In lines 119-120, the paper indicated, "there has been no research on the efficient transportation of items to shelters by multiple UAVs, that also considers the distribution equity to shelters after a disaster occurs." I do not quite agree with this description. It's a scenario in Multi-UAV cooperative task allocation; perhaps you should make the sentence more credible.

3. Model Description

Compared to other approaches, the difference in the proposed method should be described extensively and with the necessary and appropriate level of detail.

5. Numerical experiments

The experimental part should be explained clearly, including experimental motivation and an experimental scheme.

Experiments are insufficient, and I suggest authors add more results compared to the other comparable approaches introduced. For instance, the compared methods are 2017 or older, and you should use the latest techniques to reach. 

6. Conclusion and future work

I suggest presenting the principles shown by the results more adequately. In addition, the practical application field of the proposed methods and the research findings can be described, highlighting this article's contribution.

Author Response

We sincerely acknowledge the reviewer for their dedication in the evaluation of our manuscript. We have addressed the reviewer comments and provided two sections for each comment: (i) our reply to the reviewer comments, and (ii) a report of the changes in our manuscript based on the reviewer comment. In this document, we refer to the manuscript the reviewer has revised as the original manuscript, and the manuscript with the modifications generated from the reviewer comments is referred to as the revised manuscript. Besides, note that all the modifications in the manuscript are colored yellow.

Please see the attachment for our responses.

Author Response File: Author Response.pdf

Reviewer 4 Report

Short summary:

This paper focuses on the transporting problem using multiple unmanned aerial vehicles and presents a Q-learning-based method to solve the problem in complex environment. From my point of view, the approach which is taken in this paper has good potentials, and also this paper is easy to read in most parts of the text, which is another good point. However, there are several main issues and many more minor issues which cannot be ignored. Due to these issues, this paper cannot be accepted directly. However, the issues can be solved by a careful revise in the paper. Therefore, I suggest a major revision.

Major Comments:

Comment 1

        Since this paper tries to solve the collaborative transporting problem of UAVs in complex environment, it is strongly recommended to introduce the purposes or advantages of collaborative transporting in Section Abstract and Section Introduction. Meanwhile, Since Q-learning is one of the most important methods used in this paper, it should be pointed out clearly and added into the keywords.

Comment 2

From the view of the reviewer, there is not a part to express the novelty in the whole of the article. The reviewer cannot find whether the idea proposed in this paper is better than those in other papers. Please point out the novelty of the paper in the “Introduction” section.

Comment 3

The author should pay more attention to the structure of the article. Special in the Section “Introduction”, the reviewer cannot find the writing objective of Sect. “Introduction”. Sect. “Introduction” should point out clearly which problem is studied in this paper, what are the shortages of the existing methods, how to solve the problem by the authors, and what are the advantages of the approach proposed by the authors. Please rewrite this section.

Comment 4

In Section “Introduction”, I feel the current coverage of the state of the art is not satisfactory as the related work section does not cover many contributions that likely provide the building blocks of the proposed approach on path or mission planning of UAVs. For example, “A clustering-based coverage path planning method for autonomous heterogeneous UAVs” (doi: 10.1109/TITS.2021.3066240), “An Adaptive Clustering-Based Algorithm for Automatic Path Planning of Heterogeneous UAVs”(doi: 10.1109/TITS.2021.3131473), “Coverage Path Planning of Heterogeneous Unmanned Aerial Vehicles Based on Ant Colony System” (doi: 10.1016/j.swevo.2021.101005). All these works focus on the collaborative planning problem of UAVs. It is suggested to cite the above article and analyze the differences.

Comment 5

In Sect. 3, the authors try to provide the necessary problem descriptions, problem assumptions and symbol descriptions of the collaborative transporting problem. However, the authors did not write clearly what the actual problem is. Please define the problem studied in this work after system models are built.

Comment 6

In Section 5, description about the experiments is not perfect. Some experiment parameters should be introduced in this section such the readers know how to repeat the experiments and evaluate the experiment results. For example, how many times was each experiment run? Is each point in the tables represent the average value of several experiment results? Please add more sentences to introduce these parameters in the revised manuscript.

Author Response

We sincerely acknowledge the reviewer for their dedication in the evaluation of our manuscript. We have addressed the reviewer comments and provided two sections for each comment: (i) our reply to the reviewer comments, and (ii) a report of the changes in our manuscript based on the reviewer comment. In this document, we refer to the manuscript the reviewer has revised as the original manuscript, and the manuscript with the modifications generated from the reviewer comments is referred to as the revised manuscript. Besides, note that all the modifications in the manuscript are colored yellow.

Please see the attachment for our responses.

Author Response File: Author Response.pdf

Round 2

Reviewer 4 Report

The authors have addressed all my concerns.