A Hierarchical Deep Reinforcement Learning Approach for Throughput Maximization in Reconfigurable Intelligent Surface-Aided Unmanned Aerial Vehicle–Integrated Sensing and Communication Network

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

1、The article is slightly vague in introducing its innovative contributions. Although the article briefly mentions the importance and challenges of ISAC networks, the authors could clarify in more detail why this research problem was chosen and how the approach overcomes the shortcomings of existing solutions. It is recommended that the authors add a paragraph in the introduction section to clarify the comparison with the current literature and highlight the uniqueness and improvement of the proposed methodology.

2、The literature review section can be further expanded, especially regarding new developments and the latest technological approaches in the field in 2023 and 2024. It is recommended that the authors add more recent literature to strengthen the theoretical background of the article and reflect the cutting-edge of the methodology.

3、Some graphs in the Simulations section are not clear enough, such as, the curves in Fig. 4(a) and Fig. 5 are visually too close, making it difficult to distinguish the performance of different algorithms. It is suggested that different colors or markers should be used to differentiate them. In Fig. 6, the data of the algorithms fluctuates a lot, especially before 200 episodes, and the fluctuations of multiple algorithms partially overlap. It is possible to consider smoothing the curves to make the trend clearer. The location of the legend may obscure part of the data, and it is recommended that the legend be placed outside the chart or in a part that does not overlap with the data, so as not to interfere with the interpretation of the data.

4、Some sentences are overly complex and contain grammatical errors,such as, on page 13, “When using DRL, we can observe that TD3 adopts a significant range of movement, while the moving distance of the HTD3 is obviously shorter than the TD3...” This sentence is too long and complex for English writing conventions, where shorter, clearer sentences are generally preferred. I suggest breaking it into multiple sentences to improve readability. Also, there are multiple grammatical errors in this sentence, for example, optimize, choose, and perform should be three single forms. This sentence is just an example. There are multiple similar errors in the article, so I suggest you look them up and fix them.

5、Some formula explanations lack clarity, such as, in page 14, formulate 14, “a probabilistic line-of-sight channel is used” would be clearer if it explicitly described the path loss expressions PLU,k[n] and PLR,k[n]. Additionally, in section 2.4, new conditions introduced in formulas should include a detailed explanation as they were not defined previously.

Comments on the Quality of English Language

English should be improved, including corrections for grammatical errors and enhancements for language smoothing.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

-The abstract should quantize the performance of the authors proposal when compared to other solutions

-More references can be added to the introduction in order to strength the importance  of this work

-The last two figures must be better described, since the informartion provided it not enough 

-Figure 8 is place at the conclusions section and that is not supposed

-Conclusions must be improved to better describe the work carried out by the authors, since the the afirmations are too simple and does not provide a comprehensive view to the reader how this work can be seen as an incremetal contribution.

-Topics of future work are not covered  in the paper and considering its content must be addressed.

-Some reference are not well written considering the template rules.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Integrated Sensing and Communication (ISAC) is a technology that merges the capabilities of traditional communication and sensing systems into a unified framework. It enables the use of radio signals to both transmit and receive data, as well as to "sense" or detect and identify objects and surfaces within a surrounding environment. This convergence allows for more efficient use of resources, improved performance, and the creation of new applications that were not previously possible.

There are many requirements for accurate sensing such as Range and Angular Resolution, Doppler Resolution (ability to differentiate between objects with different velocities), and environmental robustness. On the other hand, communication requirements are reliability, low latency, and high throughput.

The design of sensing and communication components must be tightly integrated to optimize overall system performance. However, in this paper these two are treated independently. It is not clear how the RF resources and power are shared. Also, precise synchronization between sensing and communication operations is essential for accurate information extraction. This factor is not considered. Furthermore, environmental impacts (although mentioned in the introduction) are not incorporated in the system model. Also, the adherence to specific constraints (e.g., maximum transmission power, flight speed, and minimum sensing beampattern gain) are overly simplified.

More importantly, the sensing performances such as Doppler, Range and Angular Resolution, Sensitivity, Detection Range, False Alarm Rate and Miss Detection Rate are not evaluated.  In essence the paper focusses on communication performance while the term ISAC without proper justification. 

 

Comments on the Quality of English Language

Also, there are several typos and English mistakes.

1. **Sentence Structure and Clarity Issues: "Hence, we can also observe that when the number of UEs is 4 or 6, we achieve the highest sum-rate throughput, this is because more UEs lead to insufficient communication resources, which causes the decrease of the accumulated sum-rate throughput of the system."

- This sentence is overly long and could be clearer if split into two sentences.

2. Spelling and Grammar or Formatting Typo: "we consider the scenario with single UAV" should be "we consider the scenario with a single UAV" for grammatical correctness. There are many such issues. Redundant Phrasing and improper punctation are also found.

- "Although the aforementioned works have achieved extraordinary performance, there still exists several challenges, Firstly, the design difficulty of the system integrating UAV, ISAC, and RIS simultaneously is relatively high."

- "There still exists" should be "there still exist" to agree with the plural subject "challenges."

- "Firstly" should be "First" or "First of all" to maintain a formal tone in academic writing.

- "However, we just considered the scenario with single UAV, in the future, multi-UAV scenario will be considered..." should be "However, we have only considered the scenario with a single UAV; in the future, a multi-UAV scenario will be examined..." for improved clarity and grammatical correctness.

 

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

The paper is improved from the previous version. However, it is not enough. Authors have not evaluated the performance of sensing, therefore using the word ISAC on the title is not technically correct.  Somewhat evaluate the sensing performance or revise the title.

In Equations (5), (8) and (9), parameters E_LOS and E_NLOS are used. What are the values considered for these parameters in simulation? What is the justification for the used values. Please add a table of all used simulation parameter values.  

Furthermore, the trajectory of the UAV in Fig 7 is evaluated under a very simple scenario, only two receivers and the sensing area is away from communication area. This is not usually the case. Also, explain physically how the RIS is fixed up in the sky? Resolution of Figure 8 must be improved. I could not read the PDF irrespective of how much I zoomed. In Figure 5, the sum rate throughput increases with transmit power. This is trivial (as expected), but what is the cut-off? In fact, when the communication transmit power increases, the sum rate throughput will also increase but the sensing performance will decrease. Therefore, please incorporate the sensing performance in Figure 5. In essence, the paper cannot be accepted without some performance metric on the sensing performance.    

Too many references. Only cite the key references.

Comments on the Quality of English Language

Significant improvement of English writing is needed. For ex: Page 15, paper says, We can observed...?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf