Integrated Guidance-and-Control Design for Three-Dimensional Interception Based on Deep-Reinforcement Learning

Round 1

Reviewer 1 Report

In general, the paper submitted for review entitled „Three-dimentional guidance and control design adaptive to multi-constraints based on deep reinforcement learning” I find as interesting voice in the discussion of issues related to modern methods of guiding missiles. Nevertheless, a few issues raise my doubts, hence I kindly ask the authors to comment on the following issues:

1. Lines 36-39. PN is a generalization of all basic missile homing (i.e. two-point) methods, hence its popularity. The three-point method is not associated with homing issues discussed in the article and should be in my opinion removed from the text to keep the consistency.

2. Line 143. Assumption 4 is not clear to me. What do the authors mean by “pneumatic parameters”?

3. How are uncertainties listed in line 171 modelled?

4. Line 178: “a normal layout rudder” – what does it mean? Do the authors mean: “a normal aerodynamic scheme”?

5. Line 223. It is not entirely clear to me which element of the DDPG algorithm is the author’s idea.

6. Table 2. Target velocity -700 m/s is not a minimum value, and only informs about the direction of the velocity vector. Same with accelerations. Use value modules or change table header descriptions.

7. Eq. (4-8) and line 377. I suggest replacing the value 0.628 with pi/5. It looks better :) 

8. Figure 4. Doubts arise here. The chart suggests that the missile launches took place at different angles for PN and DDPG. This fundamentally changes the initial conditions of the guidance process. Please run simulations for the same launch angles for PN and DDPG or comment on the presented results.

9. Lines 347-351. The miss distance depends on many parameters of the guidance loop, e.g. phase delays caused by the elements of the seeker and signal processing path, missile airframe passband, inertia of the fin servos, etc., but does not depend on the chosen guidance law. As a rule, a correctly formulated guidance law considered in terms of pure kinematics (missile and aerial target considered as material points) does not introduce any guidance errors, i.e. R = 0 always occurs at the moment of meeting the missile with the target. Of course, in real conditions R = 0 does not hold for the reasons outlined above. Therefore, my questions: did the authors verify the correctness of the calculations for the case of pure kinematics? Was the missile seeker modeled and how? It seems that in this case the angle of approach to the target may be crucial for the miss distance, and as I mentioned, the Fig. 4 suggests that the missiles were launched at different angles for PN and DDPG. The authors did not provide any data on the missile model apart from the general formulas known from the literature. Therefore, it is difficult to refer to the simulation results.

10. Lines 351-354. The difference is less than 0.1 second at a distance of 3 km. The conclusion is too hasty.

11. The authors did not provide any data on the missile model apart from the general aerodynamic formulas, cf. Eq. (2-8). Therefore, again, it is difficult to refer to the simulation results. E.g., doubts in Figs. 5 and 7 are raised by the unusually large values of the angle of attack and sideslip (> 10 degrees, 40 deg in some cases!!!). In typical solutions, values up to 5-6 degrees are achieved. Please check the missile model and recalculate or provide data (calculations) to prove the correctness of the presented results.

12. In Figs. 5 and 7, strong fluctuations of the missile control fins during guidance to the target are observed. This is not energetically favorable and should have an influence on the missile velocity Vm, and thus – for the interception time (PN vs. DDPG). Did the authors investigate this aspect?

13. In my opinion, the article needs to be corrected in terms of the specialist terminology used. The authors use, for example, the words “projectile” (cf. e.g. lines: 33, 150 and others) and “bomb” (cf. e.g. line 269) interchangeably with the word “missile” (used correctly). The word “projectile” refers to issues related to artillery. Similarly, the word “rudder” describes the control element of a ship or boat, in the case of a missile, the word “fin” is more appropriate. The authors also use the phrase “off-target”, probably meaning “miss distance”. Please verify the entire text in terms of the correctness of specialist vocabulary.

14. Please remove the multiply sign from some equations, e.g. (2-6), (2-7), (4-1), (4-2), etc. (inconsistency of writing). I suggest to remove all commas from lines with equations. First of all, the authors use them not consistently. However , the most important thing is that the reader may be confused in some situations, e.g. in Eq. (2-4) the comma looks like derivative sign.

15. Please correct the text in terms of editing. There are some of typos in the text (e.g. “Wilianm”, “side slip” and “sideslip”, etc.). Please remove Section 7 (“Patents”) or provide the required information.

At this time is all for me, in the future, I hope that authors consider all of my suggestions and comments.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors designed IGC based on DRL for target interception. This is an interesting topic but it should be improved for a clear description of the method. I have following comments:

1. First, the abstract should be rewritten, highlighting the main idea, contributions and motivation. The authors describe the target interception using DRL, but it is not discussed in the abstract. The authors use the words missile, bomb, and target but the main idea or outcome is not clear which target interception. moreover, the word bomb should be removed. Because Figure 1 also depicts, the concept of missile-to-target engagement.

2. Similarly, the title should be changed which must include the main objection of the paper. So, the reader can easily find your paper after successful publication.

3. line 118 ~ 124: Please use the present tense, since it is your current contribution.

4. line 339 and 340: target has ... missile has

5. line 348: what is meant by off-target? 0.66m is a miss-distance?

6. why do the rudder deflection angles based on the DDPG fluctuate more than that of PN? rudder deflection in z-direction seems high in the case of DDPG. (Fig. 5)

7. Figure 7h, side slip angle curve seems high at the end in the case of DDPG. write the full form of DDPG first.

8. Fig 8: is the figure shows the percentage of miss-distance? what is meant by off target volume.

9. the number of references is not enough. I recommend including some references. For example:

A split target detection and tracking algorithm for ballistic missile tracking during the re-entry phase [2020]

thank you for your contribution and best of luck.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

The introduction of the paper is well-written, and the revision of the literature is reasonable. The kinematic equations for the missile trajectory and the equations for the target trajectory are simplified. In general, the variables and parameters are not described in the manuscript and the authors systematically quote reference [15]. This makes the r reading of the paper difficult and inconsistent. Reference [15] is not complete and has no date. Figure legends are rather telegraphic and are not informative. The example shown for the missile interception with the target is naively simplified.

Therefore, I cannot recommend this paper for publication in its present form. A major revision is needed.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thank you for your replies and for including the corrections.
I wish the authors success in their further scientific work.

Reviewer 2 Report

All comments have been addressed as per requirement. I have no further comment. 

Thank you and best of luck

Reviewer 3 Report

With this revision, the paper can be accepted for publication.