Review Reports - The PAPI Lights-Based Vision System for Aircraft Automatic Control during Approach and Landing

Round 1

Reviewer 1 Report

Abstract:
The abstract provides a good overview about the content of the journal and describes the design of the system. It also explains the methodology used. What could be added is the purpose or target application of the described concept, although this point is described in the subsequent Introduction. The keywords are adequately chosen.

Introduction:
The Introduction proves the need of a system for vertical automatic flight guidance during approach and landing. State of the art systems for landing assistance like ILS and GBAS are mentioned and a concluding overview about the presented concept is given. The research problem and research objectives of the journal paper are clear.

Literature Review:
Selected literature is up to date. The work is based on references to a reasonable extent.
Notice: Literature with a very similar research question exist, which is not mentioned, for example: G. Qingii, L. Jian and Z. Jinning, "Airport PAPI Lamp Image Detection Algorithm Based on Saliency and Configuration Features," 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC), 2018, pp. 1-5, doi: 10.1109/GNCC42960.2018.9019174.

Methodology:
The methodology explained in Chapter 4 allows detection of the number of red PAPI lights so that the vertical deviation from the glide path can be determined.
Problems of the methodology described here could be caused by additional red light sources in the airport area, for example red obstacle lights. A conceivable solution for this would be an additional plausibility check of the binary image by comparing the pixel distances between the detections of the red lights in Figure 8 (in case there are multiple detections of red lights in the image).
Another aspect to consider is that depending on resolution, lateral deviation and distance to PAPI lights separation between the lights might be difficult.
Rules 1 - 12 established in Chapter 5 allow the ALS module to perform a safe approach and landing, as shown in the example scenario in Chapter 6.
The procedure of first performing SIL tests and subsequently HIL tests is a common approach.

Results:
Results are provided for a sample test scenario and show that the Automatic Landing System is capable of performing the approach and landing successfully based on the results of the vision system. It would also be interesting to know if and under which circumstances the system fails or where the system limits are. An example of this would be the behavior under different lighting conditions, for example, at sunset, where there may be false segmentation due to the bright reddish sun. The problem of oversensitivity near the runway threshold is addressed, possible solutions could at least be discussed later.

Discussions:
Chapter 4 describes the determination of vertical deviation using computer vision methods. In addition to the challenges already mentioned due to changing environmental conditions (lighting conditions, reduced visibility, etc.), the proposed method might not be able to separate the individual lights from each other and determine the number of red and white lights, depending on the sensor resolution and the distance to the PAPI lights.
By an additional distance estimation to the PAPI lights (e.g. by triangulation) the distance to the PAPI system could be taken into account, so that the over-sensitivities mentioned in chapter 6 could be compensated.

Conclusions:
The Conclusion appropriately encompasses the contents and findings of the Journal Paper. Research limitations are also mentioned by the reference to oscillations in the vicinity of the runway threshold, but there is no reference to planned research activities in future.

References / Bibliography:
References are used appropriately and are consistent with the format of the journal.

Figures:
Generally appropriate use of illustration to represent the scientific problem. Some of the figures could be revised.
- Figure 3: the frame of the box “VISION SYSTEM” is cut off at the bottom
- Figure 3: Mistake in box “AUTOMATIC LANDING SYSTEM” – “vertical trajectory stabilization”
- Figure 4: text intercepts the box frame in first processing step “load video frame”
- Figure 4: Mistake in box “triple binarization” – “HSV component values”
- Figure 5: Where is electro-optical sensor mounted? It seems that it is located behind cockpit glass, which brings negative influences in the binarization step (see Figure 6b, bottom) due to undesired reflections. Moreover, there is a lack of information: Where and when is Figure 5 taken?
- Figure 20 and Figure 22: Mistake “touchdown”

Author Response

Dear

Hereby we would like to express our gratitude for your effort in reviewing our paper proposal. We highly appreciate your ideas, comments and accurate remarks which allows us to increase quality of the paper.

Thank you in advance for your guidance and expert opinion.

We took into consideration all of Your technical remarks so :

- All figures are corrected.

- Information regarding the airfield, approach direction, and time when the sample picture was taken is added.

- Information about location of the optical sensor is added.

- References are updated.

Regarding the using information about the distance to the threshold and PAPI. We agree it could be taken into consideration to mitigate the result of the variable gain. But the idea was to use only image as far as it was possible to determine the position of the aircraft to the desired glide path and to control the aircraft avoiding other system and sensors. That is why the distance from to PAPI is not introduced into rules.

We agree it would be profitable to check more detail how the system behaves in different light conditions. To be honest, the first qualitative test were conducted. They proved the system is quite sensitive to sunlight conditions. However it is planned to work more deeply with this in next research, including usage different type of cameras and some filters.

Regarding HIL not SIL. We agree that the configuration of the stand could suggest pure HIL type tests. But Please have a note the typical HIL were conducted in next step. The step before typical HIL focused on simulations run on extra PC computer. Tested functions cooperated with simulated environment. So only software was directly involved into simulation directly linked with tested functions. We agree devices, plying only, minor supplementary role but not directly involved into tests of software functions can suggest HIL tests. But we propose to keep it like is, to point the difference between these two tests series. Also clarification is added into the paper.

Yes You are right, an extra red light in the airport could be a problem. We agree with your approach mitigating its adverse impact. We haven’t take it into consideration yet.

Author Response File: Author Response.docx

Reviewer 2 Report

Manuscript "The PAPI Lights Based Vision System for Aircraft Automatic Control during Approach and Landing" presents a good description of interesting solution for automatic landing system. Manuscript is well organized and clearly written with good English.

There are several remarks:

This is obviously a continuation of the previous work by the same group, namely work presented in the references [13] and [16]. It is highly recommended to include in the introduction the details what is this manuscript bringing new in regards to previous work.
There is a MP-02A aircraft mentioned in line 309 - it is not clear what is this aircraft. Please include some details about this aircraft. Is this the aircraft from Fig. 2? Also it would be useful to include some details of its simulation model (mentioned in line 346).
For the description of rules (pages 11 and 12) I don't find appropriate for the values of pitch angle, throttle input, height or velocity usage of terms such as 'small', 'big', 'high' or 'low'. It could be presented in other, more concrete way. Maybe, for pitch angle Theta_d as a percentage of the ideal glide slope, or something similar.
According to the description given in lines 410 and 413, and Figure 16 this would also be a HIL simulation, not a SIL simulation. It would be useful to include some details or to refer to a reference about the autopilot used, what feedback it implements ...
A sensitivity problem during the final phase of landing, as pointed out in line 469, could present an issue. Beside the presented results during the final phases of landing it would be interesting to see the values of load factor. It would be valuable to present some thoughts on how to mitigate this sensitivity problem.
It would be useful to support a conclusion given in line 522: "Control system behaviour was very similar to human behaviour during the realisation of the final approach.". Is this conclusion for a specific variable? If there is available recording of manually controlled landing, their comparison could be included.
For the sake of completeness it would be useful to mention possibilities for the lateral control for automatic landing.
For references given in line 60: a recommendation is to decouple given references [5,6,7,8] - please specify what each reference refers to. Similarly in other places in the manuscript, specially in lines 116, 144, 166, 202, 210, 238, 405.
Reference to Figure 2 in line 125 is not clear.

Author Response

Dear

Thank you in advance for your guidance and expert opinion.

Regarding remarks about previous works and references to works [13] and [16]. We reconfigured and updated the references and added in the Introduction the clarification what novelty is presented in the paper.
The aircraft presented in the Figure 2. Is the same aircraft we refer to later in the text. Also a mathematical model of this aircraft is used in the paper. The model was built on the basics data gathered in dedicated test flights. Clarifications and links between some information about the aircraft are put in the text.
Regarding HIL not SIL. We agree that the configuration of the stand could suggest only HIL type tests. But Please have a note the typical HIL were conducted in the step three. The step before typical HIL tests focused on simulations run on an extra PC computer. Tested functions cooperated with simulated environment. So only software was directly involved into simulations directly linked with tested functions. We agree devices, plying only minor, supplementary role but not directly involved into tests of software functions can suggest HIL type tests. But I propose to keep it like is to point the difference between these two tests series. Also the clarification is added into the paper
Sensitivity issue near to the PAPI light line. Meanwhile we have investigated this issue. The proposal is to define a “pseudo distance” from the aircraft to the PAPI resulting from the size of the PAPI on the image. It could provide possibility to decrease the gain in control laws when the aircraft is near to the PAPI line.
The autopilot is used as a tool only its goal is to stabilize pitch angle and forward an engine control signal to an actuator only. The autopilot is not an objective of the work. It uses typical feedbacks: pitch angle, pitch rate to control the aircraft attitude in the longitudinal channel.
“Load factor” is of course a mean to evaluate the flight quality. Unfortunately we didn’t investigate it. Will be in the future.
There are pilots in the research team being experts, so as it is stated in the paper many assessments is done by pilots-experts. However recorded landings are composed of all crucial phases presented on figure 14.
Decupling of reference indexes is not always reasonable just like in cases you suggest, Because there are similar information included in cited works. We decoupled them in some cases but we propose leave as is in other ones.
Our works also take into consideration lateral control (Calvert lights are used) It is a material for other papers.

Author Response File: Author Response.docx