To minimize interference and mitigate risks associated with on-site validation during airport operations, a 1:10 scale model scenario of a Class C aircraft stand, depicted in
Figure 3b, was employed for algorithm validation. The chosen aircraft stand scale model closely mimics the real environment, incorporating accurate proportions and details of the aircraft stand size while accurately representing the connected structural elements. Furthermore, a corresponding aircraft model at a 1:10 scale was utilized to ensure experimental realism. The algorithm is implemented on a Robot Operating System (ROS)-enabled vehicular test platform, which operates on Ubuntu 18.04 and is powered by a quad-core ARM A57 processor clocked at 1.43 GHz with 4 GB of RAM. It is important to note that different hardware configurations may show varying performance. The test vehicle, also at a 1:10 scale, is equipped with a suite of sensors including 2D Lidar equipment, an RGBD camera, a distortion-free RGB camera, an odometer, and an IMU module, as illustrated in
Figure 4. The 2D Lidar equipment used is the RPLIDAR A1M8, which has a measurement radius of 0.1–12 m, a sampling frequency of 8 K, and an angular resolution of <1°. Specific device information is provided in
Appendix A.
7.1. Verification of Detection Algorithm for Unpiloted GSE
- (1)
Experiment on Virtual Channel Boundary Line Detection
Firstly, the boundary line detection algorithm’s parameters were designed with careful consideration. The color processing threshold of HSV was established based on the yellow virtual channel boundary lines allocated within the aircraft stand, as detailed in this study. The specific threshold design is presented in
Table 1.
In the Gabor filter, a kernel size of 7 × 7 was experimentally chosen, and six different Gabor filter kernels were generated in the angle interval π at a step of π/6. For the lambda parameter, ten different filter kernels were generated within 2 to 4 at a step of 0.2. The filter kernel with the best detection results was chosen, which had an angle of π/2 and a lambda of 3.
To evaluate the performance of the proposed algorithm, an experimental dataset consisting of 1000 frames was constructed. The image size of this dataset is 640 × 480, recorded at a rate of 24 frames per second.
The detection algorithm demonstrated remarkable effectiveness in most experimental scenarios. Specifically, it accurately detected 89% of the frames, missed less than 5% of the frames, falsely detected 3% of the frames, and failed to detect the remaining 3% of frames. The algorithm can handle challenging scenes, such as those under low light and exposure conditions. The average computation time on the ROS vehicle was less than 100 milliseconds, excluding camera read and decoding time.
The algorithm’s computation results are illustrated in
Figure 9 and
Figure 10. Poor detection results were classified into three categories: missed detection, false detection, and detection failure. According to the literature [
43], missed detection refers to instances wherein more than 10% of the virtual channel boundary lines in the image are undetected, false detection refers to instances wherein sections of the image not belonging to the virtual channel boundary lines are detected, and detection failure refers to instances wherein the virtual channel boundary lines are completely undetected.
Figure 10a–c represent missed detection,
Figure 10d,e represent detection failure, and
Figure 10f illustrates false detection.
During the experiment, an unusual instance of false detection occurred, wherein a section of the floor was erroneously identified as the virtual channel’s edge, as illustrated in
Figure 10f. However, this did not affect our vehicle’s performance since the inner edge of the channel was fully recognized. Considering that the GSE does not interact with the boundary’s inner edge during operation, the majority of missed detections were primarily attributed to poor lighting conditions in the environment (
Figure 10a,c). The suboptimal detection outcomes largely stemmed from the exceptionally low contrast of the markings delineating the virtual channel boundaries. From a human perspective, the virtual channel boundary lines can sometimes be challenging to discern (
Figure 10c). However, it is worth noting that the algorithm proposed in this paper effectively handles other challenging lighting conditions (
Figure 9b,c). When the vehicle is turning, the absence of an anti-shake algorithm in the camera, combined with high speed, substantial image shift, and significant distance from the virtual channel boundary lines, can potentially result in detection failure (
Figure 10d). Therefore, further research is warranted to improve the detection of virtual channel boundary lines under these specific conditions.
The quantitative results are summarized in
Table 2.
- (2)
Experiment on Turning Induction Marker Detection
Figure 11a,b illustrate the detection results of two different turning induction markers. Across 30 experimental trials conducted under diverse environmental conditions, we achieved remarkable detection performance, with a success rate surpassing 95%. Only one instance of detection failure occurred under dimly lit conditions. Considering the practicalities of apron scenarios, embedding the turning induction markers within the pavement and integrating lighting installations could serve as effective strategies to ensure reliable identification performance.
- (3)
Experiment on Object Detection in Aircraft Stands Using Improved YOLO
To satisfy the operational requirements of Unpiloted GSE, we developed an improved method for the object detection network during GSE operations, as explained in
Section 5.2. Acknowledging the scarcity of datasets dedicated to GSE operations within aircraft stands, we simulated the perspective of Unpiloted GSE operations using an aircraft stand sandbox. This simulation enabled the creation of a unique dataset for object detection in aircraft stands by Unpiloted GSE. Comprising 300 images with a resolution of 640 × 480 pixels, this dataset includes three detection object types: Passenger_door, Person, and Vehicle, as illustrated in
Figure 12. Considering the relatively small size of this database for neural network-based image processing algorithms, we employed data augmentation to increase the image count, given that the original images already exhibit diverse targets in different poses.
The accuracy of object detection is primarily evaluated based on its
Precision,
Recall, and
F1-Score. In object detection,
Precision refers to the proportion of correctly detected targets to the total number of detections during Unpiloted GSE operation.
Recall represents the proportion of samples correctly predicted as positive samples to the actual number of positive samples. Moreover, the
F1-Score combines
Precision and
Recall, serving as a measure to balance these two values. The formula for the
F1-Score is as follows:
Based on these metrics, experiments were conducted on the YOLOv5s model and the improved YOLO detection model, with the results presented in
Figure 13 and
Table 3.
The experimental results indicate that the lightweight network employed did not exhibit a significant reduction in
Average Precision (
AP) or
F1-Score. A comparison of the data presented in
Table 3 demonstrates a substantial decrease in the computational complexity of the enhanced network, evidenced by a reduction of 14.2
GFlops, which represents only 14% of the original value. Regarding the parameters, there is an 80% reduction in the improved network compared to the original YOLOv5s. In summary, the refined YOLOv5 object detection network significantly improves real-time performance without noticeably compromising the model’s detection accuracy and arguably enhances its detection stability. In the context of embedded devices for Unpiloted GSE on the apron, the revised model outperforms the original YOLOv5s model in fulfilling the operational requirements of aircraft stands.
7.2. Verification of Vehicle Control Algorithm for Unpiloted GSE
- (1)
Verification of GSE Obstacle Avoidance Algorithm
In this experiment, obstacles within the aircraft stand were categorized into two classes: Person and Vehicle.
Figure 14 presents a schematic diagram illustrating the YOLO object detection algorithm’s recognition of obstacles.
Initial experiments concentrated on the dynamic obstacle avoidance strategy, where multiple dynamic obstacles were placed at different distances along the Unpiloted GSE’s path. This setup aimed to simulate various potential obstructions within a real-world airport environment. The results demonstrated that the Unpiloted GSE could successfully execute braking maneuvers upon encountering a dynamic obstacle within its minimum stopping distance () and resume navigation once the obstacle moved beyond this safety threshold. Additionally, the experiments revealed that the GSE could still implement effective avoidance actions even when dynamic obstacles suddenly appeared on its path.
Subsequent experiments assessed the static obstacle avoidance strategy. The analysis of these experiments, depicted in
Figure 15, suggests that the strategy can accurately identify obstacle positions and initiate avoidance maneuvers at a predetermined distance (for this experiment, avoidance commenced at a distance of
a = 0.5 m, corresponding to an actual distance of 5 m, considering that the sandbox’s scale is one-tenth of the actual stand). The vehicle’s turning angle was adjusted based on the obstacle’s length, with a larger angle adopted when navigating around vehicle obstacles to maintain a safe distance, thereby enabling the GSE to re-enter its original virtual channel.
Upon analyzing the experimental results, it was observed that when a vehicle acts as the static obstacle, the outer edge of the obstacle is at a minimum of 0.15 m (corresponding to 1.5 m in real-world conditions), a maximum of 0.26 m, and an average of 0.16 m from the vehicle. Conversely, when a person is the static obstacle, the vehicle’s outer edge maintains a minimum distance of 0.16 m, a maximum of 0.22 m, and an average of 0.17 m from the person. This ensures adherence to the clearance requirement of 1.5 m between the vehicle and the obstacle, as specified by the International Civil Aviation Organization (ICAO) in Annex 14 [
44]. Additionally, the experimental data highlighted a certain degree of variation in the vehicle’s turning angle each time. For instance, in Experiment 5 of
Figure 15a, the vehicle initiated its turn before reaching the obstacle at a 0.5 m distance, possibly due to the absolute error of 0.02 m in the Lidar. Furthermore, since the experiment was conducted within the confined space of an aircraft stand sandbox, and the vehicle’s turning radius could not be proportionally reduced, this resulted in the vehicle’s turning correction occurring more slowly and the required distance being longer than the actual operational distance.
- (2)
Verification of GSE Docking Control Algorithm
The process of GSE automatic docking with the cabin door relies on real-time positioning within the ROS, using odometer and IMU sensors. The vehicle’s turning angle is calculated and adjusted based on the cabin door’s position until it reaches the docking point. Considering the clearance requirements specified by the ICAO for Ground Support Equipment during aircraft docking [
44], and taking into account the experimental environment, scaled proportionally to real airports, the experimental vehicle was considered to have reached the docking position when it was within a 3 cm radius of the docking point.
The location of the vehicle’s Lidar installation, positioned 0.9 m from the vehicle’s leading edge, requires the addition of this distance when assessing the vehicle’s proximity to the cabin door. Moreover, since this research simulates an autonomous lift platform vehicle, the distance between the lift’s front end and the vehicle’s leading edge should be accounted for after the second deceleration. Based on the average GSE size, this distance is standardized to 1 m in this study. Consequently, when the vehicle is 5.4 m from the aircraft, it should decelerate to 5.00 km/h (1.38 m/s); when it is 2.4 m away, it should decelerate to 0.80 km/h (0.22 m/s); and upon reaching a distance of 1.9 m from the aircraft, it should halt. Due to the spatial constraints of the sandbox in the experimental environment, these distances are reduced by a 1:10 scale to align with the sandbox scale. As for vehicle speed, considering the safety of the docking experiment and the braking performance of the experimental vehicle, it is scaled down by a 1:100 factor.
Tests were conducted on the automatic docking process.
Figure 16a shows the vehicle’s initial position, (b) illustrates the vehicle’s endpoint position after docking with the aircraft, and (c) denotes the standard docking position of the vehicle. The distance error between the actual point and the target point is ±3 cm, which is consistent with the set threshold and meets the industry’s docking accuracy requirements for GSE.
We plotted the trajectory of the vehicle in ten experiments, as depicted in
Figure 17. This figure showcases the vehicle’s trajectory while performing an automatic docking procedure with the cabin door. The X and Y axes in the figure represent the vehicle’s current positional coordinates, considering that the scale of the aircraft stand is one-tenth of an actual aircraft stand. Overall, the vehicle’s movement trajectory is remarkably smooth, ensuring a stable approach to the docking point during actual operations.
However, the consistency of the vehicle’s trajectory in the straight section before turning was not always maintained. This inconsistency arises from the vehicle’s need to adjust its position to remain centrally aligned within the virtual channel. Due to potential errors in the vehicle’s position and orientation each time it is situated at the starting point, the vehicle must rectify its trajectory towards the center of the virtual channel post-initiation.
The vehicle identifies the right-turn position by detecting various turning induction markers. To avoid exceeding the boundary of the virtual channel, the vehicle advances a short distance after recognizing the turning induction marker before executing the right turn. The docking position is typically not directly ahead following the vehicle’s right turn, as indicated by the figure, so the vehicle tends to make a rightward adjustment after the right turn to align with the docking point. Additionally, it was observed that the initial positions of the vehicle’s right turn may not be consistent, possibly due to variations in lighting or other environmental factors during the recognition process, leading to a minor deviation (within 0.1 m) between the actual and ideal paths. However, such a discrepancy is insignificant and can be disregarded.
An error analysis was conducted on the vehicle’s docking point accuracy, as illustrated in
Figure 18.
Figure 18a,b present histograms of the horizontal and longitudinal errors after docking by the Unpiloted GSE, while
Figure 18c shows a box plot of these errors. The experimental data reveal an average horizontal error of 0.02 m, with a maximum of −0.05 m, a minimum of 0 m, and a standard deviation of 0.02. In the longitudinal direction, the average error was 0.03 m, with a maximum of −0.07 m, a minimum of 0 m, and a standard deviation of 0.03. Further examination indicated that errors were more pronounced in the fifth and eighth trials, with the vehicle’s final halt deviating from the docking point by 0.04–0.05 m. This deviation primarily arises from the vehicle’s use of standard 2D Lidar with an inherent measurement error of approximately 0.02 m, and the absence of a GPS module, relying solely on an odometer and IMU for navigation. Such limitations may not fully compensate for human error during the vehicle’s initial placement, potentially resulting in centimeter-level discrepancies in position. Nevertheless, a maximum deviation of 0.05 m falls within operational tolerances and meets the industry’s docking accuracy requirements.
Figure 19 presents the angular deviation between the vehicle and the aircraft after docking (with positive values indicating right deviation and negative values symbolizing left deviation). Due to the presence of both positive and negative angular fluctuations in this experiment, we conducted twenty trials. The results indicate a deviation ranging from −6° to 11°, with an average of 4.58°. This angular deviation primarily arises from the horizontal displacement between the vehicle and the aircraft door during the terminal docking phase. Despite the vehicle’s continuous angle adjustments, its proximity to the aircraft prevents complete orientation correction, resulting in an angular deviation. The door position data is based on the output of the YOLO detection algorithm, and variations in the image’s center point may lead to residual horizontal errors between the vehicle and the door in the final phase—an issue that requires further investigation. From a practical standpoint, an average deviation of 5° meets the docking precision requirements.
Figure 20 illustrates the relationship between the vehicle’s speed and distance. Ten experiments were conducted to collect data, and given the minimal fluctuations observed in the experimental results, we consider these ten experiments to be representative of actual operating conditions. As shown in the graph, the vehicle can travel at a speed of 0.027 m/s when it is more than 0.54 m away from the aircraft (corresponding to an actual distance of 5.4 m). The vehicle initiates braking when it is 1 m away from the aircraft (according to actual regulations, the vehicle should take the brake test when it is 15 m away from the aircraft, which corresponds to 1.5 m in the model aircraft stand. However, the vehicle’s turning radius could not be proportionally reduced, resulting in the distance to the aircraft being less than 1.5 m after turning, hence the decision to take the brake test at 1 m). Between the distances of 0.54 m and 0.24 m from the aircraft, the vehicle operates at 0.0138 m/s, decelerating to 0.0022 m/s when the distance is less than 0.24 m, until it halts at a distance of 0.19 m from the aircraft. At the first deceleration, the maximum and minimum distance deviations are 0.02 m and 0.004 m, respectively, with an average deviation of 0.01 m. Similar values are observed at the second deceleration, with the minimum distance deviation being 0 m. These deviations can be primarily attributed to the Lidar’s measurement accuracy of 0.02 m, as all offsets fall within this range. Additionally, the vehicle’s speed remains consistent during operations, indicating effective control over docking speed.
It is important to note that the algorithm verification in this paper was conducted in the aircraft stand sandbox, which may result in some discrepancies compared to real-life scenarios. These differences can be summarized as follows:
During the obstacle avoidance experiment, the turning radius of the vehicle cannot be proportionally reduced, resulting in slower correction of the vehicle’s turning and requiring a longer distance than the actual running distance. However, this issue will be alleviated in real-life scenarios.
The trajectories of the vehicle in the straight line before turning are not always aligned. This is due to human error in placing the vehicle at the starting position for each experiment, resulting in inconsistencies in the position and angle of the vehicle. Therefore, the vehicle needs to correct its trajectory to the center of the virtual channel after starting. In real situations, this issue will be alleviated.
As the vehicle lacks a GPS module, it relies solely on an odometer and IMU for positioning, which introduces some errors. In outdoor scenarios, the vehicle may require the addition of a GPS module to assist in correcting the positioning system, thereby obtaining more accurate navigation and positioning results.