Advances in Aircraft Skin Defect Detection Using Computer Vision: A Survey and Comparison of YOLOv9 and RT-DETR Performance

Abstract

Aircraft skin surface defect detection is critical for aviation safety but is currently mostly reliant on manual or visual inspections. Recent advancements in computer vision offer opportunities for automation. This paper reviews the current state of computer vision algorithms and their application in aircraft defect detection, synthesizing insights from academic research (21 publications) and industry projects (18 initiatives). Beyond a detailed review, we experimentally evaluate the accuracy and feasibility of existing low-cost, easily deployable hardware (drone) and software solutions (computer vision algorithms). Specifically, real-world data were collected from an abandoned aircraft with visible defects using a drone to capture video footage, which was then processed with state-of-the-art computer vision models—YOLOv9 and RT-DETR. Both models achieved mAP50 scores of 0.70–0.75, with YOLOv9 demonstrating slightly better accuracy and inference speed, while RT-DETR exhibited faster training convergence. Additionally, a comparison between YOLOv5 and YOLOv9 revealed a 10% improvement in mAP50, highlighting the rapid advancements in computer vision in recent years. Lastly, we identify and discuss various alternative hardware solutions for data collection—in addition to drones, these include robotic platforms, climbing robots, and smart hangars—and discuss key challenges for their deployment, such as regulatory constraints, human–robot integration, and weather resilience. The fundamental contribution of this paper is to underscore the potential of computer vision for aircraft skin defect detection while emphasizing that further research is still required to address existing limitations.

1. Introduction

Commercial airplanes undergo regular inspections and maintenance checks to meet operational safety standards and regulatory requirements before flying. During general visual inspection (GVI) tasks, maintenance, repair, and overhaul (MRO) operators carefully examine airplanes for defects that could compromise safety and performance [1,2]. These inspections focus on critical aircraft components, including the fuselage, structural elements, landing gear, tires, and control surfaces. Identifying defects in the aircraft’s exterior (or skin), such as cracks, corrosion, dents, or missing heads (or missing rivets), is particularly relevant for maintaining aircraft’s structural integrity, preventing further deterioration, and mitigating risks to underlying systems.

The degree of each inspection varies based on the aircraft’s operational phase. Pre-flight and post-flight inspections, conducted before and after every flight during the aircraft’s turnaround, are typically brief and targeted, lasting approximately 10 to 30 min depending on the aircraft type and available turnaround time. In addition to these rapid checks, more extensive routine inspections, categorized from A to D checks are mandated by aviation regulations and occur at varying intervals [3]. Additional inspections may be necessary if reports from pilots or sensor data indicate potential defects or anomalies during operation, or following severe events. Type A checks are performed every 1 to 2 months and take approximately 6 to 12 h, while D checks, the most comprehensive, occur every 6 to 10 years and can span 3 to 7 days. These intervals are general estimates and may vary based on factors such as pilot- or crew-reported issues, aircraft usage, age, and type. Type B and C checks fall somehow in between. B checks are typically performed every 800 to 1500 flight hours (approximately 6 to 8 months), spanning 1 to 3 days of inspection. C checks occur every 3000 to 6000 flight hours (approximately 18 months to 2 years).

Currently, aircraft surface inspections are performed manually by teams of licensed MRO operators and engineers who must undergo extensive training and certification before being qualified for such tasks. While these inspections require a high level of precision, they are also physically demanding, especially when accessing challenging areas such as the upper surfaces of airplanes. Additional equipment, such as scaffolds or aerial platforms, are often necessary, adding to the cost, time, and labor-intensive nature of the process [4,5]. Moreover, manual inspections are prone to human error and inconsistencies, which can affect the overall reliability of defect detection [6].

To address these limitations, industry and academia are increasingly focused on integrating computer vision technologies into defect detection processes [7,8]. This push has given rise to a variety of innovative solution, such as (i) unmanned aerial vehicles (UAVs) for capturing real-time images of hard-to-reach defects, (ii) mobile or climbing robots for navigating aircraft surfaces, and (iii) smart hangars equipped with advanced camera systems to capture images from multiple angles.

Academic research in this field has seen rapid growth, particularly in the development and application of object detection and image segmentation algorithms for identifying surface defects on aircraft. A review of the existing literature, summarized in Section 2, identified 21 research papers focused on such algorithms. Notably, about half of these studies were published in 2024 (the year this paper is being written), highlighting the growing urgency and relevance of this topic.

In parallel, the aviation industry has been investing in research and development to operationalize computer vision-based inspection systems. Projects span both hardware, such as UAVs and robotic systems, and software, including advanced computer vision algorithms. Our review, summarized in Section 2, uncovered 18 publicly documented industry projects across different companies and regions. This likely represents only a fraction of ongoing efforts, as many initiatives are not disclosed publicly.

The surge in interest in automated aircraft defect skin inspection can be attributed to recent advancements in computer vision technologies, which have significantly improved accuracy and efficiency within a short period of time—for a detailed benchmark analysis of various object detection algorithms since 2015, refer to [9]. Moreover, the post-COVID-19 period has introduced labor challenges, with many licensed inspectors retiring or transitioning to other roles, further driving the need for automated solutions in aircraft inspection.

The aim of this research is to explore the current state-of-the-art in hardware and software solutions for aircraft skin defect detection. Beyond conducting a detailed review of the academic literature and industry innovations, we also experimentally assess the accuracy and feasibility of existing low-cost (both in terms of financial cost but also ease of deployment) hardware and software solutions. To that end, real-world data were collected from an abandoned aircraft with visible defects using a drone to capture video footage. The footage was then processed using widely adopted computer vision algorithms, specifically YOLOv9 [10] and RT-DETR [11], and their accuracy was evaluated.

In summary, the main contributions of this paper are as follows:

• Thorough review of existing hardware and software solutions for aircraft defect detection: The investigation highlights significant growth in academic research on computer vision for aircraft defect detection. Notably, all academic references are dated from 2019 onward, with over half of the publications appearing in 2024. On the industry side, numerous projects have been publicly disclosed since 2014, with half launched during or after the COVID-19 pandemic. These projects span various regions and industry sectors, including airlines (e.g., EasyJet, Air France, KLM, Singapore Airlines, Delta Airlines, Lufthansa), aircraft manufacturers (e.g., Airbus, Boeing), and organizations in the field of aerospace, robotics and artificial intelligence (e.g., Rolls-Royce, MainBlades, Donecle, SR Technics, ST Engineering, Jet Aviation, HACARUS) as well as research institutes (A*STAR).
• Collection of a new dataset of aircraft defects using a drone from an abandoned aircraft: Using an affordable and commercially available Parrot Bebop Quadcopter Drone (14-megapixel full HD 1080p, 180° field of view), we demonstrate that current readily available technology can achieve accurate aircraft defect detection. We validate this hypothesis by investigating a preserved Boeing 747 with defects located in Korat, Nakhon Ratchasima, Thailand. The collected dataset has been made publicly available (see [12]) to support testing and benchmarking of future algorithms for aircraft defect detection. To the best of our knowledge, although several other studies have collected defect data from drone imagery [13,14,15,16,17,18], their datasets remain inaccessible due to confidentiality. Existing publicly available datasets primarily focus on defect-specific images (e.g., [19]), often emphasizing close-up views of defects rather than encompassing the entire aircraft (with and without defects) or accounting for variations in image quality caused by distance and angle. The lack of publicly available data is a frequently cited challenge in research papers on this topic [18,20].
• Evaluation and Comparison of Leading Computer Vision Algorithms for Aircraft Defect Detection: This work tests and compares YOLOv9 and RT-DETR, two of the most accurate computer vision models (refer to [9]). Our implementation of these algorithms for aircraft defect detection contributes in two major ways: (i) It offers a benchmark analysis highlighting the relative strengths of these models. The results show comparable overall performance, with RT-DETR slightly outperforming YOLOv9 prior to hyperparameter tuning, whereas YOLOv9 achieves superior results post-tuning. (ii) We demonstrate the practicality of leveraging modern computer vision algorithms for real-world aircraft defect detection, achieving moderate accuracy even without extensive tuning; mAP50 scores range from 0.699 for RT-DETR to 0.685 for YOLOv9. Additionally, a comparison with the YOLOv5 extra-large (YOLOv5x) model [21], developed in 2020, reveals a 10% improvement in mAP50 with YOLOv9, emphasizing the rapid advancements in computer vision.
• Hyperparameter tuning using Bayesian optimization: To avoid a myopic, single implementation of these algorithms, we applied hyperparameter tuning and analyzed its impact on performance metrics, particularly precision, recall, and mAP50. Our results show that tuning improves accuracy by 1.6% for RT-DETR and 6.6% for YOLOv9 across the two datasets analyzed—our drone-collected dataset [12] and a publicly available dataset [19]. We demonstrate that performance can be tailored to specific priorities by emphasizing different metrics during tuning. To encourage further research and facilitate refinement of the presented models, we have made our codes publicly available in [22].
• Discussion of limitations and challenges in implementing computer vision for aircraft defect detection: Our findings highlight the potential of computer vision to enhance defect detection, particularly given the rapid evolution of computer vision algorithms. However, discussions with industry experts reveal significant challenges that remain. These include (i) regulatory constraints that vary across countries, hindering the development of standardized solutions, (ii) the need for seamless human–robot collaboration, as automation is intended to complement rather than replace human inspectors, (iii) the necessity of achieving exceptionally high accuracy levels to meet stringent safety standards. Current mAP50 levels of 0.70–0.75 may still be insufficient due to safety reasons. These findings emphasize the promise of this technology while motivating continued advancements to overcome these challenges.

The remainder of this paper is organized as follows: We begin with a review of existing work on computer vision for aircraft skin defect detection, highlighting publicly available projects from various industries in Section 2. Next, we describe the data collection process and provide details about the datasets used in this study in Section 3. The computer vision models employed for defect detection are introduced in Section 4, along with the metrics for evaluating algorithm performance in Section 4.3 and the hyperparameter tuning procedure in Section 4.4. In Section 5, we compare and analyze the performance of the object detection models, including the impact of hyperparameter tuning. We then delve into the challenges and important research directions in this field in Section 6. Finally, we conclude the paper in Section 7.

2. Research and Industry Survey

A literature review was conducted to examine existing research on the application of computer vision in aircraft defect detection. We systematically reviewed entries obtained through search engines such as Google Scholar and SciSpace, using combinations of relevant keywords, including computer vision, maintenance, repair, and overhaul (MRO), YOLO, RT-DETR, aircraft skin defect types, computer vision in aircraft MRO, and drone inspection for aircraft MRO. Each retrieved paper was individually assessed to determine its relevance to the paper.

Our review identified a total of 21 research papers, with 14 published after 2020, illustrating the rapid growth in the field. These studies employ various algorithms, ranging from direct implementations of various versions of YOLO (though version 9, used in this paper, was not implemented) [16,17,20,23,24,25,26,27,28] to more advanced neural network models. For example, some researchers have enhanced existing architectures with additional layers, such as mask-scoring region-based convolutional neural network (R-CNN) [15] and U-Net [29]. A summary of these papers is presented in Table 1. The papers also address a wide range of defect types, including stains, lightning burns, dents, rust, cracks, corrosion, scratches, and damaged and missing rivets/heads or screws, among others.

Note that these papers focus on visible defects on the aircraft’s surface or fuselage, aligning with the scope of this review. More detailed inspections, which require non-destructive testing (NDT) methods such as ultrasonic or radiographic testing, are typically conducted during B to D checks to assess internal structures, engines, or overlapping panels [30,31,32,33]. Additionally, other studies have explored object detection algorithms for identifying manufacturing defects [34,35,36,37], detecting ice formation on critical components [38], aircraft ducts defects [39], and verifying the condition of aircraft components, such as ensuring vents are closed, valves are properly positioned, and static ports are uncovered [40,41]. While these studies are not included in Table 1, which focuses on defects in visible aircraft skin, we acknowledge their relevance and importance as part of the broader inspection process.

Table 1. Related academic research for advanced aircraft defects inspection.

References (Year)	Description
[42] (2019)	Scope: Utilizing a reconfigurable climbing robot for aircraft surface defect detection. Data Source: RGB images taken by climbing robot. CV Algorithm: Enhanced SSD MobileNet. Focus: Detecting between aircraft stain and defect.
[13] (2019)	Scope: Investigating defect classification on aircraft fuselage from UAV images. Data Source: Images taken using a drone. CV Algorithm: Combination of CNN and few-shot learning methods. Focus: Detecting paint defects, lightning burns, screw rashes, rivet rashes.
[24] (2019)	Scope: Adapting YOLO3 for faster crack detection in aircraft structures. Data Source: Images from aviation company and industrial equipment. CV Algorithm: YOLOv3-Lite. Focus: Aircraft cracks.
[14] (2020)	Scope: Developing a deep neural network (DNN) to detect aircraft defects. Data Source: Images taken using drone. CV Algorithm: AlexNet and VGG-F networks with a SURF feature extractor. Focus: Detecting fuselage defects (binary classification).
[43] (2020)	Scope: Developing a Mask R-CNN to detect aircraft dents. Data Source: Images from publicly available sources and within the hangar. CV Algorithm: Mask R-CNN. Focus: Aircraft dents.
[44] (2020)	Scope: Improving MASK R-CNN with augmentation techniques. Data Source: Images from publicly available sources and within the hangar. CV Algorithm: Pre-classifier in combination with MASK R-CNN. Focus: Wing images with and without aircraft dents.
[23] (2020)	Scope: Applying and Comparing YOLOv3 with Faster-RCNN. Data Source: Self-collected dataset of aircraft defect images. CV Algorithm: YOLO Neural Network and Faster-RCNN. Focus: Detecting: skin crack, thread corrosion, skin peeling, skin deformation, and skin tear.
[15] (2022)	Scope: Developing an improved mask scoring R-CNN. Data Source: Images taken using a drone. CV Algorithm: Mask Scoring R-CNN for instance segmentation with attention and feature fusion. Focus: Detecting paint falling and scratch.
[25] (2023)	Scope: Examining the use of YOLOv5 to detect defects on aircraft surface. Data Source: Self-collected dataset of aircraft defect images. CV Algorithm: YOLOv5. Focus: Detecting: cracks, dent, missing screws, peeling, and corrosion.
[45] (2024)	Scope: Exploring the use of CNNs for aircraft defect detection. Data Source: Images taken by drones and blimps. CV Algorithm: CNN. Focus: Detecting: missing rivet/heads, corroded rivet.
[16] (2024)	Scope: Exploring the use of YOLOv8 for aircraft defect detection. Data Source: Images taken using drone, phone, and camera. CV Algorithm: YOLOv8. Focus: Detecting: panel missing, rivets/heads screws damaged or missing.
[26] (2024)	Scope: Inspecting hyperparameter tuning of YOLOv8 for aircraft defect detection. Data Source: Scanning electron microscopy (SEM) images of surface parts. CV Algorithm: YOLOv8 with MobileNet backbone and Bayesian optimization. Focus: Detecting: gap, microbridge, line collapse, bridge, rust, scratch, and dent.
[17] (2024)	Scope: Proposing an improved YOLOv5-based model for aircraft defect detection. Data Source: Publicly available images and images captured using a drone. CV Algorithm: YOLO model with deformable convolution, attention mechanisms, and contextual enhancement. Focus: Detecting: crack, dent, missing rivet/heads, peeled paint, scratch, missing caps, and lost tools.
[29] (2024)	Scope: Proposing a bio-inspired CNN model for low-illumination aircraft skin defect detection. Data Source: Images taken by an airline. CV Algorithm: U-Net with residual blocks and attention mechanisms. Focus: Detecting image pixels with defects.
[46] (2024)	Scope: Applying YOLOv8 to aircraft defect detection. Data Source: Images taken using a drone. CV Algorithm: YOLOv8. Focus: Detecting: rust, scratches, and missing rivets/heads.
[18] (2024)	Scope: Testing various deep learning architectures for aircraft defect detection. Data Source: Drone and handheld cameras used inside the hangar. CV Algorithm: Various CNN-based models with custom size-estimation. Focus: Detecting: dents, missing paint, screws, and scratches.
[20] (2024)	Scope: Improving YOLOv7 for aircraft defect detection. Data Source: Images collected using a mobile platform camera and publicly available datasets. CV Algorithm: FC-YOLO with feature fusion strategies. Focus: Detecting: paint peel, cracks, deformation, and rivet damage.
[27] (2024)	Scope: Improving YOLOv8n for detecting small aircraft defects. Data Source: Images collected using DJI OM 4 SE stabilizers paired with mobile cameras. CV Algorithm: Improved YOLOv8n with Shuffle Attention and BiFPN. Focus: Detecting: cracks, corrosion, and missing labels.
[47] (2024)	Scope: Data augmentation using Fourier GAN to improve defect detection accuracy. Data Source: Drone-collected and synthetic images. CV Algorithm: Fourier GAN for data augmentation. Focus: Detecting: loose components, corrosion, and skin damage.
[28] (2024)	Scope: Improving YOLOv8 for robust defect detection. Data Source: Publicly available images. CV Algorithm: YOLOv8 with CoTAttention and SPD-Conv modules. Focus: Detecting: cracks, dents, and rust.
[48] (2024)	Scope: Developing DyDET, a lightweight semi-supervised aircraft defect detection framework. Data Source: Images collected using camera. CV Algorithm: DyDET with dynamic attention and adaptive pseudolabeling. Focus: Detecting: scratches, paint-peeling, rivet damage, and rust.

Table 1. Related academic research for advanced aircraft defects inspection.

References (Year)	Description
[42] (2019)	Scope: Utilizing a reconfigurable climbing robot for aircraft surface defect detection. Data Source: RGB images taken by climbing robot. CV Algorithm: Enhanced SSD MobileNet. Focus: Detecting between aircraft stain and defect.
[13] (2019)	Scope: Investigating defect classification on aircraft fuselage from UAV images. Data Source: Images taken using a drone. CV Algorithm: Combination of CNN and few-shot learning methods. Focus: Detecting paint defects, lightning burns, screw rashes, rivet rashes.
[24] (2019)	Scope: Adapting YOLO3 for faster crack detection in aircraft structures. Data Source: Images from aviation company and industrial equipment. CV Algorithm: YOLOv3-Lite. Focus: Aircraft cracks.
[14] (2020)	Scope: Developing a deep neural network (DNN) to detect aircraft defects. Data Source: Images taken using drone. CV Algorithm: AlexNet and VGG-F networks with a SURF feature extractor. Focus: Detecting fuselage defects (binary classification).
[43] (2020)	Scope: Developing a Mask R-CNN to detect aircraft dents. Data Source: Images from publicly available sources and within the hangar. CV Algorithm: Mask R-CNN. Focus: Aircraft dents.
[44] (2020)	Scope: Improving MASK R-CNN with augmentation techniques. Data Source: Images from publicly available sources and within the hangar. CV Algorithm: Pre-classifier in combination with MASK R-CNN. Focus: Wing images with and without aircraft dents.
[23] (2020)	Scope: Applying and Comparing YOLOv3 with Faster-RCNN. Data Source: Self-collected dataset of aircraft defect images. CV Algorithm: YOLO Neural Network and Faster-RCNN. Focus: Detecting: skin crack, thread corrosion, skin peeling, skin deformation, and skin tear.
[15] (2022)	Scope: Developing an improved mask scoring R-CNN. Data Source: Images taken using a drone. CV Algorithm: Mask Scoring R-CNN for instance segmentation with attention and feature fusion. Focus: Detecting paint falling and scratch.
[25] (2023)	Scope: Examining the use of YOLOv5 to detect defects on aircraft surface. Data Source: Self-collected dataset of aircraft defect images. CV Algorithm: YOLOv5. Focus: Detecting: cracks, dent, missing screws, peeling, and corrosion.
[45] (2024)	Scope: Exploring the use of CNNs for aircraft defect detection. Data Source: Images taken by drones and blimps. CV Algorithm: CNN. Focus: Detecting: missing rivet/heads, corroded rivet.
[16] (2024)	Scope: Exploring the use of YOLOv8 for aircraft defect detection. Data Source: Images taken using drone, phone, and camera. CV Algorithm: YOLOv8. Focus: Detecting: panel missing, rivets/heads screws damaged or missing.
[26] (2024)	Scope: Inspecting hyperparameter tuning of YOLOv8 for aircraft defect detection. Data Source: Scanning electron microscopy (SEM) images of surface parts. CV Algorithm: YOLOv8 with MobileNet backbone and Bayesian optimization. Focus: Detecting: gap, microbridge, line collapse, bridge, rust, scratch, and dent.
[17] (2024)	Scope: Proposing an improved YOLOv5-based model for aircraft defect detection. Data Source: Publicly available images and images captured using a drone. CV Algorithm: YOLO model with deformable convolution, attention mechanisms, and contextual enhancement. Focus: Detecting: crack, dent, missing rivet/heads, peeled paint, scratch, missing caps, and lost tools.
[29] (2024)	Scope: Proposing a bio-inspired CNN model for low-illumination aircraft skin defect detection. Data Source: Images taken by an airline. CV Algorithm: U-Net with residual blocks and attention mechanisms. Focus: Detecting image pixels with defects.
[46] (2024)	Scope: Applying YOLOv8 to aircraft defect detection. Data Source: Images taken using a drone. CV Algorithm: YOLOv8. Focus: Detecting: rust, scratches, and missing rivets/heads.
[18] (2024)	Scope: Testing various deep learning architectures for aircraft defect detection. Data Source: Drone and handheld cameras used inside the hangar. CV Algorithm: Various CNN-based models with custom size-estimation. Focus: Detecting: dents, missing paint, screws, and scratches.
[20] (2024)	Scope: Improving YOLOv7 for aircraft defect detection. Data Source: Images collected using a mobile platform camera and publicly available datasets. CV Algorithm: FC-YOLO with feature fusion strategies. Focus: Detecting: paint peel, cracks, deformation, and rivet damage.
[27] (2024)	Scope: Improving YOLOv8n for detecting small aircraft defects. Data Source: Images collected using DJI OM 4 SE stabilizers paired with mobile cameras. CV Algorithm: Improved YOLOv8n with Shuffle Attention and BiFPN. Focus: Detecting: cracks, corrosion, and missing labels.
[47] (2024)	Scope: Data augmentation using Fourier GAN to improve defect detection accuracy. Data Source: Drone-collected and synthetic images. CV Algorithm: Fourier GAN for data augmentation. Focus: Detecting: loose components, corrosion, and skin damage.
[28] (2024)	Scope: Improving YOLOv8 for robust defect detection. Data Source: Publicly available images. CV Algorithm: YOLOv8 with CoTAttention and SPD-Conv modules. Focus: Detecting: cracks, dents, and rust.
[48] (2024)	Scope: Developing DyDET, a lightweight semi-supervised aircraft defect detection framework. Data Source: Images collected using camera. CV Algorithm: DyDET with dynamic attention and adaptive pseudolabeling. Focus: Detecting: scratches, paint-peeling, rivet damage, and rust.

The industry’s adoption of computer vision for defect detection is evident from the 18 publicly documented projects identified across different sectors. These initiatives span airlines (e.g., EasyJet, Air France, KLM, Singapore Airlines, Delta Airlines, Lufthansa), aircraft manufacturers (e.g., Airbus, Boeing), and organizations specializing in aerospace, robotics, and artificial intelligence (e.g., Rolls-Royce, MainBlades, Donecle, SR Technics, ST Engineering, Jet Aviation, HACARUS). Research institutes such as A*STAR are also actively contributing to advancements in this area. These projects not only focus on developing more robust object detection algorithms but also explore innovative hardware solutions for image and video collection. Emerging technologies include UAVs, drone swarms, mobile robotic platforms, climbing robots, and smart hangars equipped with high-quality cameras.

Table 2. Existing related technology solutions and projects and technology.

References (Year)	Description
[49] (2013)	Scope: A French R&D project developing an autonomous wheeled robot capable of inspecting aircraft during maintenance operations. Solution Type: Robot Platform Offered Benefits: Enhances inspection reliability and repeatability; reduces inspection time and maintenance costs.
[50] (2015)	Scope: EasyJet integrates drones for aircraft inspection and explores 3D printing to reduce maintenance costs and downtime. Solution Type: Drones Offered Benefits: Speeds up inspection; reduces dependency on external suppliers for parts.
[51] (2015)	Scope: Automated visual inspections using autonomous drones with AI-powered software. Solution Type: swarm of drones Offered Benefits: Performs GVI, identifies lightning strikes, and provides close-up inspections more efficiently than manual processes.
[52] (2017)	Scope: Mainblades offers automated drone inspections for aircraft maintenance. Solution Type: Drones Offered Benefits: Detects defects such as dents, cracks, and lightning strikes; reduces inspection time and improves consistency.
[53] (2018)	Scope: Airbus uses autonomous drones equipped with high-resolution cameras to conduct visual inspections of aircraft surfaces. Solution Type: Drones Offered Benefits: Speeds up inspection (3 h), enhances safety by reducing human exposure.
[54] (2018)	Scope: Rolls-Royce demonstrated multiple robotic solutions, including SWARM robots for visual inspections, FLARE for repairs, and remote boreblending robots for maintenance tasks. Solution Type: Swarm of drones Offered Benefits: Enables inspections and repairs without removing the engine; reduces downtime and maintenance costs.
[55] (2018)	Scope: Developed a robot to move on aircraft for inspection and deliver tools to difficult-to-access areas. Solution Type: Climbing Robot Offered Benefits: Tools can be delivered to engineers for improved maintenance efficiency.
[56] (2018)	Scope: A collaboration between Ubisense and MRO Drone to create an automated aircraft inspection and tool management system. Solution Type: Smart Hangar Offered Benefits: Decreases Aircraft on Ground (AOG) time by up to 90%; enhances efficiency and productivity in maintenance operations.
[57] (2019)	Scope: Developed automated drone inspection for exterior aircraft components on surfaces. Solution Type: Drones Offered Benefits: Reduce time and cost of aircraft maintenance. Can inspect both inside and outside of hangar.
[58] (2020)	Scope: Received CAAS authorization to use drones for aircraft inspections, ensuring faster and safer processes. Solution Type: Drones Offered Benefits: Reduces inspection time; authorized for regulatory compliance.
[59] (2022)	Scope: Korean Air’s development of a drone swarm system to inspect aircraft exteriors. Solution Type: Swarm of drones Offered Benefits: Reduces inspection time by up to 60%; improves safety and accuracy; minimizes aircraft downtime.
[60] (2023)	Scope: GE Aerospace’s soft robotic inchworm designed for on-wing jet engine inspection and repair. Solution Type: Climbing Robot Offered Benefits: Enables minimally invasive inspections; reduces maintenance burden; provides access to hard-to-reach engine areas.
[61] (2023)	Scope: A*STAR’s Smart Automated Aircraft Visual Inspection System (SAAVIS) uses robots and hybrid AI for defect detection on airframes. Solution Type: Smart Hangar Offered Benefits: Reduces inspection time and human errors; detects over 30 defect types.
[62] (2023)	Scope: Employs robotic arms and AI to inspect aircraft engines and seat tracks, improving efficiency. Solution Type: Robot platform Offered Benefits: Saves over 780 man-hours annually; reduces staff fatigue and increases inspection accuracy.
[63] (2023)	Scope: AI-powered drone inspections for a range of aircraft maintenance needs. Solution Type: Drones Offered Benefits: Conducts inspections for aircraft structure, paint quality, and pre-purchase evaluations.
[64] (2024)	Scope: Delta is the first U.S. airline to use FAA-approved drones for maintenance inspections, improving speed and safety. Solution Type: Drones Offered Benefits: Speeds up inspection by 82%; reduces technician risk during inspections.
[65] (2024)	Scope: Boeing uses small drones with AI-based damage detection software to automate exterior aircraft inspections. Solution Type: Drones Offered Benefits: Cuts inspection time by half, improves accuracy, and reduces costs.
[66] (2024)	Scope: Tests drone-based inspections, 3D scanning, and exoskeletons for aircraft overhauls, aiming to streamline maintenance. Solution Type: Drones Offered Benefits: Speeds up maintenance, improves accuracy, and reduces costs.

summarizes these projects, detailing the hardware solutions employed and their respective applications.

Figure 1a,b summarize and illustrate the evolution of research literature and industry projects over time, highlighting a growing interest in the topic in recent years, particularly in 2024.

3. Data Collection

For this study, we captured video images of an old Boeing 747 parked in the airplane park near Suranaree University of Technology (SUT), Thailand, shown in Figure 2a. We captured the video images with a Parrot Bebop 1 drone, shown in Figure 2b, with 1080p resolution using its 14-megapixel camera. Utilizing the image preprocessing tools available in Roboflow [67], we extracted 983 images from the 13 videos we took and manually annotated the visual defects into one of three defect categories: (1) missing head, (2) rust, and (3) scratch. An example image with the three different defect types from this dataset can be seen in Figure 3. The annotated images were then randomly divided into 688 training, 197 validation, and 98 test images. We applied the auto-orient preprocessing option and resized the images to 640 × 640 pixels. Note that varying image resolution or quality may influence accuracy metrics; we utilized the default resolution recommended by YOLOv9 but acknowledge that higher resolutions might yield slightly improved accuracy. The dataset is publicly available and can be downloaded from [12]. For simplicity, we will refer to this dataset as Dataset I throughout this paper.

In addition to the dataset described above, we also chose to use a publicly available dataset provided by [19] in our study to further verify the models. We refer to this additional dataset as Dataset II in this paper. The dataset consists of 9651 training, 642 validation, and 429 test images of multiple defective airplanes. The defects in the images are annotated and classified into one of the five defect categories: (1) crack, (2) dent, (3) missing head, (4) paint-off, and (5) scratch. Examples of images for these five defect types in Dataset II can be seen in Figure 4.

The summary for the number of images used in the training, validation, and testing pools together with defect instances in Datasets I and II can be seen in

Table 3. Summary of defects classification by type for Datasets I and II.

Defect Types	Dataset I (983 Images)			Dataset II (10,722 Images)
	Training	Validation	Test	Training	Validation	Test
rust	1430 instances	425 instances	183 instances	-	-	-
missing head	921 instances	327 instances	162 instances	822 instances	54 instances	32 instances
scratch	1429 instances	422 instances	211 instances	180 instances	16 instances	7 instances
crack	-	-	-	9606 instances	607 instances	386 instances
dent	-	-	-	8577 instances	554 instances	413 instances
paint-off	-	-	-	711 instances	34 instances	23 instances
number of total images	688 images	197 images	98 images	9651 images	642 images	429 images

.

4. Methodology

Object detection is a computer vision technique designed to identify and classify objects of interest within images. This capability aligns well with our goal of detecting and classifying defects on aircraft surfaces. With the development of various object detection models, numerous datasets have been proposed for benchmarking these models. The most widely used dataset for this purpose is the Microsoft Common Objects in Context (MS COCO) dataset, introduced by [68]. This dataset includes 180,000 training images, 5000 validation images, and 41,000 test images, featuring millions of labeled objects across 80 categories. Performance comparisons of different object detection models using the MS COCO dataset are detailed in [9]. For our study, we have chosen the two models well-known for their accuracy: RT-DETR and YOLOv9.

4.1. RT-DETR

DETR was first introduced by [69] in 2020, utilizing a transformer encoder-decoder architecture. Since its inception, DETR-based models have seen extensive development by various research groups (see [70] for a detailed review up to early 2023). To address the computational cost of DETR-based models, [11] proposed a new one-stage DETR variant called RT-DETR in late 2023, achieving real-time object detection. Their work introduced three key modifications to the original DETR model: (i) an efficient hybrid encoder was developed, combining attention-based intra-scale feature interaction (AIFI) and convolutional neural network (CNN)-based cross-scale feature fusion (CCFF) to alleviate computational bottlenecks; (ii) an uncertainty-minimal query selection scheme was implemented by integrating uncertainty into the loss function, reducing feature selection ambiguity; (iii) adjustments to the number of decoder layers were proposed, as the authors observed that changes in the number of decoder layers significantly impacted inference speed while maintaining accuracy. With these three modifications, RT-DETR reportedly outperformed earlier YOLO models and previous DETR series models in real-time object detection tasks on the MS COCO dataset. In this study, we implemented the RT-DETR extra-large (rtdetr-x) model available in [71].

4.2. YOLOv9

YOLO is one of the most well-known object detection models, recognized for its well-balanced tradeoff between efficiency and accuracy. The first version, YOLOv1 [72], introduced in 2015, was the first one-stage model to achieve true real-time object detection. Since then, YOLO has attracted widespread adoption. Numerous advancements and adaptations by different research groups have further improved its speed and accuracy while extending its capabilities to tasks such as image classification and segmentation. Readers interested in a detailed overview of the YOLO series models up to YOLOv8 are referred to [73]. In early 2024, ref. [10] proposed YOLOv9, a significant advancement in the YOLO series. YOLOv9 incorporates several key innovations: programmable gradient information (PGI) for more reliable gradient updates, the generalized efficient layer aggregation network (GELAN) to address data loss, and the integration of DETR and attention mechanisms to enhance detection capabilities. These improvements have reportedly resulted in superior performance on the MS COCO dataset compared to previous YOLO versions and RT-DETR. Subsequently, ref. [74] introduced YOLOv10, which reportedly achieves similar performance to YOLOv9 with 46% reduced latency and 25% fewer parameters. Additionally, in September 2024, ref. [75] released YOLOv11 for public use, which improved the YOLO-series architecture and training methods for more versatile computer vision applications. For our study, we utilized the YOLOv9 large (yolov9e) model as implemented by [76].

4.3. Performance Metrics

The performance of object detection models can be assessed using several different metrics. In this study, we evaluated the accuracy of the considered algorithms based on their precision $\left(\mathrm{P}\right)$, recall $\left(\mathrm{R}\right)$, and mean average precision $\left(\mathrm{mAP}\right)$. We specifically used the metrics quantification method described in [77]. To quantify the three mentioned metrics, it is necessary to define the notion of True Positive $\left(TP\right)$ calculated based on the Intersection over Union $\left(IoU\right)$ of the bounding boxes with a 50% threshold. Here, a bounding box is a rectangle added to tightly enclose a defect on the image. Given a predicted bounding box that intersects with a ground truth bounding box, the $IoU$ is defined as the areas of the intersection divided by the union between the two bounding boxes, i.e.,

$$IoU={\displaystyle \frac{B_{pred}\cap B_{truth}}{B_{pred}\cup B_{truth}}}$$

(1)

Here, $B_{pred}\mathrm{and}{B}_{truth}$ denote the area of the predicted and the ground truth bounding boxes, respectively, calculated in pixel units. The $\left(TP\right)$ is then defined as the number of $IoU$ instances that are higher than 50% with the correct defect classifications.

The precision $\left(\mathrm{P}\right)$ can then be computed as the ratio of the $\left(TP\right)$ to the total number of predicted bounding boxes $\left(N_{B_{pred}}\right)$, i.e.,

$$\mathrm{P}={\displaystyle \frac{TP}{N_{B_{pred}}}}$$

(2)

Here, $\mathrm{P}$ quantifies the ability of the model to correctly classify the detected defects.

On the other hand, the recall $\left(\mathrm{R}\right)$ is defined as the ratio between the $\left(TP\right)$ and the actual number of defects present in the images $\left(N_{B_{true}}\right)$, i.e.,

$$\mathrm{R}={\displaystyle \frac{TP}{N_{B_{truth}}}}$$

(3)

Thus, $\mathrm{R}$ signifies the ability of the model to detect the defects present in the given images. To encapsulate the precision and recall of the model, we can then define the area under the precision-recall curve as average precision $\left(\mathrm{AP}\right)$. The average value of $\mathrm{AP}$ across all the defect types is defined as mean average precision $\left(\mathrm{mAP}\right)$. Because we calculated $\mathrm{mAP}$ at the 50% threshold of the $IoU$, we will be using $\mathrm{mAP}50$ to denote the $\mathrm{mAP}$ of the models in this study. Additionally, we also compared the inference speed of the models to detect and classify the defects, and the convergence speed of the models during training progress.

In addition, during training procedures, we also inspected the training and validation losses of the models. Specifically, we considered the box and class losses during the training. The box loss quantifies the error in the predicted bounding boxes $\left(B_{pred}\right)$ compared to the ground truth bounding boxes $\left(B_{truth}\right)$ by subtracting the $IoU$ from one, i.e., $1-IoU$. On the other hand, the class loss measures the ability of the models to correctly classify the defects. In our experiments, we employed the $IoU$ aware classification loss, varifocal loss, proposed by [78].

4.4. Hyperparameter Tuning

Training an NN model involves a number of hyperparameters that need to be configured beforehand, and the values we choose for these hyperparameters influence the performance of the algorithm [79,80]. Given datasets for a specific application, one might resort to the default values of hyperparameters implemented by the NN packages or manually configure them based on available literature, experiences, and/or experiments.

Alternatively, a hyperparameter tuning strategy leveraging optimization techniques can also be implemented. Since this tuning strategy is generally computationally expensive, sequential model-based optimization methods, such as Bayesian optimization, are often employed [81,82,83]. In this approach, the optimization algorithm identifies the optimal point of an approximated model and iteratively updates the surrogate model, enabling the approximate optimum to be found with fewer evaluations of the computationally expensive original function.

In this study, we tuned the object detection models employing the Bayesian optimization approach by viewing the training processes as a black-box function. Specifically, we implemented the gp_minimize function available in [84]. We used the negative value of the weighted average of the accuracy metrics described in Section 4.3 as the objective function for the optimization scheme, i.e.,

$$\begin{array}{r}\underset{\mathcal{H}}{\mathrm{m}\mathrm{i}\mathrm{n}}\left(-\left[\begin{array}{ccc}{\mathrm{w}}_{\mathrm{P}}& {\mathrm{w}}_{\mathrm{R}}& {\mathrm{w}}_{\mathrm{mAP}50}\end{array}\right]{\left[\begin{array}{ccc}\mathrm{P}& \mathrm{R}& \mathrm{mAP}50\end{array}\right]}^{\mathrm{\top }}\right).\end{array}$$

(4)

where $\mathcal{H}$ denotes the hyperparameters space we considered and $\left[\begin{array}{ccc}{\mathrm{w}}_{\mathrm{P}}& {\mathrm{w}}_{\mathrm{R}}& {\mathrm{w}}_{\mathrm{mAP}50}\end{array}\right]$ are the weight factors of the accuracy metrics. Specifically, the hyperparameter space $\mathcal{H}$ comprises 29 hyperparameters, as listed in

Table 4. List of hyperparameters considered in this study (a description of the hyperparameters can be found in [77]).

No.	Argument	Values	No.	Argument	Values
1	batch	$\{4,8,12,16,24,28,32\}$	15	hsv_h	$\left[0.0,1.0\right]$
2	lr0	$\left[{10}^{-10},{10}^{-1}\right]$	16	hsv_s	$\left[0.0,1.0\right]$
3	lrf	$\left[{10}^{-10},{10}^{-1}\right]$	17	hsv_v	$\left[0.0,1.0\right]$
4	momentum	$\left[0.0,1.0\right]$	18	degrees	$\left[-180,180\right]$
5	weight_decay	$\left[0.0,0.01\right]$	19	translate	$\left[0.0,1.0\right]$
6	warmup_epochs	$\left[1.0,10.0\right]$	20	scale	$\left[0.0,\mathrm{\infty }\right)$
7	warmup_momentum	$\left[0.0,1.0\right]$	21	shear	$\left[-180,180\right]$
8	warmup_bias_lr	$\left[0.0,0.5\right]$	22	perspective	$\left[0.0,0.001\right]$
9	box	$\left[1.0,20.0\right]$	23	flipud	$\left[0.0,1.0\right]$
10	cls	$\left[0.0,1.0\right]$	24	fliplr	$\left[0.0,1.0\right]$
11	dfl	$\left[1.0,2.0\right]$	25	bgr	$\left[0.0,1.0\right]$
12	pose	$\left[10.0,15.0\right]$	26	mosaic	$\left[0.0,1.0\right]$
13	kobj	$\left[0.0,5.0\right]$	27	mixup	$\left[0.0,1.0\right]$
14	nbs	$\{4,8,12,16,24,28,32,36,$ $40,44,48,52,56,60,64\}$	28	copy_paste	$\left[0.0,1.0\right]$
			29	erasing	$\left[0.0,0.9\right]$

. For the tuning process, we set a limit of 100 function calls, where each function call involves a complete training cycle of an object detection model using Dataset I. In our analysis, we assigned weight factors of $\left[\begin{array}{ccc}0.3& 0.4& 0.3\end{array}\right]$ to maintain a balance across accuracy metrics. Additionally, we conducted a sensitivity analysis for other weight combinations (refer to Section 5.2).

Additional preprocessing was applied to Dataset II to reduce training time during the tuning process, given its large size. The training and validation data were split into 20 random pools of 500 training and 200 validation images each. At the start of each function call during the tuning iteration, a pair of training and validation datasets was randomly selected from these pools. This approach ensures that each function call during the tuning process utilized only 500 training and 200 validation images, while still leveraging the entire dataset of 9651 training and 642 validation images over the complete tuning procedure, which involved 150 function calls.

5. Experimental Results

The experiments in this study were conducted on datasets described in Section 3. The tuning and training of the models were performed on the ASPIRE2A nodes provided by the National Supercomputer Center (NSCC) Singapore. Each training iteration was conducted utilizing one AMD EPYC Millan 7713 CPU (AMD, Santa Clara, CA, USA) and one NVIDIA A100-40G SXM GPU (NVIDIA, Santa Clara, CA, USA) with 128 GB of DDR4 memory [85].

5.1. Performance Comparison

The performance results—including precision (P), recall (R), and mAP50—for the two models, RT-DETR and YOLOv9, are presented in

Table 5. Accuracy comparison of the models to detect defects on Dataset I.

Models	Before Tuning			After Tuning
	P	R	mAP50	P	R	mAP50
RT-DETR	$0.731$	$0.706$	$0.699$	$0.777$	$0.701$	$0.710$
YOLOv9	$0.713$	$0.682$	$0.685$	$0.786$	$0.720$	$0.730$
YOLOv5	$0.682$	$0.651$	$0.626$	$0.723$	$0.726$	$0.698$

and

Table 6. Accuracy comparison of the models to detect defects on Dataset II.

Models	Before Tuning			After Tuning
	P	R	mAP50	P	R	mAP50
RT-DETR	$0.749$	$0.693$	$0.716$	$0.794$	$0.664$	$0.716$
YOLOv9	$0.755$	$0.587$	$0.679$	$0.809$	$0.637$	$0.725$
From [19]	$-$	$-$	$-$	$0.795$	$0.652$	$0.696$

for Datasets I and II, respectively. These tables display results both before and after hyperparameter tuning, as described in Section 4.4. For the post-tuning results, we applied weight factors of $\left[\begin{array}{ccc}0.3& 0.4& 0.3\end{array}\right]$ to maintain a balance across accuracy metrics, with slightly more emphasis on recall (R) to minimize false negatives.

In addition to the RT-DETR and YOLOv9 results, we also include, for Dataset I, the performance of the YOLOv5 Extra-Large (YOLOv5x) model from [21] to examine the evolution of YOLO’s accuracy since 2020. For Dataset II—which is an external dataset provided by [19]—we include the results of their AI model. These results are assumed to reflect post-tuning performance, although the specific tuning method used was not disclosed.

The results show improvement of newer object detection models, i.e., RT-DETR and YOLOv9, in most of the accuracy metrics, compared to the previously employed models, i.e., YOLOv5 and model used by [19]. The performance improvement of these newer models shows the benefits that can be gained from utilizing the updated state-of-the-art object detection models.

Comparing the performance of RT-DETR and YOLOv9 in the current case study, the after-tuning results show a slight upper hand on the accuracy of YOLOv9, as seen in Table 5 and Table 6, which is consistent with the results reported by [10]. Moreover, YOLOv9 also achieved a faster inference speed of approximately 66 frames per second compared to the RT-DETR’s inference speed of around 41 frames per second. However, comparing the training progress between the two models, we observed that RT-DETR has a faster convergence speed while maintaining a comparable accuracy, as shown in Figure 5. In these plots, we show the average of the validation accuracy metrics as well as the training and validation losses from multiple training instances, at each epoch, as solid lines. The shaded areas in the plots display the one-standard deviation below and above the means of the training instances. In this comparison, we considered the training progress of RT-DETR and YOLOv9 models for both Datasets I and II. Here, RT-DETR achieved comparable “P”, “R”, and “mAP50” before 100 epochs, while YOLOv9 needed more than 150 epochs. Similarly, the losses plots show that RT-DETR achieved lower losses faster than YOLOv9.

5.2. Accuracy Metrics Emphases During Tuning

As expected, from Table 5 and Table 6, we can see that all algorithms show some performance improvement after tuning. Specifically, for YOLOv9 model, we can observe an average improvement around 8.6% in “P”, 7% in “R”, and 6.6% in “mAP50”. However, it can be argued that the benefit of hyperparameter tuning is minimal for the RT-DETR model, as it can be observed to have around 6% improvement in “P” but almost no improvement in “mAP50” and even an average of 3% decreased performance in “R”. To further inspect this, we are interested in observing the effect of different emphases on the accuracy metrics during tuning procedure.

To compare the tuning effect of different emphases on the accuracy metrics, we varied the objective function (weights) of the optimization problem (4). Specifically, we considered three additional variations of weight factors $\left[\begin{array}{ccc}{\mathrm{w}}_{\mathrm{P}}& {\mathrm{w}}_{\mathrm{R}}& {\mathrm{w}}_{\mathrm{mAP}50}\end{array}\right]$ that give different emphases to the different accuracy metrics separately, i.e., $\left[\begin{array}{ccc}1& 0& 0\end{array}\right]$, $\left[\begin{array}{ccc}0& 1& 0\end{array}\right]$, and $\left[\begin{array}{ccc}0& 0& 1\end{array}\right]$. We performed this tuning study on Dataset I. To ease the comparison, we plotted blue and red dotted lines to show the accuracy of RT-DETR and YOLOv9 models before tuning, respectively. Two different scales of the vertical axis are used to highlight the improvement in the accuracy of the models by zooming the upper part of the plots.

The comparison in Figure 6 demonstrates that a more balanced emphasis on accuracy metrics during tuning yields better overall performance in “P”, “R”, and “mAP50”. Interestingly, although “mAP50” is intended to represent both precision and recall, focusing solely on this metric during the tuning process appears to lead to suboptimal overall accuracy. This is particularly evident in the case of the YOLOv9 model. For the RT-DETR model, although emphasizing “mAP50” during tuning results in higher recall improvements, the greatest overall accuracy improvement is achieved with a balanced weight.

5.3. Generalization Issue

We have also noted that generalization issues might arise when using object detection model trained using datasets that have significant different features. For example, a model trained using images taken by a drone might not be able to predict defects on images taken using a camera. Moreover, the differences in the camera distances, environment lighting, airplane surface colors, and others might also affect the performance of the models. To inspect this issue, we experimented with the YOLOv9 model where the training and testing images were taken from different dataset. Here, we only consider the same defects that occurred in both Datasets I and II, i.e., missing heads and scratches, and dropped other defect labels. As expected, the model performs poorly when trained using a different dataset other than the testing dataset, as can be seen in

Table 7. Accuracy comparison of the YOLOv9 model with cross-training testing datasets.

Training Dataset	Testing Dataset
	Dataset I			Dataset II			Combine Dataset
	$\mathbf{P}$	$\mathbf{R}$	$\mathbf{m}\mathbf{A}\mathbf{P}50$	$\mathbf{P}$	$\mathbf{R}$	$\mathbf{m}\mathbf{A}\mathbf{P}50$	$\mathbf{P}$	$\mathbf{R}$	$\mathbf{m}\mathbf{A}\mathbf{P}50$
Dataset I	0.705	0.679	0.655	0	0	0	-	-	-
Dataset II	0.125	0.002	0.063	0.485	0.478	0.415	-	-	-
combine datasets	0.732	0.718	0.752	0.659	0.865	0.721	0.727	0.683	0.713

. The significantly worse performance of the model when trained using Dataset II can be attributed to the underrepresentation of scratch and missing head defects in Dataset II, as can be seen in Table 3. On the other hand, the model trained with images taken from both Datasets I and II combined is able to maintain its performance when tested using images taken from either Dataset I or II, as shown in Table 7. This signals the importance of training a model using various different datasets and could be one possible direction to overcome the generalization issue.

6. Discussion

The results of this research demonstrate that a relatively inexpensive UAV system equipped with a 14-megapixel camera can achieve moderately accurate results (mAP50: 0.70–0.75) in identifying aircraft defects. This suggests that existing methods have potential as supportive tools for aircraft defect detection. However, feedback from MRO experts at Bangkok Airways and Singapore Airlines indicates that, in high-demand operational scenarios, a detection accuracy above 90% is generally considered acceptable, provided the entire aircraft is inspected and the miss rate remains below 10%. While our current accuracy falls short of this benchmark, the insights highlight both the potential for real-world use and the need to develop and improve existing computer vision algorithms to meet safety and operational standards for aircraft defect detection.

In addition to algorithmic improvements to improve accuracy, a major challenge faced by computer vision researchers in this domain is the limited access to comprehensive datasets, which remains a significant bottleneck in advancing current algorithms. This issue arises for several reasons:

• Confidentiality Issues: Aircraft defect images are rarely made publicly available due to operational and safety restrictions. To help address this limitation, we have made the dataset collected in this study publicly accessible (see [12]) and identified several publicly available datasets to support future research (links are provided in the Data Availability Statement section at the end of the paper).
• Limited number of defect instances: Even when data are available, defect occurrences are relatively rare, as aircraft are typically well-maintained. To mitigate this, we highlight works from [44,47,86,87] which use data augmentation techniques to generate additional instances of defects for model training.
• Variability across aircraft models and operating conditions: Aircraft defects can vary significantly depending on the aircraft type, manufacturer, and operating environment. Consequently, developing a comprehensive dataset requires image collection across a diverse range of scenarios (as shown in Table 7). Achieving this is only possible through the widespread adoption of computer vision technologies for aircraft defect detection, supported by coordinated efforts and standardized image capture protocols. International guidelines are essential to ensure consistency across devices and settings, as poor-quality or inconsistent imagery may lead to missed defects or false positives.

Building on the previous point, the need for standardized image capture protocols raises an important question: how should defect images be captured? Ideally, capture systems should be consistent across airports and MRO facilities to ensure uniformity and reliability. We identified four main categories of hardware solutions—UAVs, smart hangars, mobile robot platforms, and climbing robots—each offering distinct advantages and limitations. A summary of the key insights from these discussions is provided below, offering a foundation for future research on hardware selection and operational integration.

• Unmanned Aerial Vehicles (UAVs):
• Advantages:

• Capable of reaching difficult areas, such as the upper fuselage and tail sections, without the need for scaffolding or lifts;
• Able to conduct inspections quickly, making them particularly useful for pre-/post-flight inspections;
• Swarms of drones can make routine inspections inside hangars even faster;
• As suggested by our review of the literature, drones have been already widely tested by both academia and industry;
• Relatively inexpensive. As shown through the experiment in this paper, even inexpensive drones and cameras can already provide good levels of accuracy;
• Easily transferable between inspections. Can be deployed across multiple aprons/stands.

• Limitations:

• UAV operations are subject to strict aviation regulations, which may limit their deployment in certain areas of the airport. Additionally, regulatory differences across airports pose challenges for developing standardized solutions;
• Drones may only be permitted for use inside hangars, reducing their utility for pre-/post-flight inspections. Solutions like tethers attached to drones could address this, but may limit their range and effectiveness;
• May obstruct other maintenance activities, and therefore human–drone coordination requires planning;
• Battery life and payload capacity can restrict flight duration and the types of sensors that can be equipped;
• Planning 3D trajectories for drones around aircraft surfaces can be challenging, particularly when coordinating a swarm of drones.

• Smart Hangars with High-Resolution Cameras:
• Advantages:

• Less constrained by regulations as the system operates within a controlled environment and does not interfere with other operations;
• Easily integrates with human-led maintenance activities without obstructing ongoing tasks;
• Once installed, these systems provide reliable, consistent coverage without needing repositioning, and maintenance costs remain relatively low after the initial setup.

• Limitations:

• Can only be used inside hangars, limiting its use at pre-/post-flight inspections;
• Fixed cameras may struggle to capture certain parts of the aircraft, especially at specific angles. Additionally, the quality of images may be reduced compared to drones or robots that capture close-up visuals;
• The system is not transferable, requiring airports to dedicate specific hangars equipped with this technology and plan aircraft inspection schedules accordingly;
• Installation and integration of high-resolution camera systems involve substantial upfront investment.

• Mobile Robot Platforms:
• Advantages:

• Relatively inexpensive and easy to implement, as they can be adapted from technologies already used in other industries (e.g., infrastructure and industrial inspections, as reviewed in [88]);
• Easily transferable between inspections, can be deployed across multiple aprons/stands;
• Can accommodate a more diverse range of equipment (different cameras, sensors, etc.) due to their greater payload capacity (not as constrained as drones or climbing robots).

• Limitations:

• Cannot access elevated areas like the top of the fuselage or tail sections without additional equipment, such as extendable cameras;
• May obstruct other maintenance activities, requiring careful planning and coordination between human workers and the robotic platform, which can be more challenging compared to drones;
• Inspections using ground-based robots may take longer than drones or fixed cameras to cover the entire aircraft.

• Climbing Robots:
• Advantages:

• Capable of traversing complex geometries, these robots can inspect detailed areas of the aircraft, including the fuselage, wings, and tail sections, without the need for scaffolding or lifts;
• Since they adhere to the aircraft’s surface rather than flying freely like drones or moving on the ground like traditional robot platforms, climbing robots may face fewer regulatory constraints during operation;
• Their operation is less likely to obstruct other maintenance activities, especially during routine inspections;
• Easily transferable between inspections. Can be deployed across multiple aprons/stands.

• Limitations:

• While effective for detailed examinations, these robots may require more time to inspect the entire aircraft surface, making them more suitable for scheduled maintenance rather than quick pre-/post-flight checks;
• Adhesion mechanisms and technologies tailored for aircraft inspection can make climbing robots more expensive compared to other inspection tools;
• Technology is still under development. Although existing models, such as those developed by SR Technics, and SUTD [42,55], demonstrate progress. Further research and development are required to address the advanced design and technology requirements of these robots.

In addition to advancing visual detection algorithms and identifying the most suitable hardware solutions for aircraft defect detection, several other critical considerations must be addressed:

• Regulatory Framework: The regulatory aspect is paramount, particularly given that aviation is a global industry. Automation of defect detection operations requires international coordination and recognition to ensure consistency across countries. Standardizing regulations would enable the development of solutions that are globally applicable rather than tailored to specific regional contexts. While the International Civil Aviation Organization (ICAO) provides overarching safety management frameworks, such as Standards and Recommended Practices (SARPs) [89], these currently lack detailed guidance and regulations specific to automated aircraft defect detection.
• Human–Robot Interaction: The interaction between humans and robotic systems in defect detection remains undefined and requires research. The role of humans in the defect detection process, once these solutions are implemented, needs clarification. Should their role be passive, focused on analyzing feedback from these systems, or active, complementing the robot/drone detection search? Additionally, these systems may obstruct other critical operations, particularly during aircraft turnaround processes. Some work has already begun to explore this topic [90].
• Path Planning: Robust path planning mechanisms are needed to address two key objectives: minimizing interference with other operations and ensuring complete aircraft coverage with high-quality images within the limited time available. Research is already underway to tackle this challenge [91,92], but further advancements are crucial.
• Weather Resilience: Weather conditions may significantly impact automated defect detection, especially during pre- and post-flight inspections. Ensuring that automated solutions remain resilient and functional under adverse weather conditions is essential to prevent disruptions. This resilience must be investigated and integrated into future systems to ensure reliability across a wide range of environmental scenarios.

7. Conclusions

This paper reviews the current state of computer vision algorithms for supporting aircraft surface defect detection during inspection routines, whether pre-/post-flight during turnarounds or at routine intervals (e.g., every few months). Specifically, we analyze advances from both academic (21 publications identified) and industry perspectives (18 projects identified).

Through this review and expert discussions, we identified existing object detection algorithms tailored for aircraft defect detection and explored hardware solutions for image and video collection, including drones (UAVs), robot platforms, climbing robots, and smart hangars equipped with cameras.

This study goes beyond a literature review by applying these concepts to real-world scenarios. We collected real data on aircraft defects using a drone, making this dataset publicly available for further research. Additionally, we evaluated and compared the performance of two state-of-the-art object detection models—RT-DETR and YOLOv9. Results show that both models achieved mAP50 scores between 0.70 and 0.75. YOLOv9 demonstrated a slight edge in accuracy and inference speed, while RT-DETR achieved similar accuracy with faster training convergence. A comparison with YOLOv5 from 2020 highlights the rapid evolution of computer vision algorithms, with YOLOv9 achieving a 10% improvement in mAP50. Despite these advancements, we acknowledge that an mAP50 of 0.70–0.75 is insufficient for meeting the risk requirements of aviation operations. Nonetheless, the continuous and rapid progress in computer vision is encouraging.

We also identify and discuss important challenges that still need to be addressed such as regulatory constraints, human–robot integration, path planning optimization for defect image collection, and weather resilience. As part of our future research, we are working with industry partners to enhance existing computer vision algorithms for aircraft defect detection—not only to improve the identification and classification of defects, but also to assess their severity, particularly in terms of size and shape, and to determine whether they require maintenance or repair, or pose any flight safety risks.

In conclusion, this paper provides a comprehensive overview of the current state of computer vision in aircraft defect detection, serving as a foundation for future research in a field that has gained significant attention in recent years.