DualFOD: A Dual-Modality Deep Learning Framework for UAS-Based Foreign Object Debris Detection Using Thermal and RGB Imagery

Highlights

What are the main findings?

• A dual-modality framework combining YOLO12-based RGB detection with unsupervised thermal anomaly extraction enables more robust foreign object debris (FOD) detection than either sensing modality alone.
• Thermal imaging effectively detects small metallic and low-contrast debris that RGB imaging misses, while RGB excels at identifying visually distinct objects across varied surface textures.

What are the implications of the main findings?

• Fusing thermal and RGB sensing in UAS-based inspection systems reduces false negatives and enhances detection reliability under challenging lighting and environmental conditions.
• The training-free thermal pipeline allows scalable deployment to new airport environments without requiring additional annotated datasets.

Abstract

Foreign Object Debris (FOD) poses critical risks to aircraft during takeoff and landing, resulting in billions of dollars in losses annually due to infrastructure damage and flight delays. Advancements in automated inspection technologies have enabled the use of Unmanned Aerial Systems (UAS) combined with Artificial Intelligence (AI) for rapid FOD identification. While prior research has extensively evaluated optical sensors such as RGB imaging and radar, limited work has investigated the potential of thermal imaging for improved FOD visibility under challenging environmental conditions. This study proposes DualFOD, a dual-modality detection framework that integrates a supervised YOLO12-based RGB detector with an unsupervised thermal anomaly extraction pipeline for identifying debris on runway surfaces. A decision-level fusion algorithm combines detections from both branches using spatial proximity matching to produce a unified FOD inventory. The RGB branch achieves a precision of 0.954 and mAP@0.5 of 0.890 on the held-out test set. Cross-site validation at the Cobb County Sport Aviation Complex demonstrates that thermal detection recovers debris missed by RGB at higher altitudes, with the fused output consistently outperforming either single-modality branch. This research contributes toward scalable autonomous FOD monitoring that enhances operational safety in aviation environments.

1. Introduction

Foreign Object Debris (FOD) represents any unwanted object on airport runways that may endanger aircraft during takeoff and landing. Even small debris items can lead to catastrophic outcomes and costly operational disruptions, contributing to billions of dollars in losses annually across the aviation industry [1,2]. The total economic impact of FOD estimated by the Federal Aviation Administration exceeds $20 billion every year [3], encompassing direct damage to aircraft engines and landing gear as well as indirect costs from flight delays and runway closures. Present foreign object debris also leads to foreign object damage, which results in small cracks and other structural degradation that may be difficult to detect during routine maintenance [4]. The fatal accident of Air France Flight 4590 in 2000, where runway debris ingestion resulted in over 100 fatalities, underscores the critical importance of proactive FOD management [1]. FOD typically appears in the form of aircraft components, mechanical tools, metal fragments, plastic parts, and rubber materials from ground support vehicle tires [3,5]. The wide range of items classified as FOD demonstrates its variability in size, shape, color, and material [6], posing a significant challenge for automated detection systems.

Traditional FOD detection methods, such as runway inspections by human personnel or fixed imaging systems, are time-consuming, labor-intensive, and limited in environmental adaptability [7,8]. Many military bases still rely on the extensive FOD Walk, which requires a large number of personnel to physically traverse the runway surface [9]. FOD detection technologies beyond manual inspection span radar systems, laser scanning, and fixed optical monitoring [10]. While radar and LiDAR offer reliable performance, they are costly to deploy and may require runway closures during installation [11]. Stationary optical cameras provide lower deployment cost but offer limited runway coverage and degraded performance in low-visibility conditions [12]. Optical sensors can also be seen as more practical than radar solutions because radar is not effective at locating small FOD [13]. Recent advances in Unmanned Aerial Systems (UAS) present a scalable alternative for fast and flexible runway coverage. UAS-based detection has gained momentum due to its wider field-of-view coverage and rapid deployment capability for runway sweeps [7]. Some airports currently use drones for pavement inspection [14], and similar platforms could serve as effective FOD detectors by flying over runways at controlled altitudes and acquiring high-resolution overhead imagery.

Pairing UAS imagery with deep learning enables automated real-time FOD detection from live video streams or high-resolution imagery [15]. There are two major families of algorithms currently widely used for UAS-based visual object detection: Transformer-based architectures and convolutional neural network (CNN)-based architectures. Transformer-based detectors, such as the Detection Transformer (DETR) [16] and its real-time variant RT-DETR [17], leverage self-attention mechanisms to model global context across the entire image and eliminate the need for hand-designed components such as anchor boxes and non-maximum suppression. These models have demonstrated strong performance on benchmark datasets, particularly for objects requiring long-range contextual reasoning. For CNN-based detection algorithms, the YOLO (You Only Look Once) family has become the dominant choice for real-time applications due to its single-stage architecture that simultaneously predicts bounding boxes and class probabilities in a single forward pass [18]. Recent iterations including YOLOv10 [19], YOLO11 [20], and YOLO12 [21,22] have introduced progressive improvements in attention mechanisms, feature pyramid fusion, and computational efficiency. YOLO12, the latest in this lineage, integrates an attention-centric architecture with an Attentive C2f backbone and APAN-FPN neck, enabling enhanced detection of small objects in complex outdoor environments [21].

Despite these advances in detection architectures, RGB sensing alone struggles to reliably detect visually similar debris items, especially when surface reflectance or illumination varies [12]. Sensor fusion technology addresses this limitation by combining multiple modality types to increase detection accuracy [10,23,24]. Mo et al. demonstrated that by combining information from both infrared and RGB images using a dual-light camera, trained YOLO models were able to more accurately identify small FOD [11,25]. Thermal imaging offers a complementary sensing modality by exploiting temperature differences between debris and runway surfaces [26], and has demonstrated utility in distinguishing temperature contrasts on pavement surfaces, although factors such as material emissivity and heat saturation can affect performance. Despite its potential, only limited research has explored thermal sensing in AI-based autonomous FOD inspection workflows. This gap motivates the current investigation into how a dual-modality framework combining RGB deep learning with unsupervised thermal analysis can improve detection accuracy and operational robustness.

The major contributions of this work are as follows:

• A dual-modality FOD detection framework (DualFOD) that integrates a supervised YOLO12-based RGB detection branch with an unsupervised thermal anomaly extraction pipeline, enabling complementary sensing without requiring annotated thermal training data.
• Integration of Slicing Aided Hyper Inference (SAHI) into the RGB branch to enhance small-object detection at variable UAS flight altitudes ranging from 5 to 30 m.
• A formal decision-level fusion algorithm that combines detections from both RGB and thermal branches using spatial proximity matching to produce a unified FOD inventory with reduced false negatives.
• Systematic evaluation of altitude effects on both RGB and thermal detection performance, including cross-site validation at the Cobb County Sport Aviation Complex demonstrating generalization beyond the training environment.

2. Materials and Methods

2.1. DualFOD Framework Overview

This work introduces DualFOD, a dual-branch sensing and detection framework that combines deep learning-based RGB object detection with unsupervised thermal anomaly analysis for FOD inspection using UAS imagery. The framework operates on synchronized RGB and thermal imagery captured from a single Skydio X10 platform equipped with a VT300-L sensor package. The RGB branch processes visible-light images using a custom-trained YOLO12 model enhanced with SAHI to localize and classify FOD instances. In parallel, the thermal branch analyzes radiometric images to extract objects displaying noticeable temperature contrast against the surrounding pavement using an ensemble of unsupervised blob detection methods. The independent detection outputs from both branches are then merged through a decision-level fusion algorithm to generate the final set of confirmed FOD candidates.

Figure 1 illustrates the complete DualFOD architecture. A single UAS platform captures both RGB and thermal imagery during flight. The outputs from the two independent processing branches are combined through spatial proximity matching to produce the final unified detection result.

Figure 1. Architecture of the proposed DualFOD framework integrating YOLO12-based RGB detection, unsupervised thermal anomaly extraction, and decision-level fusion. (a) RGB detection branch: SAHI tiling, YOLO12 backbone with attention-centric architecture, APAN-FPN neck, and detection head. (b) Thermal anomaly detection branch (unsupervised) and decision-level fusion algorithm.

The RGB detection branch utilizes the YOLO12 [21,22] architecture, which introduces architectural improvements over prior YOLO versions by integrating attention-guided feature extraction through an Attentive C2f backbone and a refined multi-scale APAN-FPN fusion strategy. These upgrades enable robust localization of low-contrast debris against runway pavements. YOLO12 was selected over prior versions (YOLOv8 [27], YOLOv9 [28], YOLOv10) based on its attention-centric design that allows the model to focus on subtle textural cues critical for distinguishing small FOD from surface noise such as cracks and stains. A quantitative comparison with alternative architectures including RT-DETR, YOLOv10, and YOLO11 is presented in Section 3.2 to validate this selection.

To address the challenge of detecting extremely small debris from high UAS altitudes, the SAHI (Slicing Aided Hyper Inference) framework [29] was integrated into the RGB detection pipeline. SAHI partitions each input image into overlapping tiles, applies YOLO12 detection on each tile independently, and then merges all predictions using confidence-weighted non-maximum suppression. The SAHI configuration parameters used in this work are summarized in Table 1.

Table 1. SAHI inference configuration parameters.

Parameter	Value
Slice dimensions	$640\times 640$ pixels
Overlap ratio	0.25 (25% overlap between adjacent tiles)
Confidence threshold	0.25
NMS IoU threshold for merging	0.45
Postprocess type	Non-maximum suppression (NMS)

2.2. FOD-UAS Dataset

2.2.1. Data Acquisition

A dataset of runway-like imagery, referred to as FOD-UAS, was collected using the Skydio X10 UAS platform equipped with the VT300-L sensor package. This payload includes a narrow RGB camera, a 1-inch wide-angle RGB camera, and a radiometric thermal sensor, enabling synchronized visible-light and thermal data acquisition. Table 2 summarizes the key specifications of the VT300-L sensor package used in this study. The primary emphasis was on small debris items such as screws, rocks, trash fragments, and loose pavement particles distributed over concrete and asphalt surfaces.

Table 2. Specifications of the Skydio X10 VT300-L Sensor Package.

Camera Type	Specifications
Narrow RGB Camera	46 mm equivalent focal length; 64 MP resolution; f/1.8 aperture; 50° field of view
1-inch Wide RGB Camera	20 mm equivalent focal length; 50 MP resolution; f/1.95 aperture; 93° field of view
Radiometric Thermal Camera	60 mm equivalent focal length; 640 × 512 px resolution; 41° field of view; <30 mK thermal sensitivity
Flashlight Module	22 lux measured at 3 m

Figure 2 illustrates representative examples of FOD items used in this study, and Figure 3 shows the UAS platform in operation acquiring top-down imagery of a simulated runway environment.

Figure 2. Typical foreign object debris items used in this study, including tools, metal fragments, and rubber materials.

Figure 3. UAS capturing top-down images of a simulated runway environment on asphalt.

All RGB frames were manually annotated under a single-class FOD scheme using the label obj. The dataset was randomly divided into training, validation, and test subsets while preserving scene diversity to reduce overfitting. The split is summarized in Table 3.

Table 3. FOD-UAS dataset split for training, validation, and testing.

Subset	Images	Proportion
Training	1379	91%
Validation	88	6%
Test	48	3%

2.2.2. Data Processing

Prior to model training, preprocessing and augmentation were applied to improve consistency and robustness across environmental conditions. Auto-orientation was applied using EXIF-based camera alignment correction, followed by resizing all frames to $640\times 640$ pixels for YOLO12 input compatibility. To improve robustness against illumination and background variation, the training data incorporated brightness augmentation ($\pm 15\%$), random horizontal flipping, and scale jittering. Mosaic augmentation was additionally applied to improve small-object representation and spatial context diversity. Table 4 summarizes the complete preprocessing and augmentation pipeline.

Table 4. Preprocessing and augmentation pipeline used prior to YOLO12 training.

Step	Description
Auto-orientation	EXIF-based camera alignment correction.
Image resizing	Frames resized to $640\times 640$ pixels for YOLO12 input compatibility.
Brightness jitter	Random brightness adjustment in the range of $\pm 15\%$.
Geometric augments	Horizontal flip, random scaling, and minor affine transforms.
Mosaic	Mosaic augmentation to improve small-object representation and context.

These steps increased effective data variability and improved recognition of minimally sized FOD across diverse runway textures and lighting. Outputs were generated at three detection scales per image to support multi-resolution feature learning, consistent with modern YOLO-based training strategies.

2.3. Unsupervised Thermal Anomaly Detection

Thermal sensing provides critical information for detecting debris that may not be visually apparent in RGB imagery. The thermal branch of the DualFOD framework operates entirely without supervised training by leveraging radiometric temperature maps captured from the VT300-L sensor package onboard the Skydio X10. This unsupervised design is a deliberate architectural choice motivated by three factors: (i) no publicly available annotated thermal FOD dataset currently exists, (ii) an unsupervised pipeline enables zero-shot deployment to new airport environments without requiring additional annotated data, and (iii) thermal imagery exhibits fundamentally different characteristics from RGB (lower spatial resolution, temperature-gradient-based contrast) for which classical blob detection methods are well-suited [26].

Each thermal image is first decoded into a calibrated temperature field where every pixel represents an estimated surface temperature in degrees Celsius. To enhance micro-contrast between debris and pavement, the thermal map is normalized and contrast-adjusted using adaptive histogram equalization. Small dark structures that deviate from the surrounding surface are then identified through a combination of three complementary blob-detection techniques: (i) Multi-scale black-hat morphology to reveal compact low-temperature artifacts, (ii) Difference-of-Gaussian (DoG) filtering to highlight localized temperature depressions, and (iii) Maximally Stable Extremal Regions (MSERs) for stable region proposal independent of object shape.

A pixel-level consensus rule is applied where a candidate region is retained only if detected by at least two of the three methods, reducing interference from runway texture patterns and thermal noise. Additional geometric filtering constrains the object dimensions and solidity to remove cracks, stains, and shadows. Finally, a temperature-contrast validation step is applied: the mean temperature inside each detected region is compared against a narrow outer boundary, and the region is accepted as potential debris only if the absolute temperature difference exceeds a defined threshold $\mathrm{\Delta }T$.

The threshold $\mathrm{\Delta }T>5$ °C was selected based on empirical analysis of the thermal characteristics observed in the FOD-UAS dataset. Metallic debris items such as bolts, wrenches, and fasteners consistently exhibited temperature differentials of 6–12 °C relative to the asphalt surface during daytime collection, while non-metallic items such as plastic fragments showed differentials of 3–7 °C. The 5 °C threshold represents the empirical operating point that maximizes the retention of true debris regions while suppressing false positives from surface texture variations, which typically produce differentials below 3 °C. A sensitivity analysis across threshold values is presented in Section 3.3.

2.4. Decision-Level Fusion Algorithm

To leverage the complementary strengths of visible-light and radiometric sensing, detections from both the YOLO12 RGB branch and the thermal anomaly extraction branch are combined through a decision-level fusion algorithm. Let ${\mathcal{D}}_{\mathrm{RGB}}=\{d_1,d_2,\dots ,d_m\}$ denote the set of m detections from the RGB branch, where each detection $d_i$ is characterized by its bounding box center coordinates $(x_i^{\mathrm{rgb}},y_i^{\mathrm{rgb}})$ and confidence score $c_i^{\mathrm{rgb}}$. Similarly, let ${\mathcal{D}}_{\mathrm{TH}}=\{t_1,t_2,\dots ,t_n\}$ denote the set of n detections from the thermal branch, where each $t_j$ is characterized by centroid coordinates $(x_j^{\mathrm{th}},y_j^{\mathrm{th}})$ and temperature contrast $\mathrm{\Delta }{T}_j$.

Since both the RGB and thermal sensors are mounted on the same Skydio X10 gimbal with fixed spatial offsets, and all imagery is captured from a nadir (top-down) perspective over approximately flat runway surfaces, the parallax between the two sensors is negligible at operational altitudes of 5–30 m. The detection coordinates from both branches therefore share an approximately common spatial reference frame without requiring explicit geometric registration. Future work will incorporate calibrated extrinsic parameters and homography-based alignment for oblique viewing angles.

For each thermal detection $t_j$, the spatial distance to every RGB detection $d_i$ is computed as:

$${\delta }_{ij}=\sqrt{{(x_i^{\mathrm{rgb}}-x_j^{\mathrm{th}})}^2+{(y_i^{\mathrm{rgb}}-y_j^{\mathrm{th}})}^2}$$

(1)

A thermal detection $t_j$ is classified as corroborated if there exists at least one RGB detection within a spatial proximity threshold $\tau$:

$$\mathrm{corroborated}\left(t_j\right)=\left\{\begin{array}{cc}\mathrm{True}\hfill & \mathrm{if}\underset{i}{min}{\delta }_{ij}<\tau \hfill \\ \mathrm{False}\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(2)

where $\tau$ is set to 50 pixels (approximately 5% of the image diagonal at $640\times 512$ resolution). Thermal detections that are not corroborated by any RGB detection are classified as thermal-only recoveries, representing debris that the RGB branch failed to identify. The fused detection set is then defined as the union:

$${\mathcal{D}}_{\mathrm{fused}}={\mathcal{D}}_{\mathrm{RGB}}\cup \{t_j\in {\mathcal{D}}_{\mathrm{TH}}\mid \neg \mathrm{corroborated}\left(t_j\right)\}$$

(3)

This union-based strategy ensures that all RGB detections are preserved while thermal-only detections supplement the final output, reducing the likelihood of missed debris. The complete fusion procedure is formalized in Algorithm 1.

Algorithm 1 DualFOD Decision-Level Fusion

Require: RGB detections ${\mathcal{D}}_{\mathrm{RGB}}$, Thermal detections ${\mathcal{D}}_{\mathrm{TH}}$, Proximity threshold $\tau$

Ensure: Fused detection set ${\mathcal{D}}_{\mathrm{fused}}$

1:  ${\mathcal{D}}_{\mathrm{fused}}\leftarrow {\mathcal{D}}_{\mathrm{RGB}}$ {Initialize with all RGB detections}

2:  for each $t_j\in {\mathcal{D}}_{\mathrm{TH}}$ do

3:    $d_{min}\leftarrow {min}_{d_i\in {\mathcal{D}}_{\mathrm{RGB}}}\sqrt{{(x_i^{\mathrm{rgb}}-x_j^{\mathrm{th}})}^2+{(y_i^{\mathrm{rgb}}-y_j^{\mathrm{th}})}^2}$

4:    if $d_{min}\ge \tau$ then

5:       ${\mathcal{D}}_{\mathrm{fused}}\leftarrow {\mathcal{D}}_{\mathrm{fused}}\cup \left\{t_j\right\}$ {Add thermal-only recovery}

6:    end if

7:  end for

8:  return ${\mathcal{D}}_{\mathrm{fused}}$

2.5. Experimental Settings

2.5.1. Training Configuration

A custom-trained YOLO12 detector was employed to localize foreign object debris from visible-light aerial imagery. The model was trained on the processed FOD-UAS RGB dataset described in Section 2.2 using the Ultralytics pipeline with adaptive momentum stochastic gradient descent. A square input resolution of $640\times 640$ was used, with batch normalization and momentum-based optimization to promote stable convergence. The core training configuration, hardware, and runtime are summarized in Table 5. Although the training schedule was set for 500 epochs, the early-stopping criterion (patience of 30 epochs) halted optimization at epoch 220 once validation performance plateaued.

Table 5. YOLO12 training configuration.

Parameter	Value
Model	YOLO12s
Epochs (max)	500
Early stopping epoch	220
Input size	$640\times 640$
Batch size	16
Optimizer	SGD with momentum (0.937)
Learning rate	0.01 (initial), cosine annealing
Device	CUDA
GPU	NVIDIA GeForce RTX 4070 (8 GB)
Training duration	≈28 h

2.5.2. Evaluation Metrics

The performance of the RGB detection branch is evaluated using standard object detection metrics. Precision measures the fraction of predicted bounding boxes that correspond to actual FOD instances. Recall measures the fraction of actual FOD instances that are successfully detected. Mean Average Precision (mAP) summarizes the precision–recall trade-off across varying confidence thresholds. Two mAP variants are reported: mAP@0.5, which uses a single Intersection-over-Union (IoU) threshold of 0.5, and mAP@0.5:0.95, which averages precision across IoU thresholds from 0.5 to 0.95 in steps of 0.05, following the COCO evaluation protocol.

For the thermal branch, which operates without bounding-box-level annotations, performance is assessed using detection-count-based metrics: the number of true detections, missed detections, and false positives are determined by visual comparison against known ground-truth FOD placements at each altitude. The fused output is evaluated by comparing the total detection count against the known ground truth to quantify the improvement gained by combining both modalities.

3. Experimental Results and Analysis

3.1. Training Convergence and Baseline Performance

The training process was monitored through multiple loss components. Figure 4 illustrates the steady reduction in bounding box regression loss, classification loss, and distribution focal loss across the training epochs, confirming proper optimization and stable learning behavior.

Figure 4. Training loss convergence for the YOLO12 custom-trained model. (a) Bounding box regression loss. (b) Classification loss. (c) Distribution focal loss.

Both mAP@0.5 and mAP@0.5:0.95 metrics showed a rising trend, stabilizing above 0.95 and 0.66 respectively during training, as shown in Figure 5. The precision-recall curve in Figure 6 remains close to the optimal upper-right boundary, achieving a peak score of 0.955 mAP@0.5 during validation. The normalized confusion matrix in Figure 7 validates that the detector maintains near-perfect background rejection while accurately identifying FOD instances, with a true-positive rate exceeding 0.94.

Figure 5. Progression of key accuracy metrics throughout model training. (a) mAP@0.5 during training. (b) mAP@0.5:0.95 during training.

Figure 6. Precision -recall curve showing strong separability for FOD classification. Note: because the dataset contains a single object class (obj), the per-class curve (light blue) and the all-classes curve (dark blue) are mathematically identical and overlap completely.

Figure 7. Normalized confusion matrix on the test set.

The YOLO12 model was evaluated on the held-out test set, with quantitative results summarized in Table 6. The evaluation demonstrates strong localization accuracy and a well-balanced trade-off between precision and recall.

Table 6. YOLO12 detector performance on test set.

Metric	Value
mAP@0.5:0.95	0.642
mAP@0.5	0.890
mAP@0.75	0.730
Precision	0.954
Recall	0.802

3.2. Comparison with State-of-the-Art Detectors

To contextualize the selection of YOLO12 for the DualFOD framework, Table 7 presents a comparison of representative small-scale detection architectures on the MS COCO val2017 benchmark [17,19,20,21]. All latency values are reported on an NVIDIA T4 GPU with TensorRT FP16, as published in the respective original papers.

Table 7. Comparison of detection architectures on the MS COCO val2017 benchmark (published results). Latency measured on T4 GPU with TensorRT FP16.

Model	mAP@0.5:0.95	Params (M)	FLOPs (G)	Latency (ms)
RT-DETR-R18	46.5	20.0	60.0	4.58
YOLOv10-S	46.3	7.2	21.6	2.49
YOLO11-S	47.0	9.4	21.5	2.46
YOLO12-S	48.0	9.3	21.4	2.61

YOLO12-S achieves the highest mAP among the compared small-scale models while maintaining competitive latency. Compared to RT-DETR-R18, YOLO12-S delivers 1.5% higher mAP while running approximately 42% faster with only 36% of the computation and 45% of the parameters [21]. Relative to YOLO11-S, YOLO12-S improves mAP by 1.0% with a marginal 6% increase in latency. These published benchmark results motivated the selection of YOLO12 as the RGB detection backbone, as its attention-centric architecture is particularly well-suited for detecting small, low-contrast objects such as runway FOD that benefit from the larger effective receptive field provided by the Area Attention mechanism.

On the FOD-UAS dataset specifically, the YOLO12-S model achieved 0.890 mAP@0.5 and 0.642 mAP@0.5:0.95 after training (Table 6). The gap between the COCO benchmark (48.0 mAP@0.5:0.95) and the FOD-UAS result (0.642) is expected given the domain shift to a single-class detection task on aerial runway imagery, where the primary challenge is distinguishing small debris from textured pavement rather than multi-class recognition.

Table 8 reports the published inference latency for each architecture on an NVIDIA T4 GPU, along with estimated throughput for the DualFOD pipeline components. Since the thermal anomaly extraction branch employs only classical image processing operations (adaptive histogram equalization, morphological filtering, DoG, MSER), its computational cost is substantially lower than the deep learning inference, enabling near-parallel execution on the UAS ground station.

Table 8. Inference latency comparison (T4 GPU, TensorRT FP16, published values) and estimated DualFOD pipeline throughput.

Pipeline Component	Latency (ms)	Est. FPS
YOLO12-S (single-shot, $640\times 640$)	2.61	∼383
YOLO12-S + SAHI (tiled, ∼9 slices)	∼23.5	∼42
Thermal anomaly extraction (CPU)	∼15–25	∼40–67
DualFOD full pipeline (parallel branches)	∼25	∼40

The SAHI tiled inference processes approximately 9 overlapping slices per frame at $640\times 640$ resolution, resulting in a throughput of approximately 42 FPS on a T4 GPU. When both branches execute in parallel, the overall DualFOD pipeline is bottlenecked by the slower branch, yielding an estimated throughput of approximately 40 FPS—sufficient for near-real-time inspection during post-flight processing or low-speed UAS sweeps. Deployment on embedded edge platforms such as the NVIDIA Jetson Orin would reduce throughput proportionally, and dedicated optimization through TensorRT INT8 quantization could partially offset this reduction [21].

3.3. Thermal Threshold Sensitivity Analysis

To validate the selection of the temperature contrast threshold $\mathrm{\Delta }T=5$ °C, a sensitivity analysis was conducted by varying the threshold across values of 2 °C, 3 °C, 5 °C, 7 °C, and 10 °C on the thermal test imagery. For each threshold, the number of true positive detections, false positives, and missed debris items were counted against the known ground truth. Table 9 presents the results.

Table 9. Thermal detection sensitivity to temperature contrast threshold $\mathrm{\Delta }T$ on the FOD-UAS test imagery (5 m altitude, 12 known debris items).

$\mathbf{\Delta }\mathit{T}$ (°C)	True Positives	False Positives	Missed	Precision
2	11	9	1	0.55
3	10	5	2	0.67
5	8	1	4	0.89
7	6	0	6	1.00
10	3	0	9	1.00

At lower thresholds ($\mathrm{\Delta }T=2$ °C), the pipeline captures more potential debris but also admits surface texture artifacts as false positives. Conversely, at higher thresholds ($\mathrm{\Delta }T\ge 7$ °C), non-metallic debris with moderate thermal contrast is missed. The threshold of 5 °C provides the best balance between detection completeness and false positive suppression for the operating conditions in this study.

3.4. RGB vs. Thermal vs. Fused Detection Comparison

To quantify the improvement provided by the dual-modality fusion strategy, the detection counts from the RGB-only, thermal-only, and fused DualFOD pipelines were compared against known ground-truth FOD placements across multiple altitudes. Table 10 summarizes the results from the primary test site.

Table 10. Detection count comparison: RGB-only vs. Thermal-only vs. DualFOD fused output at the primary test site.

Altitude	Ground Truth	RGB Only	Thermal Only	Fused	Thermal Recoveries
5 m	12	9	8	12	3
10 m	12	8	7	11	3
15 m	12	7	8	11	4
20 m	12	6	7	10	4
25 m	12	4	6	8	4
30 m	12	3	5	7	4

The “Thermal Recoveries” column indicates debris items that were missed by the RGB detector but successfully identified by the thermal branch and added to the fused output. These results demonstrate that the union-based fusion strategy consistently improves total detection count across all tested altitudes, with the greatest benefit observed at higher altitudes where RGB performance degrades due to reduced pixel resolution of small objects.

3.5. Visualization

3.5.1. Visualization of Different Sizes

Figure 8 shows an image of the asphalt surface with extremely small FOD, a metal fastener nut, captured at 5 m altitude. Thermal imaging reveals the debris much more clearly due to the significant difference in temperature between metal and asphalt, whereas the small metallic object is nearly invisible in the RGB frame due to its similarity in color to the surface. Figure 9 contains images of medium-sized debris (a screwdriver). The blue screwdriver clearly stands out in the RGB image but is difficult to locate in the thermal image due to its similar temperature to the asphalt at the time of capture. Figure 10 shows a large wrench that is easily identifiable in both modalities due to its distinct thermal signature and reflective characteristics. Figure 11 provides a close-up comparison of a small metal nut, further demonstrating the complementary nature of the two sensing modalities. We have marked the objects manually with a red circle to make the objects stand out in Figure 8, Figure 9, Figure 10 and Figure 11.

Figure 8. Thermal and RGB images of frame with small debris. (a) RGB image with small metallic debris. (b) Thermal image with small metallic debris.

Figure 9. Cropped RGB and thermal images highlighting medium debris. (a) Cropped RGB image centered on medium-sized screwdriver debris. (b) Thermal image showing medium-sized screwdriver debris.

Figure 10. Cropped RGB and thermal images highlighting large debris. (a) Cropped RGB image centered on large wrench debris. (b) Thermal image showing large wrench debris.

Figure 11. Close-up comparison of a metal nut in RGB and thermal modalities. (a) Close-up RGB image of small metallic FOD. (b) Close-up thermal image of small metallic FOD.

3.5.2. Visualization of Different Heights

The effectiveness of both detection branches at different UAS altitudes was evaluated using imagery captured between 5 and 30 m in 5-m increments on a test runway containing diverse FOD objects. The results are presented in Figure 12, Figure 13, Figure 14, Figure 15, Figure 16 and Figure 17. At lower altitudes (5–10 m), both RGB and thermal branches successfully detect most debris items. As altitude increases, RGB detection performance degrades as small debris occupies fewer pixels, while thermal detection maintains higher sensitivity due to the persistence of temperature contrast at greater distances. At 25–30 m, the thermal branch detects debris items such as the large wrench and metallic fasteners that the RGB branch fails to identify, demonstrating the altitude-dependent complementarity of the two modalities.

Figure 12. Comparison of RGB and thermal FOD detection from an altitude of 5 m. (a) RGB from 5 m. (b) Thermal from 5 m.

Figure 13. Comparison of RGB and thermal FOD detection from an altitude of 10 m. (a) RGB from 10 m. (b) Thermal from 10 m.

Figure 14. Comparison of RGB and thermal FOD detection from an altitude of 15 m. (a) RGB from 15 m. (b) Thermal from 15 m.

Figure 15. Comparison of RGB and thermal FOD detection from an altitude of 20 m. (a) RGB from 20 m. (b) Thermal from 20 m.

Figure 16. Comparison of RGB and thermal FOD detection from an altitude of 25 m. (a) RGB from 25 m. (b) Thermal from 25 m.

Figure 17. Comparison of RGB and thermal FOD detection from an altitude of 30 m. (a) RGB from 30 m. (b) Thermal from 30 m.

Both detection modalities remain functional across all tested heights, but the thermal branch proves more effective at locating metallic objects and any debris with significantly different thermal signatures than the underlying pavement. Detection accuracy decreases at heights of 20 m and above for both image types, and extremely small FOD such as small metal bolts or plastic pieces are not accurately detected above 10 m in either modality. Since most operational runways are large and accuracy at high altitudes is limited, either multiple low-altitude passes or multiple simultaneous drones would be required to effectively cover an entire runway.

Representative qualitative examples from the YOLO12–SAHI RGB detection branch are shown in Figure 18, demonstrating successful identification of multiple small FOD objects under significant illumination contrast and surface variation. Figure 19 shows corresponding thermal anomaly detection results.

Figure 18. Qualitative detection results using YOLO12 integrated with SAHI on test runway imagery. (a) Detection on RGB image with two small debris items. (b) Detection on RGB image with five debris instances.

Figure 19. Thermal anomaly detection results showing localized objects with distinct temperature differences from the runway surface. (a) Detection on thermal image with 7 small debris items. (b) Detection on thermal image with 8 debris instances.

3.6. Cross-Site Validation

To evaluate the generalizability of the DualFOD framework beyond the original training environment, a field validation was conducted at the Cobb County Sport Aviation Complex, a general aviation airfield with operational asphalt runway surfaces. Figure 20 shows the test site during data collection, with the Skydio X10 UAS in flight over the runway.

Figure 20. Data collection at the Cobb County Sport Aviation Complex, showing the Skydio X10 UAS in flight over the operational runway.

FOD items were placed on the runway surface, and synchronized RGB and thermal imagery was captured at altitudes of 10, 20, and 30 m. The detection results are shown in Figure 21, Figure 22 and Figure 23, and the detection counts are summarized in Table 11.

Figure 21. Cross-site validation: RGB and thermal detection results at 10 m altitude, Cobb County Sport Aviation Complex. (a) RGB from 10 m. (b) Thermal from 10 m.

Figure 22. Cross-site validation: RGB and thermal detection results at 20 m altitude, Cobb County Sport Aviation Complex. (a) RGB from 20 m. (b) Thermal from 20 m.

Figure 23. Cross-site validation: RGB and thermal detection results at 30 m altitude, Cobb County Sport Aviation Complex. (a) RGB from 30 m. (b) Thermal from 30 m.

Table 11. Cross-site detection count comparison at Cobb County Sport Aviation Complex.

Altitude	RGB Detections	Thermal Detections	Fused (Union)	Thermal Recoveries
10 m	8	5	10	2
20 m	6	8	11	5
30 m	4	12	14	10

The cross-site results reveal a notable trend: as altitude increases, RGB detection count decreases (from 8 at 10 m to 4 at 30 m) while thermal detection count increases (from 5 at 10 m to 12 at 30 m). This inversion occurs because the thermal sensor’s broader field of view and temperature-based contrast mechanism remain effective at higher altitudes, whereas the RGB sensor’s ability to resolve small objects diminishes with increasing distance. The fused DualFOD output consistently exceeds either single-modality branch across all altitudes, with thermal recoveries accounting for an increasing share of total detections at greater heights. At 30 m, the fusion process adds 10 thermal-only detections that the RGB branch entirely missed, more than tripling the RGB-only detection count. These results confirm that the DualFOD framework generalizes to unseen airfield environments and that the dual-modality fusion provides substantial detection improvement, particularly at operationally relevant higher altitudes.

4. Discussion

Discussion of Limitations and Recommendations

While the proposed DualFOD framework demonstrates strong performance in identifying foreign object debris across both RGB and thermal domains, several limitations were observed during experimentation. The primary challenge stems from false positives arising in thermal imagery due to irregular surface emissivity, sunlight reflections, or residual temperature gradients from recently heated pavement areas. These artifacts can occasionally mimic debris-like regions, particularly when captured under oblique viewing angles or rapidly changing illumination conditions.

Another limitation involves camera perspective and flight geometry. At extreme angles or variable altitudes, the effective pixel resolution of detected objects decreases, reducing localization precision for smaller debris. Similarly, the RGB detector may underperform when the object color closely matches the background, such as metallic or transparent debris partially embedded in the pavement texture. The current fusion approach operates at the decision level without geometric registration between RGB and thermal coordinate systems; while the nadir-viewing geometry and shared gimbal mount minimize parallax at the tested altitudes, oblique or side-looking configurations would require explicit calibration.

Environmental conditions also limit the generalizability of this study. The data were collected under specific weather and climate conditions in the southeastern United States, and this research does not account for how thermal imaging would perform across different ambient temperatures, weather conditions, and times of day. The reliance on classical unsupervised methods (DoG, MSER, black-hat morphology) for the thermal branch, while enabling zero-annotation deployment, may not achieve the detection precision of a supervised deep learning approach; however, this design choice provides practical scalability advantages as discussed in Section 2.3.

Future improvements should consider the integration of temporal filtering and multi-frame consistency checks to suppress transient anomalies in thermal frames. Incorporating a lightweight segmentation head or a transformer-based contextual attention module within the YOLO12 backbone may further enhance the model’s ability to distinguish true debris from background artifacts. Additionally, deploying on-board real-time fusion with geometric registration between RGB and thermal modalities using calibrated extrinsic parameters could enable more accurate multi-sensor decision making. Expanding the dataset to include diverse debris materials, weather conditions, and additional airfield types will further improve the generalizability of the framework for operational deployment.

5. Conclusions

This study presented DualFOD, an AI-assisted dual-modality framework for foreign object debris (FOD) detection using unmanned aerial system (UAS) imagery. The proposed method integrates a supervised YOLO12-based RGB detector enhanced with SAHI for small-object detection with an unsupervised thermal anomaly extraction pipeline, and combines their outputs through a formal decision-level fusion algorithm. The RGB model, trained on the custom FOD-UAS dataset, demonstrated strong precision and robust generalization on the held-out test set, achieving an mAP@0.5:0.95 of 0.642 and precision of 0.954. The thermal branch successfully detected rigid objects exhibiting distinct temperature deviations from their surroundings without requiring any annotated training data, providing valuable redundancy in low-contrast visual environments.

Cross-site validation at the Cobb County Sport Aviation Complex confirmed that the DualFOD framework generalizes to unseen airfield environments. At 30 m altitude, the thermal branch detected 12 objects compared to only 4 by the RGB branch, and the fused output yielded 14 total detections—more than tripling the RGB-only count at that altitude. These results demonstrate that decision-level fusion of thermal and RGB sensing substantially reduces false negatives and enhances detection reliability for automated runway inspection, particularly at higher operational altitudes where RGB performance degrades.

Future work will focus on real-time on-board implementation with calibrated RGB–thermal geometric registration, expansion of the FOD-UAS dataset to include additional debris types and environmental conditions, and systematic benchmarking on embedded edge computing platforms suitable for UAS deployment.