Bidirectional Global-Edge Refinement Network for Aerial Infrared Segmentation in Automated Industrial Inspection

Highlights

What are the main findings?

• A bidirectional global-edge refinement network is proposed for aircraft-region segmentation in physics-based simulated aerial infrared imagery. On AISD, the proposed method achieves an 86.29% mPA, 83.35% mF1, and 76.20% mIoU.
• DGAP and CFC strengthen multi-scale semantic representation, while ESSG and edge-refinement feedback improve boundary localization and target-contour consistency.

What are the implications of the main findings?

• The framework provides a useful scene-understanding component for image-assisted aviation inspection under simulated aerial infrared conditions.
• The results support the value of jointly optimizing global semantic context and local boundary structure for infrared target segmentation.

Abstract

Accurate aircraft-region segmentation in aerial infrared imagery can provide a useful scene-understanding component for image-assisted aviation inspection. However, target scale variation and ambiguous boundaries remain challenging in simulated aerial infrared scenes. To address these issues, this paper develops a bidirectional collaborative framework that couples global semantic fusion with edge optimization. The Semantics-guided Multi-scale Context Fusion branch uses adaptive multi-scale feature extraction and cross-stage calibration to produce a reliable initial segmentation, while the Edge-Sensitive Superpixel Collaborative Refinement branch improves boundary localization through semantic-guided superpixel consistency. Their interaction forms a bidirectional closed loop in which semantic context guides edge refinement and refined boundaries feed back to semantic learning. Experiments on AISD, a dedicated physics-based simulated aerial infrared dataset, achieve an 86.29% mPA, 83.35% mF1, and 76.20% mIoU. Relative to the strongest baseline, P-Mamba, the gains are 0.89 percentage points in mPA, 1.52 points in mF1, and 0.88 points in mIoU, and a sourcewise holdout comparison indicates consistent performance under unseen source content within the AISD simulation pipeline. These results concern aircraft-region segmentation rather than direct diagnosis of thermal defects.

1. Introduction

With the rapid advancement of aviation technology, infrared imaging has become indispensable in critical aviation safety applications such as aircraft fault detection [1,2,3,4], thermal anomaly monitoring, and vehicle health management. Its ability to provide reliable thermal information under low-visibility conditions offers significant advantages for proactive maintenance and safety assurance. In this context, addressing the dual challenges of multi-scale target complexity and edge ambiguity becomes essential, respectively necessitating robust global semantic understanding to identify aircraft regions and precise edge preservation to delineate target contours. Therefore, a unified framework integrating both capabilities is key to providing the reliable analysis needed for scene parsing that may support subsequent aviation inspection workflows.

In the domain of global semantic understanding [5,6,7], existing methodologies range from efficient yet limited seeded region growth techniques [8] to more advanced deep learning models such as FCN [9] and U-Net [10]. While these deep learning approaches enhance contextual modeling through multi-level feature integration [11], they often remain inadequate for handling the multi-scale target complexity and characteristically weak features present in aviation infrared images, frequently leading to incomplete target localization.

For edge preservation [12,13,14], techniques have evolved from traditional adaptive thresholding [15] and edge detection operators [16] to deep learning–based reconstruction methods [17]. However, when confronted with the significant edge ambiguity inherent to thermal imagery, these methods commonly face issues such as sensitivity to noise and limited generalization, struggling to recover crisp and accurate boundaries in complex aviation environments.

Aerial aircraft-region segmentation requires both coherent identification of aircraft structures (a global semantic task) and precise delineation of their boundaries (a local edge task). Consequently, researchers have continued to explore frameworks for integrating these two capabilities. Existing methods primarily concentrate on innovations in network architecture and optimization of feature fusion paradigms; for example, Chen et al. proposed an Edge-Enhanced U-Net network [18] that fused multi-level edge detection feature maps through skip connections and introduced an edge loss function to enhance contour learning. While it is effective on structured scenes, its reliance on localized edge cues often leads to fragmented semantic understanding and inconsistent performance in cluttered infrared backgrounds. Zhang et al. developed a Graph-CNN hybrid model [19] that innovatively combined graph convolutional networks with a CNN, utilizing graph structures to capture global semantic relationships while employing deformable convolution to enhance edge feature extraction. However, its graph-based topological modeling struggles with the high noise and low contrast typical of infrared images, resulting in suboptimal edge-semantic consistency. Ahmad et al. designed a Wavelet Transformer architecture [20] that decomposed high-frequency edge information through a wavelet transform, combined with Transformer’s long-range modeling capability. Nevertheless, the fixed nature of its frequency decomposition limits adaptability to the diverse edge profiles and scale variations present in aviation infrared scenes. Chen et al. proposed a double-branch parallel network [21], in which a high-resolution branch learns global context and a low-resolution branch captures local edge features, achieving feature interaction through cross-attention. A key limitation lies in its late-stage feature fusion, which may not fully resolve inherent conflicts between global semantic coherence and local boundary precision during learning.

In conclusion, existing approaches to global semantic understanding often fail to adequately address multi-scale target complexity in cluttered infrared scenes, while methods for edge preservation remain challenged by the edge ambiguity inherent in thermal imagery. Most importantly, although existing dual-branch networks attempt to co-optimize semantics and edges, they predominantly rely on parallel or late-fusion strategies. As illustrated in Figure 1a, this design fails to establish a bidirectional feedback mechanism between deep semantic guidance and edge detail refinement. Consequently, global and local features struggle to collaborate effectively. It should be noted that the specific requirements of aviation safety monitoring amplify these limitations: aircraft-region segmentation requires consistent semantic labeling and precise target contours as potential inputs to later inspection processes. The uniqueness of this application scenario requires that the algorithm simultaneously achieve adaptive receptive fields and accurate edge localization.

Figure 1. Comparison of segmentation results on an aerial infrared image. (a) A typical dual-branch network fails to effectively coordinate features. (b) The proposed method achieves bidirectional optimization of global and edge features.

The inherent synergy between semantics and edges suggests that they should not be treated in isolation: blurred boundaries can undermine semantic integrity, and inaccurate semantic regions can, in turn, mislead precise edge optimization. This interdependent relationship implies that addressing them as separate problems inevitably leads to a performance bottleneck. The core insight of this study, drawing inspiration from bio-inspired visual computing models [22] and multi-scale geometric analysis [23], is the establishment of a bidirectional closed-loop optimization mechanism between global semantics and local edges. As illustrated in Figure 1b, this mechanism enables continuous mutual enhancement, where semantic guidance steers edge refinement while edge feedback rectifies semantic representation.

The main contributions of this work are summarized as follows:

1.

Proposal of a bidirectional closed-loop architecture. A novel framework integrating multi-scale semantic fusion with superpixel-guided refinement is proposed, establishing a dedicated bidirectional interaction mechanism between global semantic representation and local boundary refinement beyond what conventional parallel or late-fusion paradigms provide.

2.

Novel technical components. This includes a multi-scale fusion network with Dynamic Gated Atrous Pyramid (DGAP) and Cross-stage Feature Calibration (CFC) for global context capture, as well as an Edge-Sensitive Superpixel Generation (ESSG) algorithm for boundary-aware refinement.

3.

Experimental validation on physics-based simulated aerial infrared imagery. Experiments on AISD, including a scene/sequence-disjoint standard split and a source-wise holdout protocol, evaluate the effectiveness and robustness of the proposed approach under diverse simulated aviation conditions.

2. System Framework

As shown in Figure 2, this paper proposes a bidirectional closed-loop architecture based on a dual-branch network structure. The framework maintains the parallel processing paradigm of conventional dual-branch networks while introducing a novel interactive mechanism between branches.

Figure 2. Framework of the proposed bidirectional closed-loop network. Branch (a) (top) performs global semantic understanding through DGAP and CFC, while Branch (b) (bottom) performs ESSG-based boundary refinement. The $r_1$–$r_4$ labels in DGAP denote the four fixed integer-dilation branches $\{2,4,8,16\}$; their contributions are dynamically gated rather than sampled at fractional dilation rates. The middle path shows semantic-to-edge guidance and edge-to-semantic feedback.

Branch (a) (Global Semantic Understanding) contains a Semantics-guided Multi-scale Context Fusion module designed to tackle the challenge of multi-scale target complexity. This branch focuses on capturing comprehensive contextual information from complex infrared scenes. Through Dynamic Gated Atrous Pyramid (DGAP) and Cross-stage Feature Calibration (CFC) mechanisms, it achieves robust semantic understanding of targets varying significantly in size and appearance, providing reliable global guidance for subsequent edge optimization.

In Branch (b) (Edge Optimization), an Edge-Sensitive Superpixel Collaborative Refinement method addresses edge ambiguity. This branch performs fine-grained segmentation along object boundaries by incorporating Edge-Sensitive Superpixel Generation (ESSG) and Superpixel Semantic Refinement. It is designed to improve boundary localization while preserving local structural continuity, providing geometric feedback to enhance the global semantic representation.

The core innovation lies in the bidirectional interactive mechanism between the two branches. Branch (a) provides high-level semantic guidance to Branch (b), steering the edge refinement process with global contextual understanding. Conversely, Branch (b) delivers refined edge information back to Branch (a), enabling boundary-aware correction of semantic predictions. During training, the semantic guidance signal is used by ESSG and superpixel semantic refinement, and the resulting refined edge map is detached from the discrete refinement path before it is used in the edge-feedback loss. During inference, ESSG and superpixel semantic refinement are still applied once to obtain the final refined mask, whereas the feedback loss and gradient update are disabled. This distinction makes the closed-loop training mechanism explicit while preserving the same forward refinement procedure at test time.

3. Semantics-Guided Multi-Scale Context Fusion

This section presents the semantic branch used to generate the guidance signal for subsequent superpixel refinement. As shown in Figure 2, a Dynamic Local Statistical Mapping Enhancement step is first applied to improve local contrast using patch-wise statistics; this fixed preprocessing step does not alter the supervision or the bidirectional interaction. The Dynamic Gated Atrous Pyramid (DGAP) then captures complementary receptive fields using dynamically weighted atrous branches, while Cross-stage Feature Calibration (CFC) calibrates shallow spatial details with deep semantic context.

3.1. Dynamic Gated Atrous Pyramid (DGAP)

The Dynamic Gated Atrous Pyramid (DGAP) is designed to capture multi-scale context in complex infrared scenes without imposing a fixed receptive-field preference. This is useful for aerial infrared targets because their apparent spatial support changes with target scale and viewing distance, while thermal diffusion and low local contrast can make the effective target region broader or less sharply bounded. Smaller receptive fields help preserve compact target structures, whereas larger receptive fields provide the surrounding context needed to distinguish spatially diffuse target responses from background clutter. DGAP therefore uses four complementary integer-dilation branches and adaptively emphasizes the receptive fields that are more informative for each input. Here, “dynamic” denotes sample-dependent branch weighting rather than non-integer atrous sampling.

Let $X\in {\mathbb{R}}^{H\times W\times C}$ be the input feature map. DGAP first predicts normalized branch weights $\mathbf{a}=[a_1,a_2,a_3,a_4]$ for the fixed dilation set $\mathbf{d}=[2,4,8,16]$:

$$\mathbf{a}=Softmax\left(W_2\delta \left(W_{1}GAP\left(X\right)\right)\right),\mathbf{d}=[2,4,8,16],$$

(1)

where $W_1$ and $W_2$ are learnable transformations, $\delta$ is ReLU, and ${\sum }_k{a}_k=1$. Thus, the global descriptor adjusts the relative contribution of small and large receptive fields while all dilation rates remain legal integers.

For Branch k, the gated atrous response is

$$Y_k=a_k\left[\sum _{i,j}X\left(p_i+d_{k}i,p_j+d_{k}j\right)W_k(i,j)\right]\odot \sigma \left(G_k\ast X\right),$$

(2)

where $W_k$ is the convolution kernel, $G_k$ is a $3\times 3$ depthwise separable convolution, and $\sigma$ is Sigmoid. The integer dilation $d_k$ controls receptive-field scale, $a_k$ provides sample-dependent scale weighting, and the spatial gate $\sigma (G_k\ast X)$ suppresses weak background responses. Standard atrous convolution is therefore used without fractional sampling or interpolation.

Finally, the four branch features are collaboratively fused as

$$F_{\mathrm{out}}={\displaystyle \sum _{k=1}^4}{\alpha }_{k}LN\left(Y_k\right),\mathit{\alpha }=Softmax\left({\left[v_k^{T}FC(GAP\left(Y_k\right))\right]}_{k=1}^4\right),$$

(3)

where LN is layer normalization and $\mathit{\alpha }$ denotes the fusion weights derived from the global responses of the four branches. The fused feature combines compact local structure with broader semantic context.

3.2. Cross-Stage Feature Calibration (CFC)

In infrared image segmentation, a significant gap exists between the spatially rich but semantically weak shallow features and the semantically strong but spatially coarse deep features. Traditional fusion methods such as concatenation often yield fragmented representations, limiting precise structure description and semantic guidance. To address this, the Cross-stage Feature Calibration (CFC) mechanism is proposed, as illustrated in the CFC module of Figure 2 Branch (a). The core component of this module is a spatial-channel cooperative attention unit, which leverages global semantics from deep features to adaptively calibrate shallow spatial details.

The feature $F_l\in {\mathbb{R}}^{H\times W\times C_l}$ extracted from the lth layer of the backbone network preserves detailed information like edges and textures. Simultaneously, deep features $F_{DGAP}\in {\mathbb{R}}^{H\times W\times C_d}$ are obtained from DGAP outputs, which contain higher-level semantic information describing target categories and overall structures through multi-scale feature extraction and fusion. To achieve effective fusion of shallow and deep features, a feature calibration unit is designed with the following formula:

$$F_l^{\prime }=C\left(F_l\right)=\mathrm{CA}\left(F_l\oplus \mathrm{Up}\left(F_{DGAP}\right)\right)\otimes \mathrm{SA}\left(F_l\right)$$

(4)

where shallow features $F_l$ are concatenated (⊕) with upsampled deep features $\mathrm{Up}\left(F_{DGAP}\right)$ to align them spatially, fusing shallow details with deep semantic information. A Channel Attention layer (CA) processes the concatenated features by learning weights for each channel, generating a $C_l$-dimensional weight vector that emphasizes important channel information and enhances the model’s response to crucial feature channels. A Spatial Attention layer (SA) processes the shallow features $F_l$ to generate an $H\times W$ weight map that adjusts feature distribution spatially, enhancing feature responses in target regions. Finally, elementwise multiplication (⊗) of the channel attention results with spatial attention results achieves calibration and optimization of shallow features through their cooperative action.

After calibration, the calibrated shallow features $F_l^{\prime }$ are adaptively fused with deep features $F_{DGAP}$ to obtain the final fused features using the following formula:

$$F_{\mathrm{final}}=F_{\mathrm{DGAP}}+\gamma ·{\mathrm{Conv}}_{1\times 1}\left(F_l^{\prime }\right)$$

(5)

where ${\mathrm{Conv}}_{1\times 1}$ adjusts the feature channel count to ensure compatibility between shallow and deep features in the channel dimension. The fusion coefficient $\gamma$ is dynamically computed based on global representations of both deep and calibrated shallow features:

$$\gamma =Sigmoid\left(w^T·\left[\mathrm{GAP}\left(F_{DGAP}\right);\mathrm{GAP}\left(F_l^{\prime }\right)\right]\right)$$

(6)

where GAP is applied to both deep features $F_{DGAP}$ and calibrated shallow features $F_l^{\prime }$ and the two global descriptors are concatenated. For each input sample, $w^T$ maps this concatenated descriptor to a single scalar and the Sigmoid function constrains the resulting fusion coefficient to $\gamma \in (0,1)$. Thus, $\gamma$ is a sample-dependent scalar rather than a channel-wise vector or spatial map. Compared with direct addition or a fixed fusion weight, the bounded adaptive coefficient allows the network to vary the contribution of calibrated shallow details while preserving $F_{DGAP}$ as a stable semantic residual.

Finally, the fused feature $F_{\mathrm{final}}$ is processed through convolution to obtain the semantic guidance signal $G_{sem}\in {\mathbb{R}}^{H\times W\times C}$, which will guide the superpixel segmentation in Section 4:

$$G_{sem}={\mathrm{Conv}}_{1\times 1}\left(F_{\mathrm{final}}\right)$$

(7)

4. Edge-Sensitive Superpixel Collaborative Semantic Refinement

Deep semantic segmentation models demonstrate strong pixel-level classification capabilities and effectively extract high-level semantic information. However, these models often exhibit limitations in edge region segmentation, frequently resulting in edge blurring and detail loss. To address this issue, an Edge-Sensitive Superpixel Collaborative Semantic Refinement approach is proposed. Building upon the semantic guided signal $G_{sem}\in {\mathbb{R}}^{H\times W\times C}$ output from Section 3, this method performs edge optimization on initial segmentation results. A dedicated edge refinement loss function is introduced to provide feedback to the network parameters in Section 3, establishing a reverse closed-loop optimization from edges to semantics.

4.1. Edge-Sensitive Superpixel Generation (ESSG)

The proposed Edge-Sensitive Superpixel Generation (ESSG) algorithm is designed to produce superpixels that adhere precisely to semantically meaningful boundaries. Its core innovation lies in leveraging high-level semantic guidance to inform the low-level clustering process. This semantics-aware approach enables the generated superpixels to respect color and spatial proximity while aligning with object contours indicated by deep features, thereby providing a boundary-aware geometric prior for the subsequent semantic refinement stage. The algorithm comprises the following steps:

Initialization of cluster centers: Semantic features are integrated with low-level features to define enhanced initial cluster centers:

$$C_k=\left(l_k,a_k,b_k,x_k,y_k,\mathrm{AvgPool}\left(G_{sem}\left(k\right)\right)\right)$$

(8)

where $(l_k,a_k,b_k)$ represent Lab color values, $(x_k,y_k)$ denote pixel coordinates, and $\mathrm{AvgPool}\left(G_{sem}\left(k\right)\right)$ represents the pooled semantic features.

Similarity metric calculation: Within the neighborhood of each cluster center, the distance between pixel i and cluster center k is computed as

$$\begin{array}{cc}\hfill D& =\sqrt{{\displaystyle \frac{d_c^2}{m^2}}+d_s^2+\beta ·d_{sem}^2}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill d_c& =\sqrt{{\left(l_i-l_k\right)}^2+{\left(a_i-a_k\right)}^2+{\left(b_i-b_k\right)}^2}\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill d_s& =\sqrt{{\left(x_i-x_k\right)}^2+{\left(y_i-y_k\right)}^2}\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill d_{sem}& ={∥\mathrm{AvgPool}\left(G_{sem}\left(i\right)\right)-\mathrm{AvgPool}\left(G_{sem}\left(k\right)\right)∥}_2\hfill \end{array}$$

(12)

where $d_c$, $d_s$, and $d_{sem}$ represent color, spatial, and semantic distances, respectively. In all experiments, the target number of superpixels is fixed to $K=1600$, the distance-balancing weight is $m=10$, and the semantic-distance coefficient is $\beta =0.4$. A local sensitivity check showed stable behavior for $\beta \in [0.3,0.5]$.

Update of cluster centers: Cluster center positions are updated by computing the mean of all pixels within each cluster.

For each cluster center, ESSG dynamically adjusts the window size $S_k$ based on local gradient variations, making superpixels more compact in high-gradient regions:

$$S_k=S_0\sqrt{{\displaystyle \frac{\nabla G_k}{\overline{G}}}}$$

(13)

where $S_0$ is the baseline window size determined by the $K=1600$ initial superpixel grid, $\nabla G_k$ is the average gradient in the cluster-center region, and $\overline{G}$ is the average gradient of the entire image. Thus, the local window scale is controlled directly by the target superpixel budget and the gradient ratio.

Gradient-aware optimization introduces gradient constraints into similarity metric calculation, enhancing superpixel boundary adaptation to complex texture regions:

$$D^{\prime }=D+{\lambda }_g·\nabla G$$

(14)

where $D^{\prime }$ is the improved distance, ${\lambda }_g$ is the gradient weight factor, and $\nabla G$ is the pixel gradient magnitude. We use ${\lambda }_g=0.5$ in all experiments; the observed stable range is ${\lambda }_g\in [0.4,0.7]$.

4.2. Superpixel Semantic Refinement

The Superpixel Semantic Refinement module leverages the geometric coherence of superpixels to resolve semantic ambiguities near object boundaries. Unlike methods that operate solely on pixel-level predictions, it uses superpixels as local structural units to enforce spatial consistency and boundary-aware label correction through semantic voting. The workflow is illustrated in Branch (b) of Figure 2 and detailed as follows.

4.2.1. Superpixel Semantic Category Statistics

For input image I, using the ESSG algorithm, divide it into K superpixel blocks, each containing n pixels, denoted as

$$S_i=\left\{p_1,p_2,\dots ,p_n\right\},i\in \{1,2,\dots ,K\}$$

(15)

The pixel-level semantic labels generated by the deep segmentation model are L, and so the semantic category set corresponding to superpixel $S_j$ is

$$S{p}_i=\left\{l_1,l_2,\dots ,l_n\right\},l_i\in \{0,1,2,\dots ,M\}$$

(16)

where $l_i=0$ represents background, while other values represent foreground categories; M is the total number of foreground categories. The proportion r of each category in the superpixel is calculated as

$$r_j={\displaystyle \frac{n_j}{n}},j\in \{0,1,2,\dots ,M\}$$

(17)

where $n_j$ is defined as the total count of pixels assigned to category j within the superpixel.

4.2.2. Semantic Category Ambiguity Refinement

To determine whether the semantic category of superpixel $S_j$ is clear, the following criterion is established.

Calculate the semantic category $l_{\max }$ with the largest proportion in the superpixel and its corresponding proportion $r_{\max }$:

$$l_{\max }=\arg \underset{j}{\max }\left(r_j\right),r_{\max }=\max \left(r_j\right)$$

(18)

If $r_{\max }$ falls below the ambiguity threshold ${\tau }_a=0.6$, $S_j$ is considered semantically ambiguous and requires correction.

For ambiguous superpixel $S_i$, correction rules are established based on color feature differences and semantic category distributions between superpixels: find the adjacent superpixel set $\mathrm{Neighbor}\left(S_i\right)$, where adjacency is determined based on geometric positions of superpixel blocks. Compute the color feature difference $\mu$ between $S_i$ and its nearest neighbor superpixel $\mathrm{Neighbor}\left(S_i\right)$:

$$\mu ={\displaystyle \frac{1}{n}}{\displaystyle \sum _{k=1}^n}∥C_i-C_k∥$$

(19)

where $C_i$ and $C_k$ are the average color features of $S_i$ and $\mathrm{Neighbor}\left(S_i\right)$, respectively.

The Lab-space color-similarity threshold is fixed to ${\tau }_c=15$. If $\mu >{\tau }_c$, the two superpixels are treated as dissimilar and the decision threshold T is computed from the semantic proportions within the superpixel region:

$$T={\displaystyle \frac{1}{n}}{\displaystyle \sum _{k=1}^n}{r}_k$$

(20)

If $\mu \le {\tau }_c$, $S_i$ and $\mathrm{Neighbor}\left(S_i\right)$ are treated as similar regions and the same semantic-consistency rule is applied with the ambiguity threshold ${\tau }_a$ as the confidence reference.

Semantic correction is then performed: if $r_{\max }>T$, the deep segmentation model’s result is retained; if $r_{\max }\le T$, all pixels in $S_i$ are relabeled with the semantic category $l_{\max }$.

The refined segmentation result $L_{\mathrm{refined}}$ is encapsulated as an edge optimization feedback signal $E_{\mathrm{ref}}$, which guides the optimization of the network in Section 3 during training:

$$E_{\mathrm{ref}}=\mathrm{Sobel}\left(L_{\mathrm{refined}}\right)$$

(21)

where $E_{\mathrm{ref}}$ represents the boundary map extracted from $L_{\mathrm{refined}}$ with values in the range [0, 1]. During training, it serves as a detached refinement-consistency target for the semantic prediction branch.

5. Loss Function Design

To achieve cooperative enhancement between semantic understanding and edge optimization, a multi-objective loss function comprising two key components is designed:

5.1. Base Segmentation Loss

During independent training of Section 3, a standard segmentation loss function is used to optimize parameters:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{base}}& ={\mathcal{L}}_{ce}\left(L_{\mathrm{base}},Y_{gt}\right)+{\lambda }_d{\mathcal{L}}_{\mathrm{dice}}\left(L_{\mathrm{base}},Y_{gt}\right)\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill L_{\mathrm{base}}& =\mathrm{Softmax}\left({Conv}_{1\times 1}\left(F_{\mathrm{final}}\right)\right)\hfill \end{array}$$

(23)

where ${\mathcal{L}}_{ce}$ represents the cross-entropy loss, ${\mathcal{L}}_{\mathrm{dice}}$ denotes the Dice loss, ${\lambda }_d$ is the hyperparameter controlling the Dice-loss weight, $L_{\mathrm{base}}$ is the original prediction result from the network in Section 3, and $Y_{gt}$ is the ground truth.

5.2. Edge Refinement Feedback Loss

This loss constitutes the reverse edge-to-semantic path of the bidirectional optimization. It compares the gradient of the original prediction $L_{\mathrm{base}}$ with the edge map obtained after superpixel refinement. Because ESSG and the relabeling operations are discrete, the refined edge target is explicitly detached before the loss is evaluated; gradients are propagated only through the semantic prediction branch:

$${\mathcal{L}}_{\mathrm{edge}\_\mathrm{fb}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{∥\nabla L_{\mathrm{base},i}-sg\left(E_{\mathrm{ref},i}\right)∥}_2^2$$

(24)

where $\nabla L_{\mathrm{base}}$ is the gradient map of $L_{\mathrm{base}}$ and $sg(·)$ denotes the stop-gradient operation. Consequently, prediction errors in the discrete refinement path cannot recursively update the pseudo-boundary itself. Ground-truth masks remain the direct supervision in ${\mathcal{L}}_{\mathrm{base}}$; replacing $E_{\mathrm{ref}}$ with $Sobel\left(Y_{gt}\right)$ would instead constitute an additional fully supervised boundary loss and is therefore conceptually different from the proposed refinement-consistency feedback. The configuration comparison in Section 7.3 provides supporting evidence for the feedback pathway, while a matched-refinement comparison is needed to isolate its independent effect.

The final joint training loss function is obtained as

$${\mathcal{L}}_{\mathrm{final}}={\mathcal{L}}_{\mathrm{base}}+\eta ·{\mathcal{L}}_{\mathrm{edge}\_\mathrm{fb}}$$

(25)

where $\eta$ is the weight coefficient for the edge feedback loss, controlling the guidance strength of edge optimization on network training.

The stop-gradient operation prevents differentiation through the discrete refinement target but does not guarantee that the target is error-free. As a result, the feedback term is best interpreted as a refinement-consistency constraint rather than independent ground-truth edge supervision. A controlled comparison with ground-truth-derived boundary supervision would be needed to separate these effects.

6. Experimental Setup

6.1. Experimental Dataset and Split Protocol

A prevalent challenge in evaluating aerial infrared segmentation is the scarcity of public datasets that simultaneously provide aerial viewpoints, infrared modality, and pixel-level structural annotations. To address this issue, we construct the Aircraft Infrared Simulation Dataset (AISD), a physics-based simulated aerial infrared segmentation benchmark containing 10,000 samples. The Alibaba Cloud Tianchi Laboratory Aircraft Detection Dataset (5000 source samples), Aircraft Context Dataset (2000 source samples), and MAR20 (3000 source samples) are used as source libraries for aircraft/scene diversity and geometric configuration; their original optical intensities are not directly treated as thermal measurements and are not converted to infrared by a simple pixel-level intensity remapping.

The final infrared imagery is generated with the Guanxiang V2.0 Optoelectronic Image Simulation Platform supplied by Mukun Zhihui (Suzhou, China) using a unified forward electro-optical simulation pipeline. Target/background radiative characteristics are configured from target geometry, temperature-field information, and infrared material properties, followed by atmospheric propagation and parameterized optical, detector, and electronic response modeling. Sensor-level effects, including detector noise, NETD-related sensitivity, non-uniformity, optical effects, and dynamic-range compression, are incorporated in the image-formation process, while aerial viewpoints are generated through multi-view and path-based sampling. This procedure standardizes the heterogeneous source content into a common simulated infrared image-formation domain. Accordingly, AISD is defined as physics-based simulated infrared imagery generated by the above forward image-formation chain; its grayscale values are simulation outputs rather than luminance values obtained by desaturating optical images. The pseudo-color panels in Figure 3 are visualization renderings of simulated infrared outputs only and are not used as network inputs.

Figure 3. Representative samples from AISD covering diverse aircraft types, target scales, viewpoints, flight conditions, operational backgrounds, and simulated thermal appearances. Pseudo-color panels are included only for visualization of the simulated infrared outputs and are not used as network inputs.

To ensure rigorous evaluation and prevent scene-level leakage, the 10,000 images are partitioned into 7000 training, 1000 validation, and 2000 testing images. The split is performed at the source-scene and aircraft-sequence/near-duplicate group level rather than by random image allocation: samples belonging to the same scene or sequence group are assigned entirely to one subset. Thus, the standard AISD test set is scene-/sequence-disjoint from training. In addition, Section 7.1 reports a sourcewise holdout experiment in which the entire MAR20 source is excluded from training and used only for testing, providing a stricter evaluation of cross-source generalization within the common simulation pipeline.

AISD provides a controlled, physics-based simulated infrared setting for aircraft foreground and structural-region segmentation. The simulation parameters emulate components of an electro-optical imaging chain but have not been quantitatively calibrated against a matched set of physically acquired thermal images in this study. Accordingly, the present benchmark measures performance within this simulation domain; validating radiometric fidelity and downstream fault/anomaly detection requires dedicated future evaluation.

6.2. Experimental Settings and Evaluation Metrics

During training, Adam optimizer is used with an initial learning rate of 0.0005 and Poly learning-rate scheduling. The network settings are summarized in Table 1. For ESSG, all compared superpixel methods use a matched target budget of approximately 1600 superpixels; the proposed method uses $m=10$, $\beta =0.4$, ${\lambda }_g=0.5$, ${\tau }_a=0.6$, and ${\tau }_c=15$.

Table 1. Segmentation network model training parameters.

Parameter	Value
Initial Learning Rate	0.0005
Learning Rate Decay Exponent	0.9
Weight Decay Parameter	0.00005
Batch Size	64
Maximum Training Epochs	300
Validation Interval Epochs	5

To conduct in-depth and comprehensive evaluation of the deep model’s segmentation performance, three key evaluation metrics were carefully selected: mean Pixel Accuracy (mPA), F1-score, and mean Intersection over Union (mIoU). These metrics quantitatively assess model performance in semantic segmentation tasks of different dimensions, ensuring objective and comprehensive evaluation results.

7. Results

7.1. Overall Framework Performance Evaluation

As shown in Figure 4, the Bidirectional Global-Edge Refinement results exhibit clearer boundary localization and better recovery of local details. Compared with the Semantics-guided Multi-scale Context Fusion model in Figure 4b, the refined results in Figure 4c better preserve detailed regions such as the tail wings, tail flames, and landing gear while reducing scattered background misclassifications.

Figure 4. Segmentation results on aerial infrared aircraft images. (a) Input infrared image. (b) Results of the Semantics-guided Multi-scale Context Fusion model. (c) Results of the Bidirectional Global-Edge Refinement model.

In the aircraft headlamp area shown in the first row of Figure 4, the initial prediction includes part of the lamp halo, whereas superpixel refinement yields a cleaner boundary after local label reassignment.

Additionally, superpixel semantic refinement suppresses small isolated misclassified regions in the illustrated complex backgrounds. For example, background regions between the aircraft wings in the second row and partial background pixels around the landing-gear contours in the last row are filtered out, yielding cleaner target boundaries.

Table 2 compares the proposed method with representative segmentation models, including general architectures (e.g., U-Net and DeeplabV3) and edge-/context-enhanced models (e.g., Edge-Enhanced U-Net and Graph-CNN Hybrid), using mean Pixel Accuracy (mPA), mF1-score, and mean Intersection over Union (mIoU).

Table 2. Segmentation accuracy on the standard AISD split and sourcewise holdout evaluation.

Model	mPA (%)	mF1 (%)	mIoU (%)
Standard AISD split (7000/1000/2000)
FCN	73.06	66.50	57.78
U-Net	74.25	68.25	60.18
Mask R-CNN	78.12	73.20	65.30
DeeplabV3	77.28	72.35	64.10
PSP-Net	80.02	75.58	68.40
DeeplabV3+	80.65	76.25	69.22
Edge-Enhanced U-Net [18]	79.20	74.80	67.50
Graph-CNN Hybrid [19]	81.35	77.10	70.15
Wavelet-Transformer [20]	83.70	79.40	72.80
Dual-Branch Attention [21]	84.25	80.10	73.95
P-Mamba [24]	85.40	81.83	75.32
Semantics-guided Multi-scale Fusion	84.93	81.50	74.40
Global-Edge Refinement (Proposed)	86.29	83.35	76.20
Sourcewise holdout: train on Tianchi + Aircraft Context; test on unseen MAR20
DeeplabV3+	71.25	–	63.40
P-Mamba [24]	81.30	–	71.12
Global-Edge Refinement (Proposed)	83.15	–	72.54

On the standard AISD split, the proposed model achieves the highest point estimates in Table 2: 86.29% mPA, 83.35% mF1, and 76.20% mIoU. Relative to P-Mamba, the strongest baseline in this comparison, the corresponding gains are 0.89, 1.52, and 0.88 percentage points. Across three independent training runs with different random seeds, the proposed model obtains an mIoU of $76.20\%\pm 0.18\%$, describing within-model run-to-run variation. The sourcewise holdout results in the lower panel of Table 2 provide an additional robustness check on unseen-source content.

To complement the standard split and evaluate source-level robustness, we conduct a sourcewise holdout experiment. Models are trained only on the Tianchi and Aircraft Context sources and evaluated without adaptation on the unseen MAR20 source. As shown in the lower panel of Table 2, the proposed method reaches 83.15% mPA and 72.54% mIoU, compared with 81.30%/71.12% for P-Mamba and 71.25%/63.40% for DeeplabV3+. The mIoU margin over P-Mamba therefore increases from 0.88 points on the standard split to 1.42 points under the unseen-source protocol.

7.2. Qualitative Analysis

7.2.1. Semantics Guided Multi-Scale Context Fusion

As shown in Figure 5c, the segmentation results after Semantics-guided Multi-scale Context Fusion processing show clear improvement in both global structure and local details. From a global perspective, the network accurately captures the overall contours of aircraft, particularly in regions with complex boundaries or strong background interference, such as wing and tail positions in the image, where segmentation results exhibit greater edge continuity. From a detailed perspective, compared to segmentation results of the DeeplabV3+ model (as shown in Figure 5b), multi-scale feature fusion significantly improves the ability to capture local target features, especially in small areas such as wingtips and aircraft tails, where missed and misclassified segments are substantially reduced.

Figure 5. The segmentation results on the infrared aircraft image. (a) Infrared aircraft image. (b) Segmentation results of the DeeplabV3+ model. (c) Segmentation results of the Semantics-guided Multi-scale Context Fusion model.

7.2.2. Edge-Sensitive Superpixel Collaborative Semantic Refinement

As shown in Figure 6, applying ESSG to the infrared aircraft images produces clearer boundary-aligned region partitions.

Figure 6. Comparison of superpixel segmentation. (a) Input infrared image. (b) SLIC superpixels. (c) ESSG superpixels.

To directly evaluate superpixel quality rather than downstream segmentation accuracy, Table 3 reports Boundary Recall (BR), Achievable Segmentation Accuracy (ASA), and the average number of generated superpixels $N_{sp}$. All methods are evaluated with a matched budget of approximately 1600 superpixels. Gradient-based Watershed obtains the lowest BR (72.40%) and ASA (75.10%), while SLIC reaches 81.20% BR and 83.50% ASA. ESSG achieves 87.60% BR and 89.20% ASA at a comparable superpixel count, providing direct evidence that the semantic- and gradient-aware clustering produces superpixels that adhere more closely to target boundaries.

Table 3. Intrinsic quality comparison of superpixel algorithms under a matched superpixel budget.

Method	Boundary Recall (%)	ASA (%)	${\mathit{N}}_{\mathit{sp}}$
Gradient-based Watershed	72.40	75.10	∼1580
SEEDS	78.50	81.30	∼1600
SLIC [25]	81.20	83.50	∼1600
ESSG (proposed)	87.60	89.20	∼1600

7.3. Ablation Study

Table 4 first shows the progressive contribution of the semantic modules and then compares the two directional configurations of the proposed interaction. The baseline achieves 55.30% mIoU; DGAP and CFC increase it to 62.10% and 67.40%, respectively. With semantic-to-edge guidance through ESSG but without the edge-feedback loss, performance reaches 72.50% mIoU. Conversely, retaining edge-to-semantic feedback while replacing semantic-guided ESSG with standard SLIC gives 71.82% mIoU. The full bidirectional configuration reaches 76.20% mIoU, exceeding the semantic-to-edge-only and edge-to-semantic-only variants by 3.70 and 4.38 percentage points, respectively.

Table 4. Ablation study of the proposed modules and bidirectional interaction (Unit: %).

Configuration	mPA	mF1	mIoU
Progressive and directional ablation
Baseline (no key modules)	68.50	62.80	55.30
+DGAP	73.20	68.40	62.10
+DGAP + CFC	76.80	72.15	67.40
Semantic → Edge only (DGAP + CFC + ESSG; no edge-feedback loss)	80.25	76.80	72.50
Edge → Semantic only (edge-feedback loss; standard SLIC without semantic guidance)	78.90	75.30	71.82
Full bidirectional configuration (Proposed)	86.29	83.35	76.20
Module-specific variants
DGAP: fixed dilation + standard convolution	70.10	65.20	59.80
DGAP: integer multi-rate branches + dynamic gating	73.20	68.40	62.10
CFC: feature concatenation	74.60	69.80	64.20
CFC: spatial-channel cooperative calibration	76.80	72.15	67.40
ESSG: traditional SLIC segmentation	77.30	73.50	68.90
ESSG: semantic guidance + gradient-aware adaptive clustering	80.25	76.80	72.50

The module-specific variants provide consistent supporting evidence. Integer multi-rate DGAP with dynamic gating improves mIoU from 59.80% to 62.10% relative to fixed-rate standard convolution; CFC improves from 64.20% to 67.40% relative to direct concatenation; and semantic/gradient-aware ESSG improves from 68.90% to 72.50% relative to SLIC. For ESSG, the selected settings are $m=10$, $\beta =0.4$, ${\lambda }_g=0.5$, $K=1600$, ${\tau }_a=0.6$, and ${\tau }_c=15$. A local sensitivity check around the selected values showed stable behavior for $\beta \in [0.3,0.5]$ and ${\lambda }_g\in [0.4,0.7]$, reducing sensitivity to small parameter perturbations.

8. Discussion

8.1. Interpretation of Global-Edge Refinement

The behavior shown in Figure 4 is consistent with the use of ESSG superpixels as local structural units together with the semantic predictions from the global branch. The proposed refinement uses superpixel blocks generated by ESSG as local segmentation units and performs a second-stage label correction using semantic information from the Semantics-guided Multi-scale Context Fusion branch. By dividing background areas into superpixel blocks and reassigning category labels according to semantic consistency, the refinement process removes scattered false positives while preserving the main aircraft contours.

Furthermore, shallow features extracted from intermediate layers of the backbone network, rich in edge and texture details, are combined with deep semantic features to further improve segmentation accuracy in boundary regions of complex scenes. For example, although brightness differences between aircraft wing edges and background in infrared images are small, the model effectively distinguishes targets from the background by the CFC, significantly reducing misclassified regions.

In contrast to SLIC [25], ESSG uses semantic guidance in the local adaptive clustering process, which helps the generated superpixels follow object contours in low-contrast regions such as aircraft wings and tail flames. In complex backgrounds, the same clustering strategy also produces more coherent background superpixels and reduces small isolated regions relative to SLIC.

8.2. Generalization Scope

Because AISD is generated through a physics-based electro-optical simulation pipeline, the present experiments characterize the proposed method under simulated aerial infrared imaging and source-level content shifts.

The point estimates in the lower panel of Table 2 indicate that the proposed global-edge refinement retains a higher mIoU when the test source is unseen during training, supporting transfer across source-library scene content within the shared simulation process. This protocol does not evaluate sensor-to-sensor transfer or simulation-to-real generalization.

The comparison models in Table 2 include general segmentation architectures and transferable architectures originating in other imaging tasks. They are evaluated here as architecture-level baselines on AISD, rather than as a comprehensive ranking of task-specific thermal segmentation systems. Future benchmarking against dedicated aerial thermal segmentation methods under harmonized training settings will further clarify task-level competitiveness.

8.3. Limitations and Future Work

Because corresponding repeated-run statistics for P-Mamba and the other baselines are unavailable, the point-estimate differences in Table 2 are not presented as statistically significant improvements.

The configurations in Table 4 show the performance of the implemented full system relative to two reduced variants. However, the edge-to-semantic-only variant also replaces ESSG with SLIC; therefore, this comparison jointly changes guidance and refinement design and cannot by itself isolate the causal effect of information-flow direction. A fully controlled directional study retaining the same superpixel mechanism would be useful for further evaluation.

A priority for future work is validation against physically acquired aerial thermal imagery, including quantitative assessment of simulation-to-real differences and heterogeneous sensor domains. Further work will investigate robustness under more complex thermal backgrounds and imaging conditions, broaden sourcewise validation, and develop lightweight real-time implementations for deployment on aerial inspection platforms. Additional priorities include controlled direction-only ablations, ground-truth boundary supervision comparisons, repeated runs for strong baselines, and benchmarking against task-specific thermal segmentation methods.

9. Conclusions

This paper presents a bidirectional global-edge refinement framework for aerial infrared segmentation, coupling multi-scale semantic modeling with edge-sensitive superpixel refinement. On the standard AISD split, the complete model achieves an 86.29% mPA, 83.35% mF1, and 76.20% mIoU, corresponding to gains of 0.89, 1.52, and 0.88 percentage points over P-Mamba. The reported configuration comparisons favor the full architecture, although the directional variants also differ in refinement design, while the intrinsic superpixel evaluation confirms improved BR and ASA for ESSG at a matched superpixel budget. In the sourcewise holdout context, the proposed method obtains a 72.54% mIoU on the unseen MAR20 source, compared with 71.12% for P-Mamba.