Skip to Content
Applied SciencesApplied Sciences
  • Article
  • Open Access

3 June 2026

18 Pages

Aircraft Classification via Dual-Branch Color–Shape Feature Learning and Cross-Attention Fusion

and
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
*
Author to whom correspondence should be addressed.
This article belongs to the Section Computing and Artificial Intelligence

Abstract

Aircraft type classification plays a crucial role in various applications, including remote sensing, surveillance, and aviation management. Since the development of deep learning techniques, nearly all related methods are based on neural networks, achieving excellent classification results. However, existing classification networks primarily focus on optimizing single-branch architectures, often overlooking the underlying factors driving recognition performance. Our analysis suggests that color and shape are two important and complementary visual cues for aircraft classification, with their relative importance varying across datasets and imaging scenarios. Motivated by this insight, we propose a novel dual-branch network architecture that separately processes shape and color cues, allowing each branch to emphasize one type of visual information before adaptive fusion. Specifically, we designed two dedicated modules: a Shape Feature Module (SFM) and a Color Feature Module (CFM), tailored for extracting shape and color information independently. Furthermore, we introduced a Color–Shape Cross-Attention-based Fusion Module (CSCAFM) to integrate these features. Within CSCAFM, the separated shape and color features are adaptively fused through a cross-attention mechanism, enabling the network to dynamically weigh the contributions of shape and color. Experimental results on benchmark datasets demonstrate the effectiveness of our approach.

1. Introduction

Aircraft type analysis is a fundamental task in remote sensing [1,2], aviation management [3], and defense systems [4,5]. It plays a pivotal role in applications such as airport traffic monitoring, military surveillance, and disaster response planning. Despite its importance, aircraft classification presents several significant challenges. Firstly, aircraft often exhibit high intra-class variability due to differences in design, modifications, and operational customizations. For example, variations in livery, structural adaptations, or payload configurations can lead to substantial diversity within the same type of aircraft. Secondly, inter-class similarity, particularly among certain categories of military or civilian aircraft, makes distinguishing between classes highly complex. Environmental factors such as occlusions, shadows, and varying viewing angles further compound the difficulty of accurate recognition.
To address these challenges, earlier approaches relied heavily on handcrafted features extracted from visual characteristics, such as texture, shape, and color. Traditional machine learning techniques, including support vector machines and decision trees, were then employed for classification. However, these methods often suffered from limited generalization capability and were highly sensitive to variations in the dataset. The advent of deep learning brought significant advancements, with convolutional neural networks (CNNs) becoming the standard for feature extraction and classification [6,7]. These networks demonstrated superior performance by automatically learning hierarchical features from data. Meanwhile, with the growing availability of remote sensing data, significant progress has been made using deep learning models [8,9,10] to improve classification accuracy. However, existing approaches primarily rely on single-branch networks and focus on refining network architectures [11,12], and often overlook the inherent challenges of aircraft recognition tasks. Moreover, current methods typically fail to adequately address the critical attributes that define aircraft types, such as color and shape. Single-branch architectures tend to conflate these features, limiting the model’s ability to capture their distinct contributions effectively. As a result, classification performance remains suboptimal in diverse scenarios, particularly in cases where subtle differences in color or structural design are crucial for accurate recognition.
Our analysis suggests that color and shape are two important and complementary visual cues for aircraft type recognition. In most cases, shape serves as the primary criterion for distinguishing aircraft models, as structural differences are the main features that set aircraft apart. However, color can also be critically important in specific contexts. For instance, certain aircraft types are characterized by fixed color schemes, and even within the same model, variations in color may exist due to differing uses or customizations. Specifically, as depicted in Figure 1a, color plays a pivotal role in distinguishing military and civilian aircraft, as military aircraft typically exhibit uniform, stealth-oriented colors, while civilian aircraft display vibrant and diverse liveries. On the other hand, shape is crucial when identifying specific aircraft models, where structural differences are subtle but distinctive. These observations suggest the usefulness of separately modeling color and shape cues before adaptively integrating them for aircraft classification. Existing methods [13,14,15,16] often rely on single-branch architectures that learn mixed visual representations, without explicitly modeling the complementary roles of shape and color cues, as illustrated in Figure 1b. This limitation motivates our novel dual-branch network architecture, designed to separately process shape and color features and subsequently fuse them through cross-attention, leading to improved classification performance.
Figure 1. (a) Aircraft image examples highlighting classification cues: military aircraft feature distinct color patterns from civilian types, while civilian types rely on subtle shape differences. (b) Comparison of single-branch methods vs. our dual-branch network: prior approaches extract features jointly.
In this paper, we propose a novel framework featuring a dual-branch network architecture to address these challenges. Specifically, we design two dedicated modules: a Shape Feature Module (SFM) and a Color Feature Module (CFM), tailored for extracting shape and color information independently. To enable the SFM to focus exclusively on shape features, a shape encoder is first employed to extract the features, followed by a dedicated shape decoder that predicts an edge map. Since edge maps lack color information and emphasize structural details, this approach ensures the SFM captures shape-specific features effectively. For the CFM, we preprocess the input RGB image by converting it into multiple channels representing different color spaces. This design enhances the network’s ability to learn inter-channel dependencies, thereby strengthening the extraction of color-related features. Furthermore, we introduce a Color–Shape Cross-Attention-based Fusion Module (CSCAFM) to integrate these features. Within this CSCAFM, the decoupled features are adaptively fused through a cross-attention mechanism, enabling the network to dynamically weigh the contributions of shape and color based on task requirements. Experimental results on benchmark datasets demonstrate the effectiveness of our approach, showing improved performance over the evaluated baselines and providing useful insights into shape–color feature interactions in aircraft recognition tasks. The key contributions of this work are summarized as follows:
  • We analyze color and shape as two important and complementary visual cues in aircraft classification, and propose a dual-branch framework to separately model these cues before adaptive fusion.
  • We propose a novel dual-branch network architecture featuring a color branch for channel-specific color extraction, a shape branch for enhanced structural sensitivity, and a color–shape cross-attention mechanism that adaptively integrates the separated shape and color features for final classification.
  • Extensive experiments on three benchmark datasets demonstrate the effectiveness and generalization ability of the proposed method for aircraft classification.

3. Method

3.1. Overview

To address the challenges in aircraft classification tasks, we propose a dual-branch network architecture designed to separately model shape and color cues, enabling specialized processing for each attribute. The architecture consists of three main modules: a Shape Feature Module (SFM), a Color Feature Module (CFM), and a Color–Shape Cross-Attention-based Fusion Module (CSCAFM) as illustrated in Figure 2. Each branch independently processes complementary information before dynamic integration. The Shape Module extracts structural details essential for distinguishing aircraft with similar color patterns but varying geometries. In contrast, the Color Module captures subtle chromatic variations critical for identifying aircraft types with similar shapes. The outputs from these branches are adaptively fused using a channel-level attention mechanism, allowing the model to balance the contributions of shape and color features dynamically, depending on task requirements.
Figure 2. Architecture of the proposed dual-branch network: the Shape Feature Module (SFM) extracts structural details and predicts an edge map to emphasize shape cues; the Color Feature Module (CFM) captures inter-channel dependencies across color spaces. Both features are fused via a Color–Shape Cross-Attention Fusion Module (CSCAFM), which adaptively balances their contributions for enhanced classification.

3.2. Shape Feature Module (SFM)

To explicitly extract shape-related features, the proposed Shape Module leverages an edge-detection-guided approach, emphasizing the structural attributes of the input image. The module incorporates both preprocessing and supervised learning components to ensure robust extraction of shape cues. The proposed SFM differs from directly feeding edge maps into a classifier. Instead, the original image is still used as the input, while the edge map is used as an auxiliary supervision signal. This design encourages the intermediate representation to preserve aircraft-specific structural information without discarding contextual visual cues. Therefore, SFM provides an explicit structural constraint on feature learning rather than merely introducing an additional edge input.
The first step in SFM involves generating an edge map E that serves as a structural representation of the input image I. This is achieved by applying an edge detector E D :
E = E D ( I ) .
The edge map generated by the traditional edge detector is used as a pseudo-structural label rather than manually annotated ground truth. It provides approximate boundary supervision for the shape branch, encouraging the network to focus on aircraft contours and structural cues. Next, the input image I is fed into a shape encoder E S to obtain deep shape features:
F shape = E S ( I ) .
These shape features are then passed through a shape decoder D S to predict the edge map E ^ :
E ^ = D S ( F shape ) .
A pixel-wise loss L edge compares E ^ to the pseudo-label edge map E gt :
L edge = − 1 N ∑ i = 1 N E gt , i log ( E ^ i ) + 1 − E gt , i log 1 − E ^ i ,
where N is the total number of pixels and i indexes each pixel. This loss encourages the shape branch to learn high-fidelity structural features. It should be noted that the edge map generated by a traditional detector is treated as a pseudo-label rather than an exact ground truth annotation. Such pseudo-edge labels may contain noisy responses from background regions, fragmented contours, or weak object boundaries. However, in our framework, the edge prediction loss is used only as an auxiliary supervision signal to guide the shape branch toward structural representation learning, rather than as the primary optimization objective. Therefore, moderate noise in the edge map does not directly determine the final classification result. The final training objective is still dominated by the classification loss, while the edge loss provides additional regularization for structural feature learning.

3.3. Color Feature Module (CFM)

CFM is designed to explicitly extract color features by focusing on channel-specific information, leveraging multiple color spaces and efficient convolutional operations. This module encourages the color branch to focus more on chromatic information while reducing direct dependence on structural edge cues. To capture diverse color characteristics, we transform the input image I into multiple color channels across three color spaces: RGB, HSV, and YCbCr. The selection of the eight color channels is motivated by their complementary roles in representing chromatic information. RGB preserves the original image intensity and color distribution, HSV provides a perceptually meaningful representation by separating hue and saturation from raw intensity, and YCbCr separates luminance from chrominance components. Specifically, we use R, G, B, H, S, Y, Cb, and Cr channels because they jointly encode original color responses, hue/saturation information, luminance variation, and chrominance differences. The V channel in HSV is not included because its intensity information largely overlaps with the Y component in YCbCr and the RGB channels. Therefore, the selected eight channels provide a compact yet complementary representation of aircraft appearance while avoiding excessive redundancy. To further justify this design, we conduct an ablation study with different color-space combinations, as reported in Section 4.6.
Specifically, the module employs eight channels: Red (R), Green (G), Blue (B), Hue (H), Saturation (S), Luminance (Y), Blue Difference ( C b ), and Red Difference ( C r ). These channels are defined as follows:
C = ColorTransform ( I ) ,
where C = { R , G , B , H , S , Y , C b , C r } and ColorTransform denotes the operations to derive the corresponding color channels. This multi-space strategy enriches the representation by capturing complementary color attributes, ensuring the module focuses solely on color-related features.
Since the resulting representation contains more than three channels, a preprocessing step is introduced to adapt it to standard backbone networks. Specifically, we employ a 1 × 1 convolution to project the multi-channel input into a compact three-channel feature map:
C comp = W comp ∗ C + b ,
where C comp is the transformed three-channel feature representation, W comp represents the convolutional weights, b is the bias term, and ∗ denotes the convolution operation. This step compresses the expanded color space representation into a format suitable for deeper feature extraction.
The compressed feature C comp is then fed into a color encoder E c to extract an initial color representation, denoted as F c ″ . To further enhance the discriminative capacity of color features, we introduce a channel enhancement module, as illustrated in Figure 3a. Concretely, F c ″ is first processed by a 1 × 1 convolution to produce an intermediate feature F c ′ . In parallel, F c ″ is passed through a squeeze-and-excitation (SE)-like structure, which consists of a global average pooling (GAP) layer followed by a fully connected (FC) layer and a sigmoid activation function. This branch generates adaptive channel-wise attention weights that emphasize informative color channels. The intermediate feature F c ′ is then reweighted using the learned channel attention, and subsequently combined with the original feature via element-wise multiplication and residual addition. The final refined color feature F c is thus obtained, which encodes enhanced and selectively emphasized color information for downstream tasks. This design is motivated by the inherent characteristics of color features, which are primarily associated with the channel dimension rather than spatial relationships between neighboring pixels. This operation allows the network to selectively combine and transform the input channels, encouraging the branch to emphasize color-specific information while reducing direct dependence on structural edge cues.
Figure 3. (a) Illustration of the channel enhancement module in CFM. (b) Illustration of the Color–Shape Cross-Attention-based Fusion Module (CSCAFM).

3.4. Color–Shape Cross-Attention Fusion Module (CSCAFM)

The fusion of shape and color features is critical for fine-grained classification tasks, as these attributes capture complementary aspects of the object representation, as illustrated in Figure 3b. CSCAFM is designed to attentively integrate the decoupled color and shape feature maps, enabling the network to dynamically prioritize the contributions of each feature type. The shape feature map F s ∈ R h × w × d and the color feature map F c ∈ R h × w × d are first flattened along the spatial dimension, yielding X s , X c ∈ R h w × d . For multi-head cross-attention with n heads, the projected query, key, and value features are denoted as Q s , K s , V s , Q c , K c , V c ∈ R n × h w × d h , where d h is the feature dimension of each attention head. The shape-to-color attention map is computed as follows:
M s → c = Softmax Q s K c T d h ,
where M s → c ∈ R n × h w × h w represents the spatial attention weights from shape queries to color keys. The corresponding attended color-aware shape representation is then obtained by the following:
Z s → c = M s → c V c ,
where Z s → c ∈ R n × h w × d h . Therefore, M s → c is the attention map, while Z s → c is the attended feature representation.
Similarly, the color-to-shape attention map and attended shape-aware color representation are defined as follows:
M c → s = Softmax Q c K s T d h ,
Z c → s = M c → s V s ,
where M c → s ∈ R n × h w × h w and Z c → s ∈ R n × h w × d h . The outputs from all attention heads are concatenated and linearly projected:
F ^ s → c = Reshape Concat ( Z s → c 1 , … , Z s → c n ) W o ,
F ^ c → s = Reshape Concat ( Z c → s 1 , … , Z c → s n ) W o ,
where F ^ s → c , F ^ c → s ∈ R h × w × d . Finally, the fused feature is constructed by concatenating the original and cross-attended features, followed by a 1 × 1 convolution:
F f = Conv 1 × 1 [ F s , F c , F ^ s → c , F ^ c → s ] ,
where F f ∈ R h × w × d is the final fused feature used for classification. By leveraging cross-attention, CSCAFM adaptively integrates color and shape information, ensuring that the network focuses on the most relevant features for a given task. This approach allows the network to dynamically balance the contributions of color and shape, enhancing its capability to handle complex classification scenarios.
Different from simple feature concatenation, addition, or single-direction attention, CSCAFM performs bidirectional interaction between the shape and color branches. The shape-to-color attention models which appearance cues are relevant to structural regions, while the color-to-shape attention models which structural cues should be emphasized under specific appearance patterns. This bidirectional re-alignment enables adaptive fusion of complementary aircraft cues before classification.

4. Results and Discussions

4.1. Data Preprocess

Datasets. We evaluate our method on three aircraft classification datasets, including FGVC-Aircraft [34], MTARSI [24], and MADAI [13], as shown in Figure 4. FGVC-Aircraft is a fine-grained aircraft recognition dataset containing 10,200 images from 102 aircraft variants, with 100 images for each class. Following the official protocol, the dataset is divided into 3400 training images, 3400 validation images, and 3400 testing images. MTARSI is a remote sensing aircraft recognition dataset containing 9385 aircraft images from 20 aircraft categories. The images are collected from remote sensing scenes and contain aircraft with different scales, viewpoints, backgrounds, and imaging conditions. Since the original dataset does not provide a fixed official train/test split, we adopt a stratified 80/20 train/test split, resulting in 7508 training images and 1877 testing images. MADAI is a military aircraft dataset designed for aircraft detection and classification in aerial images. It contains 2558 images from 5 aircraft categories, including 2019 training images and 539 testing images according to the original dataset split. For all three datasets, input images are resized to 224 × 224 before being fed into the network. During training, we apply the same data augmentation strategy to all methods, including random resized cropping and random horizontal flipping. During testing, images are resized and center-cropped without random augmentation. To reduce evaluation bias caused by potential class imbalance, we report not only accuracy but also precision, recall, and F1-score.
Figure 4. Examples from datasets used in the evaluation of our proposed dual-branch network. The datasets include FGVC-Aircraft, MTARSI, and MADAI.
Edge map and multi-channel image generation. To enhance structural feature extraction, we generate an edge map for each input image using a lightweight traditional detector. In the main experiments, we use the Canny edge detector to generate the edge maps for supervising the shape branch. Before edge extraction, each input image is resized to the training resolution and converted to grayscale. The Canny detector is then applied with fixed lower and upper thresholds of 100 and 200, respectively. The generated edge map is converted into a binary map and normalized to the range [0, 1], where edge pixels are assigned a value of 1 and non-edge pixels are assigned a value of 0. The edge map emphasizes object boundaries and structural contours, providing direct supervision for the shape branch. The robustness to the choice of detector is further validated in our ablation study.
To better represent color features, we transform each input image into eight distinct channels derived from three color spaces: RGB, HSV, and YCbCr. The channels include: RGB color space: Red (R), Green (G), Blue (B); HSV color space: Hue (H), Saturation (S); YCbCr color space: Luminance (Y), Blue Difference ( C b ), Red Difference ( C r ). Figure 5 illustrates the input image transformations into these channels and the corresponding edge map. This multi-channel representation ensures the comprehensive capture of color-specific information while maintaining compatibility with structural features extracted by the shape branch.
Figure 5. Illustration of multi-channel representation and edge map generation: the input image is converted into eight channels across RGB, HSV, and YCbCr spaces (R, G, B, H, S, Y, Cb, Cr) to capture diverse color cues, while an edge map highlights structural details.

4.2. Implementation Details

All experiments are implemented in PyTorch 2.6.0. All models are initialized with ImageNet pre-training and are fine-tuned on each dataset under the same protocol for fair comparison. For the proposed framework, both the shape branch and the color branch use the same backbone family for fair comparison. In the CNN-based setting, we use EfficientNet as the backbone of both branches. In the Transformer-based setting, we replace the backbone in both branches with Swin-T. The two branches do not share weights. For optimization, we use Adam with an initial learning rate of 10 − 4 , weight decay of 10 − 5 , batch size 32, and 100 epochs. All reproduced baselines used the same dataset splits, ImageNet pre-training, input resolution, optimizer, batch size, number of training epochs, and data augmentation strategy as those used for our method. The same augmentation pipeline, including random resized cropping, random horizontal flipping, and normalization, was applied to all training samples, while only resizing and normalization were used during evaluation. The reported results are not directly copied from the literature but are obtained under a consistent evaluation setting. For evaluation, we report accuracy (Acc), precision (Prec), recall (Rec), and F1-score (F1). In the robustness evaluation, JPEG compression and Gaussian noise are applied only at test time, with JPEG quality factor 60 and Gaussian noise level σ = 0.05 .

4.3. Comparisons

Comparison with CNN Backbones. We first compare our approach with representative CNN backbones that are commonly used in aircraft recognition and remote sensing classification, including VGG16 [35], ResNet18 [22], DenseNet40 [36], and EfficientNet [23].
Results are shown in Table 1. Our method consistently outperforms these CNN baselines across all datasets, achieving improvements of 1.4%, 1.6%, and 1.9% over the strongest CNN baseline (EfficientNet) on FGVC-Aircraft, MTARSI, and MADAI, respectively. Compared with earlier architectures (e.g., VGG16), the gains are more pronounced, highlighting the importance of explicitly separating shape and color cues and adaptively fusing them via cross-attention.
Table 1. Comparison of classification accuracy (%) on FGVC-Aircraft, MTARSI, and MADAI. All baselines were reproduced under the same training and evaluation protocol as our method. Best results are shown in bold in this and all subsequent tables.
Comparison with Transformer-based Models. We further benchmark against stronger modern backbones, including ConvNeXt-T [37], EfficientNetV2-S [38], DeiT-S [39], ViT-B/16 [9], and Swin-T [40]. Since Transformer-based classifiers (e.g., ViT/DeiT/Swin) typically rely on a different feature extraction paradigm than CNNs, we additionally instantiate our framework with a Transformer backbone to ensure a like-for-like comparison. Specifically, we replace the original CNN feature extractor in both branches with a Swin-Transformer backbone (Swin-T by default). All models are trained and evaluated under the same fine-tuning protocol as described above. Table 2 reports the results. First, modern backbones (ConvNeXt/Swin/ViT-family) indeed improve over classical CNNs, confirming that aircraft recognition benefits from stronger representation learning. Second, our method remains competitive even against these stronger baselines. This suggests that the proposed decoupled shape–color representation and cross-attention fusion provide complementary gains beyond simply scaling the backbone.
Table 2. Comparison with modern backbones (accuracy %).
Comparison with Fine-grained Recognition Methods. Aircraft classification is intrinsically fine-grained: categories often share global structure while differing in subtle geometric cues (e.g., winglets, engine placement) and appearance patterns (e.g., livery, camouflage). We therefore compare with representative fine-grained recognition methods, including TransFG [41], SwinFG [42], TransFGVC [43] and SFETrans [44]. Results are summarized in Table 3.
Table 3. Comparison with fine-grained recognition methods (accuracy %).
Compared with these fine-grained classifiers, our method achieves consistently strong performance across all datasets. These results indicate that the proposed framework remains effective when compared with task-specific fine-grained recognition methods. Unlike generic fine-grained classifiers, our method explicitly models aircraft-related structural and appearance cues through separate branches and then re-aligns them using cross-attention. This design provides complementary information beyond standard backbone or fine-grained feature learning, while avoiding an over-specific interpretation of any single cue as universally dominant across datasets.
Robustness and Generalization. Beyond in-domain accuracy, robustness is essential for practical remote sensing scenarios, where compression, sensor noise, and slight distribution shifts are common. We evaluate robustness under two common perturbations: JPEG compression and additive Gaussian noise. Table 4 reports the accuracy under mild and moderate perturbations.
Table 4. Robustness under image perturbations (accuracy %).
Our method achieves the best perturbed accuracy under both JPEG compression and Gaussian noise across the evaluated settings. In terms of relative degradation, it generally remains competitive with the baseline Swin-T, indicating that the proposed decoupled design improves robustness while maintaining strong generalization under common image perturbations. We attribute this to the decoupled design: (i) the shape branch is supervised by edge maps, encouraging reliance on stable structural cues when pixel-level appearance is corrupted; (ii) cross-attention adaptively re-weights the contribution of shape and color according to signal quality, mitigating over-dependence on any single attribute.

4.4. Statistical Validation

To evaluate the stability of the reported results, we further conduct three independent runs with different random seeds for the strongest baseline models and the proposed method. All runs follow the same training protocol, including the same data split, input resolution, optimizer, learning rate, batch size, and number of training epochs. Table 5 reports the mean accuracy and standard deviation across the three runs. The proposed method consistently outperforms the strongest CNN-based and Transformer-based baselines on all three datasets. The relatively small standard deviations indicate that the performance gains are stable across different random initializations rather than being caused by a single favourable run.
Table 5. Statistical validation over three independent runs with different random seeds. Results are reported as mean accuracy ± standard deviation (%).

4.5. Controlled Perturbation Analysis of Color and Shape Cues

To provide additional quantitative evidence for the role of color and shape cues in aircraft classification, we conduct a controlled perturbation analysis on the three evaluation datasets. Different from the standard comparison experiments, this analysis does not aim to introduce new training strategies. Instead, we evaluate the trained model under several test-time input transformations that selectively weaken color or structural information. Specifically, grayscale conversion suppresses chromatic information while preserving most object structures; hue and saturation perturbation disturbs color consistency; Gaussian blur mainly weakens local edges, textures, and fine-grained structural details; and the edge-only setting preserves mainly contour information while removing most chromatic appearance cues. It should be noted that these perturbations are applied to the input image while keeping the full two-branch model unchanged. Therefore, this analysis is not a branch-level ablation, but an input-level sensitivity analysis that examines how the trained model responds when chromatic or structural cues are weakened. The results are reported in Table 6. Compared with the original RGB input, suppressing or perturbing color information leads to clear performance degradation, especially on MADAI, where color patterns and camouflage-like appearance provide useful discriminative cues. For example, converting the input to grayscale reduces the accuracy from 97.3% to 93.1% on MADAI, while hue perturbation further decreases the accuracy to 92.6%. This suggests that color is not merely redundant information, but contributes to aircraft recognition in scenarios where appearance patterns are discriminative. Weakening structural information also causes substantial degradation. Gaussian blur leads to larger accuracy drops on all three datasets, indicating that fine-grained structural details such as fuselage shape, wing layout, tail configuration, and engine placement are important for aircraft classification. The edge-only setting performs worse than the original RGB input because it removes color and texture information, but it still retains non-trivial classification accuracy, showing that structural contours alone remain informative. Overall, these results support our revised claim that color and shape are two important and complementary visual cues for aircraft classification, rather than the only factors determining recognition performance.
Table 6. Controlled perturbation analysis for evaluating the contribution of color and shape cues. The performance degradation Δ is computed with respect to the original input.

4.6. Ablation Study

We conduct ablation studies on all three datasets to evaluate the effectiveness of shape–color decoupling and attention-based fusion.
Separate Shape and Color Modeling To quantify the impact of separately modeling shape and color features, we consider four variants (all trained/evaluated under the same settings): Shape Only: disable the color branch and rely solely on structural cues; Color Only: disable the shape branch and use only color cues; Both Branches (No Attn): concatenate the two branches without attention; Both Branches (With Attn): full method with cross-attention fusion.
As shown in Table 7, using either the shape branch or the color branch alone leads to clearly lower accuracy than the full model on all three datasets. This indicates that neither structural information nor chromatic information alone is sufficient for robust aircraft classification. For example, on FGVC-Aircraft, the Shape Only model achieves 89.6% accuracy, while the Color Only model achieves 86.9%, both of which are lower than the full model accuracy of 94.2%. A similar trend is observed on MTARSI and MADAI, where the full model improves over the strongest single-branch variant by 5.2% and 9.1%, respectively. Combining the two branches without attention already improves the performance compared with either single branch, suggesting that shape and color provide complementary cues. However, direct fusion remains inferior to the proposed cross-attention-based fusion. Specifically, replacing naive branch fusion with the proposed attention-based fusion improves the accuracy from 91.8% to 94.2% on FGVC-Aircraft, from 90.9% to 93.6% on MTARSI, and from 93.4% to 97.3% on MADAI. These results provide quantitative support for our motivation that color and shape are important and complementary visual cues for aircraft classification, and that adaptive fusion is beneficial for exploiting their relative contributions across different datasets.
Table 7. Ablation study on shape and color cues across three aircraft classification datasets. All results are reported as classification accuracy (%).
Color-space Combination. To verify whether the selected color channels are sufficient and effective, we compare several color-space combinations in Table 8. Using RGB alone provides the basic color representation but lacks explicit hue, saturation, and chrominance decomposition. Adding HSV or YCbCr improves the performance, indicating that perceptual color attributes and luminance–chrominance separation are both beneficial for aircraft classification. The combination of RGB, HSV, and YCbCr achieves the best performance, suggesting that these color spaces provide complementary information. We also observe that removing RGB leads to lower accuracy, which indicates that the original image color distribution remains important. These results support the effectiveness and sufficiency of the selected eight-channel design in the proposed CFM.
Table 8. Ablation study of different color-space combinations in the Color Feature Module on MADAI.
Comparison to Common Fusion Strategies. To further validate the effectiveness of the proposed fusion module, we compare it with two commonly used strategies: (i) Add: element-wise addition of features from two branches; (ii) MLP: concatenation followed by a two-layer MLP. For fairness, we keep the overall parameter budget comparable. Table 9a shows that the proposed CSCAFM consistently achieves the best performance. Cross-attention improves fusion granularity by learning feature-level correspondences between shape and color, rather than mixing them uniformly. This is particularly beneficial for aircraft categories that differ by subtle geometry or color-specific markings.
Table 9. Ablation studies on MADAI. (a) Fusion strategies. (b) Edge detectors.
Effectiveness of Edge Detector. To minimize computational overhead, we generate edge maps using traditional detectors. Table 9b shows that Sobel and Canny yield similar performance, indicating that the Shape Module can robustly leverage structural information without being sensitive to the specific edge operator.
Attention-based Diagnostic of Shape–Color Interaction. To provide a qualitative diagnostic of how the fusion module responds to shape and color features, we compute an attention-based response score for each branch. Specifically, the cross-attention outputs from the shape-to-color and color-to-shape directions are averaged over attention heads, spatial locations, and feature channels. Let F s and F c denote the two feature maps, and let A shape → color and A color → shape be the corresponding cross-attention maps. The normalized scores are then computed as follows:
W shape = A shape → color V color 1 A shape → color V color 1 + A color → shape V shape 1 ,
and
W color = 1 − W shape .
These normalized weights are used as an attention-based diagnostic score for qualitatively analyzing the relative response strength of the shape and color branches. Figure 6 visualizes these scores for several representative aircraft examples. The scores are computed by averaging the corresponding cross-attention responses over attention heads, spatial locations, and feature channels. They therefore can be interpreted as qualitative indicators of the fusion behavior, rather than direct or causal measurements of modality importance.
Figure 6. Attention-based diagnostic scores for the shape and color branches across representative aircraft examples. These scores are derived from cross-attention responses and are used only as qualitative indicators of shape–color interaction, rather than direct measurements of feature importance.

5. Conclusions

In this paper, we propose a novel framework for aircraft type classification that leverages separate shape and color feature modeling with adaptive cross-attention fusion. By introducing a dual-branch architecture, our method explicitly separates the processing of shape and color, allowing each branch to specialize and contribute uniquely to the classification task. Specifically, the Shape Feature Module (SFM) enhances structural sensitivity by leveraging an edge-detection-guided approach, ensuring robust extraction of shape features. Meanwhile, the Color Feature Module (CFM) employs multi-channel color space transformations and 1 × 1 convolutions to strengthen the learning of color-related features. The adaptive fusion mechanism dynamically integrates features from both branches, ensuring robust performance across diverse scenarios. Experiments conducted on three benchmark datasets demonstrate that the proposed approach achieves improved performance over the evaluated baselines in terms of accuracy and shows competitive robustness under the tested perturbations. Ablation studies further support the effectiveness of the separated shape–color representation and the fusion strategy, highlighting their roles in mitigating the challenges posed by inter-class similarity. Future work will focus on extending the current framework to incorporate additional modalities, such as texture or spatial information, and exploring its applicability in other fine-grained classification tasks. In addition, we will further evaluate the proposed framework under more diverse real-world conditions, including viewpoint variation, scale changes, partial occlusion, illumination changes, and background clutter, to better assess its practical robustness in remote sensing and surveillance scenarios.

Author Contributions

Conceptualization, X.Q. and P.L.; methodology, X.Q.; software, X.Q.; validation, X.Q.; formal analysis, X.Q.; investigation, X.Q.; resources, X.Q.; data curation, X.Q.; writing—original draft preparation, X.Q.; writing—review and editing, P.L.; visualization, X.Q.; supervision, P.L.; project administration, P.L.; funding acquisition, P.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China (Grant No. 62276166).

Institutional Review Board Statement

Not applicable.

Data Availability Statement

No new datasets were created in this study. The experiments were conducted using three publicly available datasets: MTARSI [24], FGVC-Aircraft [34], and MADAI [13]. These datasets can be accessed through their original publications or official release pages. Additional details regarding dataset usage and preprocessing are provided in the Datasets section of this paper.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Hu, Q.; Li, R.; Xu, Y.; Pan, C.; Niu, C.; Liu, W. Toward aircraft detection and fine-grained recognition from remote sensing images. J. Appl. Remote Sens. 2022, 16, 024516. [Google Scholar] [CrossRef] [Scilit]
  2. Yang, F.; Wang, K.; Wang, X. Robust real-time aircraft detection with multi-task cascaded calibration networks. J. Appl. Remote Sens. 2022, 16, 016504. [Google Scholar] [CrossRef] [Scilit]
  3. Zhou, D.; Zhuang, X.; Zuo, H.; Wang, H.; Yan, H. Deep learning-based approach for civil aircraft hazard identification and prediction. IEEE Access 2020, 8, 103665–103683. [Google Scholar] [CrossRef] [Scilit]
  4. Khavalko, V.; Kalapun, N.; Dokhnyak, B. Recognition and classification of military aircraft in images. In Proceedings of the 2023 IEEE 18th International Conference on Computer Science and Information Technologies (CSIT), Lviv, Ukraine, 19–21 October 2023; pp. 1–4. [Google Scholar]
  5. Kumar, A.; Singh, S.; Bharati Mohan, G. AIR-SCAN: Aircraft Identification and Recognition using Deep Learning Scanning. In Proceedings of the 2024 11th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India, 14–15 March 2024; pp. 1–6. [Google Scholar]
  6. Wu, J.; Wang, C. MCS-UNet: Sea ice classification of Sentinel-1 SAR imagery using a modified U-Net. J. Appl. Remote Sens. 2025, 19, 018504. [Google Scholar] [CrossRef] [Scilit]
  7. Wang, Y.; Li, Y.; Liu, W. Convolutional and graph convolutional neural network-based for remote sensing image urban functional area classification. J. Appl. Remote Sens. 2025, 19, 016508. [Google Scholar] [CrossRef] [Scilit]
  8. Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
  9. Dosovitskiy, A. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv 2020, arXiv:2010.11929. [Google Scholar]
  10. Xiao, F.; Li, X.; Li, W.; Shi, J.; Zhang, N.; Gao, X. Integrating category-related key regions with a dual-stream network for remote sensing scene classification. J. Vis. Commun. Image Represent. 2024, 100, 104098. [Google Scholar] [CrossRef] [Scilit]
  11. Zhao, B.; Tang, W.; Pan, Y.; Han, Y.; Wang, W. Aircraft type recognition in remote sensing images: Bilinear discriminative extreme learning machine framework. Electronics 2021, 10, 2046. [Google Scholar] [CrossRef] [Scilit]
  12. Wang, Y.; Chen, Y.; Liu, R. Aircraft image recognition network based on hybrid attention mechanism. Comput. Intell. Neurosci. 2022, 2022, 4189500. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  13. Wang, Y.; Wang, T.; Zhou, X.; Cai, W.; Liu, R.; Huang, M.; Jing, T.; Lin, M.; He, H.; Wang, W.; et al. TransEffiDet: Aircraft detection and classification in aerial images based on EfficientDet and transformer. Comput. Intell. Neurosci. 2022, 2022, 2262549. [Google Scholar] [CrossRef] [Scilit]
  14. Saeed, A.; Atif, H.B.; Habib, U.; Bilal, M. Intelligent Known and Novel Aircraft Recognition—A Shift from Classification to Similarity Learning for Combat Identification. arXiv 2024, arXiv:2402.16486. [Google Scholar]
  15. Mo, H.; Zhao, G. RIC-CNN: Rotation-invariant coordinate convolutional neural network. Pattern Recognit. 2024, 146, 109994. [Google Scholar] [CrossRef] [Scilit]
  16. Saeed, A.; Atif, H.B.; Habib, U.; Bilal, M. Remote sensing aircraft classification harnessing deep learning advancements. In Proceedings of the 2023 18th International Conference on Emerging Technologies (ICET), Peshawar, Pakistan, 6–7 November 2023; pp. 50–55. [Google Scholar]
  17. Liang, W.; Li, J.; Diao, W.; Sun, X.; Fu, K.; Wu, Y. FGATR-Net: Automatic network architecture design for fine-grained aircraft type recognition in remote sensing images. Remote Sens. 2020, 12, 4187. [Google Scholar] [CrossRef] [Scilit]
  18. Gao, K.; He, H.; Lu, D.; Xu, L.; Ma, L.; Li, J. Optimizing and evaluating swin transformer for aircraft classification: Analysis and generalizability of the mtarsi dataset. IEEE Access 2022, 10, 134427–134439. [Google Scholar] [CrossRef] [Scilit]
  19. Yan, H. Detection with fast feature pyramids and lightweight convolutional neural network: A practical aircraft detector for optical remote images. J. Appl. Remote Sens. 2022, 16, 024506. [Google Scholar] [CrossRef] [Scilit]
  20. Hsieh, J.W.; Chen, J.M.; Chuang, C.H.; Fan, K.C. Aircraft type recognition in satellite images. IEE Proc. Vis. Image Signal Process. 2005, 152, 307–315. [Google Scholar] [CrossRef] [Scilit]
  21. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012, 25, 1106–1114. [Google Scholar] [CrossRef] [Scilit]
  22. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
  23. Tan, M.; Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA, 9–15 June 2019; pp. 6105–6114. [Google Scholar]
  24. Wu, Z.Z.; Wan, S.H.; Wang, X.F.; Tan, M.; Zou, L.; Li, X.L.; Chen, Y. A benchmark data set for aircraft type recognition from remote sensing images. Appl. Soft Comput. 2020, 89, 106132. [Google Scholar] [CrossRef] [Scilit]
  25. Yang, H.; Jun, S.; Zheng, H.; Wang, J.; Bo, Y.; Cao, S. Research on Aircraft Remote Sensing Image Recognition Network Based on Attention Mechanism and TF Learning. In Proceedings of the 2022 International Conference on Cyber-Physical Social Intelligence (ICCSI), Nanjing, China, 18–21 November 2022; pp. 79–84. [Google Scholar]
  26. Denton, E.L.; Birodkar, V. Unsupervised learning of disentangled representations from video. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; Volume 30. [Google Scholar]
  27. Esser, P.; Sutter, E.; Ommer, B. A variational u-net for conditional appearance and shape generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 8857–8866. [Google Scholar]
  28. Ma, L.; Sun, Q.; Georgoulis, S.; Van Gool, L.; Schiele, B.; Fritz, M. Disentangled person image generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 99–108. [Google Scholar]
  29. Xing, X.; Gao, R.; Han, T.; Zhu, S.C.; Wu, Y.N. Deformable generator networks: Unsupervised disentanglement of appearance and geometry. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 44, 1162–1179. [Google Scholar] [CrossRef] [Scilit]
  30. Berga, D.; Masana, M.; Van de Weijer, J. Disentanglement of color and shape representations for continual learning. arXiv 2020, arXiv:2007.06356. [Google Scholar] [CrossRef] [Scilit]
  31. Lorenz, D.; Bereska, L.; Milbich, T.; Ommer, B. Unsupervised part-based disentangling of object shape and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 10955–10964. [Google Scholar]
  32. Ding, Z.; Xu, Y.; Xu, W.; Parmar, G.; Yang, Y.; Welling, M.; Tu, Z. Guided variational autoencoder for disentanglement learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 7920–7929. [Google Scholar]
  33. Uzunova, H.; Handels, H.; Ehrhardt, J. Guided filter regularization for improved disentanglement of shape and appearance in diffeomorphic autoencoders. Proc. Med. Imaging Deep. Learn. 2021, 143, 774–786. [Google Scholar]
  34. Maji, S.; Rahtu, E.; Kannala, J.; Blaschko, M.; Vedaldi, A. Fine-grained visual classification of aircraft. arXiv 2013, arXiv:1306.5151. [Google Scholar] [CrossRef] [Scilit]
  35. Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
  36. Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708. [Google Scholar]
  37. Liu, Z.; Mao, H.; Wu, C.Y.; Feichtenhofer, C.; Darrell, T.; Xie, S. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 19–20 June 2022; pp. 11976–11986. [Google Scholar]
  38. Tan, M.; Le, Q. Efficientnetv2: Smaller models and faster training. In Proceedings of the International Conference on Machine Learning, PMLR, Virtual, 18–24 July 2021; pp. 10096–10106. [Google Scholar]
  39. Touvron, H.; Cord, M.; Douze, M.; Massa, F.; Sablayrolles, A.; Jégou, H. Training data-efficient image transformers & distillation through attention. In Proceedings of the International Conference on Machine Learning, PMLR, Virtual, 18–24 July 2021; pp. 10347–10357. [Google Scholar]
  40. Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 10012–10022. [Google Scholar]
  41. He, J.; Chen, J.N.; Liu, S.; Kortylewski, A.; Yang, C.; Bai, Y.; Wang, C. TransFG: A Transformer Architecture for Fine-grained Recognition. arXiv 2021, arXiv:2103.07976. [Google Scholar] [CrossRef] [Scilit]
  42. Ma, Z.; Wu, X.; Chu, A.; Huang, L.; Wei, Z. SwinFG: A fine-grained recognition scheme based on swin transformer. Expert Syst. Appl. 2024, 244, 123021. [Google Scholar] [CrossRef] [Scilit]
  43. Shen, L.; Hou, B.; Jian, Y.; Tu, X.; Zhang, Y.; Shuai, L.; Ge, F.; Chen, D. TransFGVC: Transformer-based fine-grained visual classification. Vis. Comput. 2025, 41, 2439–2459. [Google Scholar] [CrossRef] [Scilit]
  44. Yu, Y.; Wei, W.; Zhao, C.; Qian, J.; Chen, E. Structural feature enhanced transformer for fine-grained image recognition. Pattern Recognit. 2026, 169, 111955. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.