A Context Modulation Network (CM-Net) for Ship Targets Ultra-Fine-Grained Recognition in Optical Remote Sensing Images

Abstract

Ultra-fine-grained ship recognition is a core task in optical remote sensing maritime situational awareness. Two fundamental bottlenecks persist: (1) discriminative microstructures (e.g., radar antennas, superstructures, vertical launch systems) are severely diluted or lost during repeated downsampling; (2) high-frequency background noise from sea foam, port edges and storage facilities is spectrally entangled with target signals, causing severe inter-class feature confusion. To address these issues, we propose the context modulation network (CM-Net), which unifies detail enhancement and background suppression as sequential stages within a single feature propagation pathway. Discriminative high-frequency information is first recovered in shallow-to-intermediate layers, after which uncertainty-aware modulation suppresses residual background responses in deeper layers, forming a two-stage frequency enhancement and background purification framework. Specifically, the Ultra-Fine-Grained Feature Enhancement Module (UFG-FEM) recovers multi-scale discriminative high-frequency details in shallow-to-intermediate layers, enriching representations for deeper stages. The Dual-Branch Context Modulation Module (DB-CMM) subsequently models background uncertainty via a Beta evidential distribution and selectively suppresses background interference through optimal transport-based soft alignment and an information bottleneck, forming a sequential pipeline with clear functional division. We also build the FGSC-72 dataset with 72 model-level categories and ~13,000 images. On FGSC-72, CM-Net achieves 97.49% Top-5 accuracy, 91.67% macro-Precision, 94.69% mean Average Precision (mAP), and 92.01% macro-averaged F1 score, with a 4.64-percentage-point improvement in mAP over Oriented R-CNN. Across five representative backbones, it improves mAP by 4.60–7.11 points, indicating consistent cross-architecture applicability.

1. Introduction

Maritime situational awareness has become increasingly important for traffic monitoring, resource management, and security applications. Optical remote sensing satellites provide wide-area coverage and high-resolution imagery, making them valuable data sources for ship monitoring, target identification, and maritime surveillance tasks [1,2,3].

In recent years, with advances in high-resolution optical remote sensing and the operation of sub-meter satellites such as GF-2, GF-7 and WorldView-3/4, ship perception research is shifting from coarse-grained detection to fine-grained and even ultra-fine-grained recognition [4,5,6,7]. Traditional coarse-grained detection only locates ships and confirms their presence. Current applications require model-level identification of maritime targets rather than only ship localization, enabling more precise target recognition and persistent monitoring [8,9,10].

However, compared with fine-grained recognition in natural images (e.g., birds, cars, and aircraft), ultra-fine-grained ship recognition in optical remote sensing presents several task-specific challenges, as illustrated in Figure 1.

Figure 1. Challenges in ship target classification. The morphologies of ship targets, such as arbitrary orientations, excessive aspect ratios, and dense distributions, result in feature shifts. Meanwhile, ultra-fine-grained features with a small proportion are often masked by various types of background noise.

(1)

Dilution and loss of ultra-fine-grained discriminative signals during deep downsampling. In optical remote sensing imagery, fine-grained ship identity depends on microstructural cues, such as radar antennas, superstructure geometry, and deck layouts, which mainly correspond to high-frequency components. However, repeated downsampling in conventional CNNs progressively weakens these discriminative details, causing small but critical structures to be overwhelmed by dominant low-frequency semantic features.

(2)

High-frequency background noise from complex maritime environments. Optical remote sensing ship images contain diverse backgrounds, including sea waves, port structures, storage yards, coastal regions, and atmospheric interference. These backgrounds also generate high-frequency responses that overlap with target-specific details. As a result, enhancing high-frequency information alone may simultaneously amplify background interference, making it difficult for conventional feature enhancement strategies to preserve discriminative signals while suppressing irrelevant responses.

These two challenges are inherently coupled: recovering discriminative microstructures may strengthen background responses with similar frequency characteristics. Therefore, we formulate ultra-fine-grained ship recognition as a sequential process of discriminative information recovery and background response modulation, and design the network accordingly.

To address these challenges, the community has explored multiple technical avenues. Current mainstream visual architectures follow two paths: CNN architectures represented by ResNet [11], DenseNet [12] and MobileNet [13], and vision Transformer architectures represented by ViT [14], Swin Transformer [15] and MobileViT [16]. For multi-scale modeling, traditional modules like the Inception [17] series capture multi-receptive-field features via parallel branches. Meanwhile, oriented detectors for remote sensing (e.g., RoI Transformer, ReDet, Oriented R-CNN) and hierarchical ship recognition networks (e.g., PKINet, HMS-Net) have been proposed.

Nevertheless, in-depth analysis reveals two common limitations for ultra-fine-grained ship recognition:

(1)

Convolutional, self-attention, and multi-scale fusion methods largely learn features in an end-to-end, implicit manner, leaving the retention of discriminative high-frequency information during deep propagation unmodeled. This is problematic for ultra-fine-grained ship recognition, where target microstructures and background clutter share much of the high-frequency band; recovering the former inevitably strengthens the latter. The difficulty, then, is less about frequency decomposition itself than about coordinating discriminative recovery and background modulation as one sequential process.

(2)

Conventional attention mechanisms produce a single deterministic weight at each location. Such point estimates poorly capture the ambiguity that arises near target boundaries and cluttered backgrounds, and after high-frequency enhancement they readily discard boundary responses that may still carry discriminative cues. Effective background modulation thus depends not only on estimating background probability, but also on the uncertain responses that enhancement itself introduces.

Taken together, the main difficulty in ultra-fine-grained ship recognition is not detail recovery or background suppression on its own, but their coupling along the feature path: recovering microstructural detail also strengthens background responses, whose subsequent suppression in turn governs how much detail survives. We therefore recast the task as a sequential process of discriminative recovery followed by background modulation, and design the network accordingly.

To address these problems, we propose a context modulation network (CM-Net) for ultra-fine-grained remote sensing ship recognition. Instead of treating discriminative detail enhancement and background suppression as separate objectives, CM-Net organizes them as two successive stages in a unified feature propagation process: attenuation-prone discriminative high-frequency information is first recovered in shallow and intermediate layers, and the enhanced background responses are then modulated in an uncertainty-aware manner in deeper layers, forming a sequential framework of discriminative information recovery followed by background response modulation. Two complementary modules, the Ultra-Fine-Grained Feature Enhancement Module (UFG-FEM) and the Dual-Branch Context Modulation Module (DB-CMM), are further designed to perform these two stages, respectively.

The main contributions of this paper are summarized as follows:

(1)

To address the sequential coupling between discriminative information and background responses during feature propagation in ultra-fine-grained remote sensing ship recognition, we model this task as a sequential process of discriminative information recovery and background response modulation, and establish a unified sequential context modulation framework that provides an overall methodological basis for the subsequent module design.

(2)

Building on this framework, two core modules with clearly defined roles are designed. UFG-FEM is deployed at shallow and intermediate layers to recover discriminative high-frequency information through explicit multi-scale frequency decomposition, while DB-CMM is deployed at deep layers to perform uncertainty-aware modulation of the enhanced background responses via Beta evidential modeling, optimal transport-based soft alignment, and an information bottleneck mechanism, thereby forming a sequential pipeline of information recovery and background modulation.

(3)

An ultra-fine-grained ship dataset, FGSC-72, is constructed, covering 72 model-level categories and approximately 13,000 images, providing a challenging evaluation dataset for ultra-fine-grained ship recognition.

(4)

Experiments on FGSC-72 include comparisons with representative methods, three-level ablation studies examining discriminative recovery and background modulation, and cross-backbone compatibility evaluation across five representative architectures. The results at each level provide empirical support for the proposed sequential context modulation framework within the current evaluation setting.

The rest of this paper is organized as follows. Section 2 reviews four closely related research directions: fine-grained ship recognition in optical remote sensing, general fine-grained visual categorization, frequency-domain analysis in deep networks, and uncertainty modeling with optimal transport. Section 3 details the overall CM-Net architecture, the mathematical formulation of UFG-FEM and DB-CMM, and the multi-task composite loss function. Section 4 describes the FGSC-72 dataset, experimental setup, and results from SOTA comparison, ablation studies, cross-backbone compatibility evaluation and qualitative visualization. Section 5 concludes the work and outlines future research directions.

1.1. Related Work and Motivation

This section systematically reviews four technical directions closely relevant to this study. Section 1.1.1 focuses on the research progress and bottlenecks of fine-grained ship recognition in optical remote sensing imagery. Section 1.1.2 reviews multi-scale feature learning methods for general fine-grained visual categorization. Section 1.1.3 summarizes frequency-domain analysis theories in deep neural networks, which lay the theoretical foundation for the UFG-FEM module. Section 1.1.4 outlines two core technical threads: uncertainty modeling and optimal transport.

1.1.1. Fine-Grained Ship Recognition in Optical Remote Sensing Imagery

The development of fine-grained ship recognition in optical remote sensing has been closely associated with benchmark datasets and task-specific architectures. HRSC2016 [18] mainly supports coarse-grained ship detection, whereas FGSC-42 [19] extends category granularity toward model-level recognition and ShipRSImageNet [10] covers more diverse fine-grained detection scenarios. However, the category granularity and sample distribution of existing datasets remain limited for ultra-fine-grained model recognition, motivating the construction of FGSC-72.

Methodologically, existing studies include oriented detection frameworks and dedicated fine-grained recognition networks. Representative detectors such as R³Det [20], S²ANet [21], ReDet [22], Oriented R-CNN [23], and PKINet [24] improve rotated target representation and localization through feature refinement, alignment, rotation-aware modeling, or multi-scale kernels. Fine-grained approaches, including Cog-Net [25], Efficient-prompt [26], and HMS-Net [27], further explore component relations, attention, prompt tuning, and multi-scale feature fusion. Nevertheless, these approaches do not explicitly address the sequential coupling between discriminative detail recovery and background response modulation during feature propagation, which motivates the present study.

1.1.2. Multi-Scale Feature Learning in Fine-Grained Visual Categorization

The core challenge of ultra-fine-grained recognition lies in subtle inter-class differences and prominent intra-class variations. To address this, multiple technical paths have been developed in the field of general fine-grained visual categorization (FGVC).

Early methods mainly relied on part localization and bilinear pooling. In recent years, with the rise of attention mechanisms and Transformer architectures, fine-grained recognition has entered a new stage. Context-aware attention pooling proposed by Behera et al. [28] improves discriminative part localization by modeling spatial contextual correlations. Rao et al. [29] proposed counterfactual attention learning from a causal inference perspective, learning truly discriminative attention distributions by comparing prediction differences between factual and counterfactual attention.

Under the Transformer paradigm, FFVT proposed by Wang et al. [30] enhances ViT’s ability to capture fine-grained cues via feature fusion. TransFG, proposed by He et al. [31], first systematically introduces Transformers to fine-grained recognition, focusing on discriminative regions via a part selection module. Zhu et al. [32] later proposed a dual cross-attention learning framework to mine fine-grained differences through bidirectional attention interaction. MetaFormer proposed by Diao et al. [33] builds a unified meta-learning framework integrating key components of various fine-grained recognition methods. Chou et al. [34] recently proposed a plug-and-play module to lower the deployment barrier of fine-grained methods. Liu et al. [35] proposed a knowledge-guided mechanism to alleviate attention peak suppression in Transformers.

These methods have enriched the technical foundation of fine-grained recognition, but most are designed for natural image scenarios (e.g., CUB-200, Stanford Cars). Their implicit assumption that targets occupy the main image area with relatively simple backgrounds does not hold in remote sensing scenarios. In optical remote sensing ship imagery, targets not only occupy small proportions with arbitrary orientations, but also suffer from severe high-frequency background interference such as sea waves, ports and clouds. This scenario gap causes significant performance degradation when general FGVC methods are directly transferred to remote sensing tasks, highlighting the necessity of designing dedicated methods for remote sensing imaging characteristics. Moreover, in ultra-fine-grained ship recognition, discriminative detail recovery tends to amplify background responses simultaneously, making the sequential relationship between recovery and modulation more critical than feature extraction capability alone.

1.1.3. Frequency-Domain Analysis in Deep Neural Networks

Examining the feature learning process of deep networks from the spatial frequency perspective has emerged as an important research direction in recent years. Its theoretical roots trace back to classical signal processing theories. The Laplacian pyramid proposed by Burt and Adelson [36] achieves multi-scale frequency decomposition of images via multi-level differences of Gaussian low-pass filtering, laying the engineering foundation for multi-resolution analysis. Mallat [37] later established wavelet multi-resolution analysis theory, which mathematically proves the multi-scale decomposition property of signals under orthogonal wavelet bases. The anisotropic diffusion equation proposed by Perona and Malik [38] reveals the key property of adaptive smoothing along the principal axis while preserving sharp edges in the orthogonal direction, which provides direct theoretical support for the design of the learnable anisotropic Gaussian kernel in our UFG-FEM.

Significant progress has been made in integrating frequency-domain ideas into deep neural networks. Fast Fourier Convolution (FFC), proposed by Chi et al. [39], significantly enlarges the receptive field by performing convolution in the frequency domain. Xu et al. [40] systematically proposed a new “learning in frequency domain” paradigm, which directly uses frequency-domain coefficients as network inputs for efficient recognition. FcaNet, proposed by Qin et al. [41], re-examines channel attention from the discrete cosine transform (DCT) perspective, proving that standard SE modules only utilize the lowest frequency component of DCT while discarding abundant useful frequency information. This finding directly inspired the design of the “spectrum-aware channel excitation” submodule in UFG-FEM.

Global Filter Network (GFNet), proposed by Rao et al. [42], replaces self-attention with learnable global frequency-domain filters, achieving both efficiency and global modeling capability. Recently, SpectFormer by Patro et al. [43], adaptive frequency filters by Huang et al. [44], and FFT-based dynamic token mixers by Tatsunami and Taki [45] have jointly advanced the development of frequency-domain vision Transformers. In the wavelet domain, the wavelet feature map compression method by Finder et al. [46] validates the effectiveness of wavelet transform in image-to-image tasks. Wavelet convolution, recently proposed by Finder et al. [47], achieves superior performance on multiple vision tasks via large receptive field modeling in the wavelet domain, which is highly consistent with our idea of constructing high-frequency residual decoupling based on multi-resolution analysis.

Although these frequency-domain methods perform well on general vision tasks, there is still a research gap in systematically introducing them to ultra-fine-grained ship recognition in optical remote sensing, and in specifically modeling the spectral overlap between target high-frequency discriminative signals and background high-frequency noise. The proposed method is developed under this motivation, where UFG-FEM serves as the feature recovery stage of the overall framework.

1.1.4. Uncertainty Modeling and Optimal Transport in Visual Recognition

In ultra-fine-grained recognition, the boundary between “ambiguous background” and “discriminative signals” is often indistinct. Point-estimate masks produced by conventional attention mechanisms cannot capture this inherent ambiguity, tending to cause over-suppression in low-evidence regions and accidentally damaging discriminative signals. To address this challenge, we draw theoretical foundations from two mathematical toolchains: uncertainty modeling and optimal transport.

In terms of uncertainty modeling, Kendall and Gal [48] systematically distinguished aleatoric uncertainty and epistemic uncertainty in deep learning, laying the methodological foundation for uncertainty modeling in vision tasks. Dropout Bayesian approximation, proposed by Gal and Ghahramani [49], provides a concise and efficient approach to Bayesian neural networks. The Evidential Deep Learning (EDL) framework by Sensoy et al. [50] quantifies uncertainty in a single forward pass by modeling network outputs as parameters of a Dirichlet distribution. This serves as the direct theoretical source for the evidential uncertainty mask in DB-CMM—we extend the idea from Dirichlet distributions for classification to Beta distributions for pixel-level binary background judgment. Amini et al. [51] later extended EDL to regression tasks. Trustworthy multi-view classification proposed by Han et al. [52] further demonstrates the value of evidence theory in multimodal fusion. The dynamic evidence fusion mechanism recently proposed by Han et al. [52] achieves state-of-the-art performance in multi-view classification. In the remote sensing field, recent research on uncertainty modeling for remote sensing object detection by Liu et al. [53] validates the effectiveness of uncertainty-aware mechanisms in remote sensing scenarios.

In terms of optimal transport, the entropy-regularized Sinkhorn distance proposed by Cuturi [54] reduces the computational complexity of optimal transport from super-cubic to nearly linear, enabling large-scale application of optimal transport in deep learning. The survey by Peyré and Cuturi [55] systematically summarizes the theory and algorithmic system of computational optimal transport. In recent years, optimal transport has been widely applied in vision tasks. Cao et al. [56] introduced optimal transport into multimodal knowledge graph embedding to align heterogeneous representations. Deformable attention proposed by Xia et al. [57] provides a similar mechanism in Transformers from the perspective of soft spatial alignment. Chen et al. [58] applied optimal transport to remote sensing domain adaptation to address cross-domain feature distribution alignment. The optimal transport feature alignment (OTFA) submodule in DB-CMM draws on these ideas by formulating spatial misalignment between the modulation mask and target features as a transport problem between probability distributions and obtaining a soft spatial alignment through entropy-regularized optimal transport.

In terms of information bottleneck, the information bottleneck principle proposed by Tishby and Zaslavsky [59] provides a rigorous information-theoretic foundation for representation compression in deep learning. The deep variational information bottleneck later proposed by Alemi et al. [60] realizes differentiable implementation of the information bottleneck principle in deep networks via variational approximation. These studies provide methodological support for the uncertainty-aware modulation stage adopted in the proposed framework, which is instantiated by DB-CMM in this work.

1.1.5. Motivation Summary

In summary, existing studies advance frequency-domain feature modeling and uncertainty representation learning separately, but still lack a unified formulation of the sequential relationship between discriminative information recovery and background response modulation during feature propagation. Motivated by this, we propose CM-Net, which organizes the two into a unified sequential context modulation framework, where UFG-FEM and DB-CMM correspond to these two successive stages, respectively.

2. Materials and Methods

2.1. Dataset: FGSC-72 Construction

To address the insufficient category granularity of existing ship datasets, FGSC-72 was constructed from HRSC2016, FGSC-42, and Gaofen-1 (GF-1) satellite imagery. The dataset contains 72 model-level ship categories, covering both military and civilian vessels. After deduplication and quality filtering, 13,147 images were retained. The images contain diverse illumination conditions, backgrounds, and target orientations, reflecting the variability of optical remote sensing ship imagery. The number of samples per category ranges from 5 to 1766, resulting in a severe long-tail distribution for evaluating model robustness under class imbalance.

Image-level single-label annotation was adopted, with labels assigned directly via class folders named by ship type. Ship identity was determined by cross-referencing the original labels of HRSC2016 and FGSC-42, public naval databases, and discriminative visual features including hull numbers, island structures, and superstructure profiles. For multi-ship images, the centered, largest, and most complete target was designated as the primary instance. Samples with severe occlusion, indistinguishable appearance, non-vessel content, or stitching artifacts were discarded. Annotation consistency was enforced through independent dual-annotator labeling with cross-verification, with disputed samples resolved by a third-party arbitrator. Visually confusable classes—such as Whidbey Island vs. Harpers Ferry dock landing ships, America vs. Wasp amphibious assault ships, and several Japanese destroyer classes—underwent additional spot-check verification.

Given that the three source collections may contain partially overlapping acquisition conditions, a three-stage deduplication procedure was conducted before dataset splitting to reduce the risk of sample correlation across different subsets. First, MD5 hashing was applied to identify and remove exact duplicate files. Second, perceptual hashing methods (pHash and dHash) were used to screen visually similar image pairs, and candidate pairs with Hamming distance ≤ 5 were manually inspected according to visual consistency to identify potential transformation-related duplicates, including resizing, format conversion, flipping, and minor cropping. Third, deep feature representations were extracted using an ImageNet-pretrained MobileNetV3-Large, and highly similar samples with cosine similarity > 0.98 were further manually inspected to identify possible cross-source correlations caused by similar acquisition conditions. Dataset splitting was conducted after completing the above procedure. This process reduces the possibility of near-duplicates between the training and evaluation subsets, although residual correlations cannot be completely excluded due to the characteristics of optical remote sensing image acquisition.

Stratified sampling by class was used to maintain consistent class distributions across subsets, with an 8:1:1 split yielding 10,486 training, 1281 validation, and 1380 test images. For extremely rare classes with fewer than 10 samples (e.g., Mistral class with 5 images, Charles de Gaulle with 6 images), strict proportional splitting was not applied; instead, a training-priority constraint was adopted, reserving at least one image each for validation and test to maintain their visibility during evaluation. However, due to the limited sample support of these categories, their class-specific evaluation results should be interpreted cautiously. Representative examples are shown in Figure 2, and the complete class list with per-subset sample counts is provided in Table A1.

Figure 2. Fifteen ship types in the FGSC-72 dataset. (a) Akizuki-class, (b) America-class, (c) Arleigh Burke-class, (d) cargo ship, (e) container ship, (f) crane ship, (g) Freedom-class, (h) Garibaldi-class, (i) Gwanggaeto the Great-class, (j) Hatakaze-class, (k) Horizon-class, (l) Independence-class, (m) sand carrier, (n) INS Virrat, (o) Izumo-class.

2.2. Overall Network Architecture

To simultaneously address the attenuation of high-frequency microstructural signals during deep downsampling and the interference from high-frequency background noise in ultra-fine-grained ship recognition from optical remote sensing imagery, this paper proposes a context modulation network with frequency–uncertainty collaborative optimization, termed CM-Net. The network adopts a two-stage processing architecture: discriminative details are explicitly enhanced at shallow and middle layers, while background noise is probabilistically suppressed at deep layers.

Let $\mathcal{D}={\{(I_n,y_n)\}}_{n=1}^N$ denote an optical remote sensing ship image dataset, where $I_n$ is the $n$-th RGB input image, $y_n$ is the corresponding ultra-fine-grained class label, $K$ is the total number of classes, and $N$ is the total number of training samples. CM-Net aims to learn a mapping function ${\mathcal{M}}_{\Theta }:I\to \widehat{y}$, where $\Theta$ denotes all learnable parameters and $\widehat{y}$ is the predicted class.

To investigate discriminative information recovery and background response modulation during feature propagation within a unified backbone, we adopt MobileNetV3-Large as the base network. UFG-FEM is inserted after the 4th and 8th bottleneck layers as the discriminative information recovery stage, enhancing high-frequency discriminative details before further downsampling. DB-CMM is placed after the 13th bottleneck layer as the background response modulation stage, applying uncertainty-aware background modulation to the recovered deep features.

The overall forward propagation of CM-Net is given in Algorithm 1. The internal forward propagations of UFG-FEM and DB-CMM are given in Algorithm 2 and Algorithm 3, respectively; their mathematical definitions and theoretical foundations are presented in Section 2.3 and Section 2.4. For clarity, the pseudocode uses the following abbreviations: BN denotes batch normalization, GAP denotes global average pooling, DWConv denotes depthwise separable convolution, FFT2D denotes the two-dimensional fast Fourier transform, MLP denotes a multilayer perceptron, and FC denotes a fully connected layer.

Algorithm 1 CM-Net Overall Forward Pass

Input:  I ∈ R^(3 × 224 × 224)               # RGB optical remote sensing image Output: p ∈ R^K                            # Predicted class probability                                                         # (K = 72 for FGSC-72)

Learnable parameters:        Θ_bb                                    # MobileNetV3-Large backbone weights        Θ_ufg = {Θ_ufg1, Θ_ufg2} # UFG-FEM module weights        Θ_dbc                                    # DB-CMM module weights        Θ_cls                                    # Classification head weights

---------- Stage 1: Stem and shallow feature extraction ----------

1: X_0 ← Stem(I)                           # X_0 ∈ R^(16 × 112 × 112) 2: X_1 ← BN_Block_1_to_3(X_0) # X_1 ∈ R^(24 × 56 × 56) 3: X_2 ← BN_Block_4(X_1)           # X_2 ∈ R^(40 × 28 × 28)

---------- Stage 2: First frequency decoupling ----------

4: X_2′ ← UFG-FEM_1(X_2; Θ_ufg1) # Shape unchanged: 40 × 28 × 28

---------- Stage 3: Middle feature extraction ----------

5: X_3 ← BN_Block_5_to_8(X_2′) # X_3 ∈ R^(80 × 14 × 14)

---------- Stage 4: Second frequency decoupling ----------

6: X_3′ ← UFG-FEM_2(X_3; Θ_ufg2) # Shape unchanged: 80 × 14 × 14

---------- Stage 5: Deep feature extraction ----------

7: X_4 ← BN_Block_9_to_13(X_3′) # X_4 ∈ R^(160 × 7 × 7)

---------- Stage 6: Uncertainty-aware purification ----------

8: F_cm ← DB-CMM(X_4; Θ_dbc) # Shape unchanged: 160 × 7 × 7

---------- Stage 7: Deep tail and classification ----------

9: X_5 ← BN_Block_14_to_15(F_cm) # X_5 ∈ R^(960 × 7 × 7) 10: v ← GAP(X_5)                                # v ∈ R^960 11: z ← FC_Head(v; Θ_cls)                # z ∈ R^K, class logits 12: p ← Softmax(z)                               # p ∈ R^K

13: return p

Algorithm 2 UFG-FEM Forward Pass

Input: X ∈ R^(C × H × W)

Output: X_out ∈ R^(C × H × W)

Learnable parameters:

{σ_x1, σ_y1, θ_1, σ_x2, σ_y2, θ_2}                 # LAGK kernel parameters

w_freq ∈ R^(H × W)                                       # Frequency-aware weights

W_1 ∈ R^(C/r × C), W_2 ∈ R^(C × C/r)       # SCE channel MLP

α_1, α_2 ∈ R                                                      # Band fusion coefficients

---------- Multi-scale Laplacian Decoupling ----------

1: G_1 ← Build_AnisoGaussian(σ_x1, σ_y1, θ_1)

2: G_2 ← Build_AnisoGaussian(σ_x2, σ_y2, θ_2)

3: X_low1 ← BN(DWConv(X, G_1))

4: X_low2 ← BN(DWConv(X_low1, G_2))

4: X_low2 ← BN(DWConv(X_low1, G_2)) 5: X_band1 ← X − X_low1 # High-frequency band 6: X_band2 ← X_low1 − X_low2 # Middle-frequency band 7: X_high ← α_1 · X_band1 + α_2 · X_band2

---------- Spectrum-aware Channel Excitation ----------

8: F_spec ← FFT2D(X_high) 9: z ← Mean(|F_spec|2 ⊙ w_freq, dim = spatial) # z ∈ R^C 10: W_c ← Sigmoid(W_2 · ReLU(W_1 · z)) # W_c ∈ R^C

---------- Residual fusion ----------

11: X_out ← X + W_c ⊗ X_high

12: return X_out

Algorithm 3 DB-CMM Forward Pass

Input: F_in ∈ R^(C′ × H′ × W′)

Output: F_cm ∈ R^(C′ × H′ × W′)

Learnable parameters:

Θ_eum                                    # EUM Beta parameter predictor

Θ_tgt                                    # Target branch depthwise conv

γ, ε                                    # OT cost & entropy temperature

Θ_att = {Θ_c, Θ_s}           # Channel & spatial attention

Σ_η = diag(σ_η2)              # Learnable Gaussian covariance

λ                                        # Uncertainty penalty coefficient

---------- Evidential Uncertainty Mask (EUM) ----------

1: [α, β] ← Softplus(Conv1x1_2(F_in; Θ_eum)) + 1

2: E_M ← α/(α + β)

3:  Var_M ← αβ/[(α + β)2 · (α + β+1)]

4:  M_inv ← Clip(1 − E_M + λ · sqrt(Var_M), 0, 1)

---------- Target-aware branch ----------

5: F_tgt ← ReLU(BN(DWConv3x3(F_in; Θ_tgt)))

---------- Optimal Transport soft alignment ----------

6: C ← Build_CostMatrix(F_tgt, γ) 7: μ, ν ← Compute_Marginals(F_tgt, M_inv) 8: T ← Sinkhorn(C, μ, ν, ε, T_iter) 9: M_inv_aligned ← Reshape(T · vec(M_inv))

---------- Feature modulation ----------

10: F_mod ← F_tgt ⊙ M_inv_aligned

---------- Information Bottleneck Dual-Attention ----------

11: ac ← Sigmoid(MLP(GAP(F_mod); Θ_c)) 12: a_s ← Sigmoid(Conv7x7([MaxPool_c(F_mod); AvgPool_c(F_mod)]; Θ_s)) 13: if training: 14: η ← Sample(N(0, Ση)) 15: else: 16: η ← 0 17: F_sa ← (a_c ⊗ a_s ⊗ F_mod) + η

---------- Residual fusion ----------

18: F_cm ← F_sa + F_in

19: return F_cm

Algorithm 1 describes the overall computation from the input image $I\in {ℝ}^{3\times 224\times 224}$ to the predicted class probability distribution $p\in {ℝ}^K$. The 15 bottlenecks of the backbone are divided into five consecutive segments according to the embedding positions, and the two proposed modules are inserted between the corresponding segments. The output tensor shape at each step is annotated on the right side of the pseudocode, so as to reflect the variation of feature map resolutions. Since the input and output shapes of the proposed modules are identical to those of the corresponding backbone layers, no structural modification is required for the remaining part of the backbone, and its weights pretrained on ImageNet-1K can be directly used for initialization.

Algorithm 2 describes the internal forward propagation of UFG-FEM, which consists of three parts. Multi-scale Laplacian decoupling (Lines 1–7) constructs low-frequency and middle-frequency responses through two learnable anisotropic Gaussian kernels, and derives two high-frequency bands by differencing. Spectrum-aware channel excitation (Lines 8–10) applies a two-dimensional Fourier transform and frequency-weighted pooling to the high-frequency response, so as to compute channel-wise re-weighting coefficients. Residual fusion (Line 11) adds the weighted high-frequency response back to the original input. Here, $\mathrm{DWConv}$ denotes depthwise separable convolution, and $\otimes$ denotes broadcast multiplication along the channel dimension.

Algorithm 3 describes the internal forward propagation of DB-CMM, which consists of four parts. The evidential uncertainty mask (Lines 1–4) predicts two Beta distribution parameters $\alpha$ and $\beta$ at each spatial location through a $1\times 1$ convolution, then computes the expectation $E_M$ and variance ${\mathrm{Var}}_M$ of the mask, and derives the inverse background mask $M_{\mathrm{inv}}$ as “the expectation minus $\lambda$ times the standard deviation”. The target-aware branch (Line 5) extracts target semantic features through a depthwise separable convolution. The optimal transport soft alignment (Lines 6–9) aligns $M_{\mathrm{inv}}$ and $F_{\mathrm{tgt}}$ in the feature space via the Sinkhorn algorithm. The information bottleneck dual attention (Lines 10–17) selectively re-weights the modulated features along the channel and spatial dimensions, and introduces a Gaussian perturbation $\eta$ with learnable covariance during training; at inference time, $\eta$ is set to zero to ensure deterministic prediction. The residual fusion at Line 18 preserves the original feature pathway when either the OT alignment or the EUM estimation deviates from the expected behavior.

In the pseudocode, $\otimes$ denotes broadcast multiplication and $\odot$ denotes element-wise multiplication. The learnable parameters are collectively listed in the “Learnable parameters” block at the beginning of each algorithm, and their mathematical definitions are given in Section 2.3 and Section 2.4.

2.3. Ultra-Fine-Grained Feature Enhancement Module (UFG-FEM)

In optical remote sensing ship images, the main hull contour corresponds to low-frequency components due to its smooth, large continuous regions, while the key components determining ultra-fine-grained classification belong to high-frequency components. Repeated downsampling in standard CNNs can attenuate high-frequency signals, which may weaken discriminative microstructural information in deeper features. Although conventional multi-scale modules such as Inception capture features at different scales through parallel convolutions, their implicit fusion cannot explicitly organize the propagation of discriminative details and background responses. We therefore first focus on recovering discriminative high-frequency information, establishing a more stable feature basis for the subsequent background modulation.

As the recovery module of the framework, UFG-FEM performs explicit frequency decomposition inspired by Mallat’s multi-resolution analysis and the Burt–Adelson Laplacian pyramid to recover the discriminative high-frequency information needed for subsequent propagation. It constructs multi-scale low-pass bases via cascaded learnable anisotropic Gaussian kernels, explicitly decouples high-frequency residuals at different scales in the frequency domain, and amplifies discriminative signals through a spectrum-aware channel excitation mechanism. The module structure is shown in Figure 3.

Figure 3. Structure of UFG-FEM.

Let the input feature tensor of UFG-FEM be $X\in {ℝ}^{C\times H\times W}$, where $C$ is the channel number, and $H,W$ are feature map height and width. Unlike fixed isotropic Gaussian kernels in prior works, UFG-FEM independently learns a set of directional parameters $({\sigma }_{x,c,s},{\sigma }_{y,c,s},{\theta }_c)$ for each channel $c\in \{1,2,\dots ,C\}$, where ${\sigma }_{x,c,s}$ and ${\sigma }_{y,c,s}$ are standard deviations along the major and minor axes, ${\theta }_c$ is the major-axis rotation angle, and $s\in \{1,2\}$ is the scale index. A learnable anisotropic Gaussian kernel (LAGK) $G_c^{(s)}(u,v)$ is constructed accordingly, as given in (1).

$$G_c^{(s)}(u,v)={\displaystyle \frac{1}{Z}}\exp \left(-{\displaystyle \frac{{(u^{\prime })}^2}{2{\sigma }_{x,c,s}^2}}-{\displaystyle \frac{{(v^{\prime })}^2}{2{\sigma }_{y,c,s}^2}}\right)$$

(1)

where $(u,v)$ are spatial coordinates in the kernel domain, rotated coordinates satisfy ${(u^{\prime },v^{\prime })}^{\top }=R({\theta }_c)\cdot {(u,v)}^{\top }$, $R({\theta }_c)$ is the rotation matrix, and $Z$ is the normalization factor ensuring the kernel sum equals 1. The Gaussian kernel provides a multi-scale smoothing basis for feature decomposition. Considering that ship targets exhibit strong directional structures, UFG-FEM adopts learnable anisotropic Gaussian kernels instead of fixed isotropic kernels. By adapting the smoothing direction and scale, LAGK generates direction-aware low-pass responses while preserving edge structures relevant to ultra-fine-grained recognition. The kernel parameters are initialized with a dyadic scale ratio to obtain approximately separated multi-scale responses.

Two-level low-pass responses $X_{low}^{(1)}$ and $X_{low}^{(2)}$ are obtained by cascading two LAGK layers via depthwise separable convolutions, as shown in (2).

$$\left\{\begin{array}{c}{X}_{low}^{(1)}=\mathrm{BN}\left(\mathrm{DWConv}(X;G^{(1)})\right)\\ X_{low}^{(2)}=\mathrm{BN}\left(\mathrm{DWConv}(X_{low}^{(1)};G^{(2)})\right)\end{array}\right.$$

(2)

where $\mathrm{DWConv}(\cdot ;G^{(s)})$ denotes depthwise separable convolution with kernel $G^{(s)}={\left\{G_c^{(s)}\right\}}_{c=1}^C$, $\mathrm{BN}(\cdot )$ is batch normalization, and $X_{low}^{(1)},X_{low}^{(2)}\in {ℝ}^{C\times H\times W}$ are the two low-pass feature levels. Following the Laplacian pyramid principle, high-frequency residual bands $X_{band}^{(1)}$ and $X_{band}^{(2)}$ are derived at two scales, as given in (3).

$$\left\{\begin{array}{c}{X}_{band}^{(1)}=X-X_{low}^{(1)}\\ X_{band}^{(2)}=X_{low}^{(1)}-X_{low}^{(2)}\end{array}\right.$$

(3)

where $X_{band}^{(1)},X_{band}^{(2)}\in {ℝ}^{C\times H\times W}$ are ultra-high-frequency and mid-high-frequency band features, respectively. By construction, the identity $X=X_{low}^{(2)}+X_{band}^{(2)}+X_{band}^{(1)}$ holds strictly in an element-wise sense in the spatial domain. Since the Gaussian kernel has infinite support in the frequency domain, the two band-pass components are not strictly orthogonal in the Fourier sense; nevertheless, under the dyadic-scale constraint stated above, their dominant energy supports are approximately separated, yielding an approximately decoupled multi-scale band decomposition in the feature representation sense. Learnable band fusion coefficients ${\alpha }_1,{\alpha }_2\in ℝ$ (initialized to 1.0) are introduced for adaptive weighted fusion of multi-scale high-frequency features, as shown in (4).

$$X_{high}={\alpha }_1\cdot X_{band}^{(1)}+{\alpha }_2\cdot X_{band}^{(2)}$$

(4)

where $X_{high}\in {ℝ}^{C\times H\times W}$ is the fused high-frequency discriminative feature map. ${\alpha }_1$ and ${\alpha }_2$ govern the responses to fine high-frequency details and mid-frequency structures, respectively; their adaptive learning dynamically balances discriminative signals across scales, and the fused result serves as input to the subsequent background modulation stage.

To quantify each channel’s contribution to ultra-fine-grained discrimination, UFG-FEM introduces a spectrum-aware gating (SAG) mechanism. First, $X_{high}$ is transformed to the frequency domain via 2D fast Fourier transform $\mathcal{F}\{\cdot \}$, and its power spectrum is modulated by learnable frequency weights $w_{freq}\in {ℝ}^{H\times W}$. A spectrum-aware channel descriptor $z\in {ℝ}^C$ is computed via (5).

$$z_c={\displaystyle \frac{1}{HW}}{\displaystyle \sum _{u,v}{\left|\mathcal{F}\left\{X_{high,c}\right\}(u,v)\right|}^2}\cdot w_{freq}(u,v)$$

(5)

where $X_{high,c}$ is the $c$-th channel of $X_{high}$, and $(u,v)$ are frequency-domain coordinates. Compared with conventional SE modules that only perform spatial average pooling, the spectrum-aware descriptor explicitly captures per-channel frequency-domain energy distribution and is more sensitive to high-frequency discriminative signals. The descriptor passes through a bottleneck MLP with Sigmoid activation to produce channel excitation weights, as given in (6).

$$s=\sigma \left(W_2\cdot \mathrm{ReLU}(W_1\cdot z)\right)$$

(6)

where $W_1\in {ℝ}^{(C/r)\times C}$ and $W_2\in {ℝ}^{C\times (C/r)}$ are learnable parameter matrices, $r$ is the channel reduction ratio, $\sigma (\cdot )$ is the Sigmoid function, and $s\in {[0,1]}^C$ is the channel excitation vector. Finally, the enhanced feature $X_{out}$ is synthesized via channel weighting and residual connection, as shown in (7).

$$X_{out}=X+s\otimes X_{high}$$

(7)

where $\otimes$ denotes channel-wise broadcast element-wise multiplication, and $X_{out}\in {ℝ}^{C\times H\times W}$ is the UFG-FEM output. The residual structure preserves identity mapping capability and avoids corrupting the backbone’s pretrained representations.

2.4. Dual-Branch Context Modulation Module (DB-CMM)

While enhancing target discriminative details, UFG-FEM inevitably amplifies high-frequency background noise such as sea waves and port edge structures. Conventional attention mechanisms produce point-estimate masks that cannot represent uncertainty about “whether a location belongs to background”, leading to false suppression of discriminative signals in blurred transition regions. To this end, DB-CMM serves as the background response modulation stage in the sequential context modulation framework, applying background modulation to the recovered deep features through evidential uncertainty modeling and spatial soft alignment, as shown in Figure 4.

Figure 4. Structure of DB-CMM.

Given the deep input feature $F_{in}\in {ℝ}^{C^{\prime }\times H^{\prime }\times W^{\prime }}$ of DB-CMM, the background probability at each spatial location $\left(i,j\right)$ is parameterized as $m_{bg}^{(i,j)}~\mathrm{Beta}({\alpha }_{ij},{\beta }_{ij})$. The concentration parameter ${\alpha }_{ij}$ is directly predicted by a deterministic convolutional branch following the Evidential Deep Learning (EDL) formulation. Therefore, this formulation is used as an evidential distribution parameterization rather than posterior probabilistic inference. Its closed-form mean represents the estimated background tendency, whereas the variance is used as a relative indicator of local evidential uncertainty to regulate the modulation strength.

Since pixel-level foreground/background annotations are unavailable, $M_{bg}$ is not learned through direct segmentation supervision. Instead, the Beta parameter branch is jointly optimized with the recognition network: the classification loss ${\mathcal{L}}_{cls}$ is back-propagated through $M_{inv}$ to encourage spatial modulation patterns that benefit image-level discrimination, while ${\mathcal{L}}_{KL}^{Beta}$ regularizes excessive evidence concentration and ${\mathcal{L}}_{align}$ constrains the spatial correspondence between the modulation mask and target features. Accordingly, $M_{bg}$ is interpreted as a task-dependent weakly supervised modulation map rather than a pixel-wise semantic background mask.

The network outputs the two concentration parameters $[\alpha ,\beta ]$ of the Beta distribution via (8).

$$[\alpha ,\beta ]=\mathrm{Softplus}\left({\mathrm{Conv}}_{1\times 1}^{(2)}(F_{in})\right)+1$$

(8)

where $\alpha ,\beta$ are per-position concentration parameter tensors, $\mathrm{Softplus}(x)=\ln (1+e^x)$ ensures non-negative outputs, and the +1 offset guarantees ${\alpha }_{ij},{\beta }_{ij}>1$ to avoid bimodal Beta degeneration. The corresponding expectation and variance tensors are given in (9).

$$\left\{\begin{array}{c}\mathbb{E}{[M_{bg}]}_{ij}={\displaystyle \frac{{\alpha }_{ij}}{{\alpha }_{ij}+{\beta }_{ij}}}\\ \mathrm{Var}{[M_{bg}]}_{ij}={\displaystyle \frac{{\alpha }_{ij}{\beta }_{ij}}{{({\alpha }_{ij}+{\beta }_{ij})}^2({\alpha }_{ij}+{\beta }_{ij}+1)}}\end{array}\right.$$

(9)

where $\mathbb{E}[M_{bg}]$ is the expected background mask tensor, and $\mathrm{Var}[M_{bg}]$ quantifies mask uncertainty. A large ${\alpha }_{ij}+{\beta }_{ij}$ (sufficient evidence) yields low variance and confident background judgment; a small sum (weak evidence) yields high variance and high uncertainty. An inverse background probability mask is constructed as shown in (10).

$$M_{inv}=\mathrm{clip}\left(1-\mathbb{E}[M_{bg}]+\lambda \cdot \sqrt{\mathrm{Var}[M_{bg}]},0,1\right)$$

(10)

Here, $\lambda$ is a learnable uncertainty-tolerance coefficient (initialized to 0.5); the square root, addition, and subtraction are all applied element-wise; $\mathrm{clip}(\cdot ,0,1)$ truncates the result to $[0,1]$; and $M_{inv}\in {[0,1]}^{H^{\prime }\times W^{\prime }}$ denotes the inverse mask. This formulation can be interpreted as an uncertainty-aware upper bound on the foreground-retention strength. In high-confidence background regions ($\mathrm{Var}[M_{bg}]$ small and $\mathbb{E}[M_{bg}]\to 1$), $M_{inv}\to 0$, and full suppression is applied; in high-confidence foreground regions ($\mathrm{Var}[M_{bg}]$ small and $\mathbb{E}[M_{bg}]\to 0$), $M_{inv}\to 1$, and full preservation is enforced; in regions of high uncertainty, the term $+\lambda \sqrt{\mathrm{Var}[M_{bg}]}$ actively raises the upper-bound estimate of $M_{inv}$, relaxing the retention threshold and thereby avoiding excessive erasure of discriminative signals. This realizes conservative suppression under uncertainty.

Since pixel-level background ground truth is unavailable in this task, the parameters $\alpha$ and $\beta$ are learned through a closed loop of implicit supervision at four levels. (i) The classification loss ${\mathcal{L}}_{cls}$ is back-propagated to $M_{inv}$ through the residual path $F_{cm}=F_{in}+F_{sa}$, encouraging the network to identify as background those regions whose suppression still benefits classification. (ii) ${\mathcal{L}}_{KL}^{Beta}$ takes the uniform prior $\mathrm{Beta}(1,1)$ as a reference, preventing ${\alpha }_{ij}+{\beta }_{ij}\to \infty$ and thus avoiding collapse to a point estimate. (iii) ${\mathcal{L}}_{align}={⟨T^{*},C⟩}_F$ penalizes the spatial misalignment between $M_{inv}$ and the target features through the transport cost, implicitly encouraging geometric consistency between the mask and the target semantics. (iv) The residual connection in (17) ensures that, while $M_{inv}$ has not yet converged, gradients can still be stably propagated through the original feature pathway, preventing training collapse. Together, these four mechanisms enable $M_{bg}$ to progressively converge to a semantically reasonable background distribution in the absence of pixel-level annotations.

In parallel, the target branch extracts target-specific features $F_{tgt}$ via depthwise separable convolution, as given in (11).

$$F_{tgt}=\mathrm{ReLU}\left(\mathrm{BN}\left({\mathrm{DWConv}}_{3\times 3}(F_{in})\right)\right)$$

(11)

where $F_{tgt}\in {ℝ}^{C^{\prime }\times H^{\prime }\times W^{\prime }}$ is the target-specific semantic feature. To ensure that background modulation remains spatially consistent with the recovered discriminative features, we employ optimal transport (OT) to achieve a spatial soft alignment between the background probability and the target features, rather than a direct element-wise multiplication.

Both $F_{tgt}$ and $M_{inv}$ are flattened along the spatial dimension into $N=H^{\prime }\times W^{\prime }$ locations, and the cost matrix $C\in {ℝ}^{N\times N}$ is constructed as in (12).

$$C_{i,j}={‖F_{tgt}^{(i)}-F_{tgt}^{(j)}‖}_2^2+\gamma \cdot {‖p_i-p_j‖}_2^2$$

(12)

where $F_{tgt}^{(i)},F_{tgt}^{(j)}\in {ℝ}^{C^{\prime }}$ denote feature vectors at locations $i$ and $j$, $p_i,p_j\in {ℝ}^2$ are their spatial coordinates, and $\gamma >0$ controls spatial-distance regularization. The two terms measure semantic and spatial distances, respectively. The source and target marginals $\mu ,\nu \in {\Delta }^{N-1}$ are defined as in (13).

$$\left\{\begin{array}{c}{\mu }_i={\displaystyle \frac{{‖F_{tgt}^{(i)}‖}_2}{{\displaystyle \sum _{k=1}^N{‖F_{tgt}^{(k)}‖}_2}}}\\ {\nu }_j={\displaystyle \frac{M_{inv}^{(j)}+{\epsilon }_0}{{\displaystyle \sum _{k=1}^N\left(M_{inv}^{(k)}+{\epsilon }_0\right)}}}\end{array}\right.$$

(13)

where $\mu$ represents the spatial energy distribution of target features, $\nu$ represents the foreground-preference distribution of Minv, and ${\epsilon }_0={10}^{-6}$ prevents numerical instability. Entropy-regularized optimal transport is then formulated as in (14).

$$T^{*}=\arg {\min }_{T\in \Pi (\mu ,\nu )}{⟨T,C⟩}_F-ϵ\cdot H(T)$$

(14)

where $\Pi (\mu ,\nu )=\{T:T1=\mu ,T^{\top }1=\nu \}$ denotes the coupling set satisfying the marginal constraints, $H(T)$ is the transport-plan entropy, and $ϵ$ is the entropy regularization coefficient. The optimal plan $T^{*}$ is obtained using the Sinkhorn–Knopp algorithm and applied to Minv to produce the aligned feature via (15).

$$F_{mod}=F_{tgt}\odot \mathrm{Reshape}\left(T^{*}\cdot \mathrm{vec}(M_{inv})\right)$$

(15)

Here, $\mathrm{vec}(\cdot )$ and $\mathrm{Reshape}(\cdot )$ convert between vectorized and spatial representations, and $\odot$ denotes element-wise modulation. $F_{mod}\in {ℝ}^{C^{\prime }\times H^{\prime }\times W^{\prime }}$ is then processed by the information bottleneck dual-attention submodule to reduce feature redundancy. Channel–spatial attention with learnable Gaussian perturbation forms the stochastic representation $F_{sa}$, as given in (16).

$$F_{sa}=a_s\otimes \left(a_c\otimes F_{mod}\right)+\eta ,\eta ~\mathcal{N}(0,{\Sigma }_{\eta })$$

(16)

where $a_c\in {[0,1]}^{C^{\prime }}$ and $a_s\in {[0,1]}^{H^{\prime }\times W^{\prime }}$ are channel and spatial attention weights, respectively, and $\eta$ is sampled during training and set to zero during inference. The corresponding KL regularization constrains representation redundancy. To preserve the original feature pathway, DB-CMM uses the residual connection in (17).

$$F_{cm}=F_{in}+F_{sa}$$

(17)

The resulting $F_{cm}$ is the DB-CMM output, with the residual pathway providing a stable identity connection during feature modulation.

2.5. Multi-Task Loss Function

To jointly optimize the functions of the different stages in the sequential context modulation framework, CM-Net adopts the joint optimization objective in (18).

$${\mathcal{L}}_{total}={\mathcal{L}}_{cls}+{\lambda }_1{\mathcal{L}}_{KL}^{Beta}+{\lambda }_2{\mathcal{L}}_{IB}+{\lambda }_3{\mathcal{L}}_{band}+{\lambda }_4{\mathcal{L}}_{align}$$

(18)

where ${\lambda }_1,{\lambda }_2,{\lambda }_3,{\lambda }_4$ are weighting coefficients, and each term is defined as follows:

• ${\mathcal{L}}_{cls}$ is the primary classification cross-entropy loss with label smoothing ($\epsilon =0.1$), to mitigate overfitting across ultra-fine-grained classes;
• ${\mathcal{L}}_{KL}^{Beta}$ is the KL divergence between the Beta distribution output by EUM and the uniform prior $\mathrm{Beta}(1,1)$, to prevent uncertainty modeling from degenerating to point estimation;
• ${\mathcal{L}}_{IB}$ is the closed-form Gaussian KL regularization from the information bottleneck submodule, to compress feature distributions toward the standard normal distribution;
• ${\mathcal{L}}_{band}$: $L_1$ sparsity constraint on UFG-FEM high-frequency features $X_{high}$, formulated as shown in (19), where ${\mathcal{S}}_{UFG}=\{1,2\}$ indexes the UFG-FEM modules, $X_{high}^{(m)}$ is the high-frequency feature map from the $m$-th UFG-FEM, and $C_m,H_m,W_m$ are its dimensions; this term encourages activation only at key ship components;
• ${\mathcal{L}}_{align}={⟨T^{*},C⟩}_F$ is the optimal transport cost, to constrain spatial soft alignment and preserve local perturbation properties.

  $${\mathcal{L}}_{band}={\displaystyle \frac{1}{\left|{\mathcal{S}}_{UFG}\right|}}{\displaystyle \sum _{m\in {\mathcal{S}}_{UFG}}\frac{1}{C_m{H}_m{W}_m}}\Vert X_{high}^{(m)}{\Vert }_1$$

  (19)

2.6. Experimental Setup

Given the substantial variation in raw image dimensions across FGSC-72 (longest side ranging from 295 to 1065 pixels), all images were resized to 224 × 224 using aspect-ratio-preserving scaling with zero-padding (letterbox), rather than direct resizing, to retain the geometric integrity of ship contours. During training, online augmentation was applied, including random horizontal flipping, random rotation (±15°), and color jitter (brightness, contrast, and saturation perturbation magnitude of 0.2). At test time, only center cropping and pixel normalization were performed, with mean and standard deviation taken from ImageNet statistics.

All models were implemented in PyTorch (2.12.0) and trained on a single NVIDIA RTX 3090 GPU. Unless otherwise specified, the experiments in Table 1, Table 2, Table 3, Table 4 and Table 5 followed a unified optimization protocol. Input images were resized to 224 × 224. Models were optimized using AdamW with an initial learning rate of 5 × 10⁻⁴, a weight decay of 5 × 10⁻², a batch size of 64, and 100 training epochs. A cosine annealing schedule was adopted, with a 5-epoch linear warm-up and a minimum learning rate of 1 × 10⁻⁶. Gradient clipping was applied with a maximum norm of 1.0.

Table 1. Computational complexity comparison of different CM-Net configurations.

Model	Params (M)	FLOPs (G)
MobileNetV3 baseline	5.218	0.279
Baseline + UFG-FEM	5.244	0.301
Baseline + DB-CMM	5.228	0.280
CM-Net	5.254	0.302

Training-only objectives, including KL regularization and optimal transport alignment loss, are excluded from FLOPs calculation.

Table 2. Comparison with representative methods and adapted detection frameworks on FGSC-72.

Methods	Top-5 Acc (%)	Macro-Precision (%)	mAP (%)	F1-Score (%)
Cog-Net	86.06	58.87	72.74	59.85
ResNet50	90.55	84.36	86.87	82.79
Adapted Rotated FCOS	91.25	80.12	85.10	80.55
Adapted R3Det	92.16	81.25	86.41	81.60
Adapted S2ANet	93.45	82.34	87.55	83.13
Adapted RoI Transformer	94.10	82.81	88.20	83.91
Efficient-prompt	95.40	84.34	86.38	84.52
Adapted ReDet	95.55	83.61	89.15	84.88
Adapted Oriented R-CNN	96.11 ± 0.21	84.10 ± 0.35	90.05 ± 0.28	85.51 ± 0.31
PKINet	96.41	76.12	81.03	74.99
HMS-Net	96.94 ± 0.15	81.72 ± 0.33	84.28 ± 0.29	82.11 ± 0.34
Ours	97.49 ± 0.18	91.67 ± 0.22	94.69 ± 0.33	92.01 ± 0.21

The values for CM-Net and the two strongest competing methods (Adapted Oriented R-CNN and HMS-Net) are reported as mean ± standard deviation over five independent runs with random seeds 2021–2025. The remaining baselines report single-run results obtained under identical data split, preprocessing, and optimization configurations.

Table 3. The ablation results of the two sequential processing stages in context modulation.

Base	+UFG-FEM	+DB-CMM	Top-1 Acc	Top-5 Acc	F1-Score	mAP (%)
✔			80.12 ± 0.38	91.56 ± 0.31	82.15 ± 0.35	84.63 ± 0.32
✔	✔		81.35 ± 0.44	93.1 ± 0.37	82.9 ± 0.42	83.85 ± 0.41
✔		✔	85.40 ± 0.29	95.20 ± 0.26	86.8 ± 0.31	88.42 ± 0.28
✔	✔	✔	90.68 ± 0.22	97.49 ± 0.18	92.01 ± 0.21	94.69 ± 0.33

All values are reported as mean ± standard deviation over five independent runs with random seeds 2021–2025. Table 2 primarily evaluates the individual contributions and combined effect of UFG-FEM and DB-CMM on recognition performance, while Section 2.7 further analyzes the additional computational complexity introduced by these two modules from an algorithmic perspective.

Table 4. Component-level ablation results of key implementations in UFG-FEM and DB-CMM.

Processing Stage	Configuration	Top-1 Acc (%)	Top-5 Acc (%)	F1 (%)	mAP (%)	ΔmAP
Full Model	Full CM-Net	90.68 ± 0.22	97.49 ± 0.18	92.01 ± 0.21	94.69 ± 0.33	—
Discriminative Recovery	LAGK → isotropic Gaussian	89.74 ± 0.31	97.12 ± 0.24	91.36 ± 0.30	93.72 ± 0.36	−0.97
Discriminative Recovery	Spectrum-aware excitation → standard SE	90.10 ± 0.29	97.28 ± 0.21	91.62 ± 0.27	94.07 ± 0.34	−0.62
Discriminative Recovery	Single-scale LAGK	89.58 ± 0.35	96.94 ± 0.28	91.14 ± 0.33	93.51 ± 0.41	−1.18
Background Modulation	Beta → deterministic sigmoid mask	88.96 ± 0.38	96.61 ± 0.30	90.47 ± 0.39	92.81 ± 0.44	−1.88
Background Modulation	Beta mean only	89.77 ± 0.32	97.03 ± 0.24	91.31 ± 0.31	93.76 ± 0.37	−0.93
Background Modulation	w/o OT alignment	89.21 ± 0.36	96.73 ± 0.27	90.69 ± 0.36	93.12 ± 0.40	−1.57
Background Modulation	w/o IB module	89.63 ± 0.34	96.92 ± 0.26	91.08 ± 0.34	93.57 ± 0.39	−1.12

ΔmAP denotes the mAP change relative to the full CM-Net configuration.

Table 5. The ablation results of auxiliary loss terms in the joint optimization objective.

Loss Configuration	Top-1 Acc (%)	Top-5 Acc (%)	F1-Score (%)	mAP (%)	ΔmAP
Full Joint Objective	90.68 ± 0.22	97.49 ± 0.18	92.01 ± 0.21	94.69 ± 0.33	—
w/o ${\mathcal{L}}_{KL}^{Beta}$	90.04 ± 0.28	97.18 ± 0.21	91.54 ± 0.29	93.92 ± 0.36	−0.77
w/o ${\mathcal{L}}_{IB}$	90.22 ± 0.27	97.26 ± 0.20	91.68 ± 0.27	94.11 ± 0.35	−0.58
w/o ${\mathcal{L}}_{band}$	90.37 ± 0.25	97.33 ± 0.20	91.79 ± 0.25	94.28 ± 0.34	−0.41
w/o ${\mathcal{L}}_{align}$	89.88 ± 0.30	97.09 ± 0.23	91.42 ± 0.30	93.78 ± 0.38	−0.91

All backbones were initialized with ImageNet-1K pretrained weights. Newly introduced task-specific modules were initialized independently, with UFG-FEM and DB-CMM using Kaiming normal initialization. CM-Net was trained with the joint objective defined in (18), where λ₁ = 1.0, λ₂ = 0.5, λ₃ = 0.3, and λ₄ = 0.1; these weights were determined on the validation set. Optimal transport in DB-CMM was solved via Sinkhorn iteration, with an entropy regularization coefficient of 0.05, a maximum of 50 iterations, and a channel reduction ratio of 1/4. For comparison methods without the auxiliary objectives proposed in this study, standard cross-entropy loss was used, while all other optimization settings remained consistent with those described above.

To ensure statistical reliability and reproducibility, the main CM-Net experiments are independently run under five random seeds (2021–2025) with cuDNN deterministic mode enabled within each run to reduce operator-level non-determinism. The 8:1:1 stratified data split is generated once with a fixed partitioning seed and held constant across all five runs, ensuring that the ImageNet-1K pretrained backbone weights and evaluation partitions remain identical across seeds. The five training seeds exclusively govern the Kaiming initialization of the UFG-FEM and DB-CMM modules, mini-batch shuffling order, online augmentation sampling, and the Gaussian perturbation $\eta$ in DB-CMM. The checkpoint with the highest macro-averaged F1 on the validation set is selected for each run; final CM-Net results are reported as the mean and standard deviation over five runs, with key comparisons further validated by paired two-tailed $t$-tests at a significance threshold of $p<0.05$. All baseline methods are retrained with their classification heads replaced by 72-class outputs under identical data splits, preprocessing, and optimization settings to ensure fair comparison.

Given the pronounced long-tailed class distribution of FGSC-72, overall accuracy alone is insufficient to characterize model performance across categories with substantially different sample frequencies. Therefore, Top-1 accuracy, Top-5 accuracy, macro-Precision, mean Average Precision (mAP), and macro-averaged F1 score are considered in the evaluation. Top-1 and Top-5 accuracy characterize overall classification performance, whereas the class-wise metrics provide complementary information while reducing the dominance of majority classes in aggregate evaluation.

For class $k$, Average Precision (AP) is defined from the corresponding precision–recall relationship as the area under the precision–recall curve:

$$A{P}_k={\displaystyle {\int }_0^1{P}_k}(r)dr$$

(20)

where $P_k\left(r\right)$ denotes the precision of class $k$ at recall level $r$. For FGSC-72, which contains $K=72$ categories, mean Average Precision is obtained by averaging the class-wise AP values:

$$mAP={\displaystyle \frac{1}{K}}{\displaystyle \sum _{k=1}^{K}A}{P}_k$$

(21)

Macro-Precision is defined separately from mAP. It is calculated from the final classification predictions by first computing the precision of each class and then taking the arithmetic mean across all classes:

$$P_{\mathrm{macro}}={\displaystyle \frac{1}{K}}{\displaystyle \sum _{k=1}^K\frac{T{P}_k}{T{P}_k+F{P}_k}}.$$

(22)

Accordingly, macro-Precision summarizes the average class-wise precision under the final classification decisions, whereas mAP summarizes class-wise performance over the corresponding precision–recall relationships. For extremely rare categories represented by only one or very few test samples, the resulting class-level AP, Precision, and F1 estimates have limited statistical support and should therefore be interpreted together with the class-wise sample counts reported in Table A1.

2.7. Computational Complexity Analysis

Unlike parameter count and theoretical floating-point operations, the actual inference latency, throughput, and peak memory consumption of a model are affected by multiple factors, including hardware architecture, numerical macro-Precision, batch size, software framework, memory scheduling, and operator implementation. Therefore, rather than using parameter count or FLOPs alone to infer the actual deployment performance of CM-Net on a specific edge computing platform, we analyze the theoretical complexity of the additional computations introduced by UFG-FEM and DB-CMM from an algorithmic perspective.

For an input feature $X\in {ℝ}^{C\times H\times W}$, the main computational operations of UFG-FEM include two-stage depthwise-convolution low-pass filtering, frequency band differencing, two-dimensional fast Fourier transform (2-D FFT), frequency-weighted statistics, and channel excitation. Let the depthwise-convolution kernel size be $k\times k$ and the reduction ratio of the channel excitation be $r$. Ignoring lower-order operations such as element-wise addition/subtraction, activation functions, and broadcast multiplication, the two-stage depthwise convolution has a computational complexity of $O(CHW{k}^2)$, while the 2-D FFT has a complexity of $O(CHW\log (HW))$. The complexity of the channel excitation MLP is approximately $O(C^2/r)$. Accordingly, the major additional computational complexity of a single UFG-FEM can be expressed as (23).

$${\mathcal{O}}_{UFG}=O(CHW{k}^2)+O(CHW\log (HW))+O(C^2/r).$$

(23)

In CM-Net, the two UFG-FEMs operate on intermediate features of sizes $40\times 28\times 28$ and $80\times 14\times 14$, respectively. Their additional computation is therefore mainly associated with depthwise convolution and frequency-domain transformations at moderate spatial resolutions, without introducing dense spatial convolutions on the same scale as standard full-channel convolutions.

For DB-CMM, let the input deep feature be $F_{in}$, with the number of spatial positions defined as $N=H^{\prime }{W}^{\prime }$. The main additional computations of DB-CMM arise from EUM parameter prediction, depthwise convolution in the target branch, optimal transport (OT) cost-matrix construction, Sinkhorn iterations, and dual channel–spatial attention. The computational complexities of EUM parameter prediction and depthwise convolution are $O(C^{\prime }N)$ and $O(C^{\prime }N{k}^2)$, respectively. Since optimal transport requires constructing an $N\times N$ spatial cost matrix, its main computational complexity is approximately $O(C^{\prime }{N}^2)$. If the Sinkhorn algorithm performs $T$ iterations, the corresponding iterative computation is approximately $O(T{N}^2)$. The channel-attention component has a computational complexity of approximately $O({C^{\prime }}^2/r)$. Therefore, the major additional theoretical complexity of DB-CMM can be expressed as (24).

$${\mathcal{O}}_{DBC}=O(C^{\prime }{N}^2)+O(T{N}^2)+O(C^{\prime }N{k}^2)+O({C^{\prime }}^2/r).$$

(24)

Unlike UFG-FEM, the theoretical computational bottleneck of DB-CMM mainly arises from operations in optimal transport that scale quadratically with the number of spatial positions, $N^2$. To constrain the spatial complexity of this component, DB-CMM is placed at the deeper $160\times 7\times 7$ feature stage of the backbone, where $N=49$, rather than being directly applied to shallow high-resolution features. This design preserves deep semantic information while structurally constraining the spatial scale of the OT module.

The above analysis characterizes the algorithmic complexity of different modules as the input feature dimensions vary, rather than their actual inference time on specific hardware. In particular, the practical execution efficiency of operators such as FFT and Sinkhorn depends strongly on the underlying libraries, GPU/CPU architectures, parallelization strategies, and memory-access patterns. Therefore, theoretical complexity is used here to characterize the principal computational sources and their scaling behavior across modules, rather than being treated as equivalent to actual deployment latency or peak memory consumption.

In addition to the theoretical complexity analysis above, the practical computational overhead of CM-Net was further evaluated in terms of trainable parameters and floating-point operations (FLOPs). All measurements were conducted during inference with an input resolution of 224 × 224. To analyze the contribution of each proposed component, four configurations were compared: the MobileNetV3-Large baseline, the baseline with UFG-FEM, the baseline with DB-CMM, and the complete CM-Net. The results are summarized in Table 1.

As shown in Table 1, the MobileNetV3 baseline requires 5.218 M parameters and 0.279 G FLOPs. Introducing UFG-FEM increases the parameter number by 0.026 M and the computational cost by 0.022 G FLOPs, which mainly results from the additional frequency decomposition and spectrum-aware feature modulation operations. In comparison, DB-CMM introduces only a small computational overhead, increasing the parameters by 0.010 M and FLOPs by 0.001 G, since the uncertainty estimation and feature modulation branches are implemented using lightweight operations. Overall, CM-Net requires 5.254 M parameters and 0.302 G FLOPs, indicating that the proposed frequency enhancement and uncertainty-aware modulation mechanisms can be integrated with the MobileNetV3 backbone with limited additional computational cost.

3. Results

This section provides quantitative and qualitative analyses of CM-Net on FGSC-72, focusing on the proposed sequential context modulation. Section 3.1 compares CM-Net with representative methods. Section 3.2 performs ablation studies across three levels—sequential stages, intra-stage implementation, and joint optimization constraints—to investigate discriminative information recovery and background response suppression. Section 3.3 examines the compatibility of the proposed method across different backbone architectures, and Section 3.4 analyzes stage-wise spatial response variations via Grad-CAM.

3.1. Comparison with State-of-the-Art Methods

To systematically evaluate the performance of CM-Net on ultra-fine-grained ship recognition in optical remote sensing imagery, comparisons are conducted on the FGSC-72 dataset against 11 representative methods, covering general classification backbones (ResNet50, DenseNet121, EfficientNet-B0, ConvNeXt-T, MobileViT-S), dedicated fine-grained visual categorization methods (HMS-Net, Cog-Net), a parameter-efficient fine-tuning method (Efficient-prompt), a remote sensing backbone (PKINet), and six oriented detection frameworks adapted for the image-level classification setting (Adapted Rotated FCOS, Adapted R3Det, Adapted S2ANet, Adapted RoI Transformer, Adapted ReDet, and Adapted Oriented R-CNN). The oriented detection frameworks have been widely adopted in remote sensing ship recognition studies and provide representative feature learning baselines for this task. Since their original output spaces are designed for object detection, they were adapted to the image-level classification setting of FGSC-72. The detailed adaptation strategy is described in the “Baseline Adaptation Protocol” below. All compared methods use the same data split, data augmentation, and optimization strategies. Therefore, the performance differences in Table 2 mainly reflect their capabilities in preserving discriminative information and modeling complex backgrounds.

As shown in Table 2, CM-Net attains the best results across all four metrics, with a Top-5 accuracy of 97.49%, a macro-Precision of 91.67%, an mAP of 94.69%, and an F1-score of 92.01%. In particular, CM-Net surpasses the second-best adapted detection baseline, Adapted Oriented R-CNN, by 4.64 percentage points in mAP, and surpasses HMS-Net by 9.90 percentage points in F1-score. Notably, the gains in macro-Precision and F1-score are more pronounced than that in Top-5 Accuracy. This suggests that the improvement is driven less by relaxed candidate prediction and more by a better balance between preserving discriminative information and suppressing background interference, leading to better discrimination among similar classes.

Moreover, the six adapted detection methods are originally developed for remote sensing object detection benchmarks such as DOTA. Their results in this study are evaluated under the FGSC-72 classification setting, where the task objectives and evaluation criteria differ from those of object detection. In addition, after adaptation to the unified classification task, strong baselines such as CM-Net and Oriented R-CNN achieve moderate improvements in mAP.

The original output spaces of the six oriented detection frameworks (Rotated FCOS, R3Det, S2ANet, RoI Transformer, ReDet, and Oriented R-CNN) are designed for object detection and therefore are not directly compatible with the image-level classification setting. An adaptation protocol of “preserving the discriminative mechanism while unifying the task head” is therefore adopted: only the bounding-box regression and angle regression branches are removed, whereas the core discriminative structures of each method are fully retained, including the rotation-equivariant convolutions of ReDet, the RoI transformation module of RoI Transformer, the feature alignment module of S2ANet, and the feature refinement module of R3Det. A unified classification head consisting of global average pooling, a fully connected layer, and a Softmax activation (with output dimension K = 72) is then appended to the backbone output, which is identical to the classification head used in CM-Net.

For the three two-stage methods (RoI Transformer, Oriented R-CNN, and ReDet), since each FGSC-72 sample is a cropped ship chip, the region proposals generated by the RPN are uniformly replaced by the whole image as the single region of interest, so that the RoI transformation and rotation-equivariant mechanisms continue to operate on image-level features under the classification setting. For the three single-stage methods (Adapted Rotated FCOS, Adapted R3Det, and Adapted S2ANet), the localization and angle regression branches in the dense prediction head are removed, and the deep feature maps from the classification branch are fed into the unified classification head after global average pooling; the structures and parameter update procedures of the core modules are fully retained.

All comparison methods followed the unified data preprocessing and optimization protocol described in Section 2.6. Conventional classification baselines and adapted rotated object detection methods were trained with standard cross-entropy loss, whereas CM-Net used the same optimization protocol but with an additional model-specific auxiliary loss defined in (18). Backbones of all comparison methods were initialized with ImageNet-1K pretrained weights, while the task-specific classification heads were randomly initialized. After adaptation, the six detection methods are compared under the same FGSC-72 classification protocol.

3.2. Ablation Studies

Ablation studies are conducted on FGSC-72 with MobileNetV3-Large to evaluate the proposed sequential context modulation against discriminative-background coupling in feature propagation. The analysis proceeds from coarse to fine. UFG-FEM and DB-CMM are first introduced individually and jointly to assess each stage’s contribution. Key components within each stage are then replaced or removed to examine the link between specific designs and stage-level functions. Finally, auxiliary losses are removed one by one—with the forward architecture intact—to separate the effects of forward processing from training constraints. All configurations follow the data splits and training settings in Section 2.6, with results reported as mean ± standard deviation over five runs.

Table 3 examines the relationship between discriminative information recovery and background response modulation at the stage level. Compared to the baseline, introducing UFG-FEM alone improves Top-1 accuracy from 80.12% to 81.35%, Top-5 accuracy from 91.56% to 93.10%, and F1-score from 82.15% to 82.90%, but mAP drops from 84.63% to 83.85%. Thus, high-frequency recovery alone does not yield consistent gains across all metrics. DB-CMM alone improves all metrics, with mAP reaching 88.42%. When both stages are combined in the full CM-Net, the model achieves 90.68% Top-1, 97.49% Top-5, 92.01% F1-score, and 94.69% mAP, surpassing both single-stage configurations.

These results are consistent with the analysis of discriminative-background coupling in feature propagation. UFG-FEM recovers high-frequency discriminative information attenuated by downsampling in shallow and intermediate layers, but sea surface textures, wakes, and harbor edges with similar frequency characteristics may be simultaneously enhanced. Without subsequent background modulation, the gains from high-frequency recovery may be partly offset by co-amplified background responses. DB-CMM alone performs background modulation on deep features, but the discriminative information it processes has already undergone prior downsampling. The full model organizes both into a sequential propagation pipeline, where recovered discriminative information is further refined through uncertainty-aware background modulation. The results in Table 2 are therefore better interpreted as experimental support for the sequential processing logic of discriminative recovery followed by background modulation, rather than a simple additive combination of two independent modules.

The stage-level ablation examines the overall interplay between the two stages but does not clarify whether their specific implementations align with the intended functions. Component-level ablation is thus conducted by replacing or removing one design at a time, with all other settings unchanged. For UFG-FEM, the learnable anisotropic Gaussian kernel, multi-scale frequency decomposition, and spectrum-aware channel excitation are examined. For DB-CMM, Beta distribution modeling, variance-involved mask construction, optimal transport soft alignment, and the information bottleneck submodule are evaluated. The goal is not to treat these components as independent methods, but to assess whether each supports its stage-level function within the sequential context modulation framework.

Table 4 compares component-level variants within each stage. For the discriminative recovery stage, replacing the anisotropic Gaussian kernel with an isotropic one reduces mAP by 0.97%, and single-scale decomposition reduces it by 1.18%, confirming that direction-adaptive bases and multi-scale decomposition both contribute to fine-grained structural recovery. Since ship targets have prominent principal-axis orientations and discriminative structures span multiple scales, fixed isotropic smoothing or single-scale decomposition weakens adaptability to diverse high-frequency patterns. Replacing spectrum-aware excitation with standard SE yields a smaller but consistent drop of 0.62%, indicating supplementary benefit from frequency-energy-based channel description.

The background modulation stage shows greater sensitivity. Replacing Beta modeling with a deterministic sigmoid mask produces the largest drop, with mAP falling from 94.69% to 92.81%. Using only the Beta mean reduces mAP by 0.93%, and the gap between these two variants confirms that explicit uncertainty utilization via variance-based conservative modulation provides additional suppression in target-background transition regions. Removing OT alignment and the information bottleneck submodule reduces mAP by 1.57% and 1.12%, respectively, confirming their roles in deep-layer background modulation.

All ablated configurations underperform the full model, with varying degradation across components, indicating that performance reflects the joint contribution of multiple stage-aligned designs rather than any single dominant mechanism. These results should be interpreted within the sequential propagation logic of recovery followed by modulation, where each component serves its designated stage function.

Since component-level ablation modifies forward propagation, performance changes reflect both structural and training constraint effects. To separate the two, each of the four auxiliary loss terms is removed individually with the full forward architecture intact, to assess whether each term acts as a generic regularizer or provides stage-consistent training constraints for the two sequential stages.

Table 5 shows that removing any auxiliary loss degrades all metrics. mAP drops by 0.91%, 0.77%, 0.58%, and 0.41% upon removing ${\mathcal{L}}_{align}$, ${\mathcal{L}}_{KL}^{Beta}$, ${\mathcal{L}}_{IB}$, and ${\mathcal{L}}_{band}$, respectively. As these terms act on different variables and stages, the magnitudes do not constitute a strict importance ranking; each contributes measurably under the joint optimization.

${\mathcal{L}}_{band}$ constrains high-frequency outputs of UFG-FEM against diffuse activation, while ${\mathcal{L}}_{KL}^{Beta}$, ${\mathcal{L}}_{align}$, and ${\mathcal{L}}_{IB}$ constrain DB-CMM in terms of evidence distribution, spatial correspondence, and representation compression. These losses thus reinforce existing forward mechanisms rather than forming independent processing pathways.

Cross-referencing component and loss ablations provide additional insight. Removing the IB submodule drops mAP by 1.12% versus 0.58% for removing ${\mathcal{L}}_{IB}$ alone; removing OT alignment drops mAP by 1.57% versus 0.91% for ${\mathcal{L}}_{align}$ alone. The gaps confirm that both modules contribute through their forward processing independently of their associated losses. The full CM-Net performance therefore reflects the combined effect of sequential propagation structure and stage-specific constraints, not merely regularization accumulation.

3.3. Cross-Backbone Compatibility Experiments

To assess whether the proposed sequential context modulation mechanism can be consistently integrated with different backbone architectures, experiments are conducted on five representative backbones covering compound scaling, residual learning, dense connectivity, modern convolution, and convolution–Transformer hybrids. The five backbones represent five distinct design paradigms in deep learning: EfficientNet-B0 (compound scaling paradigm); ResNet50 (residual learning paradigm); DenseNet121 (dense connectivity paradigm); ConvNeXt-T (modern convolutional paradigm); and MobileViT-S (convolutional-Transformer hybrid paradigm).

Across Table 6, the introduction of UFG-FEM and DB-CMM yields mAP improvements ranging from 4.60 ± 0.24 to 7.11 ± 0.31 percentage points. Two observations are discussed below.

Table 6. Performance comparison across different backbone architectures.

Backbone Type	Method	Top-1 Acc	Top-5 Acc	F1-Score	mAP (%)	p-Value
EfficientNet-B0	Baseline	81.50 ± 0.36	92.40 ± 0.30	83.15 ± 0.34	85.70 ± 0.36	—
	+ Ours	88.25 ± 0.24	96.80 ± 0.21	89.50 ± 0.26	92.15 ± 0.24	<0.001
ResNet50	Baseline	83.20 ± 0.31	92.15 ± 0.28	85.12 ± 0.30	86.40 ± 0.29	—
	+ Ours	89.45 ± 0.23	97.35 ± 0.19	90.80 ± 0.24	93.38 ± 0.22	<0.001
DenseNet121	Baseline	82.65 ± 0.34	91.80 ± 0.29	84.20 ± 0.33	85.34 ± 0.33	—
	+ Ours	89.10 ± 0.25	96.95 ± 0.22	89.75 ± 0.27	92.45 ± 0.25	<0.001
ConvNeXt-T	Baseline	86.80 ± 0.20	94.50 ± 0.18	88.45 ± 0.21	90.15 ± 0.19	—
	+ Ours	91.10 ± 0.17	97.85 ± 0.14	92.65 ± 0.18	94.80 ± 0.17	<0.001
MobileViT-S	Baseline	87.15 ± 0.22	94.85 ± 0.19	88.90 ± 0.23	90.65 ± 0.22	—
	+ Ours	91.85 ± 0.18	98.05 ± 0.15	93.10 ± 0.19	95.25 ± 0.18	<0.001

All values are reported as mean ± standard deviation over five independent runs with random seeds 2021–2025.

(1)

The five backbones span compound scaling, residual learning, dense connectivity, modern convolution, and convolution–Transformer hybridization, and the 95% confidence interval for ΔmAP excludes zero across all configurations. These results indicate that the performance gains are not restricted to the MobileNetV3 backbone used in the main experiments and that the proposed modules are compatible with several representative architectural paradigms. The gain on MobileViT-S (+4.60 ± 0.24 mAP) is noteworthy, as self-attention is commonly expected to render explicit spatial or frequency-domain modulation redundant. The consistent gains on MobileViT-S suggest that, when discriminative information and background responses remain coupled during feature propagation, similarity-based attention alone cannot fully replace explicit information recovery and background modulation.

(2)

Lightweight backbones such as EfficientNet-B0 (+6.45 ± 0.30 mAP) and DenseNet121 (+7.11 ± 0.31 mAP) exhibit somewhat larger absolute gains than higher-capacity counterparts such as ConvNeXt-T (+4.65 ± 0.21 mAP) and MobileViT-S (+4.60 ± 0.24 mAP), though the ordering is not strictly monotone with parameter count and ResNet50 falls between the two groups. A conservative interpretation is that, with limited representation capacity, explicit information recovery and background modulation provide more direct task priors. For high-capacity backbones, some of these behaviors can be learned by the network itself, resulting in smaller marginal gains.

3.4. Qualitative Visualization Analysis

To further assess the effects of sequential context modulation on discriminative information recovery and background response modulation, five representative samples are randomly selected from the FGSC-72 test set for Grad-CAM visualization. The Grad-CAM algorithm is adopted to generate attention heatmaps for the MobileNetV3 backbone, UFG-FEM and DB-CMM respectively for systematic comparison. The visualization results are presented in Figure 5.

Figure 5. Original ship images and attention maps from CM-Net. (a) are original input images, from left to right: Nimitz-class aircraft carrier, tank ship, KittyHawk-class aircraft carrier, Arleigh Burke-class destroyer and hospital ship; (b) are attention heatmaps of the MobileNetV3 backbone; (c) are attention heatmaps of UFG-FEM; (d) are attention heatmaps of DB-CMM. The color bar indicates relative Grad-CAM response intensity within each map, from lower responses (blue) to higher responses (red).

The Grad-CAM maps show qualitative differences in spatial response among the MobileNetV3 baseline, UFG-FEM, and DB-CMM configurations. In the dense port examples (Columns 1 and 4), the DB-CMM maps exhibit reduced responses in several surrounding background structures and relatively concentrated responses around the ship regions. In the open sea example (Column 2), UFG-FEM produces elongated response patterns approximately aligned with the hull axis. For the visually similar aircraft-carrier examples, the DB-CMM maps show relatively concentrated activation around local ship structures. Similar changes in response distribution are also observed in the remaining examples. These qualitative observations are consistent with the intended roles of UFG-FEM and DB-CMM in discriminative information enhancement and background response modulation, respectively. However, the Grad-CAM results are used only as auxiliary visualization evidence and should not be interpreted as direct proof of the internal mechanism or of deployment-level generalization.

4. Discussion

4.1. On Signal-Background Coupling and Sequential Context Modulation

The multi-level ablation provides complementary evidence for the proposed method. Stage-level results show that discriminative recovery and background modulation should not be treated as independent objectives. UFG-FEM alone improves Top-1, Top-5, and F1-score but slightly reduces mAP—consistent with the signal-background coupling problem, where recovering high-frequency structures may simultaneously enhance sea surface textures, wakes, and harbor edges; thus recovery alone does not uniformly improve ranking across categories. DB-CMM alone mitigates deep-layer background interference but operates on features already attenuated by downsampling. The full configuration outperforms both single-stage variants, indicating that the sequential pipeline better addresses the feature propagation demands of this task.

The component-level results show that neither stage depends on a single dominant component. For UFG-FEM, anisotropic modeling, multi-scale decomposition, and spectrum-aware excitation each contribute from the aspects of directional adaptation, scale coverage, and frequency selection; simplifying any of them causes consistent degradation. For DB-CMM, all simplified variants—deterministic sigmoid mask, Beta mean only, w/o OT alignment, and w/o IB—underperform the full configuration, reflecting distinct roles of uncertainty estimation, spatial correspondence, and redundancy suppression. These mechanisms are not independent contributions but instantiations of their respective stages; the experiments support design-function alignment, not the novelty of individual techniques.

Loss-term ablation adds an optimization-level perspective. Removing any auxiliary loss degrades performance, but by less than removing the corresponding forward structure—removing the full IB submodule drops mAP more than removing ${\mathcal{L}}_{IB}$ alone, and similarly for OT versus ${\mathcal{L}}_{align}$. This confirms that performance is not primarily attributable to regularization; forward propagation governs how discriminative information is recovered and modulated, while auxiliary losses impose stage-consistent constraints. Across all three ablation levels, the results consistently align with the sequential modulation logic: first recovering fine-grained discriminative information attenuated during deep propagation, then selectively suppressing the background responses concurrently retained or amplified.

4.2. On the Cross-Backbone Compatibility

The cross-backbone experiments show that the proposed sequential context modulation mechanism improves mAP by 4.60–7.11% percentage points across five representative backbone architectures. These architectures cover compound scaling, residual learning, dense connectivity, modern convolution, and convolution–Transformer hybrids, and therefore provide evidence that the proposed modules can be integrated with different feature extraction paradigms.

The consistent improvements across these architectures suggest that the observed benefit is not specific to the inductive bias of MobileNetV3. However, because all backbone experiments are conducted on FGSC-72, these results should be interpreted as evidence of cross-architecture compatibility rather than broad cross-dataset generalization. Evaluating transferability under independent data sources and label spaces remains necessary before stronger generalization claims can be made.

Notably, on MobileViT-S equipped with self-attention, the two modules still improve mAP from 90.65% to 95.25%. This result deserves further discussion. Self-attention has advantages in modeling long-range dependencies and global context, but its feature weighting is essentially spatial redistribution based on feature similarity, rather than explicit decoupling of frequency-domain signal components. When target discriminative signals and background noise are heavily spectrally overlapped, similarity-based weighting alone can barely distinguish them. UFG-FEM achieves explicit low-, mid- and high-frequency band decoupling in the frequency domain via learnable anisotropic Gaussian kernels, while DB-CMM performs selective processing on ambiguous regions in a probabilistic sense via evidential uncertainty masks. The information provided by the two modules is essentially complementary rather than redundant to that of self-attention, which may explain why the two modules still bring obvious improvements on self-attention backbones.

The inconsistent magnitude of improvement across backbones is also noteworthy. The magnitude of performance gain varies across backbone networks. Given the differences in architecture, inductive bias, and feature representation capacity among EfficientNet-B0, ResNet50, DenseNet121, ConvNeXt-T, and MobileViT-S, the marginal contributions of the two modules are not entirely consistent. Nevertheless, both UFG-FEM and DB-CMM yield consistent improvements across all five backbones, indicating that the proposed mechanisms exhibit consistent compatibility across the evaluated backbone paradigms.

4.3. On Qualitative Observations in Complex Maritime Scenarios

Grad-CAM visualization results are mutually consistent with quantitative metrics: the baseline backbone shows diffuse attention on most samples; after introducing UFG-FEM, attention converges toward the hull principal axis; after introducing DB-CMM, attention shifts from background regions such as port facilities and sea wakes to the hull itself. For the visually similar pair of Nimitz-class and KittyHawk-class aircraft carriers, the model’s attention concentrates on local component regions such as the superstructure and radar array, which matches the key discriminative regions relied on for model-level identification.

Further observation of visualization results across different scenarios reveals different response characteristics of UFG-FEM and DB-CMM. The response map generated by UFG-FEM presents a relatively fine high-frequency response pattern around the hull contour. Especially in open sea scenarios, its response forms a regular narrow-band distribution along the hull principal axis, highly consistent with the hull geometry. The response map generated by DB-CMM shows more compact high-response regions with relatively sharp edges, and retains moderate medium response in the transition zone between hull and background. These differences are consistent with the proposed sequential context modulation process, which first recovers discriminative details and then selectively modulates background responses, forming a continuous information propagation process.

In dense port scenarios, the baseline backbone’s attention is often attracted by background structures such as port building edges and storage yard facilities. This phenomenon is alleviated after introducing DB-CMM. This phenomenon is consistent with the design of the evidential uncertainty mask in DB-CMM: although port facilities produce strong high-frequency responses, their evidential strength for category discrimination is relatively weak. The observed response reduction in these regions is consistent with uncertainty-aware background modulation. Correspondingly, in the hull-background transition zone, the mask reduces suppression intensity to avoid suppressing potential discriminative edge signals together, which is consistent with the design principle of “conservative suppression in ambiguous regions”.

It should be noted that the above observations are qualitative descriptions of the visualization results. Grad-CAM reflects the spatial positions on which classification decisions depend, which do not necessarily coincide with the regions of interest in human visual discrimination. The visualization results of the same model may also vary across different Grad-CAM variants (e.g., Grad-CAM++, Score-CAM). Therefore, the above observations can only serve as auxiliary verification of the modules’ functional roles, and should not be regarded as direct evidence of the internal mechanisms of the modules.

4.4. Limitations

Although the proposed method achieves stable performance improvements on the FGSC-72 dataset, there are still several limitations, as detailed below.

(1)

Singularity of input modality. The input processed in this paper is optical RGB imagery. In scenarios with degraded optical imaging quality, such as nighttime, cloud occlusion or severe sea conditions, recognition performance relying solely on optical input is constrained by imaging conditions itself. Complementary information from modalities such as SAR, infrared and hyperspectral imagery has not been utilized in the proposed framework. Methodologically, it is feasible to extend the spatial frequency analysis idea in UFG-FEM to SAR imagery, since the spectral characteristics of speckle noise in SAR imagery can also be processed via similar band decoupling mechanisms. However, feature alignment and fusion strategies across different modalities still require dedicated design.

(2)

Rare-class evaluation under the long-tailed distribution. The number of samples per category in FGSC-72 ranges from five to 1766. For several extremely rare categories, the validation and test subsets contain only one or very few samples, resulting in limited statistical support for their class-specific metrics. The repeated-seed experiments reported in this study characterize variability caused by training stochasticity under a fixed data partition and do not represent uncertainty introduced by alternative dataset partitions. Therefore, the class-wise results of the rarest categories should be interpreted together with the corresponding sample counts reported in Table A1. More systematic sensitivity analysis based on additional rare-class samples or alternative resampling strategies may further characterize the robustness of long-tailed category evaluation in future studies.

(3)

Closed-set recognition assumption. Model training and evaluation in this paper are conducted within the closed-set scope of 72 predefined categories, without involving the detection of new ship categories outside the training set. In practical maritime situational awareness tasks, categories not covered by the training set may appear. Extending closed-set recognition to open-set or incremental learning scenarios is a direction for future work. Notably, the evidential uncertainty mask in DB-CMM inherently has the ability to model “regions with insufficient evidence”, which is conceptually related to “unknown class detection” in open-set recognition. However, there are still obvious differences between the two in mathematical modeling and training objectives, and the feasibility of direct migration needs further analysis.

(4)

Lack of temporal information from single-frame input. This paper takes single-frame static images as input and does not utilize temporal cues from remote sensing videos. Temporal information such as ship navigation trajectories and attitude changes may provide supplementary evidence for recognition and behavior analysis. Extending the proposed method to temporal input is one feasible research direction. In temporal scenarios, both UFG-FEM and DB-CMM need to be extended to the spatiotemporal dimension: frequency decoupling can be extended to 3D spectral analysis, and uncertainty modeling can incorporate temporal consistency constraints. Specific technical paths need to be explored in future work.

(5)

Limitations of experimental scope. Experiments in this paper are mainly conducted on the FGSC-72 dataset. Cross-dataset transfer performance and performance on real operational data have not been systematically evaluated. In addition, although the selected comparison methods cover representative works in general classification and remote sensing recognition, they do not exhaust all relevant methods.

(6)

Due to computational constraints, the multi-seed protocol was applied exhaustively to CM-Net, its ablation variants (Table 2), and all backbone-specific configurations in the cross-backbone experiments (Table 3). Within the SOTA comparison in Table 1, multi-seed evaluation was performed for CM-Net and the two strongest competing methods (Adapted Oriented R-CNN and HMS-Net), whereas the remaining baselines are reported as single-run results obtained under identical data split, preprocessing, and optimization configurations. This asymmetry in reporting macro-Precision should be taken into account when interpreting the numerical margins in Table 1; extending multi-seed evaluation to every baseline in the SOTA comparison remains part of ongoing benchmarking work.

(7)

This study focuses on ultra-fine-grained ship recognition, module contributions, and cross-backbone compatibility, without hardware-level evaluation on specific embedded, airborne, or spaceborne platforms. Actual inference efficiency depends on hardware architecture, numerical macro-Precision, batch size, software framework, memory management, and operator implementation, rather than solely on parameter count or theoretical computational cost. In particular, the frequency-domain operations in UFG-FEM and the optimal transport with Sinkhorn iterations in DB-CMM may exhibit runtime characteristics different from standard convolutions. Therefore, the added complexity analysis characterizes only the theoretical computational cost and scaling behavior and does not replace hardware deployment benchmarks. Future work will systematically evaluate latency, throughput, and peak memory consumption on representative embedded platforms under consistent implementation, macro-Precision, and batch-size settings.

5. Conclusions

This study proposes CM-Net for ultra-fine-grained ship recognition by organizing discriminative information recovery and background response modulation as two sequential stages. UFG-FEM enhances multi-scale high-frequency cues at shallow and intermediate layers, while DB-CMM applies Beta evidential modeling, optimal-transport-based alignment, and information bottleneck attention to modulate background responses in deeper features. FGSC-72, containing 72 model-level categories and approximately 13,000 images, is constructed for evaluation.

On FGSC-72, CM-Net achieves 97.49% Top-5 accuracy, 94.69% mAP, and 92.01% macro-F1. Ablation studies support the contributions of the sequential stages, their principal components, and the associated optimization constraints. Cross-backbone experiments further show consistent improvements across five representative architectures, while the visualization results provide qualitative observations of the resulting spatial responses.

Future work will consider multimodal fusion with SAR, infrared, or hyperspectral imagery; improved learning and evaluation for long-tailed and open-set categories; and extension from static images to temporal remote sensing sequences. These directions may further broaden the applicability of the proposed framework beyond the current FGSC-72 evaluation setting.