Two-Stage Fine-Grained Ship Recognition with a Detector Guided by Key Regions and a Multi-Patch Joint Classifier

Highlights

What are the main findings?

• Key regions play a key role in fine-grained ship recognition tasks in optical remote sensing images.
• Jointly verification on key regions contributes to improving recognition accuracy.

What are the implications of the main findings?

• Key region cognition makes fine-grained ship recognition interpretable.
• Whole-to-part hypothesis and a verification framework model the cognitive processes of humans.

Abstract

For human beings, fine-grained object recognition is a progressive process that proceeds from global outlines to local details. They can determine how to further focus on the distinctive regions based on the overall context, followed by recognition. To enhance the algorithm’s capability to capture critical features, a multi-stage recognition framework, integrated with human-attended key regions for fine-grained ship recognition, is proposed in this manuscript. First, a set of distinctive templates is constructed following human identification logic. On this basis, a supervised attention method, Key Regions Guided Yolo11 (KRGY), with part-to-whole regulation is proposed to help the model focus on critical components, leading to better recognition and location performance. Furthermore, a multi-head joint recognition classification module is proposed, with key regions of ship cropped with the distinctive templates. With the hypothesis and verification framework Key Regions Guided Yolo11-Multi Head Classifier (KRGY-MHC), the accuracy of ship recognition is significantly improved based on a challenging datasets with high inter-class similarity DCL-11.

1. Introduction

Ship detection and recognition algorithms are key to unlocking the value of remote sensing image data. The finer the granularity is, the greater the value derived from the data is. Trained with the large-scale remote sensing datasets [1,2,3,4], some single-stage detection algorithms can locate and recognize targets rapidly and effectively [5,6,7,8,9,10,11,12]. They utilize a backbone network for feature extraction, and simultaneously achieve target localization and classification, thereby effectively enhancing efficiency. Meanwhile, a number of two-stage detection and recognition algorithms [13,14,15] have improved the accuracy of target detection and recognition by first locating the target and then further classifying it. Ship detection algorithms for remote sensing image, considering the elongated and inclined characteristics of ship targets, have evolved from natural image-based algorithms to produce a series of outstanding achievements, such as rotated bounding box object detection algorithms [16,17,18,19].

Whether single-stage or two-stage algorithms, they utilize a classifier to map the features extracted by a feature extractor to specific categories and may implicitly contain global features, local features, and the relationships among them. Algorithms based on the self-attention mechanism [20,21,22] further enhance the accuracy of target detection and recognition by establishing long-distance relationships, better aligning with the way humans perceive objects. However, the approach of uniformly partitioning the target still differs from human cognitive processes.

In contrast, the process of ship recognition conducted by humans is iterative and progressive. Final conclusion are formed through the repeated hypothesis and verification of multi-dimensional information, rather than a simple multi-category classification. Additionally, humans possess the ability to infer which key components contribute to further determination. For example, given a target suspected to be an LNG carrier, key features consistent with this type of vessel will then be sought firstly, such as several spherical cargo tanks in the midsection. Once humans locate the critical areas of corresponding target, the identification and confirmation of the ship are essentially completed.

This phenomenon indicates that during the recognition process, humans achieve the directed extraction and logical integration of critical information, thereby forming a relatively explicit reasoning cognitive process. This approach significantly differs from the one-step, end-to-end recognition method typically employed by neural networks.

Related Works

To enhance target recognition accuracy, existing methods generally utilize attention mechanisms [23,24,25] to improve the model’s ability to extract key regions, attempting to simulate how humans pay more attention to critical parts. They incorporate learnable spatial or channel-wise weights into the network, automatically adjusting these weights based on input information to achieve enhancement of key features.

MSCL-Net [26] incorporates a channel-spatial attention module (CSAM) to model similarity-aware features within the same category, thereby enhancing representation learning to mitigate challenges arising from high inter-class similarity and substantial intra-class variance. Based on that, a region cropping and enlargement module is designed to capture discriminative fine-grained features from local regions, effectively reducing background interference. The CSAM is finally utilized to fuse features from both the original and cropped region images for final classification. MGANet [27] emphasizes that existing methods predominantly focus on global features while overlooking the utilization of discriminative information, thereby limiting the model’s capability in fine-grained recognition. It comprises two core components: a Local–Global Feature Alignment Module (LAM) and a Multi-Granularity Self-Attention Module (MSM). The LAM aligns part-level features with object-level representations through convolutions with varying strides, while the MSM incorporates a self-attention mechanism to effectively integrate multi-granularity features, thereby enhancing the modeling of subtle inter-class distinctions. Mid-level feature attention learning is introduced in [28] for fine-grained ship classification, which effectively combines a mid-level feature causal attention (MFCA) module and a mid-level channel attention (MCA) module, facilitating the detection of key regions and local features associated with subtle visual attributes. The MFCA module regulates the learning of mid-level features by evaluating attention maps against counterfactual attention maps. Concurrently, the MCA module utilizes a discriminative unit to derive distinctive features from channel-wise data and a diversity unit to direct feature channels toward more salient regions. Finally, an adaptive weighting mechanism is incorporated to dynamically modulate the contribution proportions of the MFCA and MCA modules in the overall model. To enhance robustness in multi-scale ship recognition, a bidirectionally fused dense feature fusion network (DFF-Net) is designed in EIRNet [25] to maximize multi-level information utilization and reduce redundancy. Subsequently, a dual-mask attention module (DMAM) refines the fused features, improving the distinction of ship features and suppressing background interference, thereby boosting performance in dense and cluttered scenes. Moreover, the Mask-RPN module, by reusing attention masks, enhances the efficiency of proposal generation. Finally, the introduction of a higher-level category concept further improves recognition accuracy by mining inter-class relationships.

Apart from that, Cog-Net [29] simulates the reasoning process of domain experts by incorporating a causal multi-grained feature learning (CMFL) module, which identifies salient regions of ship objects in full images to mimic human perceptual mechanisms. On this basis, it employs a visual concept learning (VCL) module to learn fundamental visual concepts for each ship category and generates predictions through explainable reasoning based on these concepts, thereby emulating the human cognitive process.

Additionally, some algorithms treat components of ship as detection targets, enabling explicit feature extraction from these regions [30]. Related-YOLO utilizes the mechanisms of relational attention to stress positional relationships between the components of a ship, extracting key features more accurately. Furthermore, in the context of fine-grained recognition, certain approaches [31] decompose targets into multiple regions for contrastive learning, mimicking the human cognitive process of integrating global and local information. SAFPN [32] integrates a spatial-aware alignment fusion module (SAFM) to capture discriminative fine-grained details across adjacent-scale feature maps. Meanwhile, an adaptive prototypical contrastive learning (APCL) approach is introduced to overcome the low feature discriminability resulting from significant intra-class variation and inter-class similarity among ships. By leveraging a prototypical contrastive loss, APCL strengthens the separability of features across different ship categories, thereby advancing fine-grained recognition performance. Furthermore, the studies of the keypoint-based method [33] and WakeNet [34] demonstrate that ship wakes are rich information sources for target identification, not just detection. WakeNet embeds expert knowledge via a landmark regression head to predict wake geometry for heading estimation, a frequency attention module to learn spectral features, and integrates the Radon transform. Del Prete et al.’s method uses a lightweight model for fine-grained keypoint detection of wake components. Both approaches enable the extraction of vital ship kinematic and descriptive attributes from wake signatures, advancing maritime surveillance capabilities.

However, for the fine-grained recognition of ship targets, the aforementioned algorithms still exhibit certain limitations. First, although various attention-based methods consider spatial and channel attention, they fundamentally operate under the constraints of target localization and classification losses without directly regulating the focus regions themselves. Though [25] proposed to build a Dual-Mask Attention Module to regulate the attention map, it is inconsistent with the salient aspects of human perception. Second, for methods employing contrastive learning on key regions, the selection of these regions still relies on the algorithm itself. This process introduces significant uncertainty, substantially reducing the overall reliability of the results. Third, detecting parts and the whole do not truly achieve integrated learning of global and local target features, resulting in a lack of connection between local parts and the overall target, which ultimately limits the improvement in recognition performance.

To address the above issues, a method guided by key regions is proposed in this paper. When humans recognize objects, they do not need to observe all detailed information but rely on typical, representative regions to achieve target cognition. This concept has been validated in SAR imagery, as many components of ships in SAR images appear completely disorganized and indescribable [35,36,37,38,39].

The concept of key regions is explicitly defined firstly. Based on this, the fine-grained recognition algorithms are further optimized from three perspectives, reinforcing focus capabilities through supervised attention mechanisms, explicitly constraining the extraction of key regions among components, and implementing a phased progressive approach for joint global and local recognition as shown in Figure 1.

In this manuscript, key regions are defined in terms of two aspects. First, the regions should be prominent or easily definable components of the ship itself, primarily those with distinct patterns or shapes. For example, disordered superstructures or island structures are generally not considered key regions. Areas often selected as key regions include helicopter decks, boat platforms, vertical launch systems, main guns, hull markings, smokestacks, and inclined missile launchers. Second, the regions should be areas that differ when compared with ships of the same class, serving as representative and distinctive features.

• By constructing attention map templates using manually annotated regions and establishing explicit loss functions, direct constraints are applied to spatial attention maps. It enhances the algorithm’s ability to extract and emphasize features from critical regions and fully leverages the advantage of remote sensing images capturing targets from an overhead perspective, which is not suitable for natural images.
• Based on the fact that targets in remote sensing images generally are consistent, regions of interest are accurately extracted by combining coarse-grained recognition results with masks. When the target category is misidentified, the algorithm inevitably captures key regions inconsistent with the actual category. This discrepancy is further amplified in subsequent classification stages, thereby reducing misjudgments.
• This breaks away from the approach of treating the target as a whole for convolutional recognition, and also differs from ViT’s method of random partitioning. Instead, it fully leverages prior knowledge to extract key regions, establishing a joint recognition method based on local–global relationships. This approach explicitly considers components and their interrelationships.

2. Materials and Method

2.1. Construction of Key Regions Mask

In natural images, due to variations in the relative position/pose of the target to the sensor, it is difficult to model regions of interest using universal templates even when identifiable regions are known. In contrast, targets in remote sensing images exhibit relatively stable appearances, which is determined by the nadir-looking imaging perspective, as shown in Figure 2.

For humans, the recognition of targets relies primarily on the presence of specific individual components and the spatial relationships among these components. In this work, different types of ships are grouped for the selection of distinguishable parts. On this basis, easily distinguishable parts of these targets are summarized and distinguished through intra-class comparison.

To account for deviations caused by different viewpoints, images of each type of ship are captured from multiple perspectives, and their masks are clustered to form a statistical key identification region mask, as shown in Figure 3.

Owing to this characteristic, the modeling of key identifiable components of ships is performed by incorporating expert knowledge from human recognition. Thus, it is possible to formulate the attention mask precisely with an affine transformation $RT$ from the bounding box $R=[x,y,w,h,\theta ]$ to key regions, as shown in Figure 4. $x,y,w,h,\theta$ are the parameters of the bounding box, representing the coordinates of the center point, width, height, and rotation angle, respectively. Each region $K{R}_i$ in the KRM for target is generated as Equation (1).

$$K{R}_i=R\times RT=[x,y,w,h,\theta ]\times \left[\begin{array}{ccccc}1& 0& 0& 0& 0\\ 0& 1& 0& 0& 0\\ r_{wx}^i& 0& r_w^i& 0& 0\\ 0& r_{hy}^i& 0& r_h^i& 0\\ 0& 0& 0& 0& 1\end{array}\right]$$

(1)

where $K{R}_i$ is the key region and $r_{wx}^i$, $r_{hy}^i$, $r_w^i$, $r_y^i$ are the coefficients of the affine transformation matrix.

2.2. Key-Region-Guided Attention Regulation

Unsupervised attention mechanisms are still limited in effectively conveying human focal areas. To address this, this paper proposes an expert knowledge transfer method based on supervised attention mechanisms.

Based on YOLO-OBB [7], a supervised learning branch for spatial attention is inserted after C3k2, as shown in Figure 5. Loss $L_{att}$ between scaled key regions (KR) masks and the weight maps output by the spatial attention module (SAM) are established based on the L1 norm. Neural networks extract features at different hierarchical levels in the shallow, intermediate, and deep layers. These features include edges, grayscale patterns, and others. It is noted that the spatial attention module acts on features across all channels. If all feature maps are strictly constrained to adhere to the mask distribution, this would indeed affect the perception of information such as object boundaries, as background information is a critical factor for target localization. Thus, the Huber loss [40] for attention map supervision is adopted in this work, which allows for a certain degree of deviation in the loss function.

$$L_{att}=\left\{\begin{array}{cc}{\displaystyle \frac{1}{2}}{(g{t}_a-pr{e}_a)}^2\hfill & \mathrm{if}|g{t}_a-pr{e}_a|\le \delta \hfill \\ \delta |g{t}_a-pr{e}_a|-{\displaystyle \frac{1}{2}}{\delta }^2\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(2)

where $pr{e}_a$ is the predicted weight map and $\delta$ is set to 0.5. Each key region in the ground truth attention mask $g{t}_a$ is a 2D Gaussian distribution, with the major and minor axes corresponding to the region’s width w and height h, respectively.

The loss function is composed of the localization loss, the classification loss, and the attention loss, weighted by ${\alpha }_1$ and ${\alpha }_2$, respectively.

$$L_1={\alpha }_1(L_{\mathrm{loc}}+L_{\mathrm{cls}})+{\alpha }_2{L}_{\mathrm{att}}$$

(3)

where $L_{\mathrm{loc}}$ and $L_{\mathrm{cls}}$ are the localization loss and the classification loss of the detector, respectively.

Part-to-Whole Structure Based Regulation

In this work, the application of expert knowledge relies on accurate target boundary. However, conventional detection and recognition algorithms often face the challenge of indeterminate ship bowing directions. As shown in Figure 6, to address this issue, the ship detection framework is further augmented by adding two additional categories for bow and stern sections. Each ship and its bounding box are labeled with the same index, which is used to establish a spatial constraint loss function between the parts and the overall target.

In addition to the detector’s inherent IOU loss, this function further constrains the relative positional relationships among the ship, the bow, and the stern, preventing the ship’s bounding box from being incomplete, which could affect the performance of subsequent recognition algorithms. The target is that the head and stern fall within the ship’s detection box. Once the center points of bow or stern fall outside the bounding box, the loss function should yield relatively large value. Based on this, a spatial relationship constraint function is formulated as Equation (4).

$$L_{\mathrm{relation}}=\parallel c_{\mathrm{bow}}-c_{\mathrm{ship}}{\parallel }_2+{\parallel c_{\mathrm{stern}}-c_{\mathrm{ship}}\parallel }_2-w$$

(4)

where ${\mathbf{c}}_{\mathrm{bow}}$, ${\mathbf{c}}_{\mathrm{stern}}$ and ${\mathbf{c}}_{\mathrm{ship}}$ are the center point of the prediction bounding box of bow, stern and ship. ${\parallel \ast \parallel }_2$ is the Euclidean distance and w is the width of the bounding box.

The overall detection loss combines classification loss, regression loss, and the proposed structure constraint loss,

$$L_{\mathrm{detection}}=L_1+\gamma L_{\mathrm{relation}}$$

(5)

in which $\gamma$ is set to 0.1. To prevent this loss function from interfering with the training process, the parameter $\gamma$ is set to 0 in the early stage of training, indicating that the relational constraint loss is disabled. When the regression accuracy of the detection boxes reaches a certain threshold, the relational constraint loss is enabled to further optimize the bounding boxes. In the training phase, by jointly optimizing the bow, stern, ship detection box, and their relative spatial relationships, more accurate target localization and boundary regression are achieved.

2.3. Key Region Based Ship Fine-Grained Recognition

2.3.1. Mask Based Key Regions Cropping

In the detection phase, the exact bounding boxes of ship, bow and stern are generated simultaneously. With them, an IOU-based grouping algorithm is proposed to determine the correspondences among the bow, stern, and ship, and to identify the orientation of the ship.

Given a set of detected bounding boxes for bows H, tails T, and ships S, an Intersection-over-Union (IoU)-based algorithm is employed to compute the overlapping area between bounding boxes of different categories. The algorithm traverses all bow and ship detections $S^{\ast }$ to identify the bow–ship pair with the maximum IoU.

$$T^{\ast }=arg\underset{T_l\in \mathcal{T}}{max}\mathrm{IoU}(T_l,S^{\ast })$$

(6)

$$(H^{\ast },S^{\ast })=arg\underset{\begin{array}{c}{H}_i\in \mathcal{H}\\ S_j\in \mathcal{S}\end{array}}{max}\mathrm{IoU}(H_i,S_j)$$

(7)

Subsequently, the algorithm searches for the stern detection that exhibits the maximum IoU with the same ship. This process determines a triple $(H^{\ast },S^{\ast },T^{\ast })$, representing the bow, ship, and stern, respectively, which collectively define the target’s position and spatial extent.

2.3.2. Key Regions Based Multi-Head Classifier

To further determine the class of ship, targets are cropped based on possible masks and bounding boxes to create distinctive key regions. Following identification logic of experts, a multi-head joint classifier (MHC) that takes these key regions as input are proposed in this manuscript. Compared to uniform cropping methods, such as VIT, the proposed approach is more concise, as the mask-based cropping acts as a gating mechanism that filters out non-critical information and serializes these key regions in order. Apart from that, when ships are cropped with mismatched mask, mask-based cropping acts an error amplifier, which further suppresses the wrong results.

The design of the classifier is illustrated in Figure 7. The key regions of each target are fed into a multi-head classifier, whose head consists of a ResNet50-based feature extractor. Then the features extracted by each head are concatenated into a representative feature vector and passed to an MLP to obtain the classification result. The features from 7 heads are flattened into a one-dimensional vector of 14,336 dimensions (7 × 2048) and fed into a three-layer multilayer perceptron (MLP) for dimensionality reduction and classification. The input layer accepts 14,336 dimensions, which are then reduced to 512 dimensions by the first hidden layer utilizing ReLU activation. Further reduction to 128 dimensions is achieved by the second hidden layer. Finally, 11 dimensions are produced by the output layer with Softmax activation for multi-class classification.

In the module, the parameters of the feature extractor are fixed, and only the parameters of the MLP are continuously adjusted during training, as shown in Figure 8. On the one hand, it contributes to reducing the complexity of optimization, and on the other hand, this process mimics the human’s cognitive process based on several key regions, in which the cognition of key regions and their compositional relationships is relatively independent. The construction of the feature extractor is illustrated in Section 2.3.3.

Given N classes of ships, the classifier is configured for $N+1$ classes. The key regions of the first N classes are a cropped-form ship with a mask consistent with it, while the key regions of the last class are a cropped-form ship with a mask inconsistent with it. The crops obtained with each mask are fed into a classifier, producing an $N\times (N+1)$ confidence matrix. The target category is determined by finding the maximum value along the diagonal of the $N\times N$ submatrix $\mathbf{C}$, as shown in Figure 7. The optimal category index $i^{\ast }$ is determined by

$$i^{\ast }=\underset{i\in \{1,2,\dots ,N\}}{argmax}{C}_{ii}$$

(8)

where $C_{ii}$ represents the diagonal elements of matrix $\mathbf{C}$. The corresponding ship class is

$$c_{\mathrm{final}}=c_{i^{\ast }}$$

(9)

When the mask perfectly aligns with the actual target boundaries and classes, the classifier will produce the highest confidence score for the correct category along the diagonal of the matrix. The diagonal elements $C_{ii}$ represent the confidence scores when using mask i to classify category i.

2.3.3. Build of Feature Extractor for Key Regions

To enhance feature extraction for key regions of ships, this paper develops a feature extractor generation method based on a consistency discrimination task, rather than through a classification task.

The underlying rationale is fourfold. The types of key regions could be very numerous, generally exceeding several dozen, and will increase as the number of target types increases. It is not intended that the feature extractor be retrained for each newly added category simply to adapt to new crops. Moreover, this scenario is characterized by a relatively pronounced class imbalance issue. Key regions such as the main gun, vertical launch system, and chimney are represented by a large number of samples, whereas certain specific types of structural regions constitute a very small proportion of the total. Furthermore, although distinctions between some categories are quite evident, the key components of warships from the same model but different batches often exhibit only extremely subtle variations. The feature extractor must ensure that sufficient separation is maintained between the features of these targets so that the combined features possess adequate differentiation. Finally, the discriminator tries to pull samples of the same category closer together in the feature space and push samples of different categories apart. This forces the network to learn the essential, discriminative features of the data, rather than just features that distinguish the training categories. It is not expected that this feature extractor directly map a crop to a specific category. Rather, it is only required that similar key region be mapped to a focused region within the feature space. On this basis, a stable input can be provided to the MLP layer.

With datasets, key regions with typical characteristics of each category are cropped into image slices. Given N kinds of ship, a total of S types of key regions are cropped. To accommodate key regions variations caused by inaccurate bounding boxes in practical applications, this paper employs downsampling, affine transformations, interpolation, and other methods to ensure diversity within each category of components.

Pairs of crops are grouped and fed into a Siamese neural network with ResNet50 as a backbone for training. Within each category, similar key regions are combined to form positive sample pairs, while key regions from different categories form negative sample pairs, as shown in Figure 9. Additionally, to address the issue of non-meaningful patches, numbers of patches are randomly cropped as a “non-sense” category with non-matching templates, which also participate in training. A contrastive loss is adopted in this work to constrain the training of the Siamese network.

$${\mathcal{L}}_{\mathrm{contrastive}}(\theta ;x_a,x_b,y)=(1-y)·{\displaystyle \frac{1}{2}}{D}^2+y·{\displaystyle \frac{1}{2}}{\left[max(0,m-D)\right]}^2.$$

(10)

where m is a threshold distance in the feature space that defines how far apart dissimilar sample pairs must be pushed. $x_a$, $x_b$ are pair of crops, with y indicating whether they belong to the same class. D is the normalized Euclidean distance between the feature vectors $f_{\theta }\left(x_a\right)$ and $f_{\theta }\left(x_b\right)$.

2.4. Block Diagram of the Proposed Method

As shown in Figure 10, for the construction of the KRM, key regions are manually labeled according to the principles in Section 2.1. KRMs for all 11 types of ships are then generated. For the training of the detector, ships, bows, and sterns in the (training/validation) images are annotated to form the detection dataset DCL-11. Using the KRMs, the ground-truth spatial attention maps are generated automatically. Subsequently, the detector (KRGY) is trained on this dataset. For the classifier training, the recognition dataset is constructed, which comprises key region groups cropped using the candidate class KRMs and key region ports. First, the feature extractor ResNet-50 is trained on this recognition dataset. Then, leveraging this pre-trained ResNet-50, the MHC is further trained using the key region groups cropped with the candidate class KRMs. During inference, an input image is processed by the KRGY detector (ship, stern, and bow detector) to generate bounding boxes for the ship, stern, and bow. With these bounding boxes and the KRMs, the corresponding key region slices are cropped and fed into the MHC to obtain the final recognition results.

3. Results

3.1. Datasets

3.1.1. HRSC2016

HRSC2016 [41] is an extensive collection of imagery data tailored for the task of identifying and categorizing ships in high-resolution satellite and aerial imagery. It comprises 1061 fine-detail images sourced from Google Earth, with pixel resolutions between 0.4 and 2 m. The collection covers 13 types of vessels, ranging from warships like aircraft carriers and destroyers to civilian ships such as container carriers and oil tankers. Each image is supplemented with annotations for both standard and rotated bounding boxes, supporting research in both conventional and oriented object detection.

3.1.2. FGSCR42

FGSCR42 [3] serves as a benchmark dataset for detailed, fine-grained categorization of ships within optical remote sensing scenes. It encompasses a diverse set of 42 specific ship classes, including multiple varieties of military, commercial, and specialized vessels. The dataset offers more than 10,000 accurately annotated ship instances, each provided with precise bounding boxes and class labels. Sourced from platforms like Google Earth and other commercial satellite providers, the imagery in FGSC42 features ground sample distances ranging from 0.5 to 2 m.

3.1.3. DCL-11

To valid the fine-grained recognition capabilities of proposed method, 11 classes of ships with high similarity from the FGSCR and HRSC2016 are selected to formulate a new dataset DCL-11, as shown in Figure 11. The dataset contains a total of 5246 image slices featuring 11 target classes. Among them, the Burke class is further classified into Burke II and Burke IIA, as well as its derivative models such as the Murasame class and Kongō class, all of which share highly similar ship structural designs. Additionally, the Whidbey Island class and San Antonio class have extremely close resemblances in both ship structure and exterior painting schemes, making them difficult for even experienced human observers to distinguish quickly and accurately without careful examination.

Unlike FGSCR and HRSC2016, 11 classes of target are first classified into four major categories, the Burke series, Spruance series, Littoral Combat Ship series, and Dock Landing Ship series, as shown in

Table 1. The composition of the dataset DCL-11.

Ship Type	Ship Class	Count	Total
Spruance Series	Ticonderoga	701	904
	Asagiri	203
Burke Series	Murasame	202	2079
	Arleigh Burke-II	550
	Arleigh Burke-IIA	510
	Kongo	275
	Akizuki	286
	Atago	256
Littoral Combat Ship Series	Independence	655	655
Dock Landing Ship Series	San Antonio	876	1608
	Whitby Island	732

. Specifically, the Burke series includes the the Atago class, the Kongō class, the Akizuki class, the Murasame class, the Burke I/II class, and the Burke IIA class. The Spruance series comprises the the Ticonderoga class and the Asagiri class. The Littoral Combat Ship series contains the the Freedom class. The Dock Landing Ship series encompasses the the Whidbey Island class and the San Antonio class.

3.2. Implementation Details

Based on the dataset, the proposed KR-Guided-Yolo11 (KRGY) was trained and evaluated. The training process utilized a VGPU with 32GB of memory. In terms of data augmentation, mosaic and mirror flipping techniques were disabled to preserve the authenticity of ship target data distribution, while affine transformations were employed to enhance model robustness under varying viewing angles. During the training and testing, the ratio of the train, validation, and test sets is 5:2:3. About the attention loss $L_{att}$, we selected final weighting setting, ${\alpha }_1=0.875$ and ${\alpha }_2=0.125$, through several sets of comparative experiments. The experimental results demonstrate that the model using this set of parameters achieves the better MAP performance, as shown in

Table 2. MAP of algorithms with different weighting setting about ${\alpha }_1$ and ${\alpha }_2$ with PTW and MHC-2.

	Case 1	Case 2	Case 3	Case 4	Case 5	Case 6	Case 7	Case 8
${\alpha }_1$	1.000	0.950	0.925	0.900	0.875	0.850	0.825	0.800
${\alpha }_2$	0.050	0.050	0.075	0.100	0.125	0.150	0.175	0.200
MAP	93.71	93.79	93.96	94.12	94.49	94.31	93.69	93.12

.

A group of SOTA methods including R3Det [18], RoI Transformer [19], ORCNN [16], YOLOv11-obb [7], MSCL-Net [26] and MGANet [27] are introduced and compared to demonstrate the effectiveness of the proposed method.

Further fine-grained recognition are implemented with the proposed multi-head classifier. In this stage, a dataset covering 3000 combinations of crops is constructed, with each group of crops containing 2–7 key regions. For targets with S ($S<7$) key components, the last key region is repeated $7-S$ times to maintain consistency. With the datasets, the key region feature extractor and multi-head classifier trained and tested. In the training phase, the learning rate was set to 0.001, with a decay step of 1000 iterations, reducing the learning rate to one-tenth of its previous value at each decay step. During testing, given a particular type including N classes, N masks are adopted to extract key regions from the target and fed into the multi-head classifier, respectively. In the training and testing, the ratio of the train and test sets is 7:3, in which the test set remains consistent with that of KRGY. As for the training for Resnet50, a total of 1700 crops for 20 kind of key regions and another 800 nonsense crops are prepared to build the ports of key regions. With 2500 crops, 69,455 pairs of positive samples and 34,694 pairs of negative samples are generated to train the Resnet50. In the experiment, 87 epoch of training are conducted to get the final module, in which the margin m is set to 0.5.

3.3. Experimental Results

On the test set, the visualizations of various algorithms and their quantitative results are presented in

Table 3. MAP of the proposed and SOTA detection methods.

Class	RoITrans	R3Det	O-RCNN	YOLOv11	MGANet	MSCL-Net	KRGY-MHC
C1	93.53	93.33	93.46	93.22	94.5	94.36	96.53
C2	93.71	94.29	93.17	93.19	94.88	94.21	96.19
C3	91.25	91.39	91.13	91.84	92.26	92.12	93.28
C4	93.21	93.18	93.32	93.47	94.24	94.18	96.21
C5	90.82	90.92	91.86	90.75	91.81	91.95	93.86
C6	89.76	91.16	89.98	91.22	92.30	92.16	91.08
C7	91.31	91.29	88.42	91.32	92.36	92.22	91.89
C8	93.38	93.45	93.23	93.09	94.31	94.42	96.32
C9	90.72	90.8	90.64	90.91	91.78	91.85	93.71
C10	93.78	93.92	93.25	93.37	94.09	94.21	96.08
C11	91.63	90.75	91.76	90.63	92.18	91.93	94.22

and Figure 12. Subjective results demonstrate that the proposed method outperforms the others in recognition accuracy.

Regarding recognition, for warships with similar appearance characteristics and shared design principles (such as the Arleigh Burke and Murakumo classes), the proposed method accurately distinguishes them, whereas conventional detection algorithms struggle to differentiate such vessels. Apart from that, the proposed method get high MAP in recognize two sub-class of Boke, Boke-II and Boke-IIA as shown in Table 3. The differences between them are limited to the helicopter deck and some minor details. At the observational scale of remote sensing images, many subtle distinctions are difficult to accurately capture. Through several rounds of hypothesis and verification, helicopter deck are focused and recognized, the recognition capability for these two subtypes has been significantly improved.

3.4. Computational Complexity Analysis

Regarding the trade-off of the calculation cost, we made a detailed analysis. The first stage of this method only adds an attention module to the backbone network, which has a negligible impact on overall computational complexity, and its timeliness is no different from that of YOLO. In this stage, detection of ship, bow and stern are simultaneously. In the second stage, the MHC, which consists of 7 input heads, is the main source of computational complexity, since it is needed to compute classification results of key regions groups with N kind of KRM. The overall computational complexity of the second stage is O(N), which is linear with respect to the number of possible KRM. Overall, the proposed method in this paper introduces additional computational overhead from the second-stage MHC compared to the single-stage YOLO algorithm, requiring an extra 76.8 ms to process each target when N is 6.

3.5. Ablation Study

This paper further evaluate the effect of the proposed KR-guided-attention module and part-to-whole structure based regulation in the first stage. In the second stage, Resnet50 based classifier, the multi-head classifier MHC-1 constructed with a feature extractor obtained from a classification task, and the multi-head classifier MHC-2 constructed with a feature extractor obtained from a consistency discrimination task are further compared with each other to validate the effectiveness of the proposed key-region based multi-head classifier.

3.5.1. Key-Regions Guided Attention

When supervised solely by detection loss, models may “take shortcuts” and learn attention patterns that optimize the loss but do not align with human. But in fact, models does not truly learn to recognize this target, instead, it merely found a class boundary. In the visualized feature map Figure 13a, it can be observed that areas with higher heat values are concentrated in the center of the target. Correspondingly, the target detection bounding boxes cover the central area but not completely. For ship C9, there are high heat values at the stern, and its detection bounding box fits the overall contour of the target well.

With the proposed method, as shown in Figure 13b, with the guided spatial attention module, the features transmitted to the top layer focus more on areas that humans use to identify targets, such as the helicopter deck, main gun, and vertical launching system. Correspondingly, the degree of bounding box alignment with the target is significantly better than the algorithm results obtained without the KR-guided-attention module, overall improving accuracy, as listed in

Table 4. MAP of six kind of algorithms.

	Case1	Case2	Case3	Case4	Case5	Case6
YOLO-OBB	✓	✓	✓	✓	✓	✓
KRGY		✓		✓	✓	✓
PTW			✓	✓	✓	✓
Resnet50				✓
MHC-1					✓
MHC-2		✓	✓			✓
MAP	92.28	93.65	93.71	93.82	94.28	94.49

✓ indicates Case-X is with that module. For example, Case-6 is the proposed method, with YOLO-OBB as basement, modified with KRGY, PTW and MHC-2.

.

KR-guided attention effectively enhances the localization and recognition accuracy, thereby providing higher-quality input for the recognition algorithm and ultimately improving overall performance.

As shown in Figure 14 and

Table 5. The silhouette score of three different inputs in Figure 14.

	Class0	Class1	Class2
Feature of whole ship	0.062	0.377	0.316
Feature of key regions	0.665	0.669	0.637
Feature of grouped key regions	0.480	0.723	0.436

, the silhouette scores of key regions and grouped key regions are significantly higher than that of the whole ship, which is consistent with the intuitive visualization results.

3.5.2. Part-to-Whole Relation Regulation

In the absence of constraints on the relationship between global and local parts, bounding boxes of ship often fail to correctly enclose the target. This is an typical issue for IOU-based boundary regression algorithms when dealing with slender targets, a phenomenon that can be clearly observed in Figure 13a. The YOLO11-obb algorithm produces incomplete bounding box for the aft section of the Boke-class.

The relational constraint loss force the detector to enclose the target as completely and accurately as possible. With the KR-guided-attention module in Figure 13b, the alignment between the ship’s bounding box and the target has been improved to some extent, but certain areas at the bow and stern are still uncovered, which will affect the extraction of key regions. While, incorporating the part-to-whole regulation, the bounding box of the target can more accurately conform to the overall contour of the target. As shown in the Figure 13c, the bow and helicopter deck of the Burke-class ship are completely enclosed by the detection bounding box.

3.5.3. MHC with Siamese Based Extractor

Feature maps of key regions from identical-type ship are extracted using ResNet50. With the T-SNE map in Figure 14a, it can be observed that the features from key regions are more concentrated, enabling a clearer classification boundary. This indicates that classifiers based on the entire target are susceptible to interference from non-critical regions, further leading to inaccurate classification results.

In contrast, the proposed method can directly locate the most key regions and achieve higher accuracy. As illustrated in Figure 14, the clear separability between the key regions of ships can be attributed to the distinct visual differences among these areas. Building on this, the features extracted by the MHC-based feature extractor still demonstrate a significant advantage over those from conventional classifiers, namely, a more distinct inter-class separation boundary Figure 14a,c. As shown in Table 4 and Figure 15, the MHC algorithm achieves markedly superior performance compared to the ResNet50-based classifier.

As shown in Figure 16 and

Table 6. The silhouette score of 8 kinds of classical helicopter decks with MHC-1 and MHC-2 in Figure 16.

	Deck1	Deck2	Deck3	Deck4	Deck5	Deck6	Deck7	Deck8
MHC-1	0.529	0.096	0.713	0.538	0.094	0.517	0.318	0.220
MHC-2	0.501	0.495	0.808	0.569	0.567	0.624	0.651	0.559

, the silhouette score of key regions feature extracted with MHC2 are significantly higher than that with MHC-1, which is consistent with the intuitive visualization results.

We further compared the feature mapping and classification capabilities of classifier-based feature extractors and consistency-discriminative feature extractors. Figure 16 presents the feature mapping results of eight typical helicopter decks. It can be observed that the classifier-based feature extractor is able to map distinctly different targets into relatively independent feature spaces. However, for relatively similar pairs such as deck2 and deck8, and deck5 and deck7, the features extracted by the classifier-based method (MHC-1) exhibit interweaving, indicating a certain probability of misclassification. In contrast, the consistency-discriminative feature extractor (MHC-2) yields more distinct and clear decision boundaries for the features extracted from deck2 vs. deck8 and deck5 vs. deck7. This suggests that the feature extractor obtained by this method possesses superior feature mapping capability, which consequently improves the classification accuracy of MHC, as shown in Figure 15.

3.6. Failure Cases

There is no denying that the method proposed in this paper highly relies on complete and accurate detection boxes, occlusion can adversely affect the overall performance of the algorithm as shown in Figure 17. Inaccurate and incomplete bounding boxes, when combined with masks, lead to deviations in the cropped regions, which do not align with the actual objects, thereby causing incorrect results generated by classifier in the second-stage. Apart from that, the occlusion of key components can also cause the loss of discriminative features, leading to incorrect classification results.

To prove the fault tolerance of the system, we have supplemented an experiment, selecting 200 test samples. The first scenario involves uniform scaling with scaling ratios of $-5\%,-10\%,+5\%,+10\%$. Based on the experimental results, when the scaling ratio is within $5\%$, the stability of the algorithm in the second stage remains largely unaffected, as shown in

Table 7. Impact on the recognition rate of a small amount of positioning deviation of bounding box extraction in the first stage.

Case	Case1	Case2	Case3	Case4	Case5
Positioning Deviation	$-10\%$	$-5\%$	$0\%$	$+5\%$	$+10\%$
Recognition Rate	$80.5\%$	$97.0\%$	$98.5\%$	$98.0\%$	$82.5\%$

. During the mask configuration process, allowances have been incorporated to account for factors such as viewpoint variations. The bounding boxes within the mask include certain margins, originally designed to accommodate changes in perspective. Experiments have revealed that these margins also help mitigate the effects of inaccurate bounding box extraction in the first stage.

4. Discussion

The performance gains of our method originate mainly from three aspects. The two-stage framework proposed in this paper, which simulates human cognition, enhances recognition granularity and accuracy through the following aspects. First, it utilizes key region masks to guide the model in strengthening feature extraction from critical areas, focusing more on the intrinsic characteristics of the target, thereby improving detection and localization precision. Second, through a two-stage progressive cognitive approach, it employs various masks to crop the target’s regions of interest, simulating the “hypothesis” process in human cognition. Based on the hypothesis results, the framework focuses on the target’s representative/key regions and performs comprehensive recognition to achieve category “verification.” In this work, the key region mask-supervised attention module and the local–global relationship constraint adopted in the first stage effectively improve the accuracy of ship target bounding boxes, indirectly supporting fine-grained target recognition. The “hypothesis-verification” concept in the second stage is key to achieving highly reliable fine-grained target recognition.

As depicted in [42], “Glance” and “Sequentail Decision Making” show great potential in finding where to focus and where more key regions are logically. In future work, we will further introduce large language models [43,44,45] to establish a comprehensive cognitive system, which will be combined with the perceptual capabilities of classifiers for components to achieve explainable ship target cognition.

5. Conclusions

In this manuscript, a multi-stage fine-grained target recognition method that integrates expert knowledge is proposed. A supervised attention learning branch is incorporated to force the neural network to focus on representative regions of these targets. Masks are used to obtain slices of representative target regions, which are sequentially fed into a multi-head classifier to generate class identification results. By leveraging the key regions humans focus on, it not only enhances the neural network’s capabilities but also provides screening conditions for subsequent fine classification, further amplifying the advantages of prior steps and serving as a verification mechanism, improving the performance of fine-grained ship recognition algorithm.