A Rotation Target Detection Network Based on Multi-Kernel Interaction and Hierarchical Expansion

Abstract

Remote sensing targets typically exhibit characteristics of gradual scale changes and diverse orientations. Most existing remote sensing detectors adapt to these differences by adding multi-level structures for feature fusion. However, this approach leads to incomplete coverage of the overall target by the extracted local features, resulting in the loss of critical directional information and an increase in computational complexity which affect the detector’s performance. To address this issue, this paper proposes a Rotation Target Detection Network based on Multi-kernel Interaction and Hierarchical Expansion (MIHE-Net) as a systematic solution. Specifically, we first refine scale modeling through the Multi-kernel Context Interaction (MCI) module and Hierarchical Expansion Attention (HEA) mechanism, achieving sufficient extraction of local features and global information for targets of different scales. Additionally, the Midpoint Offset Loss Function is employed to mitigate the impact of gradual scale changes on target direction perception, enabling precise regression for targets across various scales. We conducted comparative experiments on three commonly used remote sensing target datasets (DOTA, HRSC2016, and UCAS-AOD), with mean average precision (mAP) as the core evaluation metric. The mAP values of the method in this paper on the three datasets reached 81.72%, 92.43%, and 91.86% respectively, which were 0.65%, 1.93%, and 1.87% higher than those of the optimal method, significantly outperforming existing one-stage and two-stage detectors. Through multi-scale feature interaction and direction-aware optimization, MIHE-Net effectively addresses the challenges posed by scale gradation and direction diversity in remote sensing target detection, providing an efficient and feasible solution for high-precision remote sensing target detection.

1. Introduction

Remote sensing target detection is a technology that automatically identifies and locates specific ground objects by analyzing remote sensing images. With the improvement of remote sensing detection technology and imaging technology, its application scenarios have been continuously expanded, playing important roles in urban planning, agricultural monitoring, military reconnaissance and other fields. Significant progress has been made in this field with the integration of deep learning and high-resolution remote sensing data, attracting increasing attention [1,2,3,4,5,6]. While most existing methods follow the paradigm of feature extraction followed by bounding box regression/target classification, differences in the backbone networks used in the feature extraction stage lead to various differences among these methods. The feature sharing mechanism is illustrated in Figure 1.

In terms of feature extraction, approaches rooted in convolutional neural networks (CNNs) [7,8,9,10,11,12,13] focus on multiscale feature enhancement and contextual information extraction. For example, DetectoRS [14] combines the recursive feature interaction mechanism with a multi-scale atrous convolution fusion strategy. By constructing bidirectional feedback paths, it enables high-level semantic information to be reversely injected into low-level feature maps, thereby strengthening the feature representation capability for small targets. AODet [15] distinguishes foreground and background regions from multi-level feature maps through a foreground proposal network, operating only in foreground regions to reduce redundant computations and enhance the detection capability for targets in aerial images. Emerging transformer-based methods leverage the advantages of transformers in long-range dependency modeling to optimize target feature representation. For example, Swin Transformer [16] achieves cross-window connections through a sliding window mechanism, efficiently processing large-scale remote sensing images and enhancing the network’s ability to fuse global and local information. ViTDet [17] optimizes the feature extraction capability for densely arranged small targets by introducing a pyramid vision transformer architecture and combining it with a dynamic sparse attention mechanism, which significantly reduces computational complexity while maintaining global modeling capabilities.

In terms of bounding box representation, owing to the intrinsic characteristics of remote sensing targets such as multi-directional rotation and geometric elongation, directly predicting the target angle can cause the problem of discontinuous boundary in angle representation of the network loss function [18,19,20,21,22]. Therefore, GWD [23] transforms the angle prediction problem into a regression problem. Using the Gaussian weighted distance, it designs a continuously differentiable loss metric, enabling the loss function to smoothly transition at the boundary of angle representation. CSL [24] models the periodicity of angles in classification tasks by designing circular smooth labels, enabling the network to learn the cyclic characteristics of angles. Meanwhile, it uses the classification results to guide the regression task and fine-tune the angles through cosine distance metrics, which improves the accuracy and stability of bounding box representation for rotated targets. S²A-Net [25] proposes a scale-adaptive rotated bounding box representation method which decomposes the rotated bounding box into scale factors and angle parameters. By decoupling angle and scale information, it uses a multi-branch structure for separate regression. This reduces the ambiguity of angle representation, effectively solving the discontinuous boundary problem and achieving accurate positioning of remote sensing rotated targets.

In summary, although existing methods have optimized multi-scale feature fusion capabilities to a certain extent and alleviated the problem of inaccurate angle prediction in rotating target detection, they overlook the impact of gradual changes in target scale during the feature extraction process of the detector on the diversity of target orientations. In detail:

(1)

Progressive Scale Variation: In the same scene, targets of different sizes (such as various ships of different sizes) exhibit continuous progressive changes; this progressive variation in scale poses significant challenges to traditional target detection algorithms, as detectors with fixed-scale designs often struggle to accurately detect targets of all sizes simultaneously. To address the issues caused by scale variation, existing methods commonly use multi-level structures such as feature pyramids for feature fusion. Although increasing the number of stacked layers can cover a wider scale range to some extent, this significantly increases computational complexity; conversely, reducing the number of layers compromises detection accuracy. Consequently, existing methodologies face substantial challenges in striking an optimal balance between detection accuracy and computational efficiency. Against this backdrop, LSKNet [26] proposes a method aimed at selectively expanding the spatial receptive field for large-scale targets to capture more scene contextual information. This is achieved by introducing large-kernel convolutions and dilated convolutions into the backbone network. However, large-kernel convolutions may introduce substantial background noise, which negatively impacts the precise detection of small targets. On the other hand, while dilated convolutions have advantages in expanding the receptive field, they may overlook fine-grained information within the receptive field, which can lead to overly sparse feature representations and affect detection accuracy.

(2)

Coupling of Scale and Orientation: In remote sensing target detection tasks, existing methods typically model scale variation or orientation variation independently, while overlooking the deep coupling relationship between the two. When the target size is large, the features extracted within the target often fail to fully cover the entire target due to the limited local receptive field of convolutional operations, resulting in the absence of critical directional information in local features. This phenomenon makes it difficult to accurately capture the target’s orientation and shape structure by relying solely on local scale features, thereby affecting the accuracy of target detection. Therefore, the increase in target scale not only exacerbates the locality problem of feature extraction but also indirectly weakens the orientation perception ability. To effectively address this challenge, it is necessary to first perform fine-grained scale modeling during feature extraction in order to ensure that local features have sufficient global perception capabilities. Subsequently, by integrating an orientation modeling mechanism, the method can fully exploit the expressive ability of target orientation features under the background of scale variation, thereby achieving collaborative perception and enhancement of scale and orientation and improving the adaptability and robustness of detectors in complex remote sensing scenarios.

To address these issues, we propose MIHE-Net. In the feature extraction stage, we introduce the MCI module and HEA module, which utilize a set of parallel deep convolutions and depth-wise separable strip convolutions to simultaneously extract local and global features of remote sensing targets. The obtained feature maps are then fed into subsequent parameter regression branches. In the parameter regression stage, we propose a midpoint offset method to represent oriented targets and design a new loss function for bounding box regression, helping to address the problem of feature coupling between target scale and orientation in existing object detection algorithms. Our proposed MIHE-Net can be regarded as an extension of existing multi-oriented target detectors that demonstrates remarkable generality and enables compatibility with various detectors.

This paper makes the following main contributions:

(1)

This paper systematically analyzes and reveals the deep coupling relationship between target scale variation and direction perception ability in remote sensing images for the first time. When the target scale is large, local feature extraction is limited by the receptive field range, making it difficult for local features to effectively express the overall direction information of the target, which affects the accuracy of target positioning and recognition. To address this phenomenon, in this paper we begin with scale modeling, then integrate direction modeling strategies to achieve collaborative perception and enhancement of scale and direction, thereby providing a new idea that can improve comprehensive scale-direction modeling capability in remote sensing target detection.

(2)

A multi-kernel contextual interaction module is constructed to effectively address the problem of gradual target scale changes. Based on existing multi-scale feature extraction methods, this paper adopts a parallel arrangement structure of non-dilated deep convolution kernels with different sizes to achieve efficient extraction of multi-scale features and modeling of local contextual information. This design not only effectively alleviates the feature sparsity problem caused by dilated convolutions but also improves the density and integrity of feature representation, thereby enhancing the detector’s modeling capability for targets of different scales in complex remote sensing scenes.

(3)

Through a collaborative optimization framework of the multi-kernel contextual interaction module and hierarchical expansion attention mechanism, the proposed method can establish an effective complementary relationship between feature extraction at different scales and global contextual modeling, thereby significantly improving the performance of remote sensing target detection.

(4)

Finally, an anchor box convergence method is proposed; by generating boundary center offset amounts for external oriented rectangular anchor boxes using pre-extracted fine-scale feature information, this offset regression method compensates for the problem of drastic changes in loss functions caused by angle regression while making the anchor boxes converge compactly and accurately.

2. Related Works

2.1. Rotation Feature Extraction

Yuan et al. [27] addressed the issue of traditional convolutional neural network (CNN)-based detectors mainly extracting input feature maps within square windows, which can include a large amount of irrelevant information from surrounding areas when processing elongated objects. They proposed Strip R-CNN, which effectively combines the advantages of square convolutions and strip convolutions while encountering less feature redundancy. To tackle the feature misalignment challenge in the refinement phase, R³Det [28] employs feature interpolation to re-encode the positional information of the bounding box under refinement into corresponding feature points. Subsequently, it reconstructs the entire feature map at the pixel level, thereby achieving feature reconstruction and alignment. To guide the neural network model to pay more attention to effective features and suppress background noise, Li et al. [29] designed Non-local Awareness Pyramid Attention (NP-Attention). By learning both the spatial multi-scale nonlocal dependency and channel dependency, It mitigates the discrepancy between the network’s attention focus and the real-world objects in images.

Han et al. [30] proposed RoI Transformer, which transforms horizontal regions of interest (HRoIs) into rotated regions of interest (RRoIs). Subsequently, a Rotation–Position-Sensitive RoI Align (RPS-RoI-Align) module is incorporated to extract rotation-invariant regional features. Aiming at the problem of large target scale variations in remote sensing images leading to insufficient and unbalanced sampling in feature maps of different levels, Guan et al. [31] proposed an adaptive scale sampling (ADS) strategy. This strategy establishes a mapping relationship between the size of target scales and the levels of feature maps, allocating more samples from high-level feature maps to large-scale targets and the opposite for small-scale targets. This strategy achieves sufficient sampling and a more balanced distribution of samples across different scales.

2.2. Bounding Box Representation

Huang et al. [32] proposed the General Gaussian Heatmap Label Assignment method (GGHL), which employs an anchor-free target-adaptive label assignment (OLA) strategy based on 2D Gaussian heatmaps. This strategy assigns each label to Gaussian candidate positions in the feature map via a one-to-many mapping, enabling the assigned candidate objects to better align with the shape and orientation of real-world objects. R²PN [33] generates proposal regions in arbitrary directions by rotating anchor boxes to more orientations, and employs a rotated RoI pooling layer (R2oI) as the max pooling operation on arbitrary-direction regions. This ensures that all detection steps can be integrated into a unified network and enables end-to-end training. Xu et al. [34] constructed a semantic segmentation-guided RPN (sRPN) to suppress background clutter as much as possible. By introducing a semantic segmentation module, their approach obtains bounding box-based masks and semantic features; the masks guide the generation of horizontal candidate boxes, while the semantic features are fused during the ROI pooling process to enable a more accurate estimation of the bounding box. Li et al. [35] introduced a superellipse-modulated 2D Gaussian function, termed the super-Gaussian distribution, which preserves angular information by maintaining anisotropy across all aspect ratios.

Although existing rotated feature extraction methods have made some progress, they focus more on redundancy reduction, lack explicit modeling of multi-scale contextual dependencies, treat scale and orientation as independent variables, and ignore the coupling relationship between them. Thus, they fail to completely solve the key issues of gradual scale changes, scale-orientation coupling, and stable rotation regression. Based on these outstanding issues, our proposed MIHE-Net focuses more on the collaborative perception of scale and orientation, enabling progressive scale modeling combined with orientation without modifying the existing detection framework.

3. Methodology

The architecture of our proposed MIHE-Net is illustrated in Figure 2. The backbone network consists of four stages, each of which is composed of a Multi-kernel Context Interaction (MCI) module and a Hierarchical Expansion Attention (HEA) module. Subsequently, a comprehensive in-depth elaboration is provided for the proposed MCI module, HEA module, overall architecture, and Midpoint Offset Loss Function.

3.1. MCI Module

In remote sensing images, objects are generally distributed across various scales. Precise object recognition hinges not only on visual appearance cues but also on their contextual information. Contemporary approaches enhance the receptive field through the integration of large-kernel convolutions and dilated convolutions within the backbone architecture. However, the adoption of large-kernel convolutions inevitably introduces extraneous background noise, while dilated convolutions inherently overlook fine-grained spatial details within the expanded receptive field. In order to capture multi-scale texture features when objects present various different scale situations, we introduce the MCI module. First, local features are extracted through a small kernel convolution. Following this, a parallel configuration of depthwise convolution layers is utilized to extract multi-scale contextual information. Ultimately, the locally extracted feature maps and multi-scale contextual representations are fused along the channel dimension, facilitating the aggregation of spatio-contextual semantics. In detail:

Local Feature Extraction: First, a 3 × 3 convolution is applied to the input feature map to extract fine-grained local features, laying the foundation for subsequent context modeling.

Multi-Scale Context Extraction: Second, a cascade of convolutional layers with progressively increasing kernel sizes (5 × 5, 7 × 7, 9 × 9, 11 × 11) extracts hierarchical contextual representations. Each convolutional layer operates independently, ensuring that it can both cover the details of small targets and capture the overall context of large targets. The formulas are as follows:

$$L_{l-1,n}={\mathrm{Conv}}_{k_s\times k_s}\left(X_{l-1,n}^{\left(2\right)}\right),n=0,\dots ,N_l-1$$

(1)

$$Z_{l-1,n}^{\left(m\right)}={\mathrm{DWConv}}_{k^{\left(m\right)}\times k^{\left(m\right)}}\left({\mathit{L}}_{l-1,n}\right),m=1,\dots ,4$$

(2)

where $L_{l-1,n}$ represents the local features extracted by a $k_s\times k_s$ convolution, here set to $k_s=3$, and $Z_{l-1,n}^{\left(m\right)}$ denotes the contextual features extracted by the m-th $k^{\left(m\right)}\times k^{\left(m\right)}$ depthwise convolution (DWConv), here set to $k^{\left(m\right)}=2\times (m+1)+1$.

Feature Fusion: Lastly, a 1 × 1 convolution is used as a channel fusion mechanism to integrate features with different receptive field sizes, which produces the fused feature $P_{l-1,n}$. Through this methodology, our MCI module effectively captures comprehensive contextual dependencies while preserving the fidelity of local texture details. The formula is shown below.

$$P_{l-1,n}={\mathrm{Conv}}_{1\times 1}(L_{l-1,n}+\sum _{m=1}^4{Z}_{l-1,n}^{\left(m\right)})$$

(3)

3.2. HEA Module

The MCI module is primarily used to extract multi-scale local contextual information. To better incorporate contextual information over large distances and enable global context extraction, we introduce the HEA module. The process is as follows: first, global average pooling is applied to obtain local regional features; then, two depthwise separable strip convolutions are used. Meanwhile, the size of the depthwise separable strip convolution kernel is set according to the depth of the MCI module group, ensuring that the receptive field increases as the MCI module group to which the HEA module belongs becomes deeper. This strengthens the MCI module’s ability to capture interdependencies between distant pixels while enhancing feature extraction of the core parts of target objects. In detail:

Regional Feature Compression: Global average pooling is performed on the input feature map to compress the spatial dimension into a regional feature vector, thereby capturing the overall contextual trend. we obtain the local regional feature $F_{l-1,n}^{\mathrm{pool}}$ through a 1 × 1 convolution after average pooling:

$${\mathit{F}}_{l-1,n}^{\mathrm{pool}}={\mathrm{Conv}}_{1\times 1}\left({\mathcal{P}}_{avg}\left({\mathit{X}}_{l-1,n}^{\left(2\right)}\right)\right),n=0,\dots ,N_l-1$$

(4)

where $P_{avg}$ denotes the average pooling operation.

Long-Distance Dependency Modeling: We leverage a pair of depthwise separable strip convolutions to approximate the computational effect of a standard large-kernel depthwise separable convolution:

$$\begin{array}{c}{\mathit{F}}_{l-1,n}^{\mathrm{w}}={\mathrm{DWConv}}_{1\times k_b}\left({\mathit{F}}_{l-1,n}^{\mathrm{pool}}\right)\\ {\mathit{F}}_{l-1,n}^{\mathrm{h}}={\mathrm{DWConv}}_{k_b\times 1}\left({\mathit{F}}_{l-1,n}^{\mathrm{w}}\right)\end{array}$$

(5)

where $k_b$ is the size of the depthwise separable strip convolution kernel. To expand the receptive field of the HEA module as the MCI module to which the HEA module belongs grows deeper, we set $k_b=(11+2n)$, calculating the kernel size $k_b$ as a function of the MCI module depth n.

Attention Weight Generation: Lastly, our HEA module generates an attention weight $A_{l-1,n}$, where the sigmoid function ensures that $A_{l-1,n}$ falls within the range of (0,1). This weight is further used to enhance the output of the MCI module; the formula is provided below.

$$A_{l-1,n}=\mathrm{Sigmoid}\left({\mathrm{Conv}}_{1\times 1}\left({\mathit{F}}_{l-1,n}^{\mathrm{h}}\right)\right),$$

(6)

3.3. Overall Structure

The backbone network of MIHE-Net is structured into four sequential stages. The input and output of stage l are $F_{l-1}$ and $F_l$, respectively. The input $F_{l-1}$ undergoes downsampling and is then split into two parts, $X_{l-1}^{\left(1\right)}$ and $X_{l-1}^{\left(2\right)}$, along the channel dimension, which are fed into two separate paths:

$$\begin{array}{c}{\mathit{X}}_{l-1}={\mathrm{Conv}}_{3\times 3}\left(\mathrm{DS}\left({\mathit{F}}_{l-1}\right)\right)\\ {\mathbf{X}}_{l-1}^{\left(1\right)}={\mathit{X}}_{l-1}[:{\displaystyle \frac{1}{2}}{C}_l,\dots ],{\mathit{X}}_{l-1}^{\left(2\right)}={\mathit{X}}_{l-1}[{\displaystyle \frac{1}{2}}{C}_l:,\dots ]\end{array}$$

(7)

where $DS$ denotes the downsampling operation and $C_l$ denotes the number of channels. One path is a feed-forward network (FFN) that takes $X_{l-1}^{\left(1\right)}$ as input, passes it through two 3 × 3 convolutions, and outputs $X_l^{\left(1\right)}$. The other path consists of the MCI module and the HEA module, taking $X_{l-1}^{\left(2\right)}$ as input and producing $X_l^{\left(2\right)}$. Specifically, $X_{l-1,n}^{\left(2\right)}$ is processed by the MCI module to obtain the output feature $P_{l-1,n}$, while the HEA module generates the attention weight $A_{l-1,n}$. This attention weight $A_{l-1,n}$ is then used to enhance the feature resulting from the MCI module, yielding the enhanced feature $F_{l-1,n}^{\mathrm{attn}}$. The formula is as follows:

$${\mathit{F}}_{l-1,n}^{\mathrm{attn}}=({\mathit{A}}_{l-1,n}\odot {\mathit{P}}_{l-1,n})\oplus {\mathit{P}}_{l-1,n}.$$

(8)

The enhanced feature $F_{l-1,n}^{\mathrm{attn}}$ is then passed through a 1 × 1 convolution to produce $X_l^{\left(2\right)}$:

$${\mathit{X}}_{l,n}^{\left(2\right)}={\mathrm{Conv}}_{1\times 1}\left({\mathit{F}}_{l-1,n}^{\mathrm{attn}}\right).$$

(9)

The final output of the entire process is then

$${\mathit{F}}_l={\mathrm{Conv}}_{1\times 1}\left(\mathrm{Concat}({\mathit{X}}_l^{\left(1\right)},{\mathit{X}}_l^{\left(2\right)})\right),$$

(10)

where $Concat$ denotes the concatenation operation.

3.4. Midpoint Offset Loss Function

For rotated object detection, the rotation angles of bounding boxes are typically divided into several discrete intervals; however, this discretization introduces quantization errors. In addition, regression of the rotation angles is necessary in order to handle periodic problems, which increases the difficulty of training. If a complete parameter regression method is adopted, then mismatch between the physical characteristics of angle regression and the optimization mechanism of neural networks occurs, leading to unstable training and high computational overhead. Some existing anchor-free methods directly predict the center points and sizes of objects without presetting anchor boxes; however, these are prone to confusion about the center points of objects, and their detection effects are limited for multi-scale objects.

In order to design a bounding box representation scheme that is easy to train and can achieve accurate regression for objects of different scales, we first preset horizontal reference boxes of different scales through anchor boxes. This approach can inherit the multi-scale priors of horizontal anchor boxes, assist the network in learning the common direction patterns of objects, and reduce the ambiguity of anchor-free methods. Then, by introducing the offset of the midpoints at the top and right sides, direction information is added to the horizontal boxes to improve positioning accuracy. This method only needs to add two additional offset parameters to describe oriented objects on the basis of traditional bounding boxes, without the need to introduce complex angle parameters. Finally, a loss function between the offsets is designed to learn the parameters. Specifically:

Anchor Box Initialization: Across all hierarchical feature maps, we instantiate three horizontally-oriented anchor boxes per spatial position, configured with aspect ratios of 1:2, 1:1, 2:1. Each anchor box a is represented by a four-dimensional vector $a=(a_x,a_y,a_w,a_h)$, where $(a_x,a_y)$ is the central coordinate of the anchor box and where $a_w$ and $a_h$ denote the width and height of the anchor box, respectively.

Offset Prediction: One of the two sibling 1 × 1 convolutional layers is the regression branch, which outputs the offset $\delta =({\delta }_x,{\delta }_y,{\delta }_w,{\delta }_h,{\delta }_{\alpha },{\delta }_{\beta })$ of the candidate box relative to the anchor box. We can obtain the oriented candidate box by decoding the regression output. The decoding process is described as follows:

$$\begin{array}{cc}& \Delta \alpha ={\delta }_{\alpha }·w,\Delta \beta ={\delta }_{\beta }·h,\hfill \\ & w=a_w·e^{{\delta }_w},h=a_h·e^{{\delta }_h},\hfill \\ & x={\delta }_x·a_w+a_x,y={\delta }_y·a_h+a_y,\hfill \end{array}$$

(11)

among which $(x,y)$ represents the central coordinates of the predicted candidate box and $w,h$ represents the width and height of the bounding rectangle of the predicted oriented candidate box. As shown in Figure 3, $\Delta \alpha$ and $\Delta \beta$ are the offsets relative to the midpoints of the upper and right sides of the bounding rectangle, respectively.

Decoding and Loss Calculation: We generate the oriented candidate box O based on $(x,y,w,h,\Delta \alpha ,\Delta \beta )$. Then, the coordinate sets of the four vertices of each candidate box are obtained as $V=(A,B,C,D)$. The coordinates of the four vertices can be expressed as follows:

$$\begin{array}{c}A=(x+\Delta \alpha ,y-h/2),\\ B=(x+w/2,y+\Delta \beta ),\\ C=(x-\Delta \alpha ,y+h/2),\\ D=(x-w/2,y-\Delta \beta ).\end{array}$$

(12)

Leveraging this encoding scheme, we parameterize each oriented bounding box through joint prediction of its enclosing rectangle parameters $(x,y,w,h)$ and midpoint offset deltas $(\Delta \alpha ,\Delta \beta )$ to enable precise regression.

Positive and negative sample definitions are specified as follows. First, a binary label $p^{*}\in \{0,1\}$ is assigned to each anchor box, where 0 and 1 denote negative and positive samples, respectively. An anchor box is classified as a positive sample if either of the following conditions is met: the intersection over union (IoU) between the anchor box and any ground-truth box exceeds 0.7; or the anchor box achieves the highest IoU with a ground-truth box among all anchor boxes and this IoU value is greater than 0.3. Anchor boxes with IoU values lower than 0.3 relative to all ground-truth boxes are labeled as negative samples. Samples that do not fall into either category are treated as invalid and excluded from the training process. The subsequent definition of the $L_1$ loss function is as follows:

$$\begin{array}{cc}\hfill L_1& ={\displaystyle \frac{1}{N}}\sum _{i=1}^N{F}_{cls}\left(p_i,p_i^{*}\right)+{\displaystyle \frac{1}{N}}{p}_i^{*}\sum _{i=1}^N{F}_{reg}\left({\delta }_i,{\mathit{t}}_i^{*}\right)\hfill \end{array}$$

(13)

where i denotes the index of the anchor box, N (defaulting to $N=256$) represents the total number of samples within a mini-batch. $p_i^{*}$ signifies the ground-truth label for the i-th anchor box, $p_i$ is the output from the classification branch, quantifying the probability of the candidate box belonging to the foreground, and $t_i^{*}$ represents the supervised offset of the ground-truth box relative to the i-th anchor box, which is a parameterized six-dimensional vector $t_i^{*}=[t_x^{*},t_y^{*},t_w^{*},t_h^{*},t_{\alpha }^{*},t_{\beta }^{*}]$ characterizing the displacement of the predicted candidate box from the i-th anchor box:

$$\begin{array}{cc}{\delta }_{\alpha }=\Delta \alpha /w\hfill & {\delta }_{\beta }=\Delta \beta /h\hfill \\ {\delta }_w=log\left(w/w_a\right)\hfill & {\delta }_h=log\left(h/h_a\right)\hfill \\ {\delta }_x=\left(x-x_a\right)/w_a\hfill & {\delta }_y=\left(y-y_a\right)/h_a\hfill \\ t_{\alpha }^{*}=\Delta {\alpha }_g/w_g\hfill & t_{\beta }^{*}=\Delta {\beta }_g/h_g\hfill \\ t_w^{*}=log\left(w_g/w_a\right)\hfill & t_h^{*}=log\left(h_g/h_a\right)\hfill \\ t_x^{*}=\left(x_g-x_a\right)/w_a\hfill & t_y^{*}=\left(x_g-x_a\right)/h_a\hfill \end{array}$$

(14)

where $(x_g,y_g)$, $w_g$, and $h_g$ respectively define the bounding rectangle’s central coordinates, width, and height, while $\Delta {\alpha }_g$ and $\Delta {\beta }_g$ respectively quantify the top/right vertex offsets relative to the top-edge midpoint and left-edge midpoint.

4. Experiments

4.1. Datasets

The DOTA-v1.0 dataset [36] is a large-scale aerial image object detection dataset comprising 2806 images across fifteen categories: Basketball Court (BC), Roundabout (RA), Harbor (HA), Swimming Pool (SP), Helicopter (HC), Plane (PL), Baseball Diamond (BD), Bridge (BR), Ground Track Field (GTF), Soccer Field (SBF), Large Vehicle (LV), Ship (SH), Tennis Court (TC), Small Vehicle (SV), and Storage Tank (ST). Collected from diverse sensors and platforms, this dataset features images with resolutions ranging from 800 × 800 to 20,000 × 20,000 pixels, presenting objects with significant variations in scale, orientation, and shape.

The HRSC2016 dataset [37] was released by Northwestern Polytechnical University in 2016, focusing primarily on ship detection. This dataset contains 1061 images. The objects within it are categorized into three major categories and 27 subcategories, encompassing 2976 targets in total. This makes it of great research value for studying ship detection, especially detailed classification of detection tasks.

The UCAS-AOD dataset [38] is a dataset for aircraft and vehicle detection constructed from Google Earth imagery. This dataset contains 2420 images and 14,596 instances, covering two types of samples (aircraft and vehicles) along with some negative samples (background).

For these three datasets, during the preprocessing stage we normalized the image pixel values to the range [0, 1], and removed extreme noise pixels using the threshold of mean pixel count ± 3 times the standard deviation. In addition, during the training phase we adopted data augmentation strategies to enhance the generalization ability of the model, specifically including random horizontal/vertical flipping, which means randomly performing horizontal flipping or vertical flipping on training images. In order to truly reflect the model’s performance on the original data, the validation and test sets were not augmented. The dataset division methods were tailored to the characteristics of each dataset: DOTA-v1.0 was randomly divided into training, validation, and test sets in an 8:1:1 ratio; for HRSC2016, we adopted the official fixed division (436 images in the training set, 181 in the validation set, and 182 in the test set, focusing on ship detection); finally, UCAS-AOD was randomly divided in a 7:1:2 ratio.

4.2. Experimental Evaluation Metrics

In the context of remote sensing object detection, each image is likely to include objects belonging to various categories at distinct spatial locations, requiring simultaneous evaluation of a model’s classification and localization performance. Because the classification criteria in traditional image processing cannot be directly applied to object detection tasks, the mean average precision (mAP) was used to measure the quality of detectors. Specifically:

$$\mathrm{P}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}},\mathrm{R}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}},$$

(15)

where $TP$ stands for true positive, $FP$ stands for false positive, and $FN$ stands for false negative. An excellent detector is expected to achieve very good performance in both recall and precision; therefore, it is important to comprehensively consider the two factors P (precision) and R (recall). A better evaluation metric can be obtained by constructing the P-R curve for each category and calculating the average value of the area under the curve. For a dataset with $N_c$ categories, the definition of mAP is as follows:

$$mAP={\displaystyle \frac{1}{N_C}}\sum _{i=1}^{N_c}{\int }_0^1{P}_i\left(R_i\right)d_i.$$

(16)

To comprehensively evaluate the detector’s performance, we introduce a multi-dimensional evaluation metric system in which computational complexity is quantified by floating-point operations (FLOPs), while the storage requirements and structural scale of the model are measured using the number of model parameters (Params).

4.3. Parameter Setting

We adopted ResNet as the backbone of MIHE-Net, with the original ResNet fully pretrained on ImageNet. The positive sample matching threshold was set to 0.5 and the detection head confidence was configured as 0.6. The proposed MIHE-Net was optimized using the Adam optimizer with a momentum of 0.9 and a weight decay coefficient of 0.0001. All models underwent 200 epochs of training, starting with an initial learning rate of 0.01, which was decayed to 0.001 after 80 epochs. A batch size of 32 was employed throughout the training process. To ensure the reproducibility of the experiments, we fixed the random seed to 42. Experiments were conducted on a server based on the PyTorch framework with four RTX 4090D GPUs. The Python version was 3.8.13 while the PyTorch version was 1.10.1, along with torchvision 0.11.2 compatible with PyTorch (for data loading and transformation), CUDA 11.3 supporting GPU-accelerated training, numpy 1.21.5 for array operations, OpenCV-python 4.5.5 for image processing, and scikit-learn 0.24.2 for calculating the evaluation metrics. RetinaNet, a simple yet effective one-stage detector, serves as the baseline; it is worth noting that introducing any module may increase computational complexity.

To ensure the fairness and comparability of the experimental results, all comparative models were trained and tested in exactly the same hardware and software environments. Specifically, all models adopted the same dataset splitting strategy as mentioned above, with consistent training and test sets to avoid biases caused by different data divisions. In addition, all models use the same training hyperparameters, including the number of training epochs, initial learning rate and its decay strategy, batch size, optimizer parameters (momentum and weight decay), and fixed random seed. This ensures that the observed performance differences are solely due to the inherent characteristics of the model architectures rather than to changes in experimental settings.

4.4. Ablation Experiment

We conducted relevant ablation experiments on the DOTA-v1.0, HRSC2016, and UCAS-AOD datasets to validate the performance of the proposed MIHE-Net.

First, we conducted research on the multi-scale kernel design in the MCI module using the DOTA dataset.

Table 1. Multi-scale kernel design.

Kernel Design	Parameters (MB)	FLOPs (G)	mAP
(3, 3, 3, 3, 3)	12.62	62.40	$79.94\pm 0.12$
(3, 5, 7, 9, 11)	13.69	70.20	$81.72\pm 0.15$
(3, 5, 9, 13, 17)	14.99	79.57	$81.06\pm 0.12$
(11, 11, 11, 11, 11)	15.13	80.61	$80.45\pm 0.10$
(15, 15, 15, 15, 15)	17.44	92.45	$80.32\pm 0.13$

shows that when only smaller 3 × 3 kernels are used, the model achieves suboptimal performance. This can be attributed to its limited ability to extract texture information. To address this, a multi-scale kernel structure with kernel sizes ranging from 3 × 3 to 11 × 11 and a stride of 2 was implemented, under which the model achieved its highest performance. Subsequent experiments tested a configuration with increased kernel sizes and a stride of 4, but the performance fell short of the optimal level. Further investigations revealed that exclusive use of large kernels not only elevates computational complexity but also degrades performance, with mAP dropping by 0.49% and 0.84%, respectively. This suggests that large kernels may introduce background noise, leading to performance deterioration.

On the basis of the multi-scale kernel design in Table 1 that achieved optimal performance, we studied the number of kernels in the multi-scale kernel design. As shown in

Table 2. Kernel number.

Kernel Number	Parameters (MB)	FLOPs (G)	mAP
2	12.56	61.95	$78.64\pm 0.23$
3	12.78	63.57	$79.16\pm 0.18$
4	13.13	66.24	$80.07\pm 0.12$
5	13.69	70.20	$81.72\pm 0.15$
6	14.35	75.26	$81.60\pm 0.14$

, with the increment in the number of kernels, the network’s performance exhibited a gradual upward trajectory, reaching the best effect when five kernels were used.

Next, we verified the effectiveness of the HEA module. First, its impact was examined by applying HEA with different kernel sizes, with the results shown in

Table 3. Kernel size in HEA.

Kernel Design	Parameters (MB)	FLOPs (G)	mAP
(3, 3, 3)	13.50	68.95	$80.52\pm 0.14$
(5, 5, 5)	13.52	69.08	$80.66\pm 0.10$
(5, 7, 7)	13.54	69.21	$80.75\pm 0.25$
(7, 11, 11)	13.58	69.47	$80.89\pm 0.22$
Expansive	13.69	70.20	$81.72\pm 0.15$

. Empirical evidence suggests that smaller kernels struggle to model long-range dependencies, resulting in limited performance, whereas larger kernels mitigate this issue by integrating broader contextual information. Our proposed kernel size expansion strategy, which involves increasing the stripe convolution kernel size as the network blocks deepen, achieves optimal performance.

Because our MIHE-Net is composed of four stages, we subsequently investigated the deployment location of the HEA module.

Table 4. Location for implementing HEA.

Stage Apply	Parameters (MB)	FLOPs (G)	mAP
None	12.03	61.72	$80.13\pm 0.23$
1	12.19	64.04	$80.35\pm 0.25$
2	12.31	65.45	$80.48\pm 0.15$
3	12.97	66.59	$80.97\pm 0.12$
ALL	13.69	70.20	$81.72\pm 0.15$

shows that deploying the HEA module at any stage can provide performance improvements.

Table 5. Impact of individual components on the HRSC2016 dataset.

With MCI	With HEA	With Midpoint Offset	mAP
✗	✗	✗	$89.21\pm 0.17$
✓	✗	✗	$91.52\pm 0.19$
✗	✓	✗	$90.40\pm 0.15$
✗	✗	✓	$90.83\pm 0.11$
✓	✓	✓	$92.43\pm 0.19$

presents the outcomes of the baseline model incorporating our proposed MCI module, HEA module, and Midpoint Offset Loss Function on the HRSC2016 dataset. On this dataset, the baseline model yields an mAP of 89.21%, which is because traditional detection networks with fixed-scale designs struggle to simultaneously and accurately detect objects of various sizes. When combined with our proposed MCI module, the detector’s performance is optimized by 2.31%. This indicates that the MCI module can perform refined scale modeling in rotated object detection tasks, fully extracting local features of objects at different scales while incorporating feature-rich contextual information to improve the network’s detection performance. Adding HEA module to the network enables the detector to extract global contextual information. Further incorporating long-range contextual information ensures that local features have sufficient global awareness, increasing the detector’s performance by 1.19%. Adding the Midpoint Offset Loss Function allows the detector to fully exploit the expressive power of object orientation features in the context of scale changes. This achieves collaborative perception and enhancement of scale and orientation, improving the detector’s performance by 1.62%. Together, the MCI module, HEA module, and Midpoint Offset Loss Function improve the performance of MIHE-Net by 3.22% compared to the baseline model, resulting in more accurate rotated object detection.

Analogous experimental outcomes to those presented in

Table 6. Impact of individual components on UCAS-AOD dataset.

With MCI	With HEA	With Midpoint Offset	mAP
✗	✗	✗	$89.63\pm 0.28$
✓	✓	✗	$91.28\pm 0.25$
✗	✓	✓	$90.96\pm 0.19$
✓	✗	✓	$91.22\pm 0.28$
✓	✓	✓	$91.86\pm 0.22$

were observed the UCAS-AOD dataset, with networks integrating heterogeneous modules demonstrated superior performance compared to those relying solely on a single module. In addition, the results indicate no conflicts between the proposed modules. When all proposed modules are used, better regression and classification results are achieved and the detector demonstrates optimal performance, reaching an mAP of 91.86% on the UCAS-AOD dataset.

4.5. Comparison of Results for Different Datasets

4.5.1. Evaluation on DOTA

Table 7. Performance assessment on the DOTA dataset. Bold values indicate the best values with statistically significant differences from other results at p < 0.05.

Methods	Backbone	PL	BD	BR	GTF	SV	LV	SH	TC	BC	ST	SBF	RA	HA	SP	HC	mAP
DAL [39]	ResNet-101	88.68	76.55	45.08	66.80	67.00	76.76	79.74	90.84	79.54	78.45	57.71	62.27	69.05	73.14	60.11	71.44
CFC-Net [40]	ResNet-101	89.08	80.41	52.41	70.02	76.28	78.11	87.21	90.89	84.47	85.64	60.51	61.52	67.82	68.02	50.09	73.50
SLA [41]	ResNet-50	88.33	84.67	48.78	73.34	77.47	77.82	86.53	90.72	86.98	86.43	58.86	68.27	74.10	73.09	69.30	76.36
RoI Transformer [42]	ResNet-101	88.64	78.52	43.44	75.92	68.81	73.68	83.59	90.74	77.27	81.46	58.39	53.54	62.83	58.93	47.67	69.56
RIDet-O [43]	ResNet-101	88.94	78.45	46.87	72.63	77.63	80.68	88.18	90.55	81.33	83.61	64.85	63.72	73.09	73.13	56.87	74.70
Oriented R-CNN [44]	ResNet-50	89.84	85.43	61.09	79.82	78.71	85.35	88.82	90.88	86.68	87.73	72.21	70.80	82.42	78.18	74.11	80.87
TIOE-Det [45]	DarkNet-50	89.76	85.23	56.32	76.17	80.17	85.58	88.41	90.81	85.93	87.27	68.32	70.32	68.93	78.33	68.87	78.69
S2A-Net [25]	ResNet-101	89.28	84.11	56.95	79.21	80.18	82.93	89.21	90.86	84.66	87.61	71.66	68.23	78.58	78.20	65.55	79.15
LSKNet-T [26]	-	89.14	83.20	60.78	83.50	80.54	85.87	88.64	90.83	88.02	87.31	71.55	70.74	78.66	79.81	78.16	81.07
MIHE-Net	ResNet-50	87.25	81.32	58.76	78.41	77.83	83.57	86.29	90.15	86.73	85.92	70.28	69.45	76.82	80.17	83.26	79.83
MIHE-Net	ResNet-101	88.62	83.07	60.57	81.17	80.06	85.41	87.76	90.82	88.42	87.30	73.16	71.76	79.12	82.46	84.76	81.72

shows the results of our proposed MIHE-Net and other remote sensing image rotation detection methods on the DOTA dataset. For the Basketball Court (BC), Soccer Ball Field (SBF), Roundabout (RA), Swimming Pool (SP), and Helicopter (HC) categories, MIHE-Net achieved the highest detection accuracy of 88.42%, 73.16%, 71.76%, 82.46%, and 84.76%, respectively. Additionally, MIHE-Net obtained the best average result of 81.72% across all categories. LSKNet-T [26] performed the best in remote sensing rotated object detection tasks, as it not only considers the extraction of rotational features in remote sensing targets but also selectively expands the spatial receptive field of large-sized targets to capture more scene context information by combining target features with contextual features to assist target detection. However, the use of large-kernel convolutions in LSKNet-T may introduce substantial background noise, which is unfavorable for the precise detection of small targets. Meanwhile, it ignores the impact of features extracted by the detector on subsequent rotational bounding box regression. MIHE-Net remedies this deficiency, increasing the mAP by 0.65%.

We visualize some detection results of MIHE-Net on the DOTA dataset in Figure 4. The three images in the first row depict targets of various scales, for which MIHE-Net still achieves precise detection. The three images in the second row contain many small and dense targets; nonetheless, MIHE-Net still accurately detects objects such as cars and ships by utilizing contextual information. The first image in the third row contains targets arranged in a rotating and dense manner, while the second image shows rotating multi-directional targets and the third contains targets with both large and small aspect ratios. Despite this, the rotating bounding boxes generated by MIHE-Net remain well-aligned with the targets, ensuring that all target areas are included without covering excessive background areas.

These results indicate that capturing multi-scale texture features when objects exhibit various scales and achieving bounding box regression when objects are rotated in multiple directions are both crucial for accurately detecting rotated targets in remote sensing images. Our MCI module design can capture local and global contextual information at different scales, combining target features with contextual features while maintaining the extraction of fine-grained detail features. In addition, the Midpoint Offset Loss Function makes the network’s regression results for rotated bounding boxes more accurate.

4.5.2. Evaluation on HRSC2016

Table 8. Performance evaluation on the HRSC2016 dataset.

Methods	Backbone	Parameters (MB)	FLOPs (G)	Size	mAP
RRPN [46]	ResNet101	44.5	125	800 × 800	79.08
RoI-Transformer [42]	ResNet-101	55.1	200	512 × 800	86.20
RSDet [47]	ResNet-50	28.9	73	800 × 800	86.5
DAL [39]	ResNet-101	36.4	216	416 × 416	88.95
R3Det [28]	ResNet-101	41.9	216	800 × 800	89.26
SLA [41]	ResNet-101	49.0	130	768 × 768	89.51
CSL [48]	ResNet-50	37.4	236	800 × 800	89.62
GWD [23]	ResNet-101	55.2	121	800 × 800	89.85
TIOE-Det [45]	ResNet-101	48.6	139	800 × 800	90.16
S2A-Net [25]	ResNet-101	38.6	198	512 × 800	90.17
ReDet [30]	ResNet-101	31.6	89	512 × 800	90.46
Oriented RCNN [44]	ResNet-101	41.1	199	1333 × 800	90.50
MIHE-Net	ResNet-101	13.69	71	800 × 800	92.43

presents the detection results of the selected comparative methods and MIHE-Net on the HRSC2016 dataset. MIHE-Net achieves an mAP of 92.43%, with Params of 13.69 MB and FLOPs of 71 G. While achieving the highest mAP, MIHE-Net also maintains a relatively low number of parameters and computational load; thus, while ensuring detection accuracy, it also has high operational efficiency and good potential for practical deployment. Figure 5 displays some of the detection results of MIHE-Net on the HRSC2016 dataset. Most of the ship targets in the HRSC2016 dataset are rotated in multiple directions, have extreme aspect ratios, and come in different sizes (the three images in the first row). This leads to target features becoming blurred and indistinguishable when the background mimics the target’s appearance (first column of the second row), the image background is highly intricate (second column of the second row), or the image exhibits pronounced light-dark fluctuations (third column of the second row). Nonetheless, MIHE-Net demonstrates strong robustness by accurately detecting ship targets and shows some anti-interference capabilities, enabling the network to more accurately estimate target scales and boundaries.

4.5.3. Evaluation on UCAS-AOD

As indicated by the detection results in

Table 9. Performance evaluation on the UCAS-AOD dataset.

Methods	Backbone	Size	Car	Airplane	mAP
SLA [41]	ResNet-50	800 × 800	88.57	90.30	89.44
TIOE-Det [45]	ResNet-50	800 × 800	88.83	90.15	89.49
CornerNet [49]	Hourglass Network	800 × 800	89.69	89.25	89.47
FCOS [50]	ResNet-50	800 × 800	89.49	90.05	89.77
RIDet-O [43]	ResNet-50	800 × 800	88.88	90.35	89.62
DAL [39]	ResNet-50	800 × 800	89.25	90.49	89.87
S2A-Net [25]	ResNet-50	800 × 800	89.56	90.42	89.99
MIHE-Net	ResNet-101	800 × 800	91.71	92.01	91.86

, our proposed MIHE-Net outperforms all existing methods in terms of overall performance, with its mAP reaching 91.86%. The visualization results of UCAS-AOD are presented in Figure 6. In the first row, depicting airplanes of varying scales and orientations, and the second row, showcasing densely arranged small vehicles and airplanes, the detection boxes of MIHE-Net exhibit superior coverage of the targets. These outcomes further validate the efficacy of our approach. Additionally, in the third row, where some vehicles are occluded by trees and nearly devoid of texture details, our method demonstrates remarkable capability in precisely locating such targets. MIHE-Net can comprehensively extract target features to accurately capture target direction information, endowing the detector with robustness and enabling high-quality detection.

4.5.4. Failure Case Analysis

In the first image of Figure 7, MIHE-Net is only able to detect a portion of the occluded cars, resulting in missed detections. For such small-scale and severely occluded targets, the global context modeling capability of the HEA module is limited by the proportion of features in the visible area, while the multi-scale features extracted by the MCI module contain a large amount of background interference information, leading the classifier to misjudge them as background. In the second image of Figure 7, the bounding boxes output by the model are offset for targets with extreme aspect ratios. The convolution kernel size of the current MCI module (maximum 11 × 11) has limited adaptability to targets with aspect ratios exceeding 10:1, causing the features at both ends of the long axis to fail to fuse effectively. This results in “shrinking” or “stretching” deviations during bounding box regression. In the third image of Figure 7, the background of the ship is mixed with a large number of texture features similar to the ship, and the bounding boxes output by the model are offset. In the multi-scale features extracted by MIHE-Net, the high-frequency components of background noise mask the edge features of the target, making the feature anchors referenced during bounding box regression inaccurate, which ultimately leads to positioning offsets.

5. Conclusions

In the task of remote sensing object detection, it is necessary to address the inability of feature pyramid structures with fixed-scale design in the backbone network to adapt to progressive scale changes of objects, as well as the problem that relying solely on local-scale features makes it difficult for the detector to accurately capture object orientations. To solve these issues, we propose a Rotation Target Detection Network Based on Multi-kernel Interaction and Hierarchical Expansion (MIHE-Net). The proposed method extracts local features of objects through the proposed MCI module, ensures that local features have sufficient global perception capability via the proposed HEA module, and achieves precise object regression using the proposed Midpoint Offset Loss Function. Comparative experiments with other advanced detectors show that the resulting detector not only achieves excellent detection performance but also efficiently balances computational complexity and detection accuracy.

MIHE-Net demonstrates significant practical application value in remote sensing target detection, particularly due to its excellent ability to handle targets with progressive scale changes and multi-directional orientations. In military reconnaissance, it can efficiently capture the gradual scale changes of targets such as ships and aircraft (from far to near or small to large) and accurately locate moving targets with different heading angles, ensuring stable detection in complex scenarios. In agriculture, it can precisely capture plot boundaries with different orientations, such as diagonal ridges and irregular plot contours, enabling high-precision identification of crop types and plot information. However, there remains room for improvement in scenarios involving severely occluded targets and targets with extreme aspect ratios. In future research, the global context modeling capability of the HEA module could be further enhanced to capture long-range dependencies in dense scenes; alternatively, an occlusion-aware mechanism could be introduced to improve the feature integrity of partially occluded targets. Additionally, applications in synthetic aperture radar (SAR) could be explored by optimizing the feature extraction strategy of the MCI module to deal with speckle noise in SAR images.