Boosting Point Set-Based Network with Optimal Transport Optimization for Oriented Object Detection

Abstract

When handling complex remote sensing scenarios, rotational angle information can improve detection accuracy and enhance algorithm robustness, providing support for fine-grained detection. Point set representation is one of the most commonly used methods in arbitrary-oriented object detection tasks, leveraging discrete feature points to represent oriented targets and achieve high accuracy in angle prediction. However, due to the inherent discreteness of point set representation, it is prone to significant impact from isolated points and representational ambiguity in harsh application scenarios, leading to inaccurate detection. To address this issue, an efficient aerial object detector named BE-Det is proposed, which uses the optimal transport (OT) strategy to constrain the positions of isolated points. Additionally, a candidate point set quality evaluation scheme is designed to effectively assess the quality of candidate point sets. Experimental results on two challenging aerial datasets demonstrate that the proposed method outperforms several advanced detection methods.

1. Introduction

Object detection in aerial images is aimed at classifying and locating remote-sensing instances and serves as an essential module for numerous applications such as aviation control, maritime rescue, and port management. Oriented bounding boxes can more precisely define the position and boundaries of targets, reducing false positives and missed detection, which supports higher-level pose recognition and intent inference. Methods such as SCRDet [1], CADNet [2], and GWD [3] have improved the detection performance of oriented objects to a certain extent. The aforementioned methods directly regress the angle parameters of the oriented bounding boxes, resulting in a discontinuity in angle loss and inconsistency in angle and localization regression. The point set representation method characterizes the oriented target through several discrete feature points and localizes the oriented target through the conversion process between the point set and the target box, which avoids the loss discontinuity and regression inconsistency caused by directly predicting the angle of objects. However, in various remote sensing application scenarios, object detection tasks are often affected by unclear boundary problems such as adjacent similar objects and background with strong correlation features, as shown in Figure 1a, which prevent good detection results from being achieved. For point set representation methods, the trade-off between angle and position prediction results in weak localization ability, making it difficult to fully study unclear boundary geometric features when facing those situations.

As shown in Figure 1b, the above problems lead to a situation where some learned points are easily misguided and move to positions far away from the ground-truth target boundary. To effectively address the above issues, we revisited the supervision method for point set representation and the label assignment method, aiming to mitigate the impact of unclear boundaries. Previous methods typically selected a few points from the point set to construct a convex quadrilateral, using one-to-one point matching or Intersection over Union (IoU) for localization supervision. However, this supervision method cannot effectively constrain all discrete points in the point set, and inconsistencies between the point set-to-bounding box conversion methods during training and inference lead to outlier points. Additionally, in the label assignment process, the existence of unclear boundaries causes inconsistencies between classification loss and bounding box quality. Therefore, using the simple sum of classification and localization losses as the basis for positive sample matching fails to select the relatively optimal candidates for ground-truth targets.

In this work, to address practical application scenarios involving mutual interference between adjacent targets and background misguidance issues, a rotated object detector is proposed for remote sensing images, named Boundary Enhancement Detector (BE-Det). Mutual interference and background misguidance cause point set misplacement in harsh application scenarios. To achieve comprehensive constraint of the point set during localization supervision, unlike previous point set representation methods, the optimal transport theory is introduced to construct a one-to-many point matching mode based on the original one-to-one point matching method. By learning the matching relationship between ground-truth corner points and discrete feature points in the point set, we optimize the rotated box localization capability and prevent the occurrence of outlier points. Additionally, to construct target classification scores with localization awareness, an effective candidate point set quality evaluation scheme is proposed. By combining the target’s rotation attributes and the correlation of matching points, we effectively evaluate and rank candidate point sets.

To address the issue of unclear boundaries, the main contribution of this work is summarized as follows:

• An effective remote sensing image object detector is proposed, which can significantly optimize the point set localization supervision process through a one-to-many matching method.
• A novel comprehensive quality evaluation scheme for point set representation methods is proposed, which contributes to the selection of high-quality candidate point sets.

The remainder of this article is organized as follows. We introduce the related works and existing problems of oriented object detection in Section 2. Section 3 illustrates the rationale and details of the proposed framework. In Section 4, we validate the effectiveness of the proposed method and present the experimental results based on two public datasets. Finally, we draw the conclusions in Section 5.

2. Related Work

2.1. Oriented Object Detection Based on Point Set Representation

Remote sensing object detection methods primarily focus on introducing angle regression into classical general object detectors. Methods such as SCRDet [1], CADNet [2], DRN [4], R3Det [5], ReDet [6], AO2-DETR [7], RTMDet [8], LSKDet [9] and Oriented RCNN [10] directly predict the rotation angles of the bounding boxes, achieving relatively satisfactory detection performance. To address the inconsistency between angle regression and coordinate regression metrics, Gliding Vertex [11] and RSDet [12] improve detection performance by constructing new representations for rotated bounding boxes. Additionally, to alleviate the discontinuity problem in angle regression, CSL [13] and ARS-DETR [14] transforms the angle prediction regression problem into a classification problem, avoiding the abrupt changes in angle regression. GWD [3], KLD [15] and RHINO [16] utilize new Gaussian loss functions to achieve IoU-based optimization for rotated bounding box regression. However, these methods’ performance and robustness are limited by the periodicity of angle regression.

Inspired by the idea of RepPoints [17] representing bounding boxes with discrete feature points, the point set representation-based method for rotated object detection uses the positional relationships between discrete feature points to represent the position and rotation information of bounding boxes, effectively avoiding the limitations of traditional angle regression. CFA [18] leverages the spatial constraint capability of sampling points to generate a convex hull for each rotated object and optimizes it using the CIoU loss. To explore the rotational information within the point set, Oriented RepPoints [19] further designs an Adaptive Point Assessment and Allocation (APAA) scheme, evaluating each candidate point set from the aspects of classification, localization, directional alignment, and point-by-point correlation, achieving good detection performance in anchor-free methods. G-Rep [20] transforms the point set into a gaussian distribution form and optimizes the parameter regression method. The point set representation method extracts geometric feature information of the object in the form of feature points, eliminating the localization errors caused by direct angle regression and enhancing sensitivity to the object’s rotational information. While these point set representation methods overcome the limitations of angle regression on detection performance, the discrete nature of the point set regression process cannot effectively eliminate the impact of outlier points.

2.2. Quality Assessment and Sample Assignment for Object Detection

Many object detection algorithms evaluate the quality of candidate samples using object classification loss and set fixed IoU thresholds to select positive samples [21,22,23]. However, this approach does not ensure the overall quality of the training samples. Additionally, many works have attempted to use label assignment methods with self-learning optimization strategies to select high-quality samples and further improve detection performance. FreeAnchor [24] allocates k candidate anchor boxes to each ground-truth target based on IoU and then proposes a customized detection likelihood estimate to perform positive and negative sample division within each candidate set. ATSS [25] divides candidate samples using the concept of centerness and then uses the mean and variance in the IoU values of candidate samples as the threshold for positive and negative samples. PAA [26], based on ATSS [25], assumes that the joint loss distribution of positive and negative samples follows a Gaussian distribution and adaptively selects positive and negative samples through probability distribution.

In the task of rotated object detection in remote sensing images, the inclusion of rotational information leads to the issue of prediction diversity. To select higher-quality samples for training, DAL [27] introduces a matching degree parameter to evaluate the localization quality of rotated boxes and uses a matching sensitive loss to strengthen the correlation between classification and localization tasks. Oriented RepPoints [19] incorporates directional alignment information into the sample assignment process, enhancing the directional alignment capability in sample quality evaluation.

Due to issues like background interference, the classification features at the object boundary are weakened, causing severe distortion of classification loss during sample quality evaluation. To address this, VFNet [28] and GFLV1 [29] reconstruct classification loss functions with IoU evaluation capability. In this work, we need to design an effective quality evaluation and sample assignment scheme that includes directional alignment capability while achieving classification metric design guided by localization information.

3. Proposed Method

3.1. Overview

The proposed framework is illustrated in Figure 2. Similar to Oriented RepPoints [19], the widely used ResNet-50 and ResNet-101 are employed as backbones, combined with the FPN (Feature Pyramid Network) to extract features for point set representation. The point set representation is introduced in Section 3.2 as the preliminary architecture. To address the issue of representative point deviation when converting point sets into oriented bounding boxes, the OT conversion function is proposed. This function can jointly optimize and drive representative points to adaptively move to appropriate positions within an annotated bounding box. A more detailed explanation of this module is provided in Section 3.3.

Additionally, to select high-quality candidate point sets for model optimization, a quality evaluation scheme for point set representation is proposed, detailed in Section 3.4. Combining the proposed methods, we explain the loss function module in detail in Section 3.4.

3.2. Point Set Representation

The point set representation is driven by object localization and classification loss, which adaptively move feature points toward the appropriate positions and extract discriminative features. Object extent can be accurately covered while avoiding orientation ambiguity for modeling objects as point set representations. Specifically, for the i-th location on the feature maps, $X\in {\mathbb{R}}^{H\times W\times C}$, where $H$, $W$ and $C$ denote the height, width, and channel number of the feature maps, respectively. A point set models a set of adaptive sample points as

$$\mathcal{R}={\left\{\left(x_k,y_k\right)\right\}}_{k=1}^n$$

(1)

where $n$ indexes the sampled points used in the representation and $n$ is set to 9 as a $3\times 3$ feature grid.

For better matching of the ground-truth box and point set, progressively refining bounding box localization and feature extraction is important. For point set representation, the refinement can be expressed simply as

$${\mathcal{R}}_r={\left\{\left(x_k+\Delta x_k,y_k+\Delta y_k\right)\right\}}_{k=1}^n$$

(2)

where ${\left\{\left(\Delta x_k,\Delta y_k\right)\right\}}_{k=1}^n$ are the predicted offsets created by deformable convolution layers (DCNs) with respect to the previous predictions.

To take advantage of the bounding box annotations in the training phase, point set methods utilize the conversion function to transform point sets into oriented bounding boxes. During the inference stage, a standard rotated rectangle is constructed as the final prediction box based on the minimum area of the point set.

3.3. OT for Conversion Function

Conventional methods are incapable of learning complete semantic information and geometric features caused by the filtering process of point sets. For instance, Oriented RepPoints selects four out of n points to build a quadrilateral, and ConvexHull [18] constructs a convex shape by discarding some predicted points from the learning point set. Compared with those methods, we propose a new conversion function paradigm based on the optimal transport strategy that takes advantage of the overall point set features.

3.3.1. Optimal Transport

Optimal transport is a mathematical theory and computational method used to describe the distance or correspondence between two probabilities or sets. The objective of the optimal transport problem is to find the best resource transfer plan to minimize the cost of resource transfer.

Imagine there are m suppliers and n demanders in a specific area. The i-th supplier possesses $s_i$ units of goods, while the j-th demander requires $d_j$ units. The cost of transporting each unit of goods from supplier iii to demander j is represented by $c_{ij}$. The objective of the optimal transport (OT) problem is to determine a transportation plan ${\pi }^{*}=\left\{{\pi }_{i,j}\mid i=1,2,\dots m,j=1,2,\dots n\right\}$ that minimizes the total transportation cost while ensuring all goods from the suppliers are delivered to the demanders [30,31].

$$\begin{gathered} \underset{\pi }{min}\sum _{i=1}^m\sum _{j=1}^n{c}_{ij}{\pi }_{ij} \\ s.t.\sum _{i=1}^m{\pi }_{ij}=d_j,\sum _{j=1}^n{\pi }_{ij}=s_i, \\ \sum _{i=1}^m{s}_i=\sum _{j=1}^n{d}_j, \\ {\pi }_{ij}\ge 0,i=1,2,\dots m,j=1,2,\dots n. \end{gathered}$$

(3)

On the premise of maintaining data integrity and accuracy, the cost of data transmission is minimized by selecting the optimal transmission method, while ensuring the reliability and stability of transmission. The optimal transmission problem can be transformed into a linear programming problem or a convex optimization problem, which can be solved using numerical optimization algorithms.

3.3.2. Conversion Function Construction

The predicted point set and the set of four object corners can be treated as two distributions. In the context of the point set conversion process, supposing there are n representative points in a point set corresponding to an anchor and a 4 gt (ground-truth) corner corresponding to a ground-truth box, we assume each gt corner as a supplier that holds $k_i$ units of points ($\mathrm{i}.\mathrm{e}.,s_i=k_i,i=1,2,3,4$), and each representative point as a demander that needs one unit of a point ($\mathrm{i}.\mathrm{e}.,d_j=1,j=1,2,\dots ,n$). The transporting cost $C$ for one unit of a point from the gt corner to the representative point is defined as the weighted summation of pointwise correlation $P_{ij}^{corr}$ and localization distance $P_{ij}^{reg}$ of each representative point:

$$C_{ij}=P_{ij}^{corr}+\alpha P_{ij}^{reg}$$

(4)

where $\alpha$ is the balanced coefficient.

In order to construct the corresponding relationship between predicted points and ground-truth corner points in the localization aspect, we define $P_{ij}^{reg}$ as the SmoothL1 loss [32] between the predicted point and the corner point. $P_{ij}^{reg}$ is defined as follows:

$$P_{ij}^{reg}=L_{ij}^{reg}\left(R_j^{corner}\left(\theta \right),G_i^{box}\right)$$

(5)

where $\theta$ stands for model’s parameters. $L_{ij}^{reg}$ denotes the SmoothL1 loss function. $P_j^{corner}$ denotes the predicted point location and $G_i^{box}$ denotes the ground-truth corner point.

To measure the pointwise association upon a representative point for a gt corner, we extract the pointwise features and employ the cosine similarity with the corner feature vectors as the correlation for each representative point. As shown in Figure 3, we denote $e_j^{corner}$ and $e_i^{pred}$ as the averaged embedding feature vectors of each point and its adjacent points. The pointwise correlation $P_{ij}^{corr}$ can be formulated as the cosine similarity between $e_j^{corner}$ and $e_i^{pred}$ as below:

$$P_{ij}^{corr}=1-\mathrm{c}\mathrm{o}\mathrm{s}<e_j^{corner},e_i^{pred}>$$

(6)

$P_{ij}^{corr}$ and $P_{ij}^{reg}$ is only used in characterizing transporting cost. In a standard OT problem, the total supply must be equal to the total demand, which is $n$ in our case. Therefore, $k_i$ should be restricted as below:

$$\begin{gathered} \sum _{i=1}^4{k}_i\le n, \\ {1\le k}_i\le n-3 \end{gathered}$$

(7)

In order to optimize the process with every predictied point in the training phase, the above assignment should be more inclined towards equal distribution. Therefore, we can solve this OT problem via the Sinkhorn–Knopp Iteration [33] to make the transportation plan well distributed by adding an entropic regularization term in minimizing the transportation cost:

$$\underset{\pi }{min}\sum _{i=1}^m\sum _{j=1}^n{C}_{ij}{\pi }_{ij}+\gamma E\left({\pi }_{ij}\right)$$

(8)

where $E\left({\pi }_{ij}\right)={\pi }_{ij}\left(\mathrm{l}\mathrm{o}\mathrm{g}{\pi }_{ij}-1\right)$, and $\gamma$ is a constant-controlling regularization term.

Figure 3. Pointwise feature extraction for pointwise correlation measurement.

Figure 3. Pointwise feature extraction for pointwise correlation measurement.

After obtaining ${\pi }^{*}$, we decode the corresponding conversion function by assigning $k_i$ representative points to the gt corner that transports the largest number of units. The completed OT conversion procedure is described in Algorithm 1.

Algorithm 1: The pseudo-code of the optimal transport conversion strategy

Input:

I is an input image R is a point set

G represents the gt corner points for each object

T is the number of iterations in Sinkhorn–Knopp iteration

$\alpha$ is the balanced coefficient

Output:

${\pi }^{*}$ is the optimal assigning plan

1: $4\leftarrow \left|G\right|,n\leftarrow \left|R\right|$

2: pairwise localization cost: $P_{ij}^{reg}=SmoothL1Loss\left(R_j^{corner}\left(\theta \right),G_i^{box}\right)$

3: $e^{corner},e^{pred}\leftarrow Forward(I,R,G)$

4: pairwise correlation cost: $P_{ij}^{corr}=1-cos<e_j^{corner},e_i^{pred}>$

5: compute final cost matrix: $C_{ij}=P_{ij}^{corr}+\alpha P_{ij}^{reg}$

6: $v^0,u^0\leftarrow initializeunitmatrix$

7: for t = 0 to T do:

8: $u^{t+1},v^{t+1}\leftarrow SinkhornIter\left(C,u^t,v^t,R,G\right)$

9: compute optimal assigning plan ${\pi }^{*}$

10: $k_i\left(i=1,2,\dots ,m\right)\leftarrow \mathrm{Dynamic}k\mathrm{Estimation}$

11: return ${\mathit{\pi }}^{\mathbf{*}}$

By minimizing the transportation cost, the correspondence between each point in the point set and the target corners is determined, and the SmoothL1 loss is used to calculate the position loss between each two points, thereby strengthening the position constraints on each point in the point set and reducing the possibility of outlier isolated points occurring during the inference phase.

3.3.3. Dynamic $k_i$ Estimation

Intuitively, the appropriate number of points for each corner should be different and based on many factors, like object semantic information center, offset initialization, etc. It is difficult to map those factors to the representative point number; hence, we propose a simple but effective method to roughly estimate the appropriate number of representative points of each ground-truth corner point based on the calculated transporting plan matrix. Specifically, for each corner point, we first calculate the mean value of the transporting plan and assign representative points above average to the corresponding corner. If a representative point is assigned repeatedly, only the assignment with the highest transporting plan score will be retained. We name this method Dynamic $k_i$ Estimation. We present a detailed comparison between the fixed $k_i$ and Dynamic $k_i$ Estimation strategies in experiments.

3.3.4. Spatial Decoupled Sampling Strategy

OT optimization aims at directionally moving adaptive point object localization, which leads to an inability to obtain points with enriched classification features. Furthermore, the object detection task suffers from inconsistent feature sensitivity (IFS) [34] between localization and classification tasks, which makes the two separate tasks compromise each other.

In response to the above problems, we propose a concise module to decouple the regression and classification stream. As shown in Figure 2, the model extracts features through the backbone and FPN and outputs them to the dual branches of regression and classification, which decouples the tasks into two streams. After that, different sampling offsets constructed by DCNs are designed for regression and classification. The regression branch follows the coarse-to-fine routine proposed by RepPoints and predicts OBBs utilizing OT optimization. The classification branch aims to seek feature-enriched points and confirms object’s class.

Furthermore, to form a standard oriented rectangle during the inference phase, we utilize the MinAeraRect function in OpenCV to construct the minimum external bounding rectangle of the point set.

3.4. Point Set Quality Evaluation

Taking inspiration from VFNet [28], we design a classification coefficient using IoU as a soft label to address the inconsistency between point set quality and classification loss in the evaluation process. Moreover, we combine the localization coefficient, orientation alignment coefficient, and matching point correlation coefficient to construct a comprehensive quality evaluation of the candidate point set.

IoU-Guided Classification Coefficient. We implement IoU-guided classification loss by reforming VFLoss as the classification loss, and then obtain the rectified classification evaluation coefficient $C_{cls}$. Following VFNet, the classification loss is defined as below:

$$F_{cls}=\left\{\begin{array}{cc}-q\left(qlog\left(p\right)\right)\hfill & p>0\hfill \\ -\alpha p^{\gamma }\log \left(1-p\right)\hfill & p=0\hfill \end{array}\right.$$

(9)

where $p$ represents the predicted probability for the foreground, $q$ is the IoU between the predicted bbox (bounding box) and its ground truth, and $\alpha$ is the penalty term inherited from the focal loss. We first use the MinAeraRect function to obtain the rectangular bbox corresponding to the point set and calculate the IoU. Afterwards, the IoU is embedded into the classification loss to enhance the impact of boundary features on the classification loss. Unlike VFLoss, in order to adapt to the small size of most remote sensing image targets, we remove the construction form of the cross-entropy loss when the IoU is greater than zero. We employ $F_{cls}$ as the IoU-guided classification coefficient, which is defined as below:

$$C_{cls}=F_{cls}\left(r,r^{*},IoU\right)$$

(10)

where $r$ and $r^{*}$ is are predicted label and assigned ground-truth label, respectively.

Localization Coefficient. We utilize the SmoothL1 loss between predicted points and gt corner points as the localization coefficient $C_{loc}$, which is used to determine the position constraints of candidate point sets on outliers. Therefore, $C_{loc}$ is defined as below:

$$C_{loc}=\sum _{i=1}^m{F}_{\mathrm{reg}}\left({\delta }_i,t_i^{*}\right)$$

(11)

Orientation Alignment Coefficient. To enhance the orientation assessment, we adopt the orientation alignment representation [19] to evaluate the point set. The converted rectangular bbox and the gt box are outlined with $n$ points; this is carried out to evaluate the alignment by Chamfer distance [35]. The orientation alignment coefficient $C_{ori}$ is defined as below:

$$\begin{gathered} C_{ori}=\sum _{x\in R_i^p}\underset{y\in R_i^g}{min}\parallel x-y{\parallel }_2^2+ \\ \sum _{y\in R_i^g}\underset{x\in R_i^p}{min}\parallel x-y{\parallel }_2^2 \end{gathered}$$

(12)

where $R_i^p$ and $R_i^g$ denote the points that outline the converted rectangular bbox and the gt box, respectively.

Matching Point Correlation Coefficient. According to Section 3.2, the discrete points in the predicted point set are assigned to each corner point according to the position distance and pointwise feature correlation $P_{ij}^{corr}$. Therefore, we utilize the pointwise feature correlation to characterize the matching point correlation coefficient $C_{match}$.

As we have already defined four partial quality estimation coefficients, the overall quality estimation $C$ is defined as below:

$$C={{\mu }_{1}C}_{cls}+{\mu }_2{C}_{loc}+{\mu }_3{C}_{ori}+{\mu }_4{C}_{match}$$

(13)

The overall loss function of the proposed framework is constructed in two stages, and during the training phase, the proposed point set quality evaluation strategy is utilized in the refinement stage to select a high-quality candidate point set. The initialization stage and the refinement stage generate consecutive offsets to obtain an adjustment from the object center point. Following the previous methods, the loss function is as below:

$$\mathcal{L}={{\lambda }_1\mathcal{L}}_{cls}+{\lambda }_2{\mathcal{L}}_{s1}+{\lambda }_3{\mathcal{L}}_{s2}$$

(14)

where ${\lambda }_1$, ${\lambda }_2$ and ${\lambda }_3$ are weights. ${\mathcal{L}}_{cls}$ denotes the classification loss:

$$L_{cls}={\displaystyle \frac{1}{N_{gt}}}\sum _{i=1}^N{F}_{cls}\left(r_i,r^{*}\right)$$

(15)

where $i$ is the index of the candidate point set and $N_{gt}$ denotes the total number of point sets. $F_{cls}$ stands for reformed VFLoss, which was described in Section 3.

${\mathcal{L}}_{s1}$ and ${\mathcal{L}}_{s2}$ represent the localization loss from the regression branch at the initialization and refinement stage, respectively. ${\mathcal{L}}_s$ can be denoted as below:

$${\mathcal{L}}_s={\displaystyle \frac{1}{N_p}}{p}_i^{*}\sum _{i=1}^N\sum _{j=1}^k{F}_{\mathrm{reg}}\left({\delta }_{ij},t_{ik}^{*}\right)+{\mathcal{L}}_W$$

(16)

where $N_p$ denotes the total number of the positive point set samples. $t_{ik}^{*}$ denotes the corner points of ground-truth OBB, and ${\delta }_{ij}$ indicates the predicted representative points of each point set. $F_{\mathrm{reg}}$ is the SmoothL1 loss for the oriented polygon during parameter update. ${\mathcal{L}}_W$ denotes the rotated GIoU loss. OT also utilizes the SmoothL1 loss to calculate the transportation cost between points, which is not contained in the final loss function.

The proposed conversion function assigns representative points to each corner, which facilitates all points in the point set to participate in the optimization process through SmoothL1 loss. Such optimization utilizes the overall semantic information in the point set and achieves joint optimization by incorporating rotated GIoU loss.

4. Experimental Results

4.1. Datasets and Evaluation Metrics

We evaluate our method on public aerial image dataset HRSC2016 [36] and a self-constructed dataset named Opt-Aircraft.

Opt-Aircraft. This dataset is constructed for aircraft-oriented object detection. It contains 3903 aerial plane images with a wide variety of scales, orientations, and shapes of objects. These images are collected from different platforms with resolutions ranging from 600 × 600 to 1024 × 1024 pixels. The fully annotated images contain 23,012 instances. The dataset images involve the crowd and small objects in a large image.

HRSC2016. This dataset comprises numerous strip-like oriented objects with varied appearances, gathered from several well-known harbors for ship recognition. It includes a total of 1061 images, with resolutions ranging from 300 × 300 to 1500 × 900 pixels. For fair comparison purposes, 436 images are used for training, 181 images for validation, and 444 images for testing.

To evaluate the model’s performance on the Opt-Aircraft dataset, we utilize the average precision (AP) metric. The AP metric is selected to evaluate the detection performance of our method. AP, representing the area under the precision–recall curve, is defined as follows:

$$AP={\int }_0^{1}precision(recall)d(recall)$$

(17)

where recall denotes the number of TPs divided by the sum of TP and FN, and precision denotes the number of TPs divided by the sum of TP and FP; these are calculated as below:

$$precision={\displaystyle \frac{TP}{TP+FP}}$$

(18)

$$recall={\displaystyle \frac{TP}{TP+FN}}$$

(19)

Following the previous methods, the IoU threshold with the ground-truth bounding box is set to 0.5.

For practical applications, the false alarm rate (FAR) is a crucial metric that directly indicates the algorithm’s ability to meet task requirements. Hence, we also incorporate this evaluation index to assess the overall effectiveness of the proposed algorithm.

$$FAR={\displaystyle \frac{numofdetectedfalsealarms}{numofdetectedcandidates}}$$

(20)

To provide a comprehensive comparison for HRSC2016, we present the results using both the VOC2007 and VOC2012 metrics. Those two metrics are subsequently referred to as mAP(07) and mAP(12), respectively.

4.2. Implementation Details

The proposed approach is implemented on the ResNet-50 [37] backbone with the FPN [38]. The stochastic gradient descent (SGD) optimizer is used in training. Following Oriented RepPoints, the learning rate is set to 0.008 in the warming up of 500 iterations, and it will be decreased by a factor of 0.1 at each decay step. The hyperparameters of focal loss are set to $\alpha =$0.25 and $\gamma =2.0$. In Equation (14), we set the balanced weights for each stage, ${\lambda }_1=0.3$, ${\lambda }_2=0.8$ and ${\lambda }_3=1$, empirically. During the training phase, BE-Det utilizes the proposed OT conversion function to transform a point set into an oriented polygon in both the initialization stage and refinement stage. The RepPoints points assigner is used at the initialization stage. For each object, all the point set candidates are sorted according to their quality estimation scores. At the refinement stage, the proposed point set quality evaluation (PSQE) scheme is used to select the high-quality candidates. For each oriented object, the top 50 candidates are assigned as the positive samples. During the inference stage, a standard rotated rectangle is constructed as the final prediction box based on the minimum area of the point set. The experiments are trained on a server with 2 RTX 4090 GPUs using a batch size of 8, and a single RTX 4090 GPU is employed for inference.

4.3. Ablation Study

To examine the effectiveness of each component in our proposed framework, we implement ablation experiments on the HRSC2016 dataset.

Oriented RepPoints first proposed the concept of oriented conversion functions dealing with aerial objects with arbitrary orientations. Therefore, we make Oriented RepPoints our baseline. To make a reasonable comparison, we compare the different oriented conversion functions with ResNet-50 as the backbone, and the original RepPoints [17] label assigner is used to roughly select the predicted point set.

Table 1. Ablation study of different conversion functions.

Methods	Conversion Function	mAP(07)
Oriented RepPoints	NearestGTCorner	87.12
	ConvexHull	88.56
BE-Det	Optimal Transport	89.14

shows that the two oriented conversion functions in Oriented RepPoints achieves 87.12% and 88.56% mAP, respectively. Compared to them, the proposed OT conversion function achieves 89.14% mAP under the same conditions, which indicates that the OT strategy makes the point set have a good fitting effect on the objects after divergence.

The value of $k_i$ serves as the number of candidate points for each corner. When adopting $k_i$ as 1, the proposed OT conversion function is equivalent; this allows us to find the closest point from the learned point set for each corner, which has a similar effect to that of the NearestGTCorner function in Oriented RepPoints.

Table 2. Performance of different units of points assigned for each corner point.

Methods	Oriented RepPoints	BE-Det
$k_i$	\	1	2	dynamic
mAP(07)	87.12	87.19	88.82	90.59

shows that setting $k_i$ as 2 achieves better performance and that Dynamic $k_i$ Estimation achieves the best performance, which demonstrates that our proposed Dynamic $k_i$ Estimation method is effective for point set divergence.

To evaluate the effectiveness of the proposed point set label assignment, we compare it against the baseline method. As shown in

Table 3. Ablation studies on each component in BE-Det. $C_{cls}$ stands for IoU-guided classification coefficient, and $C$ stands for overall quality estimation for BE-Det.

Ablation Studies on Each Component in BE-Det
baseline	√
OT		√	√	√
$C_{cls}$			√	√
$C$				√
mAP(07)	87.12	89.14	89.79	90.59

, we observe that point set label assignment gives a decent mAP improvement. It proves that a comprehensive quality evaluation can effectively filter high-valued predicted bboxes, and the classification coefficients guided by IoU can significantly improve the localization ability at the target boundary.

4.4. Time Cost Analysis

In practical applications, timeliness and accuracy are generally considered comprehensively. For detection tasks, accuracy often determines the upper limit of the algorithm. Therefore, we take accuracy as the priority principle of algorithm design.

Table 4. Comparison of the time cost for the Opt-Aircraft.

Methods	mAP	Params	Speed
Oriented RepPoints	90.29	36.4 M	22.3 fps
ReDet	87.83	40.2 M	18.4 fps
BE-Det	92.37	36.8 M	21.9 fps

compares the time cost of BE-Det with that of the other two methods. Compared with the two-stage ReDet, the proposed BE-Det has fewer parameters and faster inference speed while ensuring improved detection accuracy. Compared with the baseline Oriented RepPoints, although there is a certain degree of decrease in inference speed, BE-Det has better detection performance.

Therefore, our algorithm can achieve the best mAP without significantly affecting the timeliness, allowing it to meet the needs of practical applications.

4.5. Comparison with the State-of-the-Art Methods

Table 5. Comparisons of state-of-the-art methods on HRSC2016.

Methods	Backbone	mAP(07)	mAP(12)
Rotated Faster-rcnn RoI-Transformer [39]	R-50-FPN	84.30	-
	R-101-FPN	86.20	-
BBAVectors [40]	R-101-FPN	88.60	-
R3Det	R-101-FPN	89.26	96.01
ReDet	ReR-50	89.92	96.63
S2A-Net [41]	R-101-FPN	90.17	95.01
Oriented RepPoints	R-50-FPN	90.38	97.26
BE-Det	R-50-FPN	90.59	97.88
BE-Det	R-101-FPN	90.63	97.95

compares our proposed method to other state-of-the-art methods on the HRSC2016 dataset. Our proposed BE-Det achieves 90.59% mAP(07) and 97.88% mAP(12), outperforming other methods with oriented predictions. Notably, it performs better than Oriented RepPoints, which shows the advantage of having comprehensive constraints on a point set. It also indicates that the optimal transport method can not only be applied in label assignment tasks, but also help detection algorithms achieve precise positioning of objects at arbitrary angles.

A visualization of results on the HRSC2016 dataset is shown in Figure 4. In comparison with the other methods, BE-Det can better distinguish targets with similar features from the background. In scenarios where multiple similar targets are close to each other, BE-Det exhibits superior boundary differentiation capability. Other point set representation models, due to the presence of isolated outlier points, are prone to false alarms and inaccurate localization issues. Additionally, the IoU-guided classification coefficient enhances BE-Det’s ability to differentiate blurry boundaries, significantly alleviating false-alarm problems caused by background resembling the target.

From the detection results, it can be seen that BE-Det can detect ship targets under various complex environmental disturbances. This effectiveness is mainly due to the ability to enhance the target feature of point set learning. In contrast, in these complex scenarios, Oriented RepPoints has difficulty effectively distinguishing between boundaries with similar features. This problem is mainly due to the insufficient penalty for outliers in this point set method; ReDet can suppress the interference caused by similar features to some extent, but there are still some missed detections and false alarms when facing strong background interference. This problem arises because this method focuses on adaptive learning of target rotation features, but the background can interfere with the target shape and angle information. Remote sensing images may have a certain degree of blurring due to degradation effects in the transmission link, which limits the performance of this method.

The results of comparison experiments on the Opt-Aircraft dataset are listed in

Table 6. Comparisons of state-of-the-art methods on Opt-Aircraft.

Methods	Backbone	mAP	FAR
RoI-Transformer R3Det	R-101-FPN	85.88	9.69
	R-101-FPN	87.62	8.06
ReDet	ReR-50	87.83	7.98
S2A-Net	R-101-FPN	88.17	7.38
Oriented RepPoints	R-50-FPN	90.29	5.01
BE-Det	R-50-FPN	92.37	3.92
BE-Det	R-101-FPN	92.67	3.77

. BE-Det also performs better than the existing methods in plane detection, where the detection of square regular objects is significant. Figure 5 shows that BE-Det predicts more accurately and with fewer false alarms for oriented bounding boxes compared to other methods.

4.6. More Discussion

Although the experiments conducted confirm that our proposed BE-Det is superior to many advanced oriented detection methods, our method still requires the use of rotated non-maximum suppression (NMS) to remove duplicate prediction results, which may limit the final performance of the model. DETR-based methods can provide a potential solution to this problem, which will be explored in future work. Moreover, our proposed BE-Det has good performance on general remote sensing datasets such as DOTA, but there is still a certain gap compared to the SOTA method, indicating that we still need to design corresponding modules for the model’s generalization ability in future work.

5. Conclusions

This paper proposed an effective aerial object detector named BE-Det based on RepPoints, which improves the detection accuracy to a great extent. For our proposed architecture, by taking advantage of the optimal transport (OT) strategy, BE-Det can constrain the positions of the isolated points and alleviate the issue of inconsistency between training and inference in generating bounding boxes in point set representation methods. Moreover, to address the issue of unclear boundaries, a candidate point set quality evaluation scheme is proposed. This scheme constructs a target classification coefficient with localization awareness, allowing for the evaluation and ranking of candidate point sets. This approach overcomes the problem of severe distortion in classification loss during sample quality assessment. Extensive experiments were performed on two testbeds, the promising results of which demonstrate the efficacy of our proposed approach.