HALD-FSOD: Hierarchical Adaptive Learning with Asymmetric Margin and Loss-Aware Dynamic Weighting for Few-Shot Object Detection in Remote Sensing Imagery

Highlights

What are the main findings?

• A hierarchical adaptive learning framework with asymmetric margin and loss-aware dynamic weighting is proposed for generalized few-shot object detection in remote sensing imagery (HALD-FSOD). It achieves state-of-the-art performance on both DIOR and NWPU VHR 10.v2 datasets without introducing any additional learnable parameters, improving novel-class mAP while maintaining or even enhancing base-class performance, effectively alleviating the catastrophic forgetting problem.
• The three core modules of HALD-FSOD work synergistically: the hierarchical adaptive learning framework balances base knowledge preservation and novel feature adaptation, the asymmetric margin-enhanced cosine classifier reduces inter-category false detections, and the loss-aware dynamic weighting mechanism mitigates the learning imbalance between base and novel classes.

What are the implications of the main findings?

• The results demonstrate that optimizing training strategies and loss functions without introducing additional learnable parameters is a practical and efficient approach for generalized few-shot object detection, avoiding the computational complexity and instability of meta-learning-based methods.
• The hierarchical adaptive learning paradigm and the proposed improved classification and weighting strategies provide generalizable solutions that can be extended to other remote sensing tasks with scarce annotations and imbalanced data, such as oriented object detection and instance segmentation.

Abstract

Recently, few-shot object detection (FSOD) in remote sensing images (RSIs) has been attracting increasing research attention. However, mainstream FSOD methods for RSIs suffer from severe base-class performance degradation when adapting to novel classes. Meanwhile, most existing methods introduce additional learnable parameters, which limit their practical deployment in remote sensing applications. In this paper, we focus on lightweight high-performance FSOD in RSIs and propose HALD-FSOD, an enhanced two-stage fine-tuning approach (TFA) based detector that achieves remarkable novel-class performance improvement without forgetting pre-trained base-class knowledge. Through a comprehensive analysis of the TFA framework’s limitations in remote sensing scenarios, a hierarchical adaptive learning framework is proposed, which achieves an optimal balance between base-class knowledge preservation and novel-class feature adaptation. Considering the low inter-class separability of geospatial objects, we design an asymmetric margin-enhanced cosine classifier to learn a more discriminative decision boundary for novel classes. Furthermore, a loss-aware dynamic class weighting mechanism is developed to alleviate the learning imbalance between base and novel classes. Extensive experiments on DIOR and NWPU VHR 10.v2 datasets demonstrate that our proposed HALD-FSOD achieves competitive overall performance, with remarkable improvement in novel-class detection and negligible base-class degradation, without introducing any additional learnable parameters.

1. Introduction

Remote sensing provides massive high-resolution earth observation data, which has been widely applied in urban planning, environmental monitoring, geographic surveying, and intelligent reconnaissance [1,2]. As a fundamental task of remote sensing image interpretation, object detection aims to accurately locate and classify ground targets of interest from complex scenes [3,4]. Benefiting from the rapid development of deep convolutional neural networks and vision transformers, fully supervised object detection methods have achieved remarkable performance breakthroughs in recent years [5,6,7,8]. However, these methods heavily rely on large-scale, high-quality annotated datasets, whose construction is extremely labor-intensive, time-consuming, and even infeasible for rare or emerging geospatial targets. In practical remote sensing applications, many rare and specific targets only have extremely limited annotated samples, making fully supervised models unable to learn robust feature representations and leading to significant performance degradation. To address this challenge, FSOD, which learns to detect novel categories with only a handful of annotated training samples, has attracted considerable research attention and has become a research hotspot in the remote sensing community [9,10,11].

Although existing FSOD methods have achieved promising results on natural image datasets, they face severe performance bottlenecks and practical limitations when migrating to remote sensing imagery [12,13]. On one hand, remote sensing images are captured from a bird’s-eye view, with inherent characteristics including complex background interference, drastic scale variations in geospatial targets, and high visual similarity between different land cover categories. These unique properties make it difficult for conventional FSOD frameworks to learn discriminative feature representations for novel targets, especially for small, sparse, or visually similar objects in complex scenes [14]. On the other hand, the extreme scarcity of annotated novel samples leads to serious feature dispersion and overfitting problems in the few-shot learning paradigm [15]. Existing mainstream FSOD methods generally struggle to balance the detection performance between base and novel categories. Most methods prioritize the performance improvement of novel classes at the cost of sacrificing base-class detection accuracy, resulting in catastrophic forgetting of pre-trained knowledge and unbalanced detection performance [10,16,17,18]. Even the widely adopted transfer learning-based fine-tuning paradigms, which are renowned for their stable base-class performance, still fail to achieve sufficient feature adaptation for novel remote sensing targets while preserving pre-trained knowledge [13,14].

To boost the generalization of novel categories, a recent line of research attempts to introduce textual semantic priors and language-guided knowledge into few-shot detection frameworks [19,20,21,22,23]. These methods leverage semantic descriptions of categories to complement visual features and enhance the recognition of scarce samples. Despite the performance gain on novel classes, such text-augmented paradigms bring non-negligible limitations in real-world remote sensing deployment. First, they heavily depend on high-quality textual prompts for each category, which is infeasible for unknown or unnamed targets in open remote sensing scenarios. Second, the cross-modal alignment between visual and linguistic features introduces redundant computation and complex model structures, which are incompatible with resource-limited edge platforms such as airborne and satellite-borne systems.

To address the above limitations, this paper proposes HALD-FSOD, a lightweight pure-vision FSOD framework dedicated to high-resolution remote sensing imagery. Our method follows a TFA paradigm [13] and achieves significant performance improvements purely through training strategy optimization and loss function enhancement, without introducing any additional learnable parameters. Specifically, we design a hierarchical adaptive learning framework, which achieves an optimal balance between pre-trained base knowledge preservation and novel-class feature adaptation through differentiated learning rate configuration for different network components. Second, considering the low inter-class separability of geospatial objects in remote sensing imagery, we develop an asymmetric margin-enhanced cosine classifier, which enforces explicit inter-class decision boundary constraints exclusively for novel classes to alleviate inter-category false detections. Third, we propose a loss-aware dynamic class weighting mechanism, which dynamically adjusts the loss weights of each category based on real-time learning difficulty, effectively mitigating the learning imbalance between easy-to-learn base classes and hard-to-learn novel classes.

The main contributions of this paper are summarized as follows:

(1)

We propose a lightweight pure-vision FSOD framework for remote sensing imagery, which achieves remarkable novel-class performance improvement while maintaining a nearly lossless base-class detection performance (as shown in Figure 1), without introducing any additional learnable parameters.

(2)

A hierarchical adaptive learning framework is proposed, which breaks through the feature adaptation bottleneck of conventional fine-tuning paradigms through differentiated learning rate configuration, and achieves an optimal trade-off between pre-trained knowledge stability and novel-class feature plasticity.

(3)

An asymmetric margin-enhanced cosine classifier is designed, which enforces explicit decision boundary constraints for novel classes, effectively alleviating the false detection problem caused by high visual similarity between geospatial objects.

(4)

A loss-aware dynamic class weighting mechanism is developed, which automatically prioritizes hard-to-learn novel classes through real-time loss statistics tracking, effectively balancing the learning progress between base and novel classes without triggering catastrophic forgetting.

Figure 1. Performance comparison of our HALD-FSOD with previous FSOD methods on the DIOR test set in the first novel/base split under the 10-shot settings: (a) base and novel-class performance comparison. The gray dashed line indicates the base mAP (69.05) of pre-trained Faster-RCNN. (b) All-class performance comparison.

Figure 1. Performance comparison of our HALD-FSOD with previous FSOD methods on the DIOR test set in the first novel/base split under the 10-shot settings: (a) base and novel-class performance comparison. The gray dashed line indicates the base mAP (69.05) of pre-trained Faster-RCNN. (b) All-class performance comparison.

2. Related Work

2.1. Object Detection

Deep learning has driven remarkable advances in generic object detection over the past decade. Existing detectors are broadly divided into two-stage and single-stage architectures [24]. Two-stage methods, typified by Faster R-CNN [6], first generate category-agnostic region proposals and then perform accurate classification and bounding box regression, delivering high detection precision. To handle multi-scale objects, Feature Pyramid Network (FPN) [25] is introduced to build hierarchical feature representations, which becomes a standard component in modern detectors. In contrast, single-stage detectors such as YOLO [26] and RetinaNet [27] directly predict object locations and categories in a one-pass manner, prioritizing real-time inference efficiency. Recently, transformer-based detectors have emerged, which model global context dependencies via self-attention mechanisms and achieve competitive performance on natural image benchmarks [28,29].

Despite their success in natural scenes, these generic detectors perform poorly when directly applied to RSIs. Unlike natural objects, aerial targets exhibit arbitrary orientations, extreme scale variations, dense arrangements, and complex background clutter. To mitigate these issues, numerous RSI-oriented detection methods have been proposed, including rotated bounding box detectors, multi-scale feature enhancement modules, dense object localization strategies, and cross-domain learning [30,31,32,33,34,35]. RPUN [36] and SCENet [37] introduce a resolution-preserving, utilization network and a similar category angle, respectively, for tiny object detection. These works effectively adapt detection models to remote sensing scenarios, but all rely on large-scale, high-quality annotated datasets. The high cost of annotating RSIs, especially for rare and fine-grained targets, severely limits the scalability of fully supervised detection methods in practical applications.

2.2. Few-Shot Object Detection

To alleviate the heavy reliance on massive annotations, few-shot object detection (FSOD) is proposed, which aims to adapt pre-trained detectors to novel classes with only a few labeled samples. Existing FSOD methods are mainly categorized into meta-learning-based [38,39,40,41,42] and fine-tuning-based paradigms [43,44,45].

Meta-learning-based methods adopt an episodic training strategy that constructs support-query task pairs to learn transferable prototype representations. These methods focus on cross-task generalization and enable fast adaptation to unseen classes. However, they prioritize episode-level transferability over discriminative object feature learning and often suffer from feature ambiguity when handling objects with large intra-class variations.

Fine-tuning-based methods have become the dominant paradigm due to their simplicity and strong performance. The representative method TFA [13] adopts a two-stage training pipeline: pre-training the detector on data-rich base classes, and then only fine-tuning the final prediction heads on few-shot novel classes. To further improve feature discriminability, FSCE [46] introduces contrastive proposal encoding to enhance the distinction between different categories. Although these methods achieve promising results on natural images, they ignore the unique characteristics of remote sensing targets and cannot be directly generalized to RSIs.

Beyond proposal-level and classification-level optimization, another line of research focuses on data augmentation and feature regularization to mitigate overfitting under low-data regimes [47]. Various geometric and semantic augmentation strategies are designed to enrich the diversity of few-shot samples, while weight regularization and feature normalization are adopted to stabilize the fine-tuning process and prevent the model from overfitting to limited novel instances. These strategies effectively improve the generalization of FSOD models, yet they are still designed for general object distributions and fail to tackle the domain-specific challenges in specialized imaging scenarios.

In addition, some recent works attempt to refine the feature alignment between base and novel classes to reduce knowledge transfer bias [48,49,50,51]. They introduce feature reweighting, channel attention, and relation modeling modules to calibrate the semantic distribution discrepancy between seen and unseen categories. While these methods narrow the domain gap between base and novel classes, they mostly focus on global feature alignment and neglect the fine-grained feature discrimination required for visually similar categories, which remains a core bottleneck for FSOD in complex visual scenarios.

2.3. Few-Shot Object Detection for Remote Sensing Imagery

In recent years, FSOD for RSIs has attracted increasing research attention, with most works focusing on addressing the domain-specific challenges of aerial images, including multi-scale variation, arbitrary orientation, and severe inter-class similarity [9,52,53].

A series of RSI-specific FSOD methods is developed based on the fine-tuning pipeline, which has become the mainstream technical route due to its simplicity and stability [14]. To handle the drastic scale variations in remote sensing targets (ranging from tiny vehicles to large buildings), many studies have dedicated efforts to multi-scale feature enhancement [54]. They introduce hierarchical feature aggregation modules, path aggregation structures, adaptive scale fusion mechanisms, and receptive field expansion modules to capture discriminative features of tiny, medium, and large aerial objects simultaneously [55]. These designs alleviate the scale mismatch problem in few-shot adaptation, but they rarely consider the semantic complementarity between different feature layers, leading to redundant feature fusion and weak semantic representation for novel classes. More importantly, almost all multi-scale enhancement methods introduce a large number of additional learnable parameters, which increase computational complexity and are not conducive to deployment on resource-constrained remote sensing platforms.

For oriented and dense aerial targets such as ships in ports and vehicles in parking lots, researchers integrate rotation-aware feature learning and shape prior modeling into the FSOD framework, improving the localization accuracy of novel aerial classes [56,57]. However, these methods usually require additional rotation annotation information or complex geometric transformation operations, which increases the difficulty of data preparation and model training.

To address the severe inter-class similarity problem in remote sensing imagery, prototype-based methods are widely adopted [19,50,58]. These methods generate class-aware visual prototypes from support samples, which are injected into the region proposal network or classification head to guide accurate proposal generation and classification. Although prototype-based methods effectively utilize the limited information of few-shot samples, they struggle to distinguish visually similar sub-categories, such as different types of aircraft, vehicles, and sports fields. Most works rely on global feature matching and prototype alignment, which cannot capture fine-grained structural and texture differences between highly similar targets, resulting in persistent classification confusion.

Despite these advances, existing remote sensing FSOD methods still have critical limitations. Most methods simply migrate natural-scene FSOD pipelines to RSIs without thoroughly optimizing the training process itself, leading to insufficient semantic modeling of novel classes. Moreover, almost all existing methods achieve performance improvements by adding complex network modules, which compromises the lightweight nature and deployability of the model. Most importantly, few works have effectively solved the fundamental challenge of balancing base-class and novel-class performance. Many methods achieve remarkable novel-class improvements at the cost of severe base-class performance degradation, leading to catastrophic forgetting [17,18].

Different from existing works that rely on adding complex network modules or cross-modal supervision, this paper proposes a lightweight pure-vision FSOD framework dedicated to remote sensing imagery. Built upon the classic fine-tuning paradigm, we optimize the training strategy and loss function from three core aspects without introducing any additional learnable parameters. Specifically, we design a hierarchical adaptive learning framework to balance base-class knowledge preservation and novel-class feature adaptation, develop an asymmetric margin-enhanced cosine classifier to alleviate inter-class confusion between similar geospatial objects, and propose a loss-aware dynamic class weighting mechanism to mitigate the learning imbalance between base and novel classes. Our method effectively addresses the core limitations of existing remote sensing FSOD methods and achieves superior overall performance while maintaining strong engineering practicability and deployment convenience.

3. Methodology

3.1. Problem Statement

Following the standard problem setup for FSOD as established in prior works [10,13,40], we adopt the same experimental configuration. Concretely, the collection of all classes is denoted as $C$. The classes with a lot of labeled samples are referred to as base classes, denoted as $C_{base}$, while the classes with a limited number of labeled samples are called novel classes, denoted as $C_{novel}$. $C_{base}$ and $C_{novel}$, and satisfy $C_{base}\cup C_{novel}=C$ and $C_{base}\cap C_{novel}=\varnothing$. The image spaces for base classes and novel classes are defined as $X_{base}$ and $X_{novel}$, respectively. In the process of few-shot learning, it is common to divide it into two stages: pretraining and fine-tuning.

During the pretraining stage, the model trains on the base dataset $D_{base}=\left\{\left(x,c\right)|x\in X_{base},c\in C_{base}\right\}$ to obtain a robust and universal representation. In the fine-tuning stage, a balanced few-shot dataset $D_{few}=\left\{\left(x,c\right)|x\in X_{base}\cup X_{novel},c\in \mathrm{C}\right\}$ containing $K$-shot annotated instances (e.g., $K=3, 5, 10, 20$) of each base and novel class is constructed for few-shot transfer.

For the inference phase in FSOD, the test set contains all classes $C$, and the model is required to detect all classes of objects. The goal is to obtain a model that achieves high detection performance on $C_{novel}$ while minimizing catastrophic forgetting on $C_{base}$.

3.2. Overview

The proposed HALD FSOD follows a classic two-stage fine-tuning (TFA) paradigm [13], and critically, no additional learnable parameters are introduced throughout the framework, which ensures the lightweight property and avoids overfitting risks in data-scarce few-shot scenarios. Specifically, the Faster R-CNN [6] with Feature Pyramid Network [25] is first trained on the abundant base dataset and then fine-tuned on the balanced few-shot dataset with the hierarchical adaptive learning framework (HALF). Moreover, in the fine-tuning stage, the asymmetric margin-enhanced cosine classifier (AMECC) and loss-aware dynamic class weighting mechanism (LADCW) are introduced to facilitate learning novel knowledge and preserving base knowledge. Specifically, in each training iteration, AMECC replaces the simple cosine classifier in TFA to classify bounding boxes and produce margin-adjusted classification scores, based on which the cross-entropy loss is computed. Meanwhile, LADCW accumulates the per-category losses across training iterations and re-weights each category accordingly, balancing the training weights between base and novel classes. No data augmentation strategies are utilized in either the training or inference phase. The overall structure of our proposed HALD-FSOD is illustrated in Figure 2.

3.3. Hierarchical Adaptive Learning Framework

The original TFA framework [13] adopts an extremely conservative fine-tuning strategy: it freezes the feature extractor (FE), region proposal network (RPN), and RoI Feature Extractor (RoI-FE) during fine-tuning and only updates the final classification and regression heads. While this strategy maximally preserves pre-trained base-class knowledge, it fails to adapt to novel classes.

To achieve an optimal balance between stability (base-class knowledge preservation) and plasticity (novel-class feature adaptation), we propose a hierarchical adaptive learning framework with differentiated learning rate multipliers (LRM) for different network modules, as shown in the fine-tuning stage of the overview figure. The detailed configuration is designed as follows:

• Full Freezing (LRM = 0.0): The FE is completely frozen during the entire fine-tuning stage. The low-level and mid-level visual features learned from massive base-class data have strong generalizability for remote sensing land cover objects, and full freezing fundamentally avoids catastrophic forgetting of base-class knowledge.
• Full Learning Rate Update (LRM = 1.0): The RPN, final classification head, and regression head are updated with the full base learning rate. Unfreezing the RPN enables the model to adjust the proposal generation strategy to novel remote sensing objects, significantly improving the recall of high-quality proposals for novel classes. The full learning rate update of the prediction heads allows the model to quickly learn the decision boundaries for novel classes with limited samples.
• Reduced Learning Rate Update (LRM = 0.1): The RoI-FE is updated with a reduced learning rate. The small learning rate enables the model to gently adjust the high-level feature space to better fit the characteristics of novel classes, without destroying the pre-trained feature distribution of base classes.

3.4. Asymmetric Margin-Enhanced Cosine Classifier

The original TFA [13] adopts a vanilla cosine similarity classifier for final classification. It performs L2 normalization on both the RoI features and classification weights, and calculates the cosine similarity as the classification score, which effectively alleviates the feature scale sensitivity problem in few-shot scenarios. However, in remote sensing imagery, novel classes often share highly similar visual features, spatial layouts, and contextual environments with base classes. With only a few annotated samples, the model cannot learn clear inter-class decision boundaries, leading to severe false detections between similar land cover objects.

To address this inter-class confusion problem, we propose an asymmetric margin-enhanced cosine classifier, which introduces an exclusive margin penalty only for novel classes to enforce more compact intra-class feature clustering and clearer inter-class decision boundaries. The full pipeline of this module is shown in the top-right of the overview figure, and the core calculation process is as follows:

First, given the high-level feature $\mathit{X}$ output by the RoI FE, we perform L2 normalization on the feature and the classification weight matrix $\mathit{W}$ to eliminate scale interference:

$${\mathit{X}}_{norm}={\displaystyle \frac{\mathit{X}}{{‖\mathit{X}‖}_2+ϵ}}$$

(1)

$${\mathit{W}}_{norm}={\displaystyle \frac{\mathit{W}}{{‖\mathit{W}‖}_2+ϵ}}$$

(2)

where $ϵ={10}^{-5}$ is a small constant to avoid division by zero.

The raw classification score is then calculated via linear mapping of the normalized features and weights, with a fixed scaling factor $scale$ to amplify the difference between similarity scores:

$$s_{i,c}=Linear\left({\mathit{X}}_{norm},{\mathit{W}}_{norm}\right)·scale$$

(3)

where $s_{i,c}$ denotes the raw classification score of the $i$-th sample for category $c$.

Subsequently, we apply an asymmetric margin penalty to the classification scores of novel classes before the cross-entropy loss calculation. The margin-adjusted score ${\tilde{s}}_{i,c}$ is formulated as:

$${\tilde{s}}_{i,c}=\left\{\begin{array}{c}{s}_{i,c}-\tau ·scale, if c\in C_{novel}\\ s_{i,c}, otherwise\end{array}\right.$$

(4)

where $\tau$ is the margin hyperparameter.

In the normalized hyperspherical feature space, subtracting the margin penalty narrows the classification decision boundary for novel classes. This forces the model to output a significantly higher cosine similarity to correctly classify novel samples, thus driving the model to learn more compact intra-class feature clustering and a larger inter-class feature distance between novel and base classes. Critically, this module only modifies the classification scores before loss calculation, without introducing any additional learnable parameters. The pseudocode of the margin-adjusted classification scores calculation is shown in Algorithm 1.

Algorithm 1. Magin-adjusted classification scores calculation
Inputs: RoI feature batch $\mathit{X}\in {\mathbb{R}}^{N\times D},$ classification weight matrix $\mathit{W}\in {\mathbb{R}}^{C\times D},$ novel class index set $C_{novel},$ margin hyperparameter $\tau ,$ scaling factor $scale,$ numerical stability constant $ϵ={10}^{-5}.$
Outputs: Margin-adjusted classification scores $\tilde{\mathit{S}}\in {\mathbb{R}}^{N\times C}.$
1:	/* Step 1: L2 normalization to eliminate feature scale interference */
2:	${\mathit{X}}_{norm}={\displaystyle \frac{\mathit{X}}{{‖\mathit{X}‖}_2+ϵ}}$
3:	${\mathit{W}}_{norm}={\displaystyle \frac{\mathit{W}}{{‖\mathit{W}‖}_2+ϵ}}$
4:	/* Step 2: Calculate raw cosine similarity classification scores */
5:	$\mathit{S}=\left({\mathit{X}}_{norm}·{\mathit{W}}_{norm}^T\right)\times scale$
6:	/* $s_{i,c}$: raw classification score of the $i$-th RoI sample for category $c$*/
7:	/* Step 3: Inject asymmetric margin penalty (only for novel classes) */
8:	for each sample $i\in \left\{1,2,\cdots ,N\right\}$ do:
9:	for each category $c\in \left\{1,2,\cdots ,C\right\}$ do:
10:	if $c\in C_{novel}$ then:
11:	${\tilde{s}}_{i,c}=s_{i,c}-\tau ·scale$
12:	else:
13:	${\tilde{s}}_{i,c}=s_{i,c}$
14:	end if
15:	end for
16:	end for
17:	return $\tilde{\mathit{S}}$ // Scores fed into cross-entropy loss for training

3.5. Loss-Aware Dynamic Class Weighting Mechanism

In remote sensing FSOD tasks, there is a significant gap in learning difficulty across different categories: base classes have massive annotated samples and clear feature patterns, making them easy to learn, while novel classes have only a few annotated samples and are often confused with similar base classes or occluded by complex backgrounds, making them extremely hard to learn. The original TFA adopts a fixed uniform loss weight, which causes the gradient update to be dominated by easy-to-learn base classes. Hard novel classes cannot receive sufficient gradient optimization, leading to a severe performance bottleneck.

To alleviate this learning imbalance problem, we propose a loss-aware dynamic class weighting mechanism. It uses an exponential moving average (EMA) to track the historical loss of each category in real time and dynamically adjusts the loss weight to automatically assign higher weights to hard-to-learn novel classes. The full pipeline of this module is shown in the bottom-right of the overview figure, and the core process is as follows:

Step 1: Real-Time Loss Statistics Update.

For each training iteration, we maintain two EMA buffers for each category $c$: the cumulative loss ${cum\_loss}_c$ and the cumulative sample count ${cum\_count}_c$. The buffers are updated with a momentum coefficient $\beta$:

$${cum\_loss}_c=\beta ·{cum\_loss}_c+\left(1-\beta \right)·{\sum }_{i,y_i=c}{l}_i$$

(5)

$${cum\_count}_c=\beta ·{cum\_count}_c+\left(1-\beta \right)·N_c$$

(6)

where $l_i$ is the per-sample cross-entropy and $y_i$ is the ground truth label of the $i$-th sample, respectively. $N_c$ is the number of samples in category $c$ in the current batch.

Step 2: Dynamic Weight Calculation and Loss Fusion.

The average loss for each category is calculated to measure its learning difficulty:

$$av{g\_loss}_c={\displaystyle \frac{{cum\_loss}_c}{{cum\_count}_c+ϵ}}$$

(7)

The raw weight for each category is generated based on the ratio of the category average loss to the global mean loss, which reflects the relative learning difficulty of the category:

$$w_c^{raw}={\displaystyle \frac{av{g\_loss}_c}{\frac{1}{\left|C\right|}\sum _{c}av{g\_loss}_c+ϵ}}$$

(8)

To avoid training instability caused by excessively large weights, we clamp the raw weight within a predefined range $\left[1.0,w_{max}\right]$. The lower bound of 1.0 ensures that simple base classes will not be neglected during fine-tuning:

$$w_c=clamp\left(w_c^{raw},1.0, w_{max}\right)$$

(9)

To further stabilize the training process, EMA smoothing is applied to the final class weights:

$${loss\_weight}_c=\beta ·{loss\_weight}_c+\left(1-\beta \right)·w_c$$

(10)

Finally, the per-sample classification loss is multiplied by the corresponding class weight, and the final weighted classification loss is obtained by averaging the weighted per-sample losses:

$${\mathcal{L}}_{LADCWM\_cls}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{l}_i·{loss\_weight}_{y_i}$$

(11)

where $N$ is the total number of samples in the batch.

This mechanism automatically identifies and prioritizes hard-to-learn novel classes without any additional learnable parameters, effectively alleviating the learning imbalance problem in remote sensing FSOD, while ensuring training stability and no degradation in base-class performance. The pseudocode of the weighted classification loss calculation is shown in Algorithm 2.

Algorithm 2. Weighted classification loss calculation
Inputs: Per-sample CE loss $\mathit{l}\in {\mathbb{R}}^N,$ ground truth labels $\mathit{y}\in {\left\{1,2,\cdots ,C\right\}}^N,$ EMA momentum $\beta ,$ max weight $w_{max},$ numerical stability constant $ϵ={10}^{-5}.$ Persistent States: EMA buffers $\mathit{c}\mathit{u}\mathit{m}\_\mathit{l}\mathit{o}\mathit{s}\mathit{s}\in {\mathbb{R}}^C,$$\mathit{c}\mathit{u}\mathit{m}\_\mathit{c}\mathit{o}\mathit{u}\mathit{n}\mathit{t}\in {\mathbb{R}}^C,$$\mathit{l}\mathit{o}\mathit{s}\mathit{s}\_\mathit{w}\mathit{e}\mathit{i}\mathit{g}\mathit{h}\mathit{t}\in {\mathbb{R}}^C$
Outputs: Weighted classification loss ${\mathcal{L}}_{LADCWM\_cls}.$
1:	/* Step 1: Aggregate per-class loss & sample count of current batch */
2:	${batch\_loss}_c={\sum }_{i,y_i=c}{l}_i,$${batch\_count}_c={\sum }_{i,y_i=c}1,$$\forall c\in \left\{1,2,\cdots ,C\right\}$
3:	/* Step 2: Update EMA statistics & compute per-class average loss */
4:	for each category $c$ do
5:	${cum\_loss}_c=\beta ·{cum\_loss}_c+\left(1-\beta \right)·{batch\_loss}_c$
6:	${cum\_count}_c=\beta ·{cum\_count}_c+\left(1-\beta \right)·{batch\_count}_c$
7:	$av{g\_loss}_c={cum\_loss}_c/\left({cum\_count}_c+ϵ\right)$
8:	end for
9:	/* Step 3: Calculate & smooth dynamic class weights */
10:	$global\_avg={\displaystyle \frac{1}{C}}{\sum }_{c=1}^{C}av{g\_loss}_c$
11:	for each category c do
12:	$w_c^{raw}=av{g\_loss}_c/\left(global\_abg+ϵ\right)$
13:	$w_c=clamp\left(w_c^{raw},1.0, w_{max}\right)$
14:	${loss\_weight}_c=\beta ·{loss\_weight}_c+\left(1-\beta \right)·w_c$
15:	end for
16:	/* Step 4: Compute final weighted loss */
17:	${\mathcal{L}}_{LADCWM\_cls}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{l}_i·{loss\_weight}_{y_i}$
18:	return ${\mathcal{L}}_{LADCWM\_cls}$

4. Experiments

4.1. Datasets

DIOR [3]: The DIOR dataset is a publicly accessible, large-scale benchmark dataset designed for object detection in optical remote sensing imagery. It contains 23,463 images sourced from Google Earth, covering 190,288 object instances across 20 categories such as airplanes, airports, baseball fields, and basketball courts, providing a rich variety of training samples and practical scenarios. Each image in the dataset has a fixed size of 800 × 800 pixels, with spatial resolutions ranging from 0.5 m to 30 m. In line with the FSOD protocol established in [10,14], the dataset is divided into four different novel/base-class splits, where each split consists of 15 base classes and five novel classes. The specific novel classes for each split are listed in

Table 1. Different novel/base split settings on the DIOR dataset and NWPU VHR 10.v2 dataset.

Dataset	Split	Novel					Base
DIOR	1	Baseball field	Basketball court	Bridge	Chimney	Ship	Rest
	2	Airplane	Airport	Expressway tool station	Harbor	Ground track field	Rest
	3	Dam	Golf course	Storage tank	Tennis court	Vehicle	Rest
	4	Expressway service area	Overpass	Stadium	Train station	Windmill	Rest
NWPU VHR 10.v2	1	Airplane	Baseball diamond	Tennis court			Rest
	2	Basketball	Ground track field	Vehicle			Rest

, with the remaining classes treated as base classes.

NWPU VHR 10.v2 [59]: This dataset consists of 1172 images across 10 geospatial object categories, with each image resized to 400 × 400 pixels. The object classes include airplane, baseball diamond, basketball, bridge, ground track field, harbor, ship, storage tank, tennis court, and vehicle. Following a setup analogous to DIOR, two novel/base-class splits are adopted, where each split contains seven base classes and three novel classes. The corresponding novel classes for each partition are presented in Table 1.

The few-shot instances in DIOR and NWPU VHR 10.v2 datasets are taken from prior work [14], ensuring a fair comparison with existing methods.

4.2. Evaluation Metrics

To validate the effectiveness of the method, mean average precision (mAP) with an IoU threshold of 0.5 is adopted as the evaluation metric. The mAP is a commonly used metric for evaluating the performance of object detection models, which denotes the mean of average precision (AP) in all categories. Higher mAP values indicate better detection performance. For a single category, AP is computed as the mean of precision (P) over the recall (R) range from 0 to 1 under a specified IoU threshold, which can be expressed as follows:

$$AP=\sum _{R\in \left(0,0.1,\cdots ,1\right)}{\displaystyle \frac{1}{11}}P\left(R\right)$$

(12)

To evaluate the performance of base classes and the few-shot novel classes, we choose the mAP of the base classes, the novel classes, and all classes as evaluation metrics to evaluate the performance of different methods.

4.3. Implementation Details

Our proposed HALD-FSOD framework is built upon the classic Faster R-CNN [6] two-stage detection architecture and employs a ResNet101 [5] network integrated with an FPN [25] as the backbone feature extractor, which is initialized with publicly available pre-trained weights to guarantee the stability of model convergence. For the base pre-training stage on abundant base-class samples, the model is trained for a total of 36 k iterations, where the learning rate starts from 0.01 and decreases to 0.001 and 0.0001 at the 24 k iterations and 32 k iterations, following prior work [14]. In the subsequent fine-tuning stage, the weights of the bounding box regression head and classification head corresponding to novel classes are randomly initialized, while other network components inherit the pre-trained weights from the base pre-training stage. The entire fine-tuning process runs for 10 k iterations, where the learning rate starts from 0.001 and decreases to 0.0001 and 0.00001 at the 4 k iterations and 8 k iterations. All images are preprocessed by resizing, normalization, and padding before training and inference. Images are resized to 800 $\times$ 800 for the DIOR dataset and up to 400 $\times$ 400 for NWPU VHR 10.v2 dataset, and the size divisor parameter for padding is set to 32 for all images.

Key hyperparameters involved in our framework are set consistently across all experiments: the scaling factor for cosine similarity calculation is fixed at 20, following prior work [14]. The margin hyperparameter τ for the asymmetric boundary constraint is set to 0.2, and the momentum coefficient β for the dynamic loss weighting module is 0.9. These two hyperparameters are selected by grid search. Throughout both the base pre-training and fine-tuning stages, all model parameters are optimized using the Stochastic Gradient Descent (SGD) algorithm, with a weight decay of 0.0001 and a momentum coefficient of 0.9. All experiments are conducted on a single NVIDIA GeForce RTX 4070Ti Super GPU, with a global batch size set to four for all training processes.

During the inference phase, the confidence threshold for preliminary detection candidates is set to 0.05, and the IoU threshold for non-maximum suppression (NMS) is fixed at 0.5 to filter redundant detection boxes. In addition, the data augmentation strategies are not utilized in either the training or inference phases. To eliminate the randomness of model training and ensure the reliability of experimental results, each experimental configuration of our proposed HALD-FSOD is independently executed for five repeated runs, and we report the average performance metrics calculated across all five repeated experiments in the subsequent sections.

4.4. Comparisons with State-of-the-Art Methods

To comprehensively validate the effectiveness and superiority of our proposed HALD-FSOD framework, we conduct extensive comparative experiments with mainstream state-of-the-art (SOTA) FSOD methods on two widely used remote sensing benchmark datasets: DIOR and NWPU VHR 10.v2. FRW [9] and Meta-RCNN [42] are classic meta-learning-based methods built on episodic training paradigms. FSDetView [41] is an advanced meta-learning method that proposes a multi-form feature aggregation scheme. TFA [13] is the fundamental baseline of transfer learning-based FSOD, and HALD-FSOD is built on TFA. P-CNN [10] and G-FSDet [14] are representative improved transfer learning methods with refined fine-tuning strategies and classification head designs, which are selected to demonstrate the performance advantages of our method over other transfer learning peers. TPG-FSOD [18] is included as a representative VLM-based remote sensing FSOD method leveraging CLIP semantic prototypes, which is selected to verify our method and can achieve competitive performance even when compared with the VLM-enhanced FSOD method. All comparative experiments are implemented under 3-shot, 5-shot, 10-shot, and 20-shot settings across all dataset splits, to verify the generalization performance of our method under different few-shot scenarios.

4.4.1. Comparisons on the DIOR Dataset

Table 2. Few-shot object detection performance on the DIOR test set under 3, 5, 10, and 20-shot settings. The red number denotes the best results, and the blue number represents the second-best results.

Split	Method	3-Shot			5-Shot			10-Shot			20-Shot
		Base	Novel	All	Base	Novel	All	Base	Novel	All	Base	Novel	All
1	FRW [9]	49.40	7.50	38.90	49.70	12.10	40.30	49.50	18.10	41.70	50.00	22.00	43.00
	Meta-RCNN [42]	60.62	12.02	48.47	62.01	13.09	49.78	65.30	17.28	53.30	65.77	18.20	53.88
	FSDetView [41]	59.54	13.19	47.95	58.58	14.29	47.51	59.64	18.02	49.24	62.69	18.01	51.52
	P-CNN [10]	47.00	18.00	39.80	48.40	22.80	42.00	50.90	27.60	45.10	52.20	29.60	46.80
	TFA w/cos [13]	70.32	11.35	55.58	70.51	11.57	55.78	70.52	15.37	56.73	71.07	17.96	57.79
	G-FSDet [14]	68.94	27.57	58.61	69.52	30.52	59.72	69.03	37.46	61.16	69.80	39.83	62.31
	TPG-FSOD [18]	62.70	34.20	55.60	62.70	35.60	55.90	64.40	40.20	58.40	65.50	43.00	59.90
	Ours	70.20	26.70	59.35	70.40	29.70	60.21	70.30	35.40	61.56	69.90	39.60	62.30
2	FRW [9]	48.50	4.80	37.60	46.80	7.00	36.90	46.40	9.00	37.10	43.50	14.10	36.20
	Meta-RCNN [42]	62.55	8.84	49.12	63.14	10.88	50.07	63.28	14.90	51.18	63.86	16.71	52.07
	FSDetView [41]	58.88	10.83	46.87	60.31	9.63	47.64	61.16	13.57	49.26	61.16	14.76	49.56
	P-CNN [10]	48.90	14.50	40.30	49.10	14.90	40.60	52.50	18.90	44.10	51.60	22.80	44.40
	TFA w/cos [13]	70.75	5.77	54.51	70.79	8.19	55.14	69.93	8.71	54.63	70.02	12.18	55.56
	G-FSDet [14]	69.20	14.13	55.43	69.25	15.84	55.87	68.71	20.70	56.70	68.18	22.69	56.86
	TPG-FSOD [18]	61.30	14.30	49.60	61.80	19.00	51.10	63.00	24.80	53.50	63.30	25.60	53.90
	Ours	69.50	14.00	55.63	69.30	19.40	56.98	69.30	21.90	57.42	69.90	23.90	58.39
3	FRW [9]	45.50	7.80	36.10	47.90	13.70	39.30	44.50	13.80	36.80	43.50	18.50	37.30
	Meta-RCNN [42]	61.93	9.10	48.72	63.44	12.29	50.66	62.57	11.96	49.92	65.53	16.14	53.10
	FSDetView [41]	61.00	7.49	47.63	61.33	12.61	49.15	61.94	11.49	49.32	65.17	17.02	53.14
	P-CNN [10]	49.50	16.50	41.30	49.90	18.80	42.10	52.10	23.30	44.90	53.10	28.80	47.00
	TFA w/cos [13]	71.95	8.36	56.05	71.64	10.13	56.26	72.56	10.75	57.11	73.13	17.99	59.30
	G-FSDet [14]	71.10	16.03	57.34	70.18	23.25	58.43	71.08	26.24	59.87	71.26	32.05	61.46
	TPG-FSOD [18]	65.60	20.10	54.20	65.10	23.10	54.60	66.40	28.90	57.00	65.90	32.60	57.60
	Ours	71.80	19.30	58.68	71.60	22.70	59.34	71.60	26.00	60.19	72.10	31.10	61.84
4	FRW [9]	48.20	3.70	37.10	48.50	6.80	38.10	45.70	7.20	36.10	44.40	12.20	36.40
	Meta-RCNN [42]	61.73	13.94	49.78	62.60	15.84	50.91	62.23	15.07	50.44	63.24	18.17	51.98
	FSDetView [41]	58.90	14.28	47.75	58.97	15.95	48.22	60.37	15.37	49.12	60.89	16.96	49.91
	P-CNN [10]	49.80	15.20	41.20	49.90	17.50	41.80	51.70	18.90	43.50	52.30	25.70	45.70
	TFA w/cos [13]	68.57	10.42	54.03	68.85	14.29	55.21	68.58	14.35	55.03	68.86	12.01	54.65
	G-FSDet [14]	69.01	16.74	55.95	67.96	21.03	56.30	68.55	25.84	57.87	67.73	31.78	58.75
	TPG-FSOD [18]	60.10	13.10	50.90	62.30	22.00	52.20	60.70	26.90	52.30	60.60	31.00	53.20
	Ours	69.60	14.70	55.90	69.30	18.50	56.59	69.10	21.90	57.28	68.90	27.30	58.47

presents the performance comparison of our HALD-FSOD with SOTA methods across four splits under 3/5/10/20-shot settings. Among all compared methods, TFA achieves the highest base-class performance except in split 4, but suffers from severe novel-class degradation. This is because TFA only updates the box predictor during the fine-tuning stage, which preserves base knowledge well but severely restricts novel feature adaptation. Our hierarchical adaptive learning framework configures differentiated learning rates for different network components, achieving 9.7% average novel mAP improvement in the 3-shot setting and 15.44% in the 20-shot setting, with less than 0.12% average base performance loss in the 3-shot setting and 0.57% in the 20-shot setting, effectively mitigating catastrophic forgetting.

Compared with meta-learning-based methods (FRW, Meta-RCNN, FSDetView), our method shows overwhelming superiority in both novel and overall performance. Meta-learning methods rely on computationally expensive episodic training and tend to overfit to training tasks, while our HALD-FSOD is simpler, more efficient, and better at generalizing complex remote sensing targets.

Against P-CNN, G-FSDet, and TPG-FSOD, HALD-FSOD achieves competitive overall performance without introducing any additional learnable parameters. Notably, our method maintains superior base-class mAP, benefiting from the asymmetric margin-enhanced cosine classifier and loss-aware dynamic weighting mechanism, which alleviate inter-class false detections and learning imbalance, respectively.

Apart from the above qualitative analysis, we also visualize the detection results under the 10-shot setting in the first novel/base split in Figure 3. Our proposed HALD-FSOD is capable of generalized few-shot object detection, which can simultaneously detect both novel and base objects.

4.4.2. Comparisons on the NWPU VHR 10.v2 Dataset

Table 3. Few-shot object detection performance on the NWPU VHR 10.v2 test set under 3-, 5-, 10-, and 20-shot settings. The red number denotes the best results, and the blue number represents the second-best results.

Split	Method	3-Shot			5-Shot			10-Shot			20-Shot
		Base	Novel	All	Base	Novel	All	Base	Novel	All	Base	Novel	All
1	FRW [9]	83.13	15.35	62.80	82.78	16.24	62.82	83.89	24.00	65.92	82.80	27.16	66.11
	Meta-RCNN [42]	87.00	20.51	67.05	85.74	21.77	66.55	87.01	26.98	69.00	87.29	28.24	69.57
	FSDetView [41]	87.68	24.56	68.75	87.77	29.55	70.31	87.75	31.77	70.96	87.83	32.73	71.30
	P-CNN [10]	82.84	41.80	70.53	82.89	49.17	72.79	83.05	63.29	78.11	83.59	66.83	78.55
	TFA w/cos [13]	89.35	8.80	65.19	89.60	9.49	64.65	89.95	9.26	65.74	89.62	10.83	65.98
	G-FSDet [14]	89.11	49.05	77.01	88.37	56.10	78.64	87.89	69.94	82.51	88.35	73.18	83.80
	TPG-FSOD [18]	89.4	56.10	79.41	89.80	64.70	82.30	89.20	75.60	85.10	88.90	75.90	85.00
	Ours	91.10	58.20	81.23	90.00	62.20	81.66	91.50	71.20	85.41	90.80	73.10	85.48
2	FRW [9]	85.34	28.61	68.32	85.75	31.45	69.45	85.49	32.53	69.60	86.58	33.90	70.79
	Meta-RCNN [42]	86.86	21.41	67.23	87.38	35.34	71.77	87.56	37.14	72.43	87.26	39.47	72.92
	FSDetView [41]	88.11	39.01	73.38	89.34	40.31	74.63	89.34	45.09	76.07	89.31	46.28	76.40
	P-CNN [10]	81.03	39.32	68.52	81.18	46.10	70.70	80.93	55.90	73.41	81.21	58.37	75.50
	TFA w/cos [13]	90.14	11.14	66.44	91.19	12.46	67.57	90.79	11.35	66.96	90.37	11.56	66.73
	G-FSDet [14]	89.99	50.09	78.02	90.52	58.75	80.99	89.23	67.00	82.56	90.61	75.86	86.13
	TPG-FSOD [18]	90.10	48.00	77.70	89.80	63.50	81.90	89.30	69.40	83.30	90.20	75.90	85.90
	Ours	91.50	51.90	79.65	91.70	62.40	82.93	91.80	65.80	83.97	91.80	71.70	85.76

presents the performance comparison of our HALD-FSOD with SOTA methods across two splits under 3/5/10/20-shot settings. TFA still achieves the highest base-class performance among existing methods but suffers from the most severe novel-class degradation. Compared with TFA, our HALD-FSOD achieves consistent improvements across all three metrics in all settings. Specifically, in the 3-shot, 5-shot, 10-shot, and 20-shot settings, our method improves the average base-class mAP by 1.56%, 0.45%, 1.28%, and 1.31%, the average novel-class mAP by 45.08%, 51.33%, 58.20%, and 61.21%, and the average all-class mAP by 14.63%, 16.19%, 18.34% and 19.27%, respectively.

Compared with the competitive SOTA method G-FSDet, our method achieves higher all-class mAP in most settings, with improvements of 2.93%, 2.48%, 2.16% and 0.66% in the 3-shot, 5-shot, 10-shot, and 20-shot settings, respectively. It can be seen that our method has more significant advantages in low-shot scenarios, which is crucial for practical remote sensing applications where labeled samples are extremely scarce. Our method also outperforms other SOTA methods, verifying the effectiveness of our proposed three modules. Figure 4 visualizes representative 10-shot detection results on NWPU VHR 10.v2 Split 1, where the first row shows base-class predictions and the second row shows novel-class predictions.

4.4.3. Comparisons on the Model Complexity

In addition to detection performance, we analyze model complexity across the compared FSOD methods. As shown in

Table 4. Model complexity comparison between HALD-FSOD and other FSOD approaches.

Model	FRW	Meta-RCNN	FSDetView	P-CNN	TFA w/cos	G-FSDet	HALD-FSOD
#Params	66.39 M	74.95 M	79.36 M	56.53 M	60.19 M	74.08 M	60.19 M

, HALD-FSOD matches the baseline TFA in complexity while achieving superior few-shot detection performance. Furthermore, compared with other few-shot detectors (P-CNN and G-FSDet), HALD-FSOD achieves the best performance without introducing additional learnable parameters. The complexity of TPG-FSOD is not reported in Table 4, as it employs CLIP, a large-scale vision-language model, whose specific version is unspecified. Regardless of the CLIP version, TPG-FSOD involves significantly more parameters than the methods listed in Table 4.

4.5. Ablation Study

We also perform an extensive ablation analysis to better understand the effectiveness of each proposed module in our HALD-FSOD. All experiments are conducted on the DIOR and NWPU VHR 10.v2 test sets in the first novel/base split under 5-shot and 10-shot settings.

4.5.1. Ablation Study on HALD-FSOD

To quantitatively verify the effectiveness of each core module in our proposed HALD-FSOD, we conduct comprehensive ablation experiments and show the performance in

Table 5. Ablation study of each proposed component in our HALD-FSOD on DIOR and NWPU VHR 10.v2 test sets in the first novel/base split under 5-shot and 10-shot settings. The bold number denotes the best results.

Dataset	Method	HALF	AMECC	LADCWM	5-Shot			10-Shot
					Base	Novel	All	Base	Novel	All
DIOR	TFA w/cos				70.51	11.57	55.78	70.52	15.37	56.73
	HALD-FSOD	√			70.90	27.00	59.88	71.20	28.50	60.52
		√	√		71.00	27.80	60.17	70.10	34.80	61.28
		√	√	√	70.40	29.70	60.21	70.30	35.40	61.56
NWPU VHR 10.v2	TFA w/cos				89.60	9.49	64.65	89.95	9.26	65.74
	HALD-FSOD	√			89.80	56.20	79.71	91.00	65.00	83.21
		√	√		89.9	60.50	81.08	90.00	68.50	83.55
		√	√	√	90.00	62.20	81.66	91.50	71.20	85.41

. We take TFA [13] as the baseline and incrementally add the HALF, AMECC, and LADCWM to analyze their individual contributions. The baseline TFA achieves high base-class performance but suffers from extremely poor novel-class detection ability. After introducing the HALF, the novel-class mAP increases by 15.43% and 13.13% on the DIOR dataset under 5-shot and 10-shot settings, respectively, and by 46.71% and 55.74% on the NWPU VHR 10.v2 dataset. This significant improvement demonstrates that configuring differentiated learning rates for different network components effectively balances base knowledge preservation and novel feature adaptation, solving the fundamental limitation of TFA’s rigid fine-tuning strategy.

On this basis, adding the AMECC further improves the novel-class mAP by 0.80% and 6.30% on the DIOR, and by 4.30% and 3.50% on the NWPU VHR 10.v2. This verifies that enforcing explicit inter-class decision boundary constraints exclusively for novel classes effectively alleviates the inter-category false detection problem caused by the high visual similarity between remote sensing targets.

Finally, incorporating the LADCWM brings additional novel-class mAP improvements of 1.90% and 0.60% to the DIOR, and 1.70% and 2.70% to the NWPU VHR 10.v2. This shows that dynamically adjusting loss weights based on real-time learning difficulty effectively mitigates the learning imbalance between easy-to-learn base classes and hard-to-learn novel classes.

In general, throughout the incremental module addition process, the base-class performance remains stable with fluctuations within 1% and even shows slight improvements in some settings. This confirms that our method effectively avoids catastrophic forgetting while significantly enhancing novel-class detection ability. The combined effect of the three modules enables HALD-FSOD to achieve competitive performance without introducing any additional learnable parameters.

4.5.2. Ablation Study on HALF

To further investigate the optimal learning rate configuration of our proposed HALF in HALD-FSOD, we conduct detailed ablation experiments on both DIOR and NWPU VHR 10.v2 datasets under 5-shot and 10-shot settings. We fix the AMECC and LADCWM and only adjust the learning rates of three core network components: FE, RPN, and RoI-FE. The baseline is TFA [13], which only fine-tunes the box predictor with all other components frozen (LRM = 0.0). The detailed results are shown in

Table 6. Ablation study of proposed HALF on DIOR and NWPU VHR 10.v2 test sets in the first novel/base split under 5-shot and 10-shot settings. The bold number denotes the best results.

Dataset	Method	Fine-Tuned LRM			5-Shot			10-Shot
		FE	RPN	RoI-FE	Base	Novel	All	Base	Novel	All
DIOR	TFA w/cos	0.0	0.0	0.0	70.51	11.57	55.78	70.52	15.37	56.73
	HALD-FSOD	0.0	0.0	0.0	71.10	20.30	58.38	71.10	22.30	58.90
		0.0	1.0	0.0	70.70	28.60	60.19	70.30	35.20	61.55
		0.0	1.0	0.1	70.40	29.70	60.21	70.30	35.40	61.56
		0.0	1.0	0.2	70.00	30.10	60.06	69.90	36.10	61.47
		0.0	1.0	0.3	69.70	30.10	59.80	69.60	35.80	61.19
NWPU VHR 10.v2	TFA w/cos	0.0	0.0	0.0	89.60	9.49	64.65	89.95	9.26	65.74
	HALD-FSOD	0.0	0.0	0.0	91.70	29.80	73.12	91.50	33.10	73.99
		0.0	1.0	0.0	89.60	54.60	79.07	91.40	60.80	82.22
		0.0	1.0	0.1	90.00	62.20	81.66	91.50	71.20	85.41
		0.0	1.0	0.2	91.10	60.90	82.05	90.90	72.00	85.20
		0.0	1.0	0.3	89.90	61.60	81.38	90.90	72.20	85.31

.

Even when FE, RPN, and RoI-FE are completely frozen, HALD-FSOD achieves significant performance improvements over TFA. Specifically, our proposed method improves novel-class mAP by 8.73% and 6.93% on the DIOR dataset under 5-shot and 10-shot settings, respectively, and by 20.31% and 23.84% on the NWPU VHR 10.v2 dataset. When we fully fine-tune the RPN (LRM = 1.0) while keeping FE and RoI-FE frozen, the novel-class mAP increases by another 8.30% and 12.90% on the DIOR dataset under 5-shot and 10-shot settings, respectively, and by another 24.80% and 27.70% on the NWPU VHR 10.v2 dataset. This indicates that updating the RPN is crucial for generating high-quality proposals for novel classes, as the RPN is designed to identify foreground regions from a vast pool of anchors. However, during the pretraining stage, novel-class objects are excluded, so the RPN learns to treat only base-class objects as foreground. This introduces a bias toward base classes and limits its ability to generalize to unseen categories. For this reason, unfreezing the RPN proves effective in enabling it to detect novel-class objects.

On this basis, introducing a small learning rate (LRM = 0.1) for RoI-FE further achieves the best overall performance. Specifically, in the 5-shot setting, this optimal configuration improves novel-class mAP by 1.10% and 7.60% on the DIOR and the NWPU VHR 10.v2 dataset, respectively, with base-class performance remaining stable. In the 10-shot setting, the novel-class mAP improves by 0.20% and 10.40%, respectively.

However, when the RoI-FE LRM increases beyond 0.1 (to 0.2 or 0.3), the base-class performance starts to degrade significantly, while the novel-class performance shows only marginal improvements or even declines. This confirms that excessive fine-tuning of the RoI-FE will lead to catastrophic forgetting of base-class knowledge. It can be observed that using a small learning rate for RoI-FE yields limited performance gains on the large-scale DIOR dataset. In contrast, on the relatively small NWPU VHR 10.v2 dataset, a small learning rate for RoI-FE can further improve the novel-class mAP, since even a small amount of sample learning can have a significant impact when data is scarce.

In conclusion, the optimal LRM configuration for HALF is 0.0 for FE, 1.0 for RPN, and 0.1 for RoI-FE. This hierarchical strategy effectively balances base knowledge preservation and novel feature adaptation, which is the core reason for the superior performance of HALD-FSOD.

4.5.3. Hyper-Parameter Analysis on AMECC

To determine the optimal value of the asymmetric margin parameter in our proposed asymmetric margin-enhanced cosine classifier (AMECC), we conduct comprehensive hyper-parameter experiments on both the DIOR and the NWPU VHR 10.v2 datasets under 5-shot and 10-shot settings. We fix all other modules and parameters of HALD-FSOD and only adjust the margin value from 0.0 to 1.0 with a step size of 0.2. The detailed results are shown in Figure 5.

The experimental results show that the asymmetric margin parameter has a significant impact on the performance balance between base and novel classes. When the margin value increases from 0.0 to 0.2, the novel-class mAP achieves consistent improvements across all settings. Specifically, on the DIOR dataset, the novel-class mAP increases by 0.20% and 1.00% under 5-shot and 10-shot settings, respectively. On the NWPU VHR 10.v2 dataset, the improvements are more substantial, reaching 4.20% and 1.30%, respectively. Meanwhile, the base-class performance remains stable.

When the margin value exceeds 0.2, the performance starts to degrade gradually. As the margin increases from 0.2 to 1.0, the base-class mAP shows a continuous downward trend in most settings, with a maximum decrease of 1.90% on the DIOR dataset. More importantly, the novel-class mAP also begins to decline after reaching the peak at a margin = 0.2. Particularly on the NWPU VHR 10.v2 dataset, when the margin increases to 1.0, the novel-class mAP drops sharply by 4.60% in the 5-shot setting and 8.00% in the 10-shot setting.

This phenomenon can be explained as follows: base classes have well-clustered feature distributions because of training with abundant samples in the pretraining stage, while novel classes, with only a few samples, suffer from loose and scattered feature distributions. This imbalance causes the native symmetric decision boundary to be biased toward the novel-class side, leading to frequent misclassification of novel-class features into base classes. The introduced margin $\tau$ explicitly shifts the decision boundary toward the base-class side. For a sample to be classified into a novel class, its cosine similarity to the novel-class weight must exceed the similarity to base-class weights by a certain value. Geometrically, this widens the decision region of novel classes and adds a hard constraint on inter-class separability, which directly alleviates the false detection problem caused by insufficient discriminability of novel-class features. An appropriate asymmetric margin effectively enforces explicit inter-class decision boundary constraints exclusively for novel classes, which alleviates the inter-category false detection problem caused by high visual similarity between remote sensing targets. However, an excessively large margin will lead to over-constraint of novel-class features, compressing the feature space of novel classes and reducing their generalization ability. At the same time, it will also cause the model to forget the learned base-class knowledge, resulting in significant performance degradation of base classes.

In conclusion, the optimal value of the asymmetric margin parameter is 0.2. This setting achieves the best balance between base-class knowledge preservation and novel-class discrimination, which is consistent across both datasets and different few-shot settings.

4.5.4. Hyper-Parameter Analysis on LADCWM

To investigate the impact of the moment parameter β in our proposed LADCWM, we conduct comprehensive hyper-parameter experiments on both the DIOR and the NWPU VHR 10.v2 datasets under 5-shot and 10-shot settings. We fix all other modules and parameters of HALD-FSOD and only adjust the β value from 1.0 to 0.0 with a step size of 0.1. The β parameter controls the weight ratio between base classes and novel classes in the total loss function, where β = 1.0 represents equal weighting for all classes (equivalent to disabling the LADCWM). The detailed results are shown in Figure 6.

The experimental results show that introducing the LADCWM consistently improves the novel-class detection performance across all settings. When β decreases from 1.0 to 0.9, the novel-class mAP achieves the most significant single-step improvement. Specifically, on the DIOR dataset, the novel-class mAP increases by 1.90% and 0.60% under 5-shot and 10-shot settings, respectively. On the NWPU VHR 10.v2 dataset, the improvements are even more substantial, reaching 1.70% and 2.70%, respectively. This demonstrates that appropriately increasing the loss weight of novel classes can effectively alleviate the learning imbalance problem caused by the extreme sample scarcity of novel classes.

As β continues to decrease from 0.9 to 0.0, the novel-class performance remains at a high level with small fluctuations, while the base-class performance stays remarkably stable with variations within 0.5% across all settings. The optimal β value varies slightly across different experimental settings: on the DIOR dataset, the highest novel-class mAP is achieved at β = 0.7 in the 5-shot setting and at β = 0.3 in the 10-shot setting; on the NWPU VHR 10.v2 dataset, the highest novel-class mAP is achieved at β = 0.4 in the 5-shot setting and at β = 0.0 and β = 0.3 in the 10-shot setting.

Notably, the performance differences between β values in the range of 0.0–0.9 are minimal. This indicates that our LADCWM module is highly robust to the choice of the β parameter within a wide range. Considering the comprehensive performance across all datasets and settings, we recommend β = 0.9 as the default value. Although it is not the absolute best in every individual setting, it achieves the most balanced performance: it brings the most significant initial performance improvement while maintaining almost identical base-class performance compared to the baseline without LADCWM. Moreover, this value ensures excellent generalization ability across different datasets and few-shot scenarios, making it the most practical choice for real-world applications.

5. Discussion

5.1. Comparison with Previous Studies and Interpretation of Results

This section interprets our experimental results in the context of existing FSOD methods, explaining the underlying reasons for the performance differences and validating our working hypotheses.

Compared with the baseline TFA [13], HALD-FSOD achieves significant improvement in novel-class mAP while maintaining or even enhancing base-class performance across all settings. This significant improvement directly addresses the fundamental limitation of TFA’s rigid fine-tuning strategy, which only updates the box predictor while freezing all other network components. As demonstrated in our ablation study, TFA’s approach fails to generate high-quality proposals for novel classes and cannot adapt feature representations to remote sensing-specific target characteristics. Our HALF solves this problem by fully fine-tuning the RPN and applying a small learning rate to the RoI-FE, enabling the model to learn novel-class features without catastrophic forgetting of base knowledge.

In contrast to meta-learning-based methods such as FRW [9], Meta-RCNN [42], and FSDetView [41], our transfer learning-based pipeline achieves superior performance with substantially lower computational cost. Meta-learning methods rely on computationally expensive episodic training regimes that require thousands of training episodes, making them impractical for large-scale remote sensing datasets. Moreover, these methods often suffer from unstable training and poor generalization to complex remote sensing targets with high intra-class variance and arbitrary orientations. Our results demonstrate that optimizing the training strategy and loss function of pre-trained models is a more practical and effective approach for remote sensing FSOD, aligning with recent findings in the computer vision community that transfer learning outperforms meta-learning in most real-world scenarios.

Compared with other state-of-the-art transfer learning-based methods, including P-CNN [10], G-FSDet [14], and TPG-FSOD [18], HALD-FSOD shows particularly significant advantages in low-shot scenarios. This superiority can be attributed to two key innovations: the AMECC and the LADCWM. The AMECC explicitly enforces inter-class decision boundaries exclusively for novel classes, effectively reducing false detections between visually similar remote sensing targets. The LADCWM further ensures that the model focuses more on learning novel classes with limited samples, dynamically adjusting loss weights based on real-time learning difficulty. These two modules work synergistically to address the unique challenges of remote sensing imagery, where target classes often exhibit high visual similarity and extreme sample imbalance.

5.2. Broader Implications of the Research

The findings of this study have several important implications for the broader field of remote sensing image analysis and few-shot learning.

Firstly, our results challenge the common belief that introducing additional learnable parameters is necessary to improve FSOD performance. We demonstrate that careful design of training strategies and loss functions can achieve competitive performance without increasing model complexity. This parameter efficiency is particularly valuable for remote sensing applications, where computational resources are often limited, and models need to be deployed on edge devices with constrained processing capabilities. Our approach provides a blueprint for developing lightweight yet effective FSOD methods that can be easily integrated into existing remote sensing processing pipelines.

Secondly, we empirically validate that the hierarchical fine-tuning paradigm with differentiated learning rates provides a highly effective general framework for transferring pre-trained models to specialized remote sensing tasks under scarce annotations. Through comprehensive ablation experiments, we show that freezing the FE (LRM = 0.0), fully fine-tuning the RPN (LRM = 1.0), and applying a small learning rate (LRM = 0.1) to the RoI-FE achieve the optimal balance between base knowledge preservation and novel feature adaptation. This configuration can be readily extended to various remote sensing tasks beyond object detection, including semantic segmentation, change detection, and object tracking, where labeled data is often difficult and expensive to obtain. Our results suggest that this hierarchical fine-tuning strategy offers a promising alternative to computationally expensive meta-learning approaches for a wide range of remote sensing applications with limited annotations.

Finally, our work highlights the critical importance of addressing the unique characteristics of remote sensing data when developing few-shot learning methods. Remote sensing imagery presents several distinct challenges compared to natural images, including high visual similarity between classes, extreme class imbalance, arbitrary target orientations, and large variations in scale and resolution. In this study, we specifically address two of the most pressing challenges: high inter-class visual similarity and severe class imbalance. The proposed AMECC explicitly enforces wider decision boundaries for novel classes, effectively reducing false detections between visually similar remote sensing targets. The LADCWM dynamically adjusts the contribution of each class to the total loss based on its learning difficulty, mitigating the problem where abundant base classes dominate the training process at the expense of scarce novel classes. The effectiveness of these two domain-specific designs demonstrates that incorporating targeted solutions for remote sensing data characteristics is crucial for achieving optimal performance in few-shot detection tasks. This insight can guide the development of future few-shot learning methods for other remote sensing tasks with similar data challenges, such as small target detection, multi-temporal image analysis, and cross-domain transfer learning.

5.3. Limitations

Despite the promising results, our study has several limitations that need to be acknowledged and addressed in future research.

Firstly, the current method was only evaluated on horizontal object detection tasks. However, most remote sensing targets have arbitrary orientations, and oriented object detection is more suitable for practical applications such as ship detection, vehicle detection, and infrastructure monitoring. Extending HALD-FSOD to support oriented object detection remains an important future work. This will involve modifying the RPN and box predictor to handle oriented bounding boxes and adapting the asymmetric margin classifier to account for orientation variations between targets.

Secondly, the current method was tested on two widely used public datasets. While these datasets are standard benchmarks for remote sensing object detection, they do not fully represent the diversity of real-world remote sensing data. Further validation on more diverse and challenging datasets with different resolutions, sensors, geographical regions, and target types is needed to fully demonstrate the generalization ability of HALD-FSOD. In particular, testing on datasets with complex backgrounds, occlusions, and atmospheric interference would provide a more comprehensive evaluation of the method’s robustness.

Finally, our current implementation does not explicitly address the problem of domain shift between base and novel classes. In real-world applications, novel classes may come from different geographical regions or be captured by different sensors than the base classes, leading to significant domain differences. Developing domain adaptation techniques that can be integrated into our HALD-FSOD framework to handle cross-domain few-shot detection is another important area for future investigation.

6. Conclusions

In this paper, we propose HALD-FSOD, a hierarchical adaptive learning framework with asymmetric margin and loss-aware dynamic weighting for few-shot object detection in remote sensing imagery. Our method addresses three critical limitations of existing transfer learning-based few-shot detectors: rigid fine-tuning strategies that fail to balance base knowledge preservation and novel feature adaptation, insufficient inter-class discrimination for visually similar remote sensing targets, and severe learning imbalance between abundant base classes and scarce novel classes. Extensive experiments on two widely used remote sensing datasets (DIOR and NWPU VHR 10.v2) demonstrate that HALD-FSOD achieves competitive performance across all few-shot settings.

For future work, we will focus on several promising directions based on the limitations identified in this study. First and foremost, we will extend HALD-FSOD to support oriented object detection by modifying the RPN and box predictor to handle oriented bounding boxes and adapting the asymmetric margin classifier to account for orientation variations. Secondly, we will develop domain adaptation techniques that can be integrated into our framework to handle cross-domain few-shot detection scenarios where novel classes come from different geographical regions or are captured by different sensors. Finally, we will validate our method on more diverse and challenging remote sensing datasets and apply it to real-world applications such as disaster response and environmental monitoring.