MSDR-Net: Multiscale Dynamic Reasoning for Multi-Label Remote Sensing Image Classification

Highlights

What are the main findings?

• The proposed unified framework integrating multiscale feature representation and semantic reasoning achieves 95.88% mAP on the DIOR dataset.
• The difficulty-weighted loss improves recognition performance for long-tail and small-scale categories by incorporating category frequency and sample difficulty.

What are the implications of the main findings?

• Joint modeling of multiscale features and semantic dependencies is essential for multi-label classification in complex remote sensing scenarios.
• Task-specific loss reweighting effectively mitigates class imbalance and improves performance on underrepresented categories.

Abstract

With the rapid advancement of Earth observation technologies and the growing demand for intelligent remote sensing applications, high-resolution remote sensing imagery provides critical data support for a range of downstream applications, including land monitoring and disaster assessment. In this context, multi-label remote sensing image classification has become an important research task, because a single image may contain multiple ground-object categories with complex spatial distributions and semantic co-occurrence relationships. However, challenges such as the coexistence of multiscale objects, complex semantic dependencies, and long-tail category distributions impose significant limitations on existing methods in terms of feature representation capacity and class-balanced modeling. To address these challenges, a Multiscale Dynamic Reasoning Network (MSDR-Net) is proposed. Different from methods that focus on localized optimization for a single challenge, MSDR-Net establishes a task-driven modeling framework that jointly integrates multiscale feature extraction, label-aware semantic reasoning, and long-tail category optimization within an end-to-end architecture. The proposed network consists of three core modules. The Multiscale Feature Enhancement (MSFE) module incorporates a Feature Pyramid Network-based fusion mechanism, integrating deep semantic information with shallow, detailed features to effectively enhance the representation of multiscale objects. The Dynamic Semantic Reasoning (DSR) module introduces a Transformer-based global attention mechanism that models long-range dependencies among image features, enabling the capture of complex global semantic relationships. In the loss optimization stage, a Difficulty-Weighted Loss (DW-Loss) is introduced, which jointly incorporates category frequency weights and prior difficulty coefficients to dynamically regulate the contributions of rare classes and hard samples during training, thereby mitigating bias induced by class imbalance. Experiments conducted on the large-scale Detection in Optical Remote Sensing Images dataset demonstrate that the proposed method achieves superior performance. Ablation studies validate the effectiveness of each component, while comparative experiments indicate that MSDR-Net achieves a mean Average Precision of 95.88%, outperforming existing state-of-the-art methods. An improvement of approximately 1.74% is observed over the strongest baseline, MSCA, with consistent advantages demonstrated across Overall F1 and Class-wise F1 metrics. By unifying multiscale feature extraction, global semantic reasoning, and balanced loss optimization within a single framework, MSDR-Net provides a robust and efficient solution for multi-label classification in complex remote sensing scenarios.

1. Introduction

With the rapid advancement of remote sensing technologies and satellite sensors, high-resolution remote sensing imagery has become a fundamental data source for global land cover monitoring, urban management, disaster assessment, and ecological environment analysis [1,2]. The authoritative international report Earth Intelligence for All highlights that future intelligent analysis of remote sensing imagery requires substantial improvements in understanding complex semantic scenes and in information-extraction capabilities to support sustainable development and refined governance. High-resolution remote sensing imagery often captures complex scenes featuring coexisting multiscale and multi-category objects, rendering traditional single-label classification methods inadequate for practical applications. Consequently, multi-label remote sensing image classification has emerged as an important research direction in the intelligent interpretation of remote sensing imagery [3]. By enabling the identification of multiple semantic categories within a single image, the accuracy of scene understanding can be improved, while providing critical support for downstream tasks such as change detection, object recognition, and land cover analysis.

However, multi-label remote sensing image classification still faces two fundamental challenges: (i) the representation of multiscale objects struggles to capture local details and global semantics simultaneously [4]; and (ii) long-tail category distributions hinder the effective learning of low-frequency classes [5,6]. Although existing studies have attempted to alleviate these issues through deep networks, attention mechanisms, or data augmentation, most methods remain focused on addressing challenges along a single dimension. As a result, a unified solution capable of jointly optimizing multiscale feature extraction, long-tail sample balancing, and complex semantic dependency modeling within an end-to-end framework is still lacking [4,6]. Although notable progress has been achieved in feature fusion, semantic modeling, and long-tail optimization, these approaches are still predominantly designed to tackle isolated challenges, and an integrated framework that simultaneously addresses multiscale feature extraction, long-tail sample balancing, and complex semantic dependency modeling remains absent [5,7]. This limitation constitutes the primary bottleneck that the present study aims to address.

The Multiscale Dynamic Reasoning Network (MSDR-Net), an end-to-end multi-label classification network, is proposed to address the challenges posed by multiscale objects, complex semantic dependencies, and long-tail category distributions in remote sensing images. A task-driven unified modeling framework is established for multi-label remote sensing image classification, integrating multiscale feature enhancement, label-aware dynamic semantic reasoning, and difficulty-weighted loss optimization. Consequently, scale variations, semantic dependencies, and long-tail category distributions can be jointly addressed within a unified end-to-end architecture. The proposed network consists of three core modules. First, the Multiscale Feature Enhancement (MSFE) module is constructed using ResNet-34 and a Feature Pyramid Network (FPN) fusion. Deep features are stabilized via residual representation learning, while multi-branch convolution captures both local and global semantic information across multiple scales.

Furthermore, a top-down cross-layer fusion mechanism is adopted to jointly model high-level semantics and low-level details, thereby significantly enhancing the representation of small-scale, fine-grained objects. Second, the Dynamic Semantic Reasoning (DSR) module is built on a Transformer encoder, which incorporates two-dimensional positional encoding and label embeddings. Feature dependencies within the image are adaptively modeled via a multi-head attention mechanism, while cross-layer multilayer perceptrons (MLPs) and Dropout are employed to improve multiscale feature representation and training stability. As a result, the modeling of multi-label semantic dependencies in complex scenarios is effectively enhanced. Finally, to address the long-tail distribution and hard-sample challenges commonly observed in remote sensing data, a Difficulty-Weighted Loss (DW-Loss) is proposed. Category frequency weights and prior difficulty coefficients are jointly incorporated to dynamically regulate the loss contributions of rare classes and hard samples during training, thereby enhancing the model’s focus on underrepresented and challenging categories. Based on the aforementioned modules, MSDR-Net achieves synergistic modeling of multiscale representation learning, label-aware semantic reasoning, and long-tail category optimization. Experimental results demonstrate that, while maintaining high overall classification accuracy, MSDR-Net significantly improves recognition robustness for complex scenes, small-scale targets, and rare categories, thereby validating the effectiveness of the proposed task-driven unified modeling framework.

The main contributions of this study are summarized as follows:

• The MSFE module is proposed, which jointly models deep semantic information and shallow, detailed features to effectively capture multiscale object characteristics in remote sensing imagery, thereby improving the representation of small-scale and complex targets.
• The DSR module is designed to adaptively model multi-label semantic dependencies based on a Transformer encoder and a multi-head attention mechanism, enabling efficient integration of global semantic information and local details in complex scenarios; and
• DW-Loss is introduced within an end-to-end training framework, termed MSDR-Net, in which the loss contributions of long-tail categories and hard samples are dynamically regulated to enable collaborative optimization across multiple modules, thereby significantly enhancing the classification robustness of rare categories and small-scale objects.

2. Related Work

Multi-label remote sensing image classification is primarily challenged by the diversity of land-cover categories, significant scale variations, long-tail distributions of categories, and latent semantic dependencies among labels. In complex scenarios, Multi-Label Remote Sensing Image Classification (MLRIC) continues to face two fundamental issues: the representation of multiscale objects and label imbalance under long-tail distributions.

From a multiscale feature representation perspective, remote sensing imagery exhibits wide spatial coverage and complex viewing angles, where large-scale structures and small objects often coexist within a single image. This substantial scale variation imposes higher demands on feature representation. To address this issue, Tan and Le proposed a compound scaling strategy in EfficientNet to enhance cross-scale feature modeling capability; however, the computational complexity remains relatively high [8]. Subsequently, Zhu et al. introduced deformable attention in Deformable DETR, enabling flexible multiscale alignment through dynamic sampling across feature scales [9]. In the remote sensing domain, li et al. presented a spatial-topological-semantic alignment paradigm to enhance domain adaptability for few-label cross-domain scene classification [10]. Pandey et al. explored a joint super-resolution and multi-label classification framework for remote sensing images, demonstrating that preserving high-resolution spatial details is beneficial for multi-label recognition in low-resolution satellite imagery [11]. More recently, Zhao et al. developed a multiscale sparse cross-attention network, in which sparse attention mechanisms facilitate the integration of local details and global context, leading to significant performance improvements in complex scenarios [12]. Nevertheless, it has been reported that repeated downsampling in deep convolutional networks leads to substantial resolution degradation, weakening the representation of small objects during feature fusion and imposing inherent limitations on multiscale modeling [13].

From a semantic dependency modeling perspective, multi-label remote sensing imagery typically exhibits pronounced semantic co-occurrence patterns. To capture such relationships, hierarchical semantic structures have been explored to model inter-category dependencies. For instance, Zhang et al. proposed a hierarchical knowledge graph-based approach that models multi-level semantic relationships to enhance understanding of multiscale objects [14]. Meanwhile, studies in general computer vision have demonstrated that Transformer-based architectures are highly effective in modeling complex semantic dependencies. Carion et al. established inter-object relationship modeling within the Transformer-based DETR framework, providing an effective paradigm for multi-semantic relationship learning [15]. Subsequently, Dosovitskiy et al. demonstrated that the Vision Transformer can effectively capture long-range semantic dependencies [16]. Inspired by these advances, Transformer-based interaction mechanisms have been introduced into remote sensing methods. Ou et al. proposed a view–category interactive sharing mechanism that jointly models multi-view information and category-level semantic relationships, effectively alleviating label dependency issues under incomplete annotation conditions [17]. Cao et al. introduced the pioneering CLIP-Mamba framework, integrating a pre-trained Vision-Language Model (CLIP) with a State Space Model (Mamba) for efficient and comprehensive feature fusion and semantic extraction [18]. In addition, Xia et al. proposed a latent semantic dependency model that jointly infers explicit and implicit label relationships to improve classification performance; however, its applicability in complex scenarios is constrained by limited feature representation stability and high computational cost [19]. Moreover, due to the inherent deep downsampling process in remote sensing imagery, small-object information is often lost, making it difficult for conventional Transformer architectures to achieve full-scale perception from local details to global semantics.

From the perspective of the long-tail distribution, large-scale remote sensing datasets commonly exhibit severe class imbalance, with low-frequency categories underrepresented and difficult to learn effectively. It has been demonstrated that such an imbalance leads to a pronounced bias toward high-frequency categories, thereby degrading generalization performance on rare classes [2]. To mitigate this issue, Cui et al. proposed a class-balanced loss based on the effective number of samples, in which class weights are redefined to counteract frequency imbalance; however, it remains insufficient to simultaneously address sample difficulty and distribution disparities in remote sensing scenarios [20]. Wang et al. introduced a diffusion-based noise augmentation strategy to improve tail-class performance, while Du et al. proposed a category-selective feature enhancement mechanism to enhance tail-class responses adaptively [21,22]. Wang et al. proposed DMRS, a foundation-model-based framework for long-tailed remote sensing scene recognition, further demonstrating the importance of robust representation learning for imbalanced remote sensing data [22]. In the broader computer vision domain, robust loss function designs, such as asymmetric loss and distributionally robust loss, have provided theoretical support for addressing long-tail learning [23,24]. In addition, Zhang et al. proposed a unified learning framework to address multi-label classification under long-tailed distributions and partial-label conditions [25]. Nevertheless, directly transferring these general approaches to remote sensing scenarios remains challenging, as long-tail distributions are often coupled with multiscale variations. Consequently, relying solely on loss optimization or data augmentation is insufficient to address the complexity of jointly modeling multiscale features and long-tail distributions.

Despite substantial progress in multiscale feature representation, semantic dependency modeling, and long-tail learning, most existing methods address these challenges separately. In complex multi-label remote sensing scenes, however, these issues are often coupled: small objects require fine-grained spatial details, label prediction depends on global semantic context and inter-class correlations, and rare categories are more likely to be suppressed during training. Therefore, a unified framework is required to coordinate feature representation, semantic reasoning, and class-balanced optimization within an end-to-end learning process.

Compared with existing approaches, MSDR-Net is designed to address these coupled challenges collaboratively. The MSFE module enhances multiscale spatial representation by integrating shallow details and deep semantic features. The DSR module further models long-range spatial dependencies and inter-class semantic relationships through Transformer-based reasoning. The DW-Loss introduces category-frequency weighting and prior difficulty coefficients to improve learning of rare and difficult categories. In this way, MSDR-Net does not simply stack existing modules, but coordinates feature extraction, semantic reasoning, and loss optimization for multi-label remote sensing image classification.

3. Methods

A multiscale feature extraction framework is constructed by integrating residual learning, multiscale convolution, and an FPN. Dynamic semantic reasoning is further achieved by incorporating label embedding and Transformer-based modeling. In addition, DW-Loss is proposed to effectively alleviate class imbalance and hard-sample issues, thereby improving the performance of multi-label remote sensing image classification, as shown in Figure 1.

To provide a clear description of the computational workflow and logical architecture of the proposed method, Algorithm 1 summarizes the overall training and inference procedures of MSDR-Net.

Algorithm 1: The Proposed MSDR-Net Framework

Input: Remote Sensing Image $RSI\in {\mathbb{R}}^{3\times H\times W}$, Ground truth labels y. Output: Predicted probabilities $\widehat{y}$, Total Loss LDW.

1  Feature Extraction and Fusion: Extract hierarchical features from RSI via ResNet34 and fuse them using FPN-PAN to yield the enhanced feature map Hi 2  Sequence Generation: Flatten Hi and incorporate Positional Encoding and label embeddings to construct the input sequence GIN 3  Global Reasoning: for l = 1 to L (Transformer Layers) do 4                       Iteratively update the sequence representations via Multi-Head Self-Attention and MLP blocks. 5                       end for 6  Classification: Aggregate global features and derive prediction probabilities $\widehat{y}$ through a linear projection layer. 7  Optimization: Compute the Dynamic Weighting Loss LDW based on label difficulty and update network parameters via backpropagation.

3.1. Multiscale Feature Enhancement Module

3.1.1. Residual Representation Learning

Deep convolutional networks exhibit significant advantages in the high-level semantic representation of remote sensing imagery. However, as network depth increases, learning complete mappings solely through stacked convolutions can lead to vanishing gradients, optimization difficulties, and feature degradation. To ensure stable gradient propagation in deep architectures, a residual learning mechanism is introduced that employs identity mappings to facilitate the training of deep nonlinear transformations, as illustrated in Figure 2.

The input remote sensing image is represented as a batch tensor. The residual structure consists of multiple convolutional layers and activation functions, and the overall residual transformation can be formulated as a functional mapping

$$R_{\mathrm{n}}(RSI;W_R)$$

(1)

where n denotes the number of residual units, and $W_R$ represents the corresponding parameter set.

By introducing an identity shortcut connection, the output of the residual module can be expressed as

$$F_1=RSI+R_n(RSI;W_R)$$

(2)

The advantage of this structure lies in the fact that, when the optimal mapping approaches an identity function, the residual term. $R_n(·)$ only needs to be driven toward zero, thereby significantly reducing the optimization difficulty of deep architectures. Meanwhile, the identity shortcut ensures that low-level spatial structures and texture information are preserved and directly propagated to deeper semantic representations, which is particularly critical for complex high-resolution remote sensing scenarios where local objects and global backgrounds coexist.

3.1.2. Multiscale Convolutional Feature Extraction

Remote sensing imagery exhibits significant multiscale characteristics, encompassing rich information ranging from local details to global structural patterns. To overcome the limitations of single-scale feature representation, a hierarchical multiscale feature extraction strategy is proposed. The proposed strategy exploits the hierarchical representation capabilities of convolutional neural networks (CNNs) to model land-cover features across different spatial scales explicitly.

As illustrated in Figure 3, the network is composed of multiple sequential convolutional stages. The $k$-th stage, denoted as $M_k(·)$, is responsible for extracting feature representations at a specific semantic level. Let the input to the current stage be the output feature from the previous stage. The feature extraction process at the $k$-th stage can then be formulated as:

$$F_k=M_k(F_{k-1};W_k)$$

(3)

where $W_k$ denotes the learnable parameters of the $k$-th stage, and $F_{k-1}$ represents the input feature map. For the first stage, the input corresponds to the original RSI. The output $F_k$ represents the deep feature representation extracted at the corresponding stage. Through this cascaded architecture, multiple convolutional stages jointly form a hierarchical multiscale semantic extraction sequence from shallow to deep representations:

$$F_1=M_1(RSI),F_2=M_2(F_1),F_n=M_n(F_{n-1})$$

(4)

As the network depth increases, the receptive field is progressively enlarged, enabling the extraction of hierarchical dependency features ranging from local textures and geometric boundaries to high-level regional semantic structures. The final feature representation $F_n$ serves as a comprehensive, deep, multiscale semantic descriptor and is subsequently fed into the downstream decision network for efficient modeling of complex remote sensing scene structures.

3.1.3. Multiscale Feature Pyramid Fusion and Fixed-Scale Representation

Deep convolutional features exhibit significant differences in semantic richness and spatial resolution across different layers. Shallow features preserve rich spatial details but lack strong semantic discriminability, whereas deep features contain more discriminative semantic information but suffer from reduced spatial resolution. This cross-layer semantic gap limits the effective integration of multiscale information. To address this issue, a FPN-based fusion mechanism is introduced that achieves multiscale semantic enhancement via top-down information propagation. Furthermore, a fixed-scale feature representation is constructed to facilitate subsequent decision-making modules.

As illustrated in Figure 4, the constructed feature pyramid adopts a bidirectional fusion architecture that combines top-down and bottom-up pathways, enabling the joint modeling of cross-layer semantic and spatial information. In the top-down pathway, high-level features are progressively upsampled and fused with corresponding low-level features via lateral connections, resulting in feature representations with both high spatial resolution and strong semantic expressiveness. Let the feature map at the $l$-th layer of the backbone be denoted as $F_l$. The feature alignment, top-down fusion, and smoothing operations are defined as:

$$P_l=\phi ({\varphi }_l(F_l)+U(P_{l+1}))$$

(5)

where ${\varnothing }_l(·)$ denotes the feature embedding function for achieving isomorphic representation across feature levels; $U(·)$ represents the upsampling operator for spatial alignment; and $\phi (·)$ denotes a smoothing operation to mitigate aliasing effects introduced by upsampling.

Subsequently, to enhance the contribution of low-level high-resolution features, a bottom-up PAN is introduced. In the bottom-up pathway, fused low-level features are progressively downsampled and propagated to higher layers, where they are combined with corresponding high-level features through element-wise addition and convolutional smoothing, which can be recursively formulated as:

$$N_{l+1}={\varphi }^{PAN}(\stackrel{\wedge }{P_{l+1}}+D(N_l))$$

(6)

where $D_1(·)$ denotes the downsampling operator, and ${\varnothing }_{\mathrm{PAN}}(·)$ represents the convolutional smoothing operation after bottom-up fusion. This bidirectional fusion strategy effectively enhances spatial perception capability, enabling the resulting features to preserve fine-grained spatial details while maintaining strong semantic representations.

From the multiscale feature set, an optimal level $l^{\ast }$ is selected as the output representation. The fused feature map is partitioned into subregions. $\{R_i{\}}_{i=1}^K$, followed by normalization and nonlinear transformation to obtain the final feature representation, which can be formulated as:

$$H_i=\sigma (N({\displaystyle \frac{1}{\left|R_i\right|}}{\displaystyle \sum _{x\in R_i}{P}^{\ast }({\widehat{N}}_{l^{*}})}))$$

(7)

$$H=Flatten(H_i)$$

(8)

The resulting feature representation preserves both spatial structural information and semantic content, providing stable and informative inputs for downstream classification or detection tasks.

3.2. Dynamic Semantic Reasoning Module

3.2.1. Label Indexing System and Positional Encoding

As illustrated in Figure 5, After visual feature extraction and initialization, a label-indexing system is constructed to support supervised learning for multi-label classification.

• Category index generation: Let the DIOR dataset contain $M$ object categories (e.g., airplanes, ships, and vehicles). A set of unique category indices $i=\{\mathrm{0,1},\dots ,M-1\}$ is first defined, and a mapping from category names to integer indices is established.
• Index set expansion and alignment: To achieve dimensional alignment between label information and visual features, the category index set is replicated to construct an index sequence of length N, the number of feature locations.
• Label embedding mapping: For each input sample, the corresponding category indices derived from ground-truth annotations are mapped to high-dimensional label embedding vectors, forming the label representation space.

Mathematically, let the predicted category at the k-th feature location be denoted as $y_k$. The corresponding label embedding can be expressed as:

$$E_{label}=W_{emb}\left[y_k\right]$$

(9)

where $W_{\mathrm{emb}}\in {\mathbb{R}}^{M\times D}$ denotes a learnable category embedding matrix. This initialization strategy ensures that each feature location is associated with explicit semantic priors at the early stage of training, enabling rapid alignment between visual features and category labels, thereby accelerating convergence and improving classification performance.

Within the Transformer framework, the self-attention mechanism processes each feature by attending to all other positions, thereby enabling the effective modeling of global feature dependencies [14]. However, this property implies the absence of an inherent sequential processing mechanism, preventing the direct encoding of absolute positional information for input features.

$$WD=\exp \left(-{\displaystyle \frac{f}{D_{\mathrm{mod}el}/2}}·\log \left(10,000\right)\right) , f=1,\cdots ,D_{\mathrm{mod}el}/2$$

(10)

$$P_{PH\times PV}=\left\{\begin{array}{c}\sin \left(posH·WD\right)\\ \cos \left(posH·WD\right)\\ \sin \left(posV·WD\right)\\ \cos \left(posV·WD\right)\end{array}\right.$$

(11)

where $f$ denotes the index of the frequency term, $W$ and $H$ represent the spatial width and height of the feature map output by the ResNet backbone, and $D_{model}$ denotes the feature dimensionality. The feature representation augmented with positional encoding is concatenated with the embedded label indices. $E_{\mathrm{label}}$ to construct a label-aware feature representation, which serves as the input tensor to the Transformer encoder. The formulation is defined as follows:

$$G_{IN}=H+P_{PH\times PV}+E_{label}$$

(12)

3.2.2. Multi-Head Attention Mechanism

As illustrated in Figure 4, to enhance the model’s learning capacity and representation depth, a Multi-Head Attention mechanism with $H$ attention heads is adopted and stacked across $L$ layers within the Transformer architecture. Through this design, input features are jointly processed in parallel and sequential manners, enabling comprehensive feature refinement. In addition, a Dropout mechanism is incorporated to mitigate overfitting and improve model robustness. The computation process of the multi-head attention mechanism is defined as follows:

$$Q_i,K_i,V_i=G_{IN}{W}_{Qi},G_{IN}{W}_{Ki},G_{IN}{W}_{Vi} , i=1,\cdots ,I$$

(13)

$$h_i=Attention\left(Q,K,V\right)=soft\max \left({\displaystyle \frac{Q_i{K}_j^T}{\sqrt{Z}}}\right)V_j , j=1,\cdots ,I$$

(14)

$$\begin{array}{l}{H}_k=D\mathrm{ropout}\left(MultiHead\left(Q_i,K_i,V_i\right)\right)\\ =D\mathrm{ropout}\left(Concat\left(h_1,\cdots ,h_i,\cdots ,h_I\right)W_C\right)\end{array} \begin{array}{c}k=1,\cdots ,K\end{array}$$

(15)

where $W_Q,W_K,W_V$ denote the learnable projection matrices that map the input features into query, key, and value representations, respectively; ${\displaystyle \frac{1}{\sqrt{\mathrm{Z}}}}$ is a scaling factor introduced to stabilize gradients across different feature dimensions; $W_c$ represents the output projection matrix used to aggregate multiple attention heads; ${\mathrm{head}}_i$ denotes the $i$-th attention head; and $H_k$ denotes the output of the Multi-Head Attention module.

Between the stacked attention layers, an MLP block and a Dropout mechanism are incorporated to enhance further the Multi-Head Attention module’s ability to capture multiscale information. This design improves the representational capacity, training stability, and the joint modeling of local and global dependencies. The output after the MLP block and Dropout operation is defined as follows:

$$\begin{array}{c}{H}_{IN{k}_1}\\ \end{array}\begin{array}{c}=\\ =\end{array}\begin{array}{c}Dropout\left(MLP\left(H_k\right)W_P+b_1\right)\\ Dropout\left(\mathrm{Re}LU\left(H_k+b_0\right)W_P+b_1\right)\end{array}\begin{array}{c} , k_1=2,\cdots ,K\end{array}$$

(16)

where $W_{\mathrm{p}}$ denotes the weight matrix used for inter-layer transformation, and $b_1$ and $b_2$ represent the corresponding bias terms.

3.3. Supervised Learning and Loss Optimization

3.3.1. Probability Prediction

The label-related components of the Transformer encoder output tensor are extracted via an index selection module, after which logits are computed and transformed through a Sigmoid function to obtain probability values corresponding to each category. The logit transformation and Sigmoid function are defined as follows:

$$\left\{\begin{array}{l}L{g}_l=Logit\left(G_{OUT}\right)=W_O\cdot G_{OUT}+b_2\\ {\widehat{y}}_i=sigmoid(L{g}_l)={\displaystyle \frac{1}{1+e^{-L{g}_l}}}\end{array}\begin{array}{c}, l=1,\cdots ,L\end{array}\right.$$

(17)

where $W_o$ and $b_2$ denote the weight matrix and bias term of the logit transformation, respectively, and ${\widehat{y}}_i$ represents the predicted probability vector. A binary vector with a length equal to the number of categories is constructed, in which the positions corresponding to the ground truth labels are set to 1, forming the multi-label ground truth vector. Since the predicted probabilities ${\widehat{y}}_i$ lie within the range $\left[\mathrm{0,1}\right]$ aligning the output structure with the ground truth vector enables each element to directly represent the probability of its corresponding category.

3.3.2. Difficulty-Weighted Loss

To mitigate class imbalance and improve learning of hard samples, a DW-Loss is proposed based on the Binary Cross-Entropy (BCE) loss. This loss function dynamically adjusts the contributions of different categories during optimization by jointly incorporating category-frequency weights and prior-difficulty coefficients. The DW-Loss is formulated as:

$$L_{DW}=-{\displaystyle \frac{1}{N}}{\displaystyle {\sum }_{i=1}^N\left[w_{freq}^{(c)}·{\alpha }_{hard}^{\left(c\right)}·y_i\log ({\widehat{y}}_i)+(1-y_i)\log (1-{\widehat{y}}_i)\right]}$$

(18)

where $N$ denotes the total number of samples, and $y_i$ and ${\widehat{y}}_i$ represent the ground truth label and predicted probability of the i-th sample, respectively. For positive samples, a composite weighting term is introduced:

• Category frequency weight: introduced to address long-tail distribution. For category $c$, the weight is computed based on the ratio of positive and negative samples in the training set:

  $$w_{freq}^{(c)}={\displaystyle \frac{N_{neg}^{(c)}}{N_{pos}^{(c)}+\epsilon }}$$

  (19)

  where $N_{pos}^{\left(c\right)}$ and $N_{neg}^{\left(c\right)}$ denote the number of positive and negative samples for category $c$, respectively, and $ϵ$ is a small smoothing constant to avoid division by zero. Categories with fewer samples are assigned larger weights.
• Difficulty coefficient: introduced to emphasize hard categories, such as small-scale objects or structurally complex targets. For instance, categories like vehicles, despite having sufficient samples, are assigned higher coefficients due to their small scale and ambiguity. This coefficient is empirically set to encourage the model to focus on hard samples during early training.

4. Experiments

4.1. Dataset

To validate the effectiveness of the proposed MSDR-Net, experiments are conducted on the publicly available DIOR dataset [26]. As shown in Figure 6, DIOR is a large-scale, high-resolution remote sensing imagery dataset that encompasses diverse, complex scenes and a wide range of object categories and has been widely adopted for remote sensing image understanding tasks. The dataset contains 20 object categories, including large-scale targets such as Airplane, Airport, Ship, and Stadium, as well as small-scale or structurally complex targets such as Vehicle, Bridge, and Overpass. Significant variations across categories are observed in scale distribution, spatial density, and semantic co-occurrence patterns, reflecting typical characteristics of multi-label remote sensing scenarios.

Based on the object-level annotations, each image is reformulated as an image-level multi-label sample, allowing a single image to correspond to multiple category labels. This transformation better reflects the real-world characteristics of remote sensing scenarios, where multiple land-cover types often coexist within a single scene. No external training data are introduced in the experiments, and a fixed 8:2 split is adopted to construct the training and validation sets, ensuring the fairness and reproducibility of the experimental comparisons.

In addition to DIOR, MLRSNet is also used as a multi-label remote sensing benchmark to evaluate the cross-dataset generalization ability of MSDR-Net [1]. MLRSNet contains diverse high-resolution remote sensing scenes with multiple land-cover categories and multi-label annotations. In this study, 20 categories are selected for evaluation, including Airplane, Airport, Baseball Diamond, Basketball Court, Bridge, Freeway, Golf Course, Ground Track Field, Harbor&Port, Overpass, Parking Lot, Railway, Railway Station, Shipping Yard, Stadium, Storage Tank, Tennis Court, Terrace, Transmission Tower, and Wind Turbine. To construct the experimental subset, the first 25% samples of each category are retained, resulting in 12,012 images. The dataset is split into 9609 training and 2403 validation images at 80%/20%.

4.2. Evaluation Metrics

To comprehensively evaluate the performance of the model in multi-label remote sensing image classification, Mean Average Precision (mAP), Hamming Accuracy (HA), Overall F1-score (OF1), Class-wise F1-score (CF1), Overall Precision/Recall (OP/OR), and Class-wise Precision/Recall (CP/CR) are adopted. Among these metrics, mAP reflects the overall ranking capability of the model across different thresholds; HA measures label-wise prediction consistency; OF1 and OP/OR evaluate prediction quality from a sample-level perspective; and CF1 and CP/CR assess model performance from a class-level perspective, particularly in terms of class balance and long-tail category recognition. The corresponding formulations are defined as follows:

$$mAP={\displaystyle \frac{1}{q}}{\displaystyle \sum _{i=1}^{q}AP(i)} HA={\displaystyle \frac{1}{n}}{\displaystyle \sum _{j=1}^n\frac{1}{q}\left|Y_j\cap h(X_j)\right|}$$

(20)

$$CP={\displaystyle \frac{1}{q}}{\displaystyle \sum _{i=1}^q\frac{T{P}_i}{{TP}_i+{FP}_i}} CR={\displaystyle \frac{1}{q}}{\displaystyle \sum _{i=1}^q\frac{{TP}_i}{{TP}_i+{FN}_i}} CF1={\displaystyle \frac{2\times CP\times CR}{CP+CR}}$$

(21)

$$OP={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^q\frac{{TP}_i}{{TP}_i+{FP}_i}} OR={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^q\frac{{TP}_i}{{TP}_i+{FN}_i}} OF1={\displaystyle \frac{2\times OP\times OR}{OP+OR}}$$

(22)

Given that single-run experiments are susceptible to stochastic variations from random initialization and batch sampling, all overall evaluation metrics, except class-wise Average Precision (AP), are reported based on statistics from multiple repeated runs. Specifically, deep learning models are trained using five random seeds ([42, 2022, 7, 123, 999]), whereas traditional and comparative methods, including Support Vector Machine (SVM), Extremely Randomized Trees (ERT), Relation Network, Deep Multi-Attention, MSCA, and SFNet, are evaluated using three random seeds. All results are reported in the form of $\mu \pm \sigma$, where $\sigma$ denotes the sample standard deviation. For the validation mAP, a 95% confidence interval (CI) is additionally reported. To assess the statistical significance of performance differences between MSDR-Net and MSCA, a two-sided $t$-test is conducted with a significance level of $\alpha =0.05$.

4.3. Experimental Setup

Considering the significant scale variations and diverse land-cover types in high-resolution remote sensing imagery, a customized data preprocessing strategy is designed. During training, the following data augmentation strategies are applied to improve model generalization: Random Resized Crop (scale range: 0.7–1.0; aspect ratio: 0.85–1.15), Random Horizontal Flip (probability: 0.5), and rotations at multiples of 90°. The images are subsequently converted into tensors and normalized using dataset-specific statistics, with the mean. [0.485 0.456 0.406] and standard deviation [0.229 0.224 0.225]. For the validation set, only deterministic preprocessing is applied, including resizing to $512\times 512$ pixels, center cropping, tensor conversion, and normalization, to ensure the stability and consistency of evaluation metrics.

The model is trained on a single NVIDIA GeForce RTX 4070 Ti Super GPU using the PyTorch V1.12.1 framework. The key hyperparameters are set as follows: a batch size of 40, 100 training epochs, and the AdamW optimizer. A hybrid learning rate scheduling strategy is adopted, with linear warmup during the first five epochs, followed by dynamic adjustment based on validation loss, enabling more refined convergence in later training stages.

The proposed DW-Loss is employed, and consistent experimental settings, including input resolution, number of training epochs, optimizer configuration, and data splits, are maintained across all compared deep learning models. For methods originally designed for different task settings, their core architectures are retained while the output layers are uniformly replaced with a 20-dimensional Sigmoid classification head to conform to the DIOR multi-label protocol. Additionally, identical data augmentation strategies, training epochs, and validation protocols are applied to ensure fair and consistent comparisons.

The DSR module is implemented as a Transformer encoder with four layers, each with eight attention heads and a hidden embedding dimension of 256. A dropout rate of 0.1 is applied to prevent overfitting. The label embedding dimension is set to 20, corresponding to the number of categories in the multi-label classification task.

The DW-Loss, formulated as a difficulty-weighted binary cross-entropy, incorporates two complementary weighting mechanisms to improve learning on rare and challenging categories. First, the category frequency weight $w_{\mathrm{f}\mathrm{r}\mathrm{e}\mathrm{q}}^{\left(c\right)}$ is calculated as the ratio of negative to positive samples for each category in the training set, with a small smoothing constant $ϵ=1\times {10}^{-6}$ to avoid numerical instability. Second, the prior difficulty coefficient ${\alpha }_{\mathrm{h}\mathrm{a}\mathrm{r}\mathrm{d}}^{\left(c\right)}$ is empirically assigned according to target scale, geometric complexity, and observed training difficulty. Categories that are inherently challenging—such as Vehicle, Bridge, and Overpass—are assigned higher coefficients within the range [1.0, 2.5] to emphasize their contribution during training. The final weighting vector applied in the BCE loss is obtained by multiplying the category frequency weight and the difficulty coefficient element-wise, with a maximum clamp of 50 to maintain training stability.

This configuration ensures that the MSDR-Net framework effectively emphasizes complex, small-scale, or semantically ambiguous categories, while maintaining stable convergence for frequently occurring classes.

4.4. Performance Evaluation and Ablation Studies

4.4.1. Overall Performance and Fine-Grained Category Analysis

Table 1. Overall performance metrics of the proposed method on the DIOR validation set.

	mAP	HA	OF1	CF1	CP	CR	OP	OR
Value $\mu \pm \sigma$	0.9588 ± 0.0029	0.9724 ± 0.0018	0.8241 ± 0.0051	0.8738 ± 0.0042	0.7942 ± 0.0054	0.8746 ± 0.0041	0.7775 ± 0.0057	0.9589 ± 0.0025
95% CI	[0.9570, 0.9642]	[0.9602, 0.9646]	[0.8178, 0.8304]	[0.8686, 0.8790]	[0.7875, 0.8009]	[0.8695, 0.8797]	[0.7704, 0.7846]	[0.9558, 0.9620]

presents the overall performance statistics of MSDR-Net on the DIOR validation set. Based on repeated experiments across five random seeds, the proposed model achieves a favorable balance between precision and recall in the multi-label classification task, indicating strong overall discriminative capability while maintaining robustness in both label-level consistency and class-level balance.

To further evaluate category-level performance,

Table 2. Detailed statistics of the AP and HA for each category over five repeated experiments.

labels	Airplane	Airport	Baseball Field	Basketball Court	Bridge
AP	0.9905	0.9918	0.9917	0.9354	0.8543
HA	0.9978	0.9957	0.9945	0.9859	0.9667
labels	Chimney	Dam	Expressway Service Area	Expressway Toll Station	Golf Field
AP	0.9967	0.9779	0.9882	0.9747	0.9669
HA	0.9991	0.9940	0.9961	0.9927	0.9945
labels	Ground Track Field	Harbor	Overpass	Ship	Stadium
AP	0.9511	0.9777	0.8709	0.9929	0.9974
HA	0.9765	0.9932	0.9680	0.9914	0.9987
labels	Storage Tank	Tennis Court	Train Station	Vehicle	Windmill
AP	0.9463	0.9744	0.9889	0.8444	1
HA	0.9885	0.9906	0.9979	0.9014	0.9999

reports the AP and HA results for all 20 DIOR categories. Overall, MSDR-Net achieves high AP and HA values for most categories, indicating strong category discrimination and stable label-wise prediction consistency. For categories with distinctive structures and large spatial scales, such as Airplane, Airport, Baseball Field, Chimney, Ship, Stadium, and Windmill, the AP values are close to 1.0, while the HA values are around 0.99. This demonstrates the model’s stable recognition of salient, well-structured targets. For medium-scale structured categories, including Dam, Expressway Service Area, Expressway Toll Station, Golf Field, Harbor, Tennis Court, and Train Station, most AP values exceed 0.97, further confirming the robustness of MSDR-Net for regular scene objects. Relatively lower AP values are observed for Bridge, Overpass, and Vehicle, at 0.8543, 0.8709, and 0.8444, respectively. These categories are more challenging due to small object sizes, large appearance variations, and strong background coupling. In particular, Vehicle obtains a lower HA of 0.9014, indicating that dense small-object recognition remains difficult. For relatively imbalanced categories such as Storage Tank and Train Station, MSDR-Net still maintains high AP and HA values, suggesting that DW-Loss helps improve learning for rare and difficult categories.

As illustrated in Figure 7, the Transformer module’s attention responses are visualized to evaluate MSDR-Net’s spatial attention capability in multi-label remote sensing image classification. Since the feature pyramid produces a $7\times 7$ feature representation, the resulting heatmaps correspond to coarse-grained token-level semantic responses, which are subsequently upsampled and overlaid onto the original images for visualization. The high-attention regions effectively cover both large target areas and densely distributed small-scale objects, while also accurately focusing on critical regions of structurally organized targets with linear or block-like spatial distributions. In complex multi-label scenarios, multiple semantic regions can be simultaneously attended, indicating that the combination of multiscale feature representation and Transformer-based semantic reasoning effectively captures target co-occurrence patterns and inter-class spatial relationships.

Overall, MSDR-Net demonstrates superior performance on both global and fine-grained target categories. Large-scale and common categories are accurately recognized, while substantial improvements are achieved for small objects, structurally complex targets, and rare categories. Furthermore, the attention visualization results demonstrate that the proposed model can simultaneously focus on multiple targets and semantically important regions, further validating the synergistic advantages of multiscale feature fusion, Transformer-based semantic reasoning, and DW-Loss.

4.4.2. Ablation Study

To further investigate the underlying mechanisms contributing to the performance improvements of MSDR-Net and to validate the necessity of each component within the network architecture and training strategy, a systematic cumulative ablation study is conducted. The experiments are conducted using ResNet-34 as the baseline model. Under consistent hyperparameter settings, the Transformer-based global attention module, multiscale feature pyramid, DW-Loss, and data augmentation strategies are incrementally incorporated, allowing a quantitative evaluation of the cumulative contributions of each component to the final classification performance.

The baseline model employs ResNet-34 as the feature extractor without incorporating additional attention mechanisms or multiscale fusion modules, and is optimized using the BCE loss. As shown in

Table 3. Results of stepwise cumulative ablation experiments for MSDR-Net (mAP %).

No.	Network Configuration	Seeds	Validation mAP%	Validation 95% CI	Improvement
Baseline	ResNet34	5	90.71 ± 0.53	[90.05, 91.37]	–
Exp-1	ResNet34+ DSR	5	92.31 ± 0.44	[91.76, 92.86]	+1.60
Exp-2	FPN+DSR	5	94.82 ± 0.38	[94.44, 95.20]	+4.09
Exp-3	FPN+ DSR +DW-Loss	5	95.53 ± 0.35	[95.18,95. 88]	+4.82
MSDR-Net (Ours)	MSDR-Net	5	95.88 ± 0.29	[95.59, 96.17]	+5.17

, this configuration achieves an mAP of 90.71%. Although the backbone provides reasonable feature extraction capability, its representation of small, densely distributed targets remains limited in remote sensing scenarios characterized by significant scale variations and complex background interference. In Exp-1, a Transformer-based global attention module is incorporated into the baseline model. By leveraging self-attention to capture long-range dependencies, the model’s ability to model contextual relationships among objects is enhanced. As a result, the mAP improves from 90.71% to 92.3% (+1.8%). This improvement highlights the importance of global contextual information for understanding complex remote sensing scenes, particularly in distinguishing categories with similar local features but distinct global semantics. In Exp-2, the FPN is further integrated based on Exp-1. By combining deep semantic information with shallow spatial details, the model’s ability to represent multiscale targets is significantly enhanced. Consequently, the mAP increases to 94.8% (+3.5% compared to Exp-1), demonstrating the effectiveness of multiscale feature fusion in addressing large-scale variations, particularly for structurally distinctive targets such as bridges and overpasses. To address class imbalance and hard-sample challenges in remote sensing datasets, the proposed DW-Loss is introduced in Exp-3. By dynamically adjusting category- and sample-level weights, the model is guided to focus more on rare and difficult samples. As shown in Table 2, the mAP further increases to 95.5% (+0.8% compared to Exp-2). Although the overall gain is moderate, substantial improvements are observed for long-tail categories such as vehicles and storage tanks, effectively mitigating category bias. In the final configuration, customized data augmentation strategies, including random cropping, flipping, and rotation, are incorporated to increase the diversity of the training data. This leads to improved generalization and robustness, resulting in a final mAP of 95.88% (+0.3% compared to Exp-3). Overall, the progressive ablation results clearly demonstrate that each component of MSDR-Net—from the global attention mechanism and multiscale feature fusion to tailored loss optimization and data augmentation strategies—positively and significantly contributes to overall performance. These components operate synergistically to form a high-performance framework for multi-label remote sensing image classification.

4.5. Cross-Dataset Validation on MLRSNet

Table 4. Overall performance metrics of the proposed method on the MLRSNet validation set.

	mAP	HA	OF1	CF1	CP	CR	OP	OR
Value $\mu \pm \sigma$	0.9805 ± 0.0016	0.9872 ± 0.0011	0.9044 ± 0.0091	0.9296 ± 0.0042	0.8881 ± 0.0068	0.9753 ± 0.0037	0.8451 ± 0.0064	0.9725 ± 0.0056
95% CI	[0.9789, 0.9821]	[0.9861, 0.9883]	[0.8954, 0.9135]	[0.9254, 0.9338]	[0.8949, 0.8813]	[0.9790, 0.9716]	[0.8515, 0.8387]	[0.9781, 0.9669]

summarizes the overall performance metrics of MSDR-Net on the MLRSNet validation set, including mAP, HA, OF1, CF1, CP, CR, OP, and OR, together with the corresponding standard deviations and 95% confidence intervals. To further evaluate category-level performance,

Table 5. Detailed statistics of the AP and HA for each category over five repeated experiments.

labels	Airplane	Airport	Baseball Field	Basketball Court	Bridge
AP	0.9517	0.9950	0.9993	0.9843	0.9847
HA	0.9888	0.9929	0.9992	0.9767	0.9942
labels	Freeway	Golf Course	Ground Track Field	Harbor	Overpass
AP	0.9968	0.9999	0.9665	0.9999	0.9813
HA	0.9975	0.9963	0.9754	0.9996	0.9908
labels	Parking Lot	Railway	Railway Station	Shipping Yard	Stadium
AP	0.8950	0.9807	0.9529	0.9881	0.9733
HA	0.9097	0.9863	0.9875	0.9788	0.9921
labels	Storage Tank	Tennis Court	Terrace	Transmission Tower	Wind Turbine
AP	0.9999	0.9754	0.9999	0.9949	0.9998
HA	0.9996	0.9938	0.9988	0.9975	0.9996

presents the AP and HA results for all 20 categories in MLRSNet. Overall, MSDR-Net achieves high AP and HA values across most categories, demonstrating strong category discrimination and stable label-wise prediction consistency.

For large-scale, geometrically distinctive categories, such as Airport, Baseball Diamond, Basketball Court, Harbor, Golf Course, Terrace, Storage Tank, Transmission Tower, and Wind Turbine, AP values are above 0.98, and HA values are around 0.99, confirming that MSDR-Net can reliably recognize prominent, clearly structured targets. For medium-scale structured categories, including Freeway, Ground Track Field, Railway Station, Shipping Yard, and Stadium, AP values remain above 0.95, further demonstrating the robustness of MSDR-Net on regular scene objects. Small-scale and challenging categories, such as Parking Lot, Vehicle, Bridge, Overpass, and Railway, have relatively lower AP values (0.8950–0.9807) and slightly lower HA values (0.9097–0.9863) due to their dense spatial distribution, large appearance variation, and complex backgrounds. Nevertheless, compared to prior benchmarks, MSDR-Net achieves noticeable improvement in small-scale target recognition, indicating the effectiveness of multiscale feature enhancement and DW-Loss in improving small and difficult object predictions. In addition, long-tail categories such as Storage Tank and Railway Station, despite having relatively fewer samples, still achieve high AP and HA, demonstrating that MSDR-Net mitigates the influence of long-tail category imbalance through difficulty-weighted optimization. Collectively, these results show that MSDR-Net effectively handles multiscale objects, models complex semantic dependencies, and addresses long-tail distribution issues.

Compared to DIOR, MLRSNet shows higher AP for medium and large-scale categories, likely due to more balanced class distributions and higher image quality, while small-scale and long-tail categories remain more challenging, confirming the importance of global reasoning and DW-Loss in these scenarios.

4.6. Comparative Experiments

To comprehensively evaluate the effectiveness and competitiveness of MSDR-Net, comparisons are conducted under the DIOR multi-label protocol against a diverse set of approaches, including traditional machine learning models, classical deep learning models, relation modeling methods, and recent multiscale and semantic enhancement techniques. Specifically, the compared methods include Support Vector Machine (SVM) [27], Extremely Randomized Trees (ERT) [27], CNN [28], ResNet-34 [29], Relation Network [30], Deep Multi-Attention [31], MSCA [12], and SFNet [21]. It should be noted that the original task settings of methods such as Relation Network, Deep Multi-Attention, MSCA, and SFNet are not fully aligned with the multi-label classification setting considered in this study. Therefore, to ensure fair comparison, their core architectures are re-implemented under a unified experimental protocol, including consistent input resolution, optimizer configuration, training epochs, data augmentation strategies, and a standardized multi-label Sigmoid classification head. The comparative results are reported in

Table 6. Performance comparison of MSDR-Net and baseline methods on the DIOR validation set (mAP%).

No.	Network	Seeds	Validation mAP%	Validation 95% CI
Exp-1	SVM	3	68.42 ± 0.08	[68.36, 68.50]
Exp-2	ERT	3	73.55 ± 0.31	[73.24, 73.86]
Exp-3	CNN	3	86.37 ± 0.71	[94.44, 87.08]
Exp-4	ResNet50	3	91.16 ± 0.46	[90.70,91.62]
Exp-5	RelationNet	3	92.46 ± 0.61	[91.85,93.07]
Exp-6	Deep MultiAttention	3	91.88 ± 0.68	[91.20,92.56]
Exp-7	MSCA	3	94.24 ± 0.41	[93.83,94.65]
Exp-8	SFNet	5	93.8 ± 0.37	[93.43,94.17]

.

As illustrated in Figure 8, Overall, traditional machine learning methods exhibit substantially inferior performance. Specifically, Support Vector Machine (SVM) and Extremely Randomized Trees (ERT) achieve mAP values of 68.42% and 73.55% on the validation set, respectively. This indicates that methods relying on hand-crafted features are insufficient for capturing the complex spatial structures and semantic co-occurrence patterns present in high-resolution remote sensing imagery. In contrast, deep learning-based approaches significantly improve classification performance. The CNN achieves an mAP of 86.37%, while ResNet-50 further improves it to 91.16%, validating the effectiveness of deep convolutional architectures and residual learning in feature representation. However, these methods primarily rely on local receptive fields, limiting their ability to model long-range semantic dependencies in multi-label scenarios.

To further enhance multi-label classification performance, subsequent methods incorporate relation modeling and attention mechanisms. The Relation Network explicitly models inter-label dependencies, achieving an mAP of 92.46% on the validation set, while Deep Multi-Attention enhances regional feature representations through multi-branch attention, achieving an mAP of 91.88%. Overall, these approaches improve performance beyond 92%, demonstrating that modeling label relationships and incorporating attention mechanisms effectively enhance multi-label discrimination capability. However, these methods still exhibit notable instability, as reflected in relatively high standard deviations of 0.56–0.68, indicating increased sensitivity to random initialization.

In recent years, research has shifted from relational modeling toward integrated, multiscale, and semantic fusion frameworks. MSCA leverages sparse cross-scale attention to enable multiscale feature interaction, achieving an mAP of $94.24\pm 0.41$ on the validation set, while SFNet enhances category discrimination through semantic-assisted feature fusion, achieving $93.85\pm 0.37$. Although these approaches effectively mitigate challenges arising from scale variations in remote sensing imagery, a unified optimization framework that jointly models multiscale information, semantic relationships, and long-tail distributions remains lacking.

Compared with the aforementioned methods, the proposed MSDR-Net achieves the highest validation mAP of $95.88\pm 0.29$, outperforming all competing approaches. Meanwhile, a training mAP of $97.87\pm 0.27$ is obtained, with a train–validation gap of approximately 2.03%, which is significantly smaller than that of other deep models. This indicates that, while maintaining strong fitting capability, the model effectively suppresses overfitting and demonstrates superior generalization performance. In terms of performance gains, MSDR-Net improves mAP by approximately 1.64% compared with the current best-performing method, MSCA, and by 2.08% compared with SFNet. Notably, an improvement exceeding 1.5% is still achieved in the high-performance regime (above 94% mAP), further demonstrating the effectiveness of the proposed approach.

Table 7. The AP results of MSDR-Net and MSCA for each category.

labels	Airplane	Airport	Baseball Field	Basketball Court	Bridge
MSDR	0.9903	0.9843	0.9908	0.9345	0.8429
MSCA	0.9881	0.9772	0.9825	0.9245	0.7240
labels	Chimney	Dam	Expressway Service Area	Expressway Toll Station	Golf Field
MSDR	0.9877	0.9765	0.9881	0.9710	0.9632
MSCA	0.9743	0.9345	0.9549	0.9464	0.9574
labels	Ground Track Field	Harbor	Overpass	Ship	Stadium
MSDR	0.9548	0.9800	0.8941	0.9894	0.9970
MSCA	0.8903	0.9666	0.7948	0.9798	0.9913
labels	Storage Tank	Tennis Court	Train Station	Vehicle	Windmill
MSDR	0.9509	0.9767	0.9932	0.8370	1
MSCA	0.9138	0.9513	0.9672	0.7884	0.9999

presents the AP results of MSDR-Net and MSCA for each category. Overall, MSDR-Net consistently outperforms MSCA across most categories, particularly for challenging categories with small-scale objects or long-tail distributions. For instance, small-scale targets such as Vehicle, Bridge, and Overpass exhibit notable improvements, with MSDR-Net achieving AP values of 0.8370, 0.8429, and 0.8941, respectively, compared to MSCA’s 0.7884, 0.7240, and 0.7948, respectively. These gains indicate that MSDR-Net’s multiscale feature enhancement and dynamic semantic reasoning effectively capture subtle spatial details, improving recognition of small or densely distributed targets.

For long-tail categories, including Storage Tank, Tennis Court, and Train Station, MSDR-Net also demonstrates superior performance (AP = 0.9509, 0.9767, and 0.9932) relative to MSCA (AP = 0.9138, 0.9513, and 0.9672), highlighting the effectiveness of the DW-Loss in mitigating class imbalance and enhancing learning on rare categories. Large-scale categories, such as Airplane, Baseball Field, Ship, and Windmill, are already well recognized by both methods, but MSDR-Net still provides small yet consistent improvements.

In summary, these results indicate that MSDR-Net achieves notable gains on small-scale, difficult, and long-tail categories, while maintaining or slightly improving performance on large and medium-scale categories. This underscores the advantages of integrating multiscale feature extraction, dynamic semantic reasoning, and difficulty-weighted loss in a unified framework.

4.7. Computational Complexity Analysis

To further evaluate the practical applicability of the proposed method, the computational complexity of MSDR-Net was compared with representative baseline and competing methods, including ResNet-34, SFNet, and MSCA. The comparison was conducted under the same input resolution of $3\times 512\times 512$. The number of parameters, FLOPs, single-image inference time, and frames per second (FPS) were reported. FLOPs were calculated using THOP, and inference time was measured on the same GPU platform with a batch size of 1. Data loading and image preprocessing were excluded from the timing process.

As shown in

Table 8. Computational complexity comparison of different methods under $512\times 512$ input size.

Method	Params(M)	FLOPs(G)	Inference Time (ms/Image)	FPS
ResNet-34	21.80	19.12	6.02	166.0
SFNet	16.26	21.57	6.67	149.9
MSCA	38.32	35.54	12.02	83.2
MSDR-Net	39.46	24.61	8.34	119.9

, ResNet-34 has the lowest inference cost among the compared CNN-based baselines, with 21.80 M parameters, 19.12 GFLOPs, and an average inference time of 6.02 ms per image. However, its mAP is only 90.71%, indicating that the standard backbone alone is insufficient for classifying complex multi-label remote sensing images. SFNet achieves better classification performance than ResNet-34 with relatively low parameters and inference time, but its mAP remains lower than that of MSDR-Net.

Compared with MSCA, MSDR-Net achieves a better trade-off between accuracy and efficiency. Although the parameter count of MSDR-Net is slightly higher than that of MSCA, increasing from 38.32 M to 39.46 M, the FLOPs are reduced from 35.54 GFLOPs to 24.61 GFLOPs, corresponding to a reduction of approximately 30.8%. Meanwhile, the average inference time decreases from 12.02 ms to 8.34 ms, yielding an approximately 30.6% improvement in inference speed. More importantly, MSDR-Net achieves a higher mAP of 95.88%, outperforming MSCA by approximately 1.64 percentage points.

These results indicate that the proposed MSDR-Net does not simply improve performance by introducing excessive computational overhead. Instead, by efficiently fusing multiscale features, employing Transformer-based semantic reasoning, and applying difficulty-weighted optimization, MSDR-Net achieves higher classification accuracy while maintaining moderate computational complexity. The average inference speed of 119.9 FPS further demonstrates that the proposed method has practical potential for high-resolution remote sensing image interpretation, especially in offline and near-real-time application scenarios.

4.8. Discussion

An end-to-end MSDR-Net is proposed to address three major challenges in multi-label remote sensing image classification, including multi-scale variation, complex semantic dependencies, and long-tail category distributions. Extensive experiments conducted on the Dataset for DIOR and MLRSNet datasets demonstrate the effectiveness of the proposed framework. Existing approaches generally focus on isolated optimization objectives, where CNN-based methods emphasize multi-scale feature extraction, while Transformer-based methods primarily focus on long-range dependency modeling. In contrast, MSDR-Net unifies MSFE, DSR and DW-Loss within a unified end-to-end framework. On the DIOR dataset, MSDR-Net achieves an mAP of 95.88%, outperforming the state-of-the-art MSCA method by approximately 1.64%. Notable improvements are observed for small-scale targets, such as vehicles and bridges, as well as long-tail categories, including storage tanks, demonstrating the robustness of the proposed joint modeling strategy. Compared with traditional machine learning methods, such as SVM, deep learning-based approaches exhibit substantially stronger capability in modeling the complex spatial structures of high-resolution remote sensing imagery, further highlighting the limitations of hand-crafted feature representations.

Beyond performance improvements, the proposed framework also provides methodological insights for multi-label remote sensing image classification. Experimental analysis indicates that simply increasing network depth, such as adopting ResNet-34, is insufficient for effectively addressing missed detections of small-scale targets. After introducing the Transformer-based DSR module, long-range contextual dependencies can be captured through global attention mechanisms, enabling improved recognition of occluded and spatially scattered targets. This observation is consistent with findings reported in Vision Transformer studies, where global contextual modeling has been shown to play a critical role in visual understanding.

Despite the promising performance achieved by MSDR-Net, several limitations remain. Although the inference speed is improved to 119.9 FPS through architectural optimization, the incorporation of FPN/PAN structures and the Transformer encoder increases the model complexity to 39.46 M parameters, compared with lightweight CNN models such as ResNet-34. This increased computational cost may restrict real-time deployment on resource-constrained edge devices, such as small unmanned aerial vehicles. In addition, for extremely dense and small-scale targets, such as vehicles, the HA still remains improvable, indicating that feature discriminability under severe background clutter requires further enhancement. Future work will focus on lightweight model design, including knowledge distillation and neural architecture search, as well as self-supervised and large-scale pretraining strategies, to reduce computational cost while maintaining high accuracy and improving generalization capability under challenging remote sensing scenarios.

5. Conclusions

To address the challenges of multiscale variation, complex semantic dependencies, and long-tail distributions in multi-label remote sensing image classification, an end-to-end framework, MSDR-Net, is proposed that integrates MSFE, DSR, and DW-Loss into a unified architecture for representation learning and imbalance-aware optimization. The MSFE module enables robust multiscale feature extraction through residual learning and feature pyramid fusion, while the DSR module captures long-range dependencies and inter-class correlations via positional encoding, enhancing global semantic modeling. The proposed DW-Loss further improves robustness by dynamically reweighting category contributions, effectively mitigating long-tail and hard-sample issues. Extensive experiments on DIOR and MLRSNet demonstrate that MSDR-Net achieves favorable multi-label classification performance and shows promising robustness and generalization potential on the evaluated datasets.

Despite its favorable performance on the DIOR multi-label remote sensing image classification task, MSDR-Net still has several limitations. First, due to the introduction of FPN/PAN, the Transformer encoder, and multiscale feature enhancement structures, the model’s complexity remains higher than that of lightweight CNN-based models, and further optimization is required for deployment on resource-constrained edge platforms. In addition, the recognition of small-scale, densely distributed, or background-coupled categories, such as Vehicle, Bridge, and Overpass, still leaves room for further improvement. Future work will explore lightweight model design, self-supervised or large-scale pretraining strategies, and multimodal data integration to enhance scalability and practical applicability.