Rebalancing in Supervised Contrastive Learning for Long-Tailed Visual Recognition

Abstract

In real-world visual recognition tasks, long-tailed distribution is a pervasive challenge, where the extreme class imbalance severely limits the representation learning capability of deep models. Although supervised learning has demonstrated certain potential in long-tailed visual recognition, these models’ gradient updates dominated by head classes often lead to insufficient representation of tail classes, resulting in ambiguous decision boundaries. While existing Supervised Contrastive Learning variants mitigate class bias through instance-level similarity comparison, they are still limited by biased negative sample selection and insufficient modeling of the feature space structure. To address this, we propose Rebalancing Supervised Contrastive Learning (Reb-SupCon), which constructs a balanced and discriminative feature space during model training to alleviate performance deviation. Our method consists of two key components: (1) a dynamic rebalancing factor that automatically adjusts sample contributions through differentiable weighting, thereby establishing class-balanced feature representations; (2) a prototype-aware enhancement module that further improves feature discriminability by explicitly constraining the geometric structure of the feature space through introduced feature prototypes, enabling locally discriminative feature reconstruction. This breaks through the limitations of conventional instance contrastive learning and helps the model to identify more reasonable decision boundaries. Experimental results show that this method demonstrates superior performance on mainstream long-tailed benchmark datasets, with ablation studies and feature visualizations validating the modules’ synergistic effects.

1. Introduction

The explosive development of deep learning has enabled Convolutional Neural Networks (CNNs) to achieve groundbreaking advances across multiple computer vision domains. Notably, CNNs have demonstrated superhuman-level performance in fundamental vision tasks including image classification [1], object detection [2], and semantic segmentation [3]. Progress in neural architecture search (NAS) techniques [4,5] has further enhanced CNN capabilities by automating architectural optimization, particularly improving their effectiveness on large-scale datasets.

These achievements largely rely on the powerful computational resources and high-quality large-scale datasets enabled by the Internet era, such as ImageNet [6] and MS COCO [7]. These datasets are carefully designed to ensure sufficient and balanced training samples for each category. However, real-world data distributions often deviate from this ideal scenario and instead exhibit long-tailed characteristics [8], as shown in Figure 1a. In a long-tailed distribution, a small number of categories dominate the majority of the data, while a large number of tail classes contain only limited samples. This imbalanced distribution pattern is prevalent in many practical applications, such as rare object recognition in image classification and niche topic detection in text classification.

In long-tailed recognition tasks, the imbalanced data distribution substantially influences both the model training process and final performance. This imbalance results in strong recognition performance for head classes, while significantly degrading learning efficacy for tail classes due to their scarcity of samples, ultimately degrading overall classification accuracy [9,10,11,12]. Our in-depth analysis identifies the root cause as a systematic bias in classifier weight norms [13], which critically influences the model’s feature response capability. Specifically, larger weight norms maintain strong sensitivity to head class features, whereas smaller norms result in inadequate responses to tail class features, leading to misclassification. As clearly illustrated by the blue lines in Figure 2, our visualization analysis of classifier weight norms in the baseline Supervised Contrastive Learning (SCL) [14] method demonstrates that tail classes exhibit significantly smaller weight norms compared to head classes. This observation provides crucial evidence from a parameter space perspective for understanding the performance bottleneck in long-tailed recognition.

To address this challenge, conventional approaches primarily include sampling methods [15,16], cost-sensitive learning techniques [9,12,17], and classifier optimization strategies [18,19,20]. These methodologies aim to mitigate classification bias toward head classes while improving recognition performance for tail classes. However, these solutions typically operate under the assumption of well-separated inter-class differences, focusing solely on optimizing sample distribution while paying insufficient attention to the latent representations learned from imbalanced data. Crucially, they fail to adequately account for inter-class feature correlations and intrinsic relationships. Moreover, their excessive emphasis on tail classes often comes at the expense of head class performance, resulting in an overall imbalance in classification effectiveness.

The emergence of self-supervised learning [21,22] has inspired new approaches to learning without label dependency, offering novel perspectives for addressing long-tailed problems. As a fundamental paradigm in self-supervised learning, contrastive learning [23,24] effectively models instance-level similarity relationships, enabling the extraction of highly discriminative feature representations without requiring labeled supervision. Khosla et al. [14] generalized the conventional self-supervised contrastive loss to its supervised counterpart (SCL) by integrating label information, yielding significant performance improvements. While SCL has achieved notable success on balanced datasets [25], its effectiveness on imbalanced distributions remains limited. The standard contrastive learning framework, which critically depends on the selection of positive–negative sample pairs, is inherently biased toward majority classes [13,26], consequently amplifying the challenges induced by data imbalance.

Based on the above analysis, this paper proposes a Rebalancing Supervised Contrastive Learning (Reb-SupCon) method for long-tailed visual recognition. The approach mitigates the learning difficulties associated with long-tailed data by breaking the dominance of head classes in model optimization within the feature space and establishing balanced category representations. The main contributions are as follows:

• We propose an adaptive focal gradient-weighted rebalancing factor that dynamically integrates class frequency statistics and gradient magnitude information. It automatically adjusts the importance weight of each sample during training, simultaneously enhancing gradient focus on tail classes while preventing representation degradation of head classes, thereby constructing a balanced feature space.
• Leveraging the balanced feature representations enabled by the rebalancing factor, we introduce a prototype-aware discriminative enhancement module. This module constrains the geometric structure of prototypes through an aggregation–separation loss and incorporates swapped prediction to achieve dual-branch supervision alignment between features and prototypes. This approach reconstructs the feature space from global balance to local discriminability, alleviating the limitation of traditional contrastive learning that relies solely on inter-sample similarity.
• Extensive experiments on multiple benchmark datasets demonstrate that the proposed method achieves state-of-the-art performance.

2. Related Work

2.1. Long-Tailed Visual Recognition

Early approaches to addressing the long-tailed problem have mainly revolved around two core strategies: re-sampling and re-weighting. Re-sampling [27,28] either oversamples low-frequency classes or undersamples high-frequency classes. Re-weighting [12,17,29] assigns different loss values to different training samples based on their class or instance characteristics. Furthermore, module enhancement methods, such as BBN [30], introduce dual-branch networks combining conventional learning and rebalancing to address long-tailed recognition. Similarly, Hybrid-SC and Hybrid-PSC [31] adopt this dual-branch architecture to decouple feature learning from classification tasks, thereby mitigating inter-task interference. Notably, Hybrid-PSC further introduces a class prototype contrastive mechanism, achieving computational efficiency without compromising discriminative power on tail classes. Metric learning has also been widely applied in long-tailed recognition. Its core objective is to design task-specific distance metrics to better capture similarities and differences between objects, thereby promoting the learning of more discriminative feature spaces. For instance, KCL [32] proposes a k-positive contrastive loss to alleviate class imbalance by learning a balanced feature space, improving model generalization. In decoupled training approaches, He et al. [33] proposed an adaptive calibration method that operates through two key phases. During the first-phase feature space decoupling, it constructs a uniformly distributed hyperspherical prototype space. For the second-phase adaptive fine-tuning, the method develops a dynamic temperature scaling algorithm based on prototype distance metrics derived from the first phase, enabling automatic calibration of decision boundaries according to each class’s specific characteristics. Recent Vision Transformer (ViT)-based approaches have demonstrated unique advantages in long-tailed recognition. Lin et al. [34] proposed rebalanced modal gradient modulation (ReGrad) for the modal imbalance problem in multimodal face anti-counterfeiting (FAS). By adaptively adjusting the gradient amplitudes of each mode, the convergence speed is balanced. Through interpretable reliability modeling and gradient control mechanisms, this provides new possibilities for multimodal long-tailed learning.

2.2. Contrastive Learning

Self-supervised Contrastive Learning (SSCL) and SCL, as two major branches of contrastive learning, learn effective feature representations through contrastive loss functions in scenarios with unlabeled and labeled data, respectively. In recent years, SSCL has achieved remarkable progress. SimCLR [21] pioneered the training paradigm by optimizing feature representations through positive and negative sample pairs, demonstrating that contrastive learning can achieve performance comparable to supervised learning even without labels. Subsequently, BYOL [35] further refined the contrastive learning framework by proposing a method that relies solely on positive sample pairs, eliminating the challenges associated with negative sample selection. Other related models, such as MoCo [23,24] and SwAV [26], further improved performance through momentum encoding and online clustering, respectively. Collectively, these advancements have driven unsupervised representation learning to approach the performance of supervised learning.

In contrast to SSCL, SCL relies on labeled data to pull samples of the same class closer while pushing apart those from different classes [14]. In long-tailed recognition tasks, numerous variants have been proposed. KCL [32] adopts a two-stage learning paradigm, ensuring an equal number of positive examples per class within each batch. Wang et al. [31] introduced a hybrid network architecture combining contrastive loss for feature learning and cross-entropy loss for classifier learning, which both improves representation quality and mitigates classifier bias toward head classes. Subsequently, several methods [10,13,25] incorporating class-complementary mechanisms for constructing positive and negative pairs have been developed to ensure adequate participation of tail classes during the learning process. ProCo [26] introduces a probabilistic contrastive learning framework based on von Mises–Fisher distribution modeling, which generates infinite contrastive pairs by estimating class-conditional feature distributions online and optimizes the expected loss in closed form, effectively addressing the inherent limitations of batch constraints. DSCL [36] proposes a decoupled Supervised Contrastive Learning framework, which alleviates intra-class distance bias and underrepresentation of tail class features by separating optimization objectives for two types of positive samples and introducing a patch-based self-distillation mechanism. Regarding the design of distribution-aware mechanisms, DaSC [37] innovatively consolidates distribution-aware weighting strategies and mixed-sample augmentation techniques into a contrastive learning framework. By dynamically fusing global data distribution information with local sample confidence, it effectively liberates the model from traditional methods’ dependence on high-purity samples.

Our work differs from previous approaches in several key aspects. Existing methods typically employ an equal-treatment strategy for both types of positive samples, which has been demonstrated to lead to cross-category optimization bias [13]. To address this, we propose an adaptive focal gradient weighting mechanism that constructs category-aware rebalancing factors, enabling precise modeling of long-tailed data. Secondly, our proposed prototype-aware discriminative enhancement module innovatively integrates aggregation–separation loss with a swapped prediction mechanism, achieving explicit optimization of the feature space geometry. Most importantly, unlike existing methods that require complex two-stage tuning, our end-to-end framework accomplishes both feature balancing and discriminative enhancement through single-stage training.

2.3. Prototype Learning for Long-Tailed Recognition

Prototype learning methods provide an effective solution for long-tailed recognition tasks by constructing category-specific feature prototypes. The evolution of these methods demonstrates a clear progression from static prototype modeling to dynamic adaptive learning. The groundbreaking development of Open Long-Tail Recognition (OLTR) [8] introduced a novel framework for long-tailed recognition in open environments. By maintaining a visual memory bank that stores discriminative feature prototypes and utilizing retrieved memory features to enhance original representations, this approach not only strengthens the discriminative power of the feature space, but also effectively pushes novel-class samples away from the memory bank region. This design enables simultaneous recognition of known classes and detection of open-set classes.

Building upon OLTR, subsequent research has introduced various extensions and improvements. Expanded Episodic Memory (EEM) [38] implements a meta-embedding memory with dynamic update mechanisms, assigning each class a distinct memory block to record its most discriminative prototypes. This dynamic memory design significantly mitigates class imbalance effects by selectively retaining highly discriminative prototypes. Recent advancements have driven prototype learning toward more sophisticated implementations. PCL [39] innovatively employs category prototypes to represent semantic information, constructing a learnable prototype classifier through dual calibration mechanisms at both prototype and instance levels, effectively addressing data imbalance issues. Meanwhile, PADE [40] breaks the conventional assumption about test distributions by implementing momentum prototypes for balanced compactness learning, combined with instance-level domain detection and multi-expert parameter customization, achieving dynamic adaptation to arbitrary test distributions.

3. Method

In this section, we present Reb-SupCon, a novel framework designed to address feature bias in SCL for long-tailed visual recognition. Our approach consists of four key components: (1) fundamental concepts of Supervised Contrastive Learning (Section 3.1), (2) systematic analysis of classification bias in long-tailed scenarios (Section 3.2), (3) implementation details with proposed loss functions (Section 3.3), and (4) model training methodology (Section 3.4).

3.1. Preliminaries

In the visual recognition task, our objective is to establish a mapping from the input space $\mathcal{X}$ to the target space $\mathcal{Y}$. This complex mapping process consists of two fundamental components: (1) an encoder $f_{\theta }(·)$ that extracts features from input data $x\in \mathcal{X}$, and (2) a linear classifier $g_{\varphi }(·)$ that processes the extracted features $z=f_{\theta }\left(x\right)\in {\mathbb{R}}^d$ to produce the final classification output $\widehat{y}=g_{\varphi }\left(z\right)\in \mathcal{Y}$. The quality of feature extraction plays a pivotal role in determining the model’s overall performance. Consequently, enhancing the encoder’s capability and learning discriminative feature representations are primary objectives for optimizing classification performance. To facilitate our subsequent analysis, we first formalize the following key definitions:

Supervised Contrastive Learning (SCL). The goal of SCL is to optimize recognition performance by contrasting similarities between samples from the same class and those from different classes. Given a sample $x_i$ with its embedding representation $z_i$, the similarity between two embedding vectors is measured using cosine similarity, defined as

$$\mathrm{sim}(z_i,z_j)={\displaystyle \frac{z_i·z_j}{\parallel z_i\parallel \parallel z_j\parallel }},$$

(1)

where $z_i·z_j$ denotes the dot product between vectors $z_i$ and $z_j$, and $\parallel z\parallel$ represents the L2-norm of vector z. The similarity range is normalized to [−1, 1], with higher values indicating greater similarity. This similarity metric serves as the foundation for constructing contrastive loss functions that pull together embeddings from the same class while pushing apart those from different classes.

The Supervised Contrastive Loss is formally defined as

$${\mathcal{L}}_{\sup }\left(\theta \right)=-\sum _{i=1}^N{\displaystyle \frac{1}{\left|P\right(i\left)\right|}}\sum _{p\in P\left(i\right)}log{\displaystyle \frac{exp(s(z_i,z_p)/\tau )}{{\sum }_{\begin{array}{c}j=1\\ j\ne i\end{array}}^{N}exp(s(z_i,z_j)/\tau )}},$$

(2)

where $z_i$ denotes the embedding vector of sample $x_i$, $P\left(i\right)$ represents the set of positive samples sharing the same label with $x_i$, $\tau$ is the temperature parameter, and $exp(·)$ is the exponential function.

3.2. Analysis

When applying SCL to long-tailed visual recognition tasks, frequent classes typically achieve lower loss values and dominate the model training process [13]. This leads to excessive attention being focused on features from frequent classes while those from rare classes are neglected, thereby exacerbating the data imbalance problem. In SCL, the selection of negative samples is determined by their label difference from the current anchor sample. Specifically, only samples sharing the same class label with the anchor are considered positive samples, while all other samples are treated as negatives. However, in class-imbalanced datasets, the number of head class samples significantly exceeds that of tail class samples. This imbalance is particularly pronounced in the MoCo [23,24] framework, where it causes two critical issues during the negative sample $queue$ update process:

• The negative sample $queue$ exhibits a skewed class distribution. Let $N_h$, $N_m$, and $N_t$ denote the number of head, medium, and tail class samples in the dataset, respectively. The head class proportion in the dataset is

  $$p_h={\displaystyle \frac{N_h}{N_h+N_m+N_t}}$$

  (3)
  Within the $queue$ of size Q, the empirical head class proportion is $Q_h/Q$. When $N_h\gg N_t$ in the dataset, the queue proportion $Q_h/Q$ approaches 1, causing the negative sampling process to become heavily biased toward head classes while suppressing gradient contributions from tail classes. This imbalance ultimately degrades model performance.
• The class imbalance in negative sampling critically distorts gradient updates during long-tailed learning. Following BCL [25], we reformulate the supervised contrastive loss at the class level for theoretical analysis:

  $$L_{\sup }=-\sum _{i}log\left({\displaystyle \frac{exp\left(s\left(z_i,z_{y_i}\right)/\tau \right)}{{\sum }_{j\ne i}exp\left(s\left(z_i,z_{y_j}\right)/\tau \right)}}\right),$$

  (4)

  where $z_{y_i}$ denotes anchor features from the same class as $z_i$. During backpropagation, the gradient update for each sample i can be derived via the chain rule, with its mathematical formulation expressed as

  $${\displaystyle \frac{\mathsf{\partial }{L}_{\sup }\left(i\right)}{\mathsf{\partial }\theta }}={\displaystyle \frac{\mathsf{\partial }{L}_{\sup }\left(i\right)}{\mathsf{\partial }{z}_i}}·{\displaystyle \frac{\mathsf{\partial }{z}_i}{\mathsf{\partial }\theta }}$$

  (5)
  To account for the long-tailed distribution, the impact on gradient updates can be approximated as

  $${\displaystyle \frac{\mathsf{\partial }{L}_{\sup }\left(i\right)}{\mathsf{\partial }{z}_i}}\propto -{\displaystyle \frac{1}{\tau }}{\displaystyle \frac{\mathsf{\partial }\tilde{s}\left(i\right)}{\mathsf{\partial }{z}_i}}·\left(1-{\displaystyle \frac{exp(\tilde{s}\left(i\right)/\tau )}{{\sum }_{j\in head}exp(\tilde{s}\left(j\right)/\tau )+{\sum }_{j\in tail}exp(\tilde{s}\left(j\right)/\tau )}}\right),$$

  (6)

  where $\tilde{s}\left(i\right)=s(z_i,z_{y_i})$ denotes the positive sample pair similarity, while $\tilde{s}\left(j\right)=s(z_i,z_{y_j})$ represents the negative sample pair similarity. For analytical simplicity, the above formulation presents only the approximated gradient updates for head and tail classes. When ${\sum }_{j\in head}\gg {\sum }_{j\in tail}$, this bias causes gradient updates to concentrate predominantly on head class samples while suppressing updates from tail classes, consequently impairing the model’s ability to learn discriminative features for tail classes. Furthermore, the temperature parameter $\tau$ governs the concentration of similarity distributions. In long-tailed data scenarios, an excessively small $\tau$ value exacerbates class imbalance by further strengthening the dominance of head class samples.

3.3. Rebalancing Supervised Contrastive Learning

Based on the analysis in the previous section, to achieve rebalancing for Supervised Contrastive Learning in long-tailed recognition tasks, we need to address two key issues: (1) the excessive proportion of head class samples in the negative sample feature queue causes the gradient update process to skew toward head classes, thereby weakening the model’s learning capability for tail classes; (2) existing methods rely on label differentiation for local feature comparison, lacking further optimization of the feature space, which amplifies the negative impact of imbalanced data distribution.

To address this, we incorporate two additional terms into the original loss function: an adaptive focal gradient-weighted rebalancing factor to establish balanced feature representations, and a prototype-aware discriminative enhancement module to refine prototype representations in order to construct a more discriminative feature space. As shown in Figure 3, our framework is built on the two-branch momentum contrast (MoCo) architecture [23], integrating the collaborative optimization interaction between the rebalancing module and the prototype-aware constraints. The prototype alignment dynamically adjusts the sample reweighting strategy, and the resulting balanced gradient iteratively optimizes the prototype position, forming a coupled optimization cycle.

3.3.1. Rebalancing Factor

Traditional contrastive learning methods typically assume that negative samples are relatively easy to distinguish, thus focusing primarily on maximizing the similarity of positive samples. Under this assumption, existing studies often employ simple class-frequency-based weighting strategies to balance the training process [41]. However, such approaches fail to adequately account for the complex relationship between class distribution and sample difficulty. A common practice is to treat samples from head classes as inherently easy and those from tail classes as uniformly hard—a simplistic frequency-based division with clear limitations. In reality, accurately assessing sample difficulty should incorporate multiple factors, including their distribution characteristics in feature space and gradient information, rather than relying solely on class frequency [42].

Motivated by this observation, we propose an adaptive focal gradient-weighted rebalancing factor, which introduces a dynamic gradient-aware mechanism into supervised contrastive loss to enable adaptive focus across different classes. Formally, given a batch of samples $\left\{x_i,y_i\right\}$, the enhanced loss function is defined as

$${\mathcal{L}}_i=-\sum _{c\in C}\omega \left(c\right)·{\mathbb{I}}_{c=c_y}·log\left({\displaystyle \frac{exp\left(z_c·\mathcal{G}\left(x_i\right)/\tau \right)}{{\sum }_{z_j\in \mathcal{R}\left(i\right)}exp\left(z_j·\mathcal{G}\left(x_i\right)/\tau \right)}}\right),$$

(7)

where C denotes the original set of categories. $\mathcal{R}\left(i\right)=S_i^{+}\cup S_i^{-}$ represents the positive and negative sample sets for instance i, excluding its own feature $z_i$. $\mathcal{G}\left(·\right)$ denotes a multilayer perceptron composed of two fully-connected layers. The rebalancing factor $\omega \left(c\right)$ is defined as

$$\omega \left(c\right)={\displaystyle \frac{1}{1+\alpha ·{\nabla }_{\mathrm{c}}}}·{\omega }_f\left(c\right),$$

(8)

where $\alpha$ is a hyperparameter controlling the rebalancing intensity, and the gradient term ∇_c represents the average gradient magnitude for class c, reflecting the model’s learning difficulty on this category. It is calculated as follows:

$${\nabla }_c={\displaystyle \frac{1}{\left|{\mathcal{B}}_c\right|}}\sum _{i\in {\mathcal{B}}_c}{∥{\nabla }_{\theta }{\mathcal{L}}_i∥}_2,$$

(9)

where ${\nabla }_{\theta }{\mathcal{L}}_i$ denotes the loss gradient of sample i with respect to model parameters $\theta$ (calculated while freezing $\omega \left(c\right)$ to avoid second-order gradients), and ${\mathcal{B}}_c$ represents the batch of samples from class c. We allocate more attention to classes with higher learning difficulty to prevent them from being “forgotten” or inadequately learned.

To further enhance focus on hard samples, we introduce a focal weight ${\omega }_f\left(c\right)$, which depends on the class frequency $f\left(c\right)$, along with hyperparameters $\beta$ and $\epsilon$ to control the adjustment intensity:

$${\omega }_f\left(c\right)={\displaystyle \frac{\beta }{f\left(c\right)+\epsilon }},$$

(10)

where $\beta$ controls the adjustment intensity of the focal weight, and $\epsilon$ = 1 × 10⁻⁵ is added to prevent division-by-zero errors. Furthermore, to ensure that only positive sample pairs of the current instance are computed, we introduce an indicator function ${\mathbb{I}}_{c=c_y}$, which equals 1 when c matches the label of sample i and 0 otherwise.

In summary, the rebalanced supervised contrastive loss function can be expressed as follows:

$${\mathcal{L}}_{\mathrm{reb}}=\sum _i^N{\displaystyle \frac{1}{\left|\left\{S_i^{+}\right\}\right|}}{\mathcal{L}}_i$$

(11)

3.3.2. Prototype-Aware Discriminative Enhancement Module

Building upon the balanced feature representations established by the aforementioned rebalancing factor, we introduce feature prototypes to further optimize the discriminative capability at the feature space level. By jointly employing aggregation and separation losses, we explicitly constrain the geometric structure of the prototype space. Additionally, we incorporate a swapped prediction loss to enforce bidirectional alignment between features and dynamic prototypes, thereby achieving refined feature space reconstruction on top of the balanced distribution.

As illustrated in Figure 3, for input features $z_q$ and $z_k$, we first introduce a set of K trainable global prototype vectors $\mathbf{C}=[{\mathbf{c}}_1,{\mathbf{c}}_2,\dots ,{\mathbf{c}}_K]\in {\mathbb{R}}^{K\times D}$ stored in the network through the dual-branch architecture. Each prototype ${\mathbf{c}}_i$ dynamically encodes the global data distribution via cross-batch gradient updates, while supporting multiple prototypes per class to capture fine-grained subclass structures. To address the common prototype collapse issue in long-tailed learning, we abandon traditional clustering methods and instead employ optimal transport theory [43], efficiently computing soft assignment matrices ${\mathbf{Q}}_q,{\mathbf{Q}}_k\in {\mathbb{R}}^{K\times B}$ through the Sinkhorn algorithm. This ensures proper prototype distribution in the feature space while preventing prototype collapse and achieving balanced class representation. The final optimization objective can be formally expressed as

$$\underset{\mathbf{Q}\in \mathcal{Q}}{max}Tr\left({\mathbf{Q}}^{\top }{\mathbf{C}}^{\top }\mathbf{Z}\right)+\psi H\left(\mathbf{Q}\right),$$

(12)

where $Tr\left({\mathbf{Q}}^{\top }{\mathbf{C}}^{\top }\mathbf{Z}\right)$ maximizes the trace between the prototype assignment matrix $\mathbf{Q}$ and the feature–prototype similarity matrix ${\mathbf{C}}^{\top }\mathbf{Z}$, directly optimizing the matching degree between prototypes and features. The term $\psi H\left(\mathbf{Q}\right)$ imposes entropy regularization to constrain the uniformity of the assignment matrix, where H denotes the entropy function, $H\left(\mathbf{Q}\right)=-{\sum }_{ij}{\mathbf{Q}}_{ij}log{\mathbf{Q}}_{ij}$. An approximate optimal solution is efficiently obtained through three Sinkhorn iterations while computational efficiency is maintained. Furthermore, to enable online updates within each batch, the constraint set $\mathcal{Q}$ for the optimal transport problem is defined as

$$\mathcal{Q}=\left\{\mathbf{Q}\in {\mathbb{R}}_{+}^{K\times B}\left|\mathbf{Q}{\mathbf{1}}_B={\displaystyle \frac{1}{K}}{\mathbf{1}}_K\right.,{\mathbf{Q}}^{\top }{\mathbf{1}}_K={\displaystyle \frac{1}{B}}{\mathbf{1}}_B\right\}$$

(13)

where ${\mathbf{1}}_K$ denotes a K-dimensional all-ones vector, and ${\mathbf{1}}_B$ denotes a B-dimensional all-ones column vector. These constraints enforce that each prototype is selected at least ${\displaystyle \frac{B}{K}}$ times on average within the batch, preventing all samples from clustering around a few prototypes and ensuring uniform prototype coverage across samples.

To facilitate cross-branch knowledge transfer, we implement a symmetric swapped prediction mechanism that enforces mutual consistency between the prototype assignment distributions predicted by both branches. The computation proceeds as follows: first, we calculate the normalized similarity matrices between the features and prototypes for each branch:

$${\mathbf{S}}_q={\mathbf{Z}}_q{\mathbf{C}}^{\top }/\tau ,{\mathbf{S}}_k={\mathbf{Z}}_k{\mathbf{C}}^{\top }/\tau$$

(14)

These similarity scores are then transformed into probabilistic assignment distributions ${\mathbf{P}}_q$ and ${\mathbf{P}}_k$ through softmax normalization, which subsequently participate in the loss computation:

$${\mathcal{L}}_1={\displaystyle \frac{1}{2B}}\sum _{i=1}^B[\mathcal{L}\left({\mathbf{Q}}_k^i,{\mathbf{P}}_q^i\right)+\mathcal{L}\left({\mathbf{Q}}_q^i,{\mathbf{P}}_k^i)\right]$$

(15)

here, $\mathcal{L}(·,·)$ represents a divergence measure between probability distributions, implemented as cross-entropy loss.

During gradient computation, we prevent backpropagation through ${\mathbf{Q}}_q$ and ${\mathbf{Q}}_k$ to avoid interference with the prototype updating process. Subsequently, to enhance the discriminative power of the feature space, we design an aggregation loss that compels samples from the same class to cluster around their corresponding prototypes:

$${\mathcal{L}}_2={\displaystyle \frac{1}{B}}\sum _{i=1}^B\sum _{k=1}^K{\mathbf{Q}}_q^{ki}{∥{\mathbf{z}}_q^i-{\mathbf{c}}_k∥}_2^2,$$

(16)

where the soft assignment probability ${\mathbf{Q}}_q^{ki}$ serves as weighting coefficients, giving higher-confidence samples greater importance.

Building upon the rapid prototype–sample association established by the aggregation loss, we further design a separation loss to maximize the distances between different prototypes:

$${\mathcal{L}}_3=\sum _{k=1}^K\sum _{k^{\prime }\ne k}max\left(0,\delta -{∥{\mathbf{c}}_k-{\mathbf{c}}_{k^{\prime }}∥}_2^2\right),$$

(17)

where $\delta$ is a hyperparameter controlling the minimum separation threshold between prototypes. The complete objective function combines these components with weighting hyperparameters ${\lambda }_1$, ${\lambda }_2$, and ${\lambda }_3$ to balance their contributions:

$${\mathcal{L}}_{\mathrm{proto}}={\lambda }_1{\mathcal{L}}_1+{\lambda }_2{\mathcal{L}}_2+{\lambda }_3{\mathcal{L}}_3$$

(18)

3.4. Model Training

Our proposed feature rebalancing framework comprises two key components: (1) an adaptive focal gradient-weighted loss function with rebalancing factors, and (2) a discriminative enhancement loss based on feature prototypes. The complete objective function can be expressed as

$$\mathcal{L}={\mathcal{L}}_{\mathrm{proto}}+{\mathcal{L}}_{reb}$$

(19)

These components work synergistically to address the class imbalance problem in long-tailed datasets by ensuring sufficient learning signals for tail classes while simultaneously enhancing the separation and discriminability between head and tail classes, thereby improving overall recognition performance.

4. Experience

4.1. Dataset

We evaluated our method against three widely-used long-tailed recognition benchmarks: CIFAR-100-LT [44], ImageNet-LT [8], and iNaturalist 2018 [45], as illustrated in

Table 1. Basic information of the datasets.

Datasets	Number of Classes	Training/Test Samples	IF
CIFAR-LT	100	10.8 K/10 K	{100, 50, 10}
ImageNet-LT	1000	115.8 K/50 K	256
iNaturalist 2018	8142	437.5 K/24.4 K	500

. Unlike their balanced counterparts (CIFAR-100 [46] and ImageNet-2012 [1]), CIFAR-100-LT and ImageNet-LT are synthetically constructed imbalanced class distributions. In accordance with the standard evaluation protocol in long-tailed recognition [8], we trained the models on the original long-tailed training data and evaluated their performance on uniformly sampled test sets, strictly adhering to the official train–test splits provided for each dataset.

CIFAR-100-LT. CIFAR-100 contains 60,000 images—50,000 for training and 10,000 for validation across 100 classes. For fair comparison, we adopted the long-tailed version of CIFAR with identical experimental settings to those specified in [9,10]. These studies employed an imbalance factor $IF$ to control the degree of data imbalance, defined as $IF=N_{max}/N_{min}$, where $N_{max}$ and $N_{min}$ denote the number of training samples in the most and least frequent classes, respectively. In our experiments, we configured $IF\in \{100,50,10\}$ to conduct comprehensive evaluations. Classes were categorized into head, medium, and tail groups based on sample counts, with the classification thresholds set at 100 and 20.

ImageNet-LT. As a long-tailed version of the ImageNet dataset [8], ImageNet-LT was constructed by sampling a subset following the Pareto distribution with a power value $\alpha =6$ [1]. It comprises 115.8 K images across 1000 categories, with the number of samples per class ranging from 5 to 1280.

iNaturalist 2018. iNaturalist 2018 [45] is a large-scale species classification dataset with extreme label imbalance, comprising 437.5 K images across 8142 categories. Beyond its severe class imbalance, the dataset presents fine-grained recognition challenges—significant visual variations may exist among different individuals or variants of the same species, substantially complicating the classification task. We selected iNaturalist 2018 as our experimental testbed to better simulate real-world species classification scenarios.

4.2. Experimental Setup

Evaluation Metrics. We employ the Top-1 accuracy across the entire dataset as our primary performance metric. Additionally, we report the class-average accuracy to provide a comprehensive evaluation of model performance, along with the accuracy breakdown for three category subsets partitioned by training sample size: head classes (including categories with >100 training samples), medium classes (including categories with sample sizes between 20 and 100), and tail classes (consisting of categories with fewer than 20 samples).

Experimental Basic Setup. For fair comparison with prior works, all experiments followed the same basic configurations as described in [8]. Additional experimental details are provided in

Table 2. Basic information of the datasets.

Datasets	CIFAR-100-LT	ImageNet-LT	iNaturalist 2018
Backbone	ResNet-32	ResNet-50/ResNext-50	ResNet-50
Input resolution	32 × 32	224 × 224	224 × 224
Epochs	400	400	400
Batch size	128	256	512
Initial learning rate	0.02	0.1	0.1
Temperature $\tau$	0.05	0.2	0.2

.

We constructed the feature encoder based on the ResNet series backbone networks. For the CIFAR-LT dataset, ResNet-32 [46] was used as the backbone, with the downsampling layers removed from the original design to accommodate a 32 × 32 input resolution. For the ImageNet-LT dataset, we used standard ResNet-50 [46] and ResNeXt-50 [47] as the encoder backbones. For the iNaturalist 2018 dataset, standard ResNet-50 was used as the encoder backbone. Following MoCo [23,24], the projection head was a two-layer MLP (input dimension 128 → hidden layer 256 → output 128), with ReLU activation applied to the hidden layer, and the output layer normalized the features to the unit hypersphere using $L_2$ normalization. The prototype vector parameter matrix had dimensions of K128, aligned with the feature dimensions to ensure the validity of similarity calculations.

We trained all the Reb-SupCon models using the SGD optimizer, with a momentum coefficient of 0.9 and weight decay set to 5 × 10⁻⁴. The learning rate was decayed from 0.02 to 0 using a cosine scheduler. Contrastive learning typically requires a long training time to converge, with MoCo [23,24] and SWAV [43] training for 800 epochs to achieve model convergence. Supervised Contrastive Learning [14] trains for 350 epochs for feature learning, with an additional 350 epochs for classifier learning. We trained Reb-SupCon for 400 epochs with a batch size of 128 on 4 GPUs. The temperature parameter $\tau$ was set to 0.2, and for CIFAR-100-LT, in order to strictly follow the settings of [23] for fair comparison, we used a lower temperature of 0.05.

For prototype-related parameters, the Sinkhorn algorithm iterated 3 times, the entropy regularization coefficient was set to $\psi =0.05$, and the threshold hyperparameter was set to $\delta =1$. To balance the weights of the loss components, we set the coefficients as ${\lambda }_1=0.5$, ${\lambda }_2=0.3$, and ${\lambda }_3=0.2$, with a gradient rebalancing strength of $\alpha =0.5$ and a focal weight of $\beta =0.1$. Furthermore, following [13], we adopted a differentiated augmentation strategy: the online branch used high-disturbance augmentation with RandAugment to uncover potential patterns in hard samples, while the momentum branch used low-disturbance augmentation with SimAugment to maintain the historical consistency of the negative sample queue. All images were normalized, with the mean and standard deviation parameters consistent with the original settings of the corresponding datasets.

4.3. Comparison with State-of-the-Art Methods

We evaluated Reb-SupCon against several state-of-the-art long-tailed recognition approaches on three benchmark datasets.

Results on CIFAR-100-LT. Table 3 presents the experimental results of Reb-SupCon on CIFAR-100-LT with $IF$ of 100, 50, and 10. Our Reb-SupCon demonstrates consistent improvements across all imbalance scenarios, achieving significantly higher Top-1 accuracy than the CE baseline. Compared to the original SupCon method, Reb-SupCon shows accuracy gains of 6.9% and 4.5% under $IF=100$ and $IF=50$, respectively, validating the effectiveness of integrating rebalancing strategies with contrastive learning. However, in near-balanced scenarios, the improvement reduces to 0.4%, suggesting that while our method specializes in extreme imbalance cases, conventional contrastive learning remains sufficiently effective under mild imbalance with diminishing marginal returns from rebalancing. Compared to the BCL [25], Reb-SupCon still achieves competitive results. Furthermore, it can be observed that the improvement brought by Balanced Softmax (BS) is limited, indicating that the static balancing strategy lacks sufficient adaptability to changes in the data distribution. In contrast, the dynamic gradient adjustment mechanism proposed in this chapter can adapt to varying degrees of class distribution skew, validating its effectiveness.

Results on ImageNet-LT. As shown in Table 4, the experimental results achieved with the large-scale dataset align with the observations from CIFAR-100-LT. Under RandAugment [48] with 400-epoch training, Reb-SupCon achieves 56.9% Top-1 accuracy, still surpassing both baseline and state-of-the-art methods. Crucially, it exhibits a 1.5% accuracy gain on tail classes and a 1.0% improvement on medium classes over BS, while preserving comparable head class performance. This demonstrates that Reb-SupCon significantly mitigates the optimization suppression of tail class samples while maintaining the representational ability of head classes, achieving global balanced optimization. Additionally, as shown in Figure 4, Reb-SupCon further increases the overall accuracy to 59.1% on the ResNeXt-50 backbone network, validating the method’s generalization ability on complex backbone networks. The consistent improvements across architectures demonstrate that our core innovations—dynamic gradient awareness and prototype semantic clustering—are architecture-agnostic, universally improving feature discriminability for long-tailed data.

Table 3. Top-1 accuracy on CIFAR-100-LT.

Imbalance Factor	100	50	10
Cross Entropy (CE)	38.6	44.0	56.4
CE-DRW	41.1	45.6	57.9
LDAM-DRW [9]	41.7	47.9	57.3
BBN [30]	42.6	47.1	59.2
CMO [27]	43.9	48.3	59.5
MoCo v2 [24]	44.6	50.2	63.1
SupCon [14]	45.8	52.0	64.4
Hybrid-SC [31]	46.7	51.8	63.0
ResLT [49]	48.2	52.7	62.0
BCL [25]	51.9	56.4	64.6
SBCL [50]	44.9	48.7	57.9
CC-SAM [51]	49.2	51.9	62.0
GLC-E [52]	47.9	52.4	62.2
GLC-E [52]	47.9	52.4	62.2
BS † [53]	50.8	54.2	63.0
ProCo † [26]	52.6	57.0	65.0
Reb-SupCon (Ours) †	52.7	56.5	64.8

† indicates that the model was trained with RandAugment [48] for 400 epochs.

Table 3. Top-1 accuracy on CIFAR-100-LT.

Imbalance Factor	100	50	10
Cross Entropy (CE)	38.6	44.0	56.4
CE-DRW	41.1	45.6	57.9
LDAM-DRW [9]	41.7	47.9	57.3
BBN [30]	42.6	47.1	59.2
CMO [27]	43.9	48.3	59.5
MoCo v2 [24]	44.6	50.2	63.1
SupCon [14]	45.8	52.0	64.4
Hybrid-SC [31]	46.7	51.8	63.0
ResLT [49]	48.2	52.7	62.0
BCL [25]	51.9	56.4	64.6
SBCL [50]	44.9	48.7	57.9
CC-SAM [51]	49.2	51.9	62.0
GLC-E [52]	47.9	52.4	62.2
GLC-E [52]	47.9	52.4	62.2
BS † [53]	50.8	54.2	63.0
ProCo † [26]	52.6	57.0	65.0
Reb-SupCon (Ours) †	52.7	56.5	64.8

† indicates that the model was trained with RandAugment [48] for 400 epochs.

Table 4. Top-1 accuracy on ImageNet-LT.

Methods	All	Many	Medium	Few
CE	41.6	64.0	33.8	5.8
Focal Loss [54]	43.7	64.3	37.1	8.2
LWS [55]	49.9	60.2	47.2	30.3
LADE [56]	51.9	62.3	49.3	31.2
SBCL [50]	52.5	-	-	-
TSC [10]	52.4	63.5	49.7	30.4
CC-SAM [51]	54.4	-	-	-
GLC-E [52]	53.6	-	-	-
DSCL [36]	57.5	68.3	54.9	35.2
BS † [53]	55.4	65.8	53.2	34.1
ProCo † [26]	57.5	-	-	-
Reb-SupCon (Ours) †	56.9	64.9	54.2	35.6

† indicates that the model was trained with RandAugment [48] for 400 epochs.

Table 4. Top-1 accuracy on ImageNet-LT.

Methods	All	Many	Medium	Few
CE	41.6	64.0	33.8	5.8
Focal Loss [54]	43.7	64.3	37.1	8.2
LWS [55]	49.9	60.2	47.2	30.3
LADE [56]	51.9	62.3	49.3	31.2
SBCL [50]	52.5	-	-	-
TSC [10]	52.4	63.5	49.7	30.4
CC-SAM [51]	54.4	-	-	-
GLC-E [52]	53.6	-	-	-
DSCL [36]	57.5	68.3	54.9	35.2
BS † [53]	55.4	65.8	53.2	34.1
ProCo † [26]	57.5	-	-	-
Reb-SupCon (Ours) †	56.9	64.9	54.2	35.6

† indicates that the model was trained with RandAugment [48] for 400 epochs.

Results on iNaturalist 2018. As presented in

Table 5. Top-1 accuracy on iNaturalist 2018.

Methods	All	Many	Medium	Few
CE	61.0	73.9	63.5	55.5
LDAM-DRW [9]	66.1	-	-	-
BS [53]	70.0	70.0	70.2	69.9
Remix [57]	70.5	-	-	-
RIDE(3E) [58]	72.2	70.2	72.2	72.7
TSC [10]	69.7	72.6	70.6	67.8
PC [59]	70.6	71.6	70.6	70.2
SHIKE [41]	74.5	-	-	-
DSCL [36]	72.0	74.2	72.9	70.3
BS † [53]	71.8	-	-	-
Reb-SupCon (Ours) †	73.0	71.9	73.2	72.5

† indicates that the model was trained with RandAugment [48] for 400 epochs.

, using the ResNet-50 backbone without knowledge distillation, Reb-SupCon demonstrates remarkable advantages in extreme long-tailed scenarios. Compared to the baseline method BS, Reb-SupCon achieves a 3.0% improvement in overall performance, while outperforming the recently proposed DSCL [36] by 1.0%. Fine-grained analysis reveals that Reb-SupCon attains 73.2% and 72.5% accuracy for medium and tail classes, respectively, representing improvements of 0.3% and 2.2% over DSCL—this particularly highlights its superior capability in learning tail class features. Notably, although SHIKE [41] shows marginally higher aggregate accuracy, it relies on complex multi-expert architectures and knowledge distillation strategies. In contrast, Reb-SupCon achieves lightweight yet high performance solely through enhanced supervised contrastive loss, further validating our method’s applicability and efficiency advantages for large-scale open datasets.

4.4. Ablation Study

To validate the effectiveness of the rebalancing factor and the aggregation–separation loss for feature space optimization in Reb-SupCon, we conducted systematic ablation experiments on the CIFAR-100-LT dataset. The experimental results in

Table 6. Ablation study results for primary components of Reb-SupCon.

Imbalance Factor	100	50	10
SupCon	45.8	52.0	64.4
+ Rebalancing factor	50.2	55.1	64.5
+ Prototype module	48.7	54.7	64.5
Reb-SupCon	52.7	56.5	64.8

demonstrate that both components contribute significantly to performance improvement. Quantitative analysis shows that with an imbalance factor of 100, introducing the rebalancing factor alone improves Top-1 accuracy by 4.4%, confirming its effectiveness in mitigating model optimization bias. Further incorporating the aggregation–separation module leads to additional performance gains, with the approach ultimately achieving a 6.9% Top-1 accuracy improvement over the baseline, which fully validates the importance of explicit global class distribution modeling for long-tailed learning.

To better visualize the improvements, we employed multi-dimensional analysis methods. For gradient distributions, Figure 2 compares the average gradient magnitude distributions on ImageNet-LT before and after applying the rebalancing factor, clearly showing that our gradient balancing strategy significantly reduces the disparity between head and tail classes. For feature space analysis, Figure 5 presents t-SNE visualizations on CIFAR-100-LT, revealing deeper insights into feature distributions. The visualization experiment selected 10 representative subclasses (evenly sampled by frequency across head, medium, and tail classes) and projected their 1024-dimensional feature vectors into 3-dimensional space. Comparative analysis shows the following: (1) the baseline model exhibits obvious feature overlap at head–tail class boundaries, indicating insufficient feature learning for tail classes; (2) our method demonstrates superior feature distribution characteristics, achieving local manifold compactness through prototype clustering while maintaining global equilibrium via optimal transport planning, effectively preserving head class discriminability while recovering the underlying manifold structures of tail classes.

4.5. Hyperparameter Sensitivity Analysis

Rebalancing Strength $\mathit{\alpha }$ and Focal Weight $\mathit{\beta }$. We conducted systematic experiments on CIFAR-100-LT with $IF$ of 100, 50, and 10 using a ResNet-32 backbone while fixing other parameters.

Table 7. Ablation study on rebalancing factor in Reb-SupCon.

$\mathit{\alpha }$	$\mathit{\beta }$	Top-1 Accuracy	Class-Avg Accuracy
0.1	0.1	49.3	25.8
0.5	0.1	52.7	30.1
1.0	0.1	51.1	28.5
0.5	0.05	51.7	28.9
0.5	0.5	50.4	26.7

presents the Top-1 and Class-Avg accuracy for combinations of $\alpha \in \{0.1,0.5,1.0\}$ and $\beta \in \{0.05,0.1,0.5\}$. The results show that the combination of $\alpha =0.5$ and $\beta =0.1$ achieves optimal performance. Excessively high $\alpha =1.0$ leads to accuracy degradation, while increasing $\beta$ to 0.5 significantly deteriorates tail class performance, indicating that over-reliance on class-frequency weighting may compromise gradient stability.

Prototype count $\mathit{K}$ and loss weights $\mathit{\lambda }$. As shown in

Table 8. Ablation study results for prototype-related parameters.

K	$\mathit{\lambda }$	All	Many	Medium	Few
800	(0.5, 0.3, 0.2)	55.1	64.3	52.7	33.9
5000	(0.5, 0.3, 0.2)	56.5	64.7	53.9	35.1
3000	(0.5, 0.3, 0.2)	56.9	64.9	54.2	35.6
3000	(0.6, 0.2, 0.2)	56.2	65.1	53.6	34.8
3000	(0.4, 0.4, 0.2)	55.9	64.5	53.2	34.3

, we evaluated the performance under different combinations of K and $\lambda$, using the ResNet-50 backbone with the other parameters fixed. The experimental results show that the model achieves optimal performance when $K=3000$, with the overall accuracy improving by 1.8% compared to $K=800$ and the tail class accuracy increasing by 1.7%. Under this setting, sufficient prototypes effectively cover all 1000 classes in ImageNet-LT, avoiding the prototype assignment conflicts observed at $K=800$, while $K=5000$ leads to performance saturation and increased computational cost, indicating that excessive K introduces redundant prototypes and reduces optimization efficiency.

For loss weights $\lambda$, the model achieves the best balance between inter-class separation and intra-class compactness when the aggregation loss weight ${\lambda }_1=0.5$, the separation loss weight ${\lambda }_2=0.3$, and the alignment loss weight ${\lambda }_3=0.2$. Increasing ${\lambda }_1$ to 0.6 (emphasizing intra-class aggregation) weakens tail class discriminability due to over-constrained intra-class variance, leading to decreased tail class performance. Conversely, raising ${\lambda }_2$ to 0.4 (strengthening inter-class separation) reduces the head class accuracy by 0.4%, suggesting that excessive inter-class loss may compromise the representation stability of head classes. Furthermore, increasing the alignment loss weight ${\lambda }_3$ to 0.3 causes slight performance degradation, indicating that excessive alignment constraints may suppress the adaptive optimization capability of prototype clustering.

In summary, the combination of $K=3000$ and $\lambda =(0.5,0.3,0.2)$ achieves coordinated optimization of global–local features in long-tailed scenarios through balanced prototype granularity and loss weight allocation, providing a robust and efficient learning paradigm for large-scale imbalanced data.

5. Conclusions

To address the optimization bias in Supervised Contrastive Learning for long-tailed image recognition, this paper proposes a Rebalancing Supervised Contrastive Learning (Reb-SupCon) method that integrates gradient reweighting and prototype calibration. Through theoretical analysis, we first reveal the fundamental causes of performance degradation in conventional methods under long-tailed data. Subsequently, we design a dynamic gradient adjustment mechanism and global feature distribution constraints to achieve coordinated optimization of head and tail class representations. Visualization results of features align with experimental expectations, while comprehensive evaluations on multiple benchmark datasets demonstrate that our method achieves state-of-the-art performance.

Limitations. Although our method shows progress, it comes with some limitations. The rebalancing factor enhances tail class discriminability, but slightly reduces head class performance, reflecting long-tail learning’s inherent trade-off. At the same time, while the prototype calibration mechanism is effective, it introduces additional computational overhead and increases training time, limiting its applicability in resource-constrained environments. Future work will focus on exploring lightweight prototype allocation algorithms and adaptive prototype prediction mechanisms to further optimize feature learning for long-tailed data.