Meta-Label-Corrected Knowledge Distillation for Partial Multi-Label Learning

Abstract

Partial multi-label learning (PML) assigns each instance a candidate label set that contains all relevant labels but may also include irrelevant noisy ones, making reliable disambiguation essential. Although a small number of verified clean labels is often available in practice, existing PML methods rarely exploit such information to explicitly guide candidate-label correction. Meanwhile, directly applying knowledge distillation (KD) to PML is highly vulnerable to noisy supervision during representation learning, which can aggravate error accumulation under overlapping candidate labels. To address these issues, we propose a meta-guided distillation framework for PML that integrates teacher–student learning with nested meta-optimization. Specifically, the teacher is optimized with large-scale noisy data under the guidance of limited clean labels, so that it can learn calibrated probabilistic label semantics and generate corrected soft targets for student training. To make this meta-correction process scalable, a truncated meta-gradient approximation is further adopted to reduce computational overhead. The resulting corrected teacher outputs are then used to drive robust multi-label distillation for the student. Experiments on multiple benchmark multi-label image datasets demonstrate consistent improvements over seven representative PML methods across standard evaluation metrics. These results show that meta-guided calibration effectively reduces semantic ambiguity and mitigates noise-induced error propagation in partial multi-label learning.

1. Introduction

In recent years, multi-label learning has attracted growing research interest. This task focuses on determining whether each label category is relevant to a given sample [1]. In real-world applications, obtaining accurately annotated datasets is often difficult because manual labeling requires substantial labor cost. For this reason, crowd-sourcing platforms are widely used for data collection. However, non-expert annotators may assign labels that are not truly relevant to the sample. This behavior leads to over-annotation. This phenomenon is illustrated in Figure 1. Due to visual occlusion, the annotator cannot clearly judge whether the image contains sunflowers or daisies. The annotator also fails to distinguish whether the background is forest or mountain. As a result, the annotator assigns all four labels as possible annotations for the image. Such cases directly give rise to the Partial Multi-label Learning (PML) problem [2]. In the PML setting, each instance is associated with a candidate label set. This set contains all relevant labels, but it also includes some irrelevant ones. To address this issue, many PML methods have been proposed to resolve label ambiguity and to learn robust models. These methods have achieved strong performance on PML benchmark table datasets [3,4,5]

Figure 1. This illustrates a case of partial multi-label learning. In the given candidate label set, the image’s true labels are sunflower and leaf, while mountain and daisy are incorrect annotations.

Motivated by the success of deep neural networks, researchers have begun to explore PML methods based on DNNs, especially for image classification tasks [5]. However, the number of existing deep PML methods remains limited. To address the PML problem, existing studies have explored different strategies to alleviate the negative impact of noisy supervision in candidate label sets. However, clear limitations still remain in terms of supervision utilization and optimization mechanisms.The first category of methods originates from traditional multi-label learning frameworks. Representative approaches such as UNM [6] enhance the overall robustness under diverse data conditions through loss re-weighting and regularization strategies. These methods usually assume that training labels are statistically reliable and, therefore, focus on global-level robust optimization. Since they do not explicitly model candidate label sets, they struggle to handle scenarios where all samples are only partially labeled. As a result, they cannot effectively exploit limited but reliable supervision to guide the learning process. The second category consists of non-deep-learning-based PML methods. Early studies mainly focus on label structure and noise modeling, aiming to distinguish reliable labels from unreliable ones within candidate label sets. PML-NI [3] performs true label recovery and noisy label identification within a unified framework. This method jointly models label correlations and feature structures through regularization, enabling candidate label selection and noise suppression during training. PML-MD [7] further introduces meta-learning to the label correction process, allowing the correction strategy to adapt throughout training. However, these methods mainly operate on explicit labels or weight assignments and rely on global constraints, which limits their flexibility in complex scenarios. Building upon these efforts, PML-NLR [8] models noisy label identification as a constrained regression problem and selectively processes candidate labels according to estimated noise levels. As a more recent non-deep-learning approach, PML-NLR provides a clearer modeling formulation. Nevertheless, it still depends on rule-driven mechanisms. Its ability to utilize supervision remains limited under high noise levels or complex label structures.The third category includes deep-learning-based PML modeling strategies. PASE [9] addresses label noise at the feature level by introducing label-specific feature correction, which reduces the negative influence of noisy labels. This type of method relies on global consistency assumptions in the feature space. As a result, its stability in label correction remains limited in complex settings. Other studies introduce curriculum learning to improve the training stability by controlling the order in which samples are used. PML-CDCR [10] adopts a training strategy from easy to hard, allowing the model to prioritize more reliable supervision. PML-CD [11] reduces systematic bias by enforcing consistency in prediction confidence. Although these methods improve the training behavior to some extent, most of them rely on heuristic designs. They lack explicit modeling of supervision transfer and fail to fully leverage a small amount of clean data to guide overall learning. Multi-Label Knowledge Distillation (MLKD) [12], built upon the knowledge distillation (KD) paradigm [13], aims to improve the performance of a student model by transferring supervisory signals from a teacher model. In weakly supervised settings, such as semi-supervised learning, a small amount of accurately labeled data is often used to guide the model in capturing underlying structures and discriminative patterns from large-scale unlabeled or weakly labeled data. However, existing MLKD methods lack an effective mechanism to distinguish noisy supervision from reliable supervision during training. The teacher model is typically optimized on contaminated data and may propagate unreliable supervisory signals to the student model, which significantly limits the applicability of MLKD in high-noise scenarios such as Partial Multi-Label Learning. As a result, an important and unresolved challenge is how to simultaneously exploit abundant noisy data and a small amount of clean data to properly constrain and guide the distillation process. To provide intuitive evidence for this limitation, Figure 2 visualizes the attention responses obtained under different training strategies. When only knowledge distillation is applied, the model tends to produce dispersed and less discriminative activation patterns, which indicates that noisy supervision may mislead the distillation process. In contrast, incorporating meta-guided calibration enables the model to focus on more semantically consistent object regions. This observation suggests that conventional KD alone is insufficient to ensure reliable supervision under noisy labels, highlighting the necessity of introducing additional mechanisms to enhance label reliability during distillation.

Figure 2. Visualization comparison of attention responses under different training strategies under 70% noise. Each column corresponds to a target category (left: bicycle, right: person). For each example, the top-left image shows the original input image. The top-right image shows the attention map obtained using knowledge distillation only (KD-only). The bottom-left image shows the attention map obtained using meta-learning correction only (Meta-only). The bottom-right image shows the attention map produced by the proposed MCKD framework. Compared with the baseline variants, MCKD produces more concentrated and semantically consistent activation regions around the true object areas.

To address the aforementioned limitations, we propose a novel PML method termed MCKD (Meta-Corrected Knowledge Distillation for Partial Multi-Label Learning). MCKD enhances the robustness of Multi-Label Knowledge Distillation (MLKD) by integrating a meta-learning framework into the distillation process. Different from existing meta-learning-based approaches, such as meta-sample weighting, meta-robustification for artificially injected noise, meta-label correction, meta-soft label correction, and meta-label-specific learning, MCKD focuses on improving the optimization of the teacher model itself. We refine the conventional nested optimization procedure by deriving a more accurate meta-gradient for teacher model optimization. Specifically, we adopt the exact meta-gradient formulation in the one-step look-ahead setting and further propose an improved approximation strategy for the multi-step look-ahead scenario. This design enables more stable and efficient meta-optimization in multi-label knowledge distillation. Moreover, we introduce a novel teacher architecture for multi-label knowledge distillation. The teacher model first generates initial predictions through feature extraction enhanced by a dedicated teacher enhancer. By further incorporating supervision signals from noisy labels, the teacher produces refined soft labels. These refined soft labels are then used for both iterative self-training of the teacher model and knowledge transfer to the student model. Notably, the label correction mechanism is activated only during the pretraining stage of the teacher model. Afterward, the trained teacher is fixed and used solely to guide label correction and learning in the student model, without further updating its parameters. Our main contributions are summarized as follows:

We propose a meta-learning-enhanced knowledge distillation framework, termed MCKD, for Partial Multi-Label Learning. The proposed framework is designed for scenarios where training samples are partially labeled, and clean annotations are scarce. By jointly exploiting abundant partially labeled data and a small clean meta set, MCKD provides a principled way to refine candidate label supervision for student learning.

We develop a meta-gradient computation strategy tailored to the multi-label knowledge distillation objective, which enables more stable optimization of the teacher model under the nested learning framework.

We design a dedicated teacher architecture and training objective for multi-label knowledge distillation in PML. The teacher model performs label refinement during the pretraining stage and is then fixed to provide corrected soft supervision for student learning.

2. Related Work

In recent years, multi-label image classification has attracted increasing attention in the computer vision community, as real-world images often contain multiple semantically related objects or attributes simultaneously [14]. Compared with single-label classification, multi-label learning not only requires models to recognize multiple targets within an image but also demands effective modeling of semantic dependencies and structural relationships among labels. This requirement better reflects the complexity of real-world visual understanding. To exploit label correlations, a variety of network architectures and learning mechanisms have been proposed based on convolutional neural networks.

Recurrent neural networks have been employed to model dependencies among label sequences [15]. Graph convolutional networks construct label graphs and encode semantic relationships between labels as graph structures, enabling feature propagation across labels [16,17]. Graph propagation methods further introduce cross-label information transfer mechanisms to strengthen contextual semantic modeling [18]. Meanwhile, Transformer-based architectures have demonstrated strong capability in capturing long-range dependencies and complex label interactions through self-attention mechanisms [1,19]. More recently, label relationship modeling methods have partitioned class labels into subgroups according to co-occurrence and mutual exclusivity patterns. Within each group, multiple prompt tokens and mixture-of-experts models are employed to balance correlation learning and discriminative representation among labels. This strategy effectively alleviates overfitting caused by excessively strong label dependencies [20]. Attention mechanisms have also shown practical value beyond visual recognition. Recent work applied an attention-based fault diagnosis model with Bayesian multi-parameter joint optimization to a CO₂ heat pump system, further demonstrating the application potential of attention-based learning frameworks in complex engineering scenarios [21].

Despite these advances, most existing multi-label image classification methods focus on modeling label correlations under clean annotation assumptions. Research on robustness to label noise or partial annotation remains relatively limited. To address this gap, the label-adaptive selection and correction meta-learning method proposed in this work introduces adaptive mechanisms from the label perspective within a meta-learning framework. This design enables the dynamic evaluation and correction of candidate label reliability, thereby improving the robustness and generalization performance in noisy and partial multi-label scenarios. Partial Multi-Label Learning studies a learning scenario where each sample is associated with a candidate label set, while only a subset of these labels is truly relevant. Early studies mainly rely on global structural assumptions, such as the manifold assumption or label correlation assumption, to disambiguate candidate labels [4,15,22]. These methods attempt to reduce label ambiguity by modeling similarities among samples or labels. However, their ability to assess supervision reliability remains limited when facing high noise levels or complex label interactions.Another line of research is based on the low-rank decomposition assumption, which assumes that the ground-truth label matrix exhibits a low-rank structure and that noisy labels can be treated as perturbations [7,23]. By imposing low-rank constraints on noisy label data, these methods aim to separate true labels from noise. Nevertheless, in practical high-dimensional label spaces, the low-rank assumption is often difficult to satisfy. This limitation may lead to information loss or biased estimation, thereby restricting the applicability of such approaches in complex scenarios.

With further development, several methods attempt to combine manifold-based and low-rank assumptions in a more refined manner to improve candidate label modeling [8,23,24,25]. Although these approaches enhance modeling capacity at the theoretical level, they still face challenges when integrated into deep learning frameworks. Moreover, their ability to effectively exploit supervision under high noise conditions remains insufficient. In particular, when all samples are only partially labeled, these methods lack reliable supervision to properly constrain the training process. Recently, researchers have begun to explore the applicability of PML in image classification tasks. Some methods leverage the memorization effect of deep neural networks to progressively filter supervision signals during training [26]. Others introduce curriculum-based strategies to mitigate the influence of noisy labels [5,6]. In addition, several studies analyze curriculum learning from a theoretical perspective by interpreting curriculum weights as a form of prediction confidence calibration and further design transferable calibration modules to improve training stability. However, these methods mainly operate at the training strategy level and often rely on heuristic designs. They do not explicitly model supervision transfer or optimization dynamics in PML.

In contrast to the above approaches, this work proposes a knowledge-distillation-based meta-learning framework, termed MCKD, for Partial Multi-Label Learning. By integrating multi-label knowledge distillation into a meta-learning framework, MCKD explicitly constrains and optimizes the distillation process under noisy supervision. The proposed method is able to jointly exploit abundant partially labeled data and a small amount of clean data under high annotation cost settings. Through this design, MCKD enables the teacher model to generate more reliable soft supervision and further improves the training effectiveness and generalization performance of the student model, demonstrating clear advantages over conventional PML methods.

3. Preliminaries

In PML, each instance x is drawn from an input space $\chi \subset {\mathbb{R}}^d$, where d is the feature dimension. Each instance may be associated with multiple labels from a finite set $\{1,\dots ,L\}$, where L is the number of labels. The unknown ground-truth multi-label vector is denoted by $y\in {\{0,1\}}^L$, where $y^{\left(l\right)}=1$ indicates that label l is relevant. An annotator provides a candidate label vector $\tilde{y}\in {\{0,1\}}^L$. Under the standard PML assumption, $\tilde{y}$ is a superset of y, namely $y^{\left(l\right)}=1\Rightarrow {\tilde{y}}^{\left(l\right)}=1$. The candidate set may include irrelevant labels and does not provide confirmed negatives. Figure 1 illustrates this ambiguity.

Let ${\mathcal{D}}_{\mathrm{noisy}}={\left\{(x_i,{\tilde{y}}_i)\right\}}_{i=1}^n$ denote a partially labeled training set, and let ${\mathcal{D}}_{\mathrm{clean}}={\left\{(x_j,y_j)\right\}}_{j=1}^{n_c}$ denote a small clean set, where n and $n_c$ are the numbers of noisy and clean samples, respectively. We consider a student model parameterized by $\theta$. Given x, the student produces logits $s_{\theta }\left(x\right)\in {\mathbb{R}}^L$ and probabilities $p_{\theta }\left(x\right)=\sigma \left(s_{\theta }\left(x\right)\right)\in {(0,1)}^L$ via the element-wise sigmoid, i.e., $p_{\theta }^{\left(l\right)}\left(x\right)=\sigma \left(s_{\theta }^{\left(l\right)}\left(x\right)\right)$. We also introduce a meta-teacher model parameterized by $\phi$, which outputs a soft correction target for each candidate label. We denote the corrected target by $q_{\phi }(x_i,{\tilde{y}}_i)\in {[0,1]}^L$ and use $q_{\phi }^{\left(l\right)}(x_i,{\tilde{y}}_i)$ for its l-th element.

We use $\mathrm{BCE}(a,b)$ to denote the binary cross-entropy between a target $a\in [0,1]$ and a predicted probability $b\in (0,1)$. Since PML provides no confirmed negatives, the inner correction loss is computed only on candidate labels. Specifically, we define a candidate mask $M\left({\tilde{y}}_i\right)\in {\{0,1\}}^L$ with $M^{\left(l\right)}\left({\tilde{y}}_i\right)={\tilde{y}}_i^{\left(l\right)}$ and set

$${\ell }_{\mathrm{corr}}(\theta ,\phi )={\displaystyle \frac{1}{n}}\sum _{i=1}^n\sum _{l=1}^L{M}^{\left(l\right)}\left({\tilde{y}}_i\right)\mathrm{BCE}\left(q_{\phi }^{\left(l\right)}(x_i,{\tilde{y}}_i),p_{\theta }^{\left(l\right)}\left(x_i\right)\right).$$

(1)

The meta outer loss evaluates the induced student on clean labels:

$${\ell }_{\mathrm{meta}}\left(\theta \right)={\displaystyle \frac{1}{n_c}}\sum _{j=1}^{n_c}\sum _{l=1}^L\mathrm{BCE}\left(y_j^{\left(l\right)},p_{\theta }^{\left(l\right)}\left(x_j\right)\right).$$

(2)

The nested objective minimizes ${\ell }_{\mathrm{meta}}\left({\theta }^{*}\left(\phi \right)\right)$ subject to

$${\theta }^{*}\left(\phi \right)=arg\underset{\theta }{min}{\ell }_{\mathrm{corr}}(\theta ,\phi ).$$

(3)

We use t to index outer iterations and use $\tau$ to index inner-loop updates. The student learning rate in the inner loop is denoted by ${\eta }_{\theta }$. The discount factor used in the multi-step approximation is denoted by ${\gamma }_{\theta }$. We use ${\partial }_{\theta }$ and ${\partial }_{\phi }$ to denote partial derivatives with respect to $\theta$ and $\phi$, respectively.

4. Knowledge Distillation in Partial Multi-Label Learning

An overview of MCKD is shown in Figure 3. Our framework involves three distinct model roles. First, a distillation teacher, parameterized by ${\theta }_t$, is used only in Stage (1) to provide supervision for knowledge distillation pretraining. Second, a meta-teacher, parameterized by $\phi$, is learned to generate corrected soft targets for noisy candidate labels. Third, a student, parameterized by $\theta$, is trained using these corrected targets. To avoid ambiguity, we use the term teacher only when referring to the distillation teacher ${\theta }_t$, and the term meta-teacher when referring to the label-correction network $\phi$. During Stage (1), we denote the student in the distillation objective by ${\theta }_s$. This notation is used only within the pretraining stage. In Stages (2) and (3), the student parameters are denoted by $\theta$.

Figure 3. Overview of the MCKD framework. Stage (1) uses a distillation teacher ${\theta }_t$ to pretrain the network that initializes the meta-teacher. Stage (2) optimizes the meta-teacher $\phi$ through nested meta-learning, where the meta-teacher generates corrected soft targets for noisy candidate labels, and the student $\theta$ is updated in the inner loop. Stage (3) fixes the learned meta-teacher and trains the student using the corrected targets.

In this subsection, $\phi$ denotes the parameters of the meta-teacher used for label correction, and $\theta$ denotes the parameters of the final PML student. During the distillation pretraining stage, we use $({\theta }_t,{\theta }_s)$ to denote the distillation teacher and student, respectively. The pair $({\theta }_t,{\theta }_s)$ is only used in Stage (1) and is independent from $(\phi ,\theta )$.

For each training instance $(x_i,{\tilde{y}}_i)\in {\mathcal{D}}_{\mathrm{noisy}}$, the meta-teacher outputs a corrected soft target vector $q_{\phi }(x_i,{\tilde{y}}_i)\in {[0,1]}^L$, where $q_{\phi }^{\left(l\right)}(x_i,{\tilde{y}}_i)$ denotes the l-th element. These corrected targets are used in the inner correction loss ${\ell }_{\mathrm{corr}}(\theta ,\phi )$ defined in (18).

The meta-teacher consists of three components. (1) Disentangled feature extraction. The meta-teacher uses its own feature extractor $h_{\phi }:\chi \to {\mathbb{R}}^D$ and classifier head $C_{\phi }:{\mathbb{R}}^D\to {\mathbb{R}}^L$. Given $x_i$, the teacher prediction is

$$c_{\phi }\left(x_i\right)=\sigma \left(C_{\phi }\left(h_{\phi }\left(x_i\right)\right)\right),$$

(4)

where $\sigma (·)$ is the element-wise sigmoid. The meta-teacher does not share parameters with the student network, which reduces confirmation bias and prevents directly inheriting noise memorization from the student. (2) Noise-aware label embedding. We embed the candidate label vector ${\tilde{y}}_i\in {\{0,1\}}^L$ as

$$e_{\phi }\left({\tilde{y}}_i\right)=W_e{\tilde{y}}_i,$$

(5)

where $W_e$ is a learnable projection. We concatenate $e_{\phi }\left({\tilde{y}}_i\right)$ with $h_{\phi }\left(x_i\right)$ and feed the result into a gating MLP ${\pi }_{\phi }(·)$. (3) Adaptive label correction. The gating MLP takes the concatenated feature-label representation $[h_{\phi }\left(x_i\right);e_{\phi }\left({\tilde{y}}_i\right)]\in {\mathbb{R}}^{D+D_e}$ as input and outputs an instance-wise scalar gate:

$${\alpha }_{\phi }^{\left(i\right)}=\sigma \left({\pi }_{\phi }\left([h_{\phi }\left(x_i\right);e_{\phi }\left({\tilde{y}}_i\right)]\right)\right),$$

(6)

where ${\pi }_{\phi }(·)$ denotes a lightweight MLP with a one-dimensional output, and $\sigma (·)$ is the sigmoid function. Therefore, ${\alpha }_{\phi }^{\left(i\right)}\in (0,1)$ is a scalar coefficient shared by all labels of instance $x_i$. No additional normalization across labels is applied. The corrected target interpolates between the teacher prediction and the candidate label vector:

$$q_{\phi }(x_i,{\tilde{y}}_i)={\alpha }_{\phi }^{\left(i\right)}{c}_{\phi }\left(x_i\right)+\left(1-{\alpha }_{\phi }^{\left(i\right)}\right){\tilde{y}}_i,$$

(7)

where both $c_{\phi }\left(x_i\right)$ and ${\tilde{y}}_i$ are L-dimensional vectors, and the scalar ${\alpha }_{\phi }^{\left(i\right)}$ is broadcast to each label dimension. A larger ${\alpha }_{\phi }^{\left(i\right)}$ places more weight on the meta-teacher prediction, while a smaller ${\alpha }_{\phi }^{\left(i\right)}$ retains more candidate-label information. In implementation, ${\pi }_{\phi }$ is instantiated as a two-layer MLP.

Three-stage training procedure. Our training consists of the following stages.

Stage (1): MLKD pretraining. We first initialize the meta-teacher parameters $\phi$ via a multi-label knowledge distillation objective. In this stage only, the network that will later serve as the meta-teacher is trained as the student in the distillation objective, and we denote its parameters by ${\theta }_s$ with ${\theta }_s\leftarrow \phi$. The distillation teacher is parameterized by ${\theta }_t$. After pretraining, the obtained ${\theta }_s$ is used to initialize the meta-teacher parameters $\phi$. If needed, we disable the label-embedding and gating pathway and pretrain only $h_{\phi }$ and $C_{\phi }$.

Stage (2): Nested meta-learning. We optimize the meta-teacher parameters $\phi$ by solving the nested problem in (16). The outer loss ${\ell }_{\mathrm{meta}}$ is evaluated on the clean set ${\mathcal{D}}_{\mathrm{clean}}$, and the inner correction loss ${\ell }_{\mathrm{corr}}$ trains the student on ${\mathcal{D}}_{\mathrm{noisy}}$ using the corrected targets $q_{\phi }(x_i,{\tilde{y}}_i)$.

Stage (3): Student training. After meta-learning, we fix $\phi$ and train the student on ${\mathcal{D}}_{\mathrm{noisy}}$ using $q_{\phi }(x_i,{\tilde{y}}_i)$ as supervision.

The meta-teacher objective can be summarized as

$${\ell }_{\mathrm{teach}}\left(\phi \right)={\mathcal{L}}_{\mathrm{MLKD}}({\theta }_t,{\theta }_s=\phi )+\lambda {\ell }_{\mathrm{meta}}\left({\theta }^{*}\left(\phi \right)\right),$$

(8)

where $\lambda \ge 0$ balances the distillation pretraining term and the nested meta-learning term.

5. Multi-Label Knowledge Distillation

We introduce a multi-label knowledge distillation objective to transfer both semantic predictions and representation structure from a distillation teacher to a student. Let $({\theta }_t,{\theta }_s)$ denote the parameters of the distillation teacher and student, respectively. For an input x, the teacher and student output logits $s_{{\theta }_t}\left(x\right),s_{{\theta }_s}\left(x\right)\in {\mathbb{R}}^L$ and probabilities $p_{{\theta }_t}\left(x\right)$, $p_{{\theta }_s}\left(x\right)\in {(0,1)}^L$ via the element-wise sigmoid.

Given a training set $\left\{(x,\tilde{y})\right\}$ with noisy candidate labels, we optimize the student using the following composite loss:

$${\mathcal{L}}_{\mathrm{MLKD}}={\mathcal{L}}_{\mathrm{BCE}}+{\lambda }_{\mathrm{MLD}}{\mathcal{L}}_{\mathrm{MLD}}+{\lambda }_{\mathrm{CD}}{\mathcal{L}}_{\mathrm{CD}}+{\lambda }_{\mathrm{ID}}{\mathcal{L}}_{\mathrm{ID}},$$

(9)

where ${\lambda }_{\mathrm{MLD}},{\lambda }_{\mathrm{CD}},{\lambda }_{\mathrm{ID}}\ge 0$ are balancing coefficients. Following the multi-label knowledge distillation framework in [12], we define the four components of ${\mathcal{L}}_{\mathrm{MLKD}}$ as follows.

Let a mini-batch contain b training instances, and let q denote the number of labels. For the i-th instance and the k-th label, we use $y_{ik}$ to denote the supervision target, ${\widehat{y}}_{ik}^T$ and ${\widehat{y}}_{ik}^S$ to denote the teacher and student predicted probabilities, respectively, and $e_{ik}^T$ and $e_{ik}^S$ to denote the corresponding teacher and student label embeddings.

The binary cross-entropy term is defined as

$${\mathcal{L}}_{\mathrm{BCE}}=-{\displaystyle \frac{1}{b}}\sum _{i=1}^b\sum _{k=1}^q\left[y_{ik}log{\widehat{y}}_{ik}^S+(1-y_{ik})log\left(1-{\widehat{y}}_{ik}^S\right)\right].$$

(10)

The multi-label distillation term is defined as

$${\mathcal{L}}_{\mathrm{MLD}}={\displaystyle \frac{1}{b}}\sum _{i=1}^b\sum _{k=1}^{q}D\left({\widehat{y}}_{ik}^T,{\widehat{y}}_{ik}^S\right),$$

(11)

where $D(·,·)$ denotes the divergence between the teacher and student binary predicted probabilities for each label.

To preserve class-aware structural consistency across instances, we define

$${\mathcal{L}}_{\mathrm{CD}}=\sum _{k=1}^q\sum _{i,j\in \left[b\right]}{\ell }_{\mathrm{hub}}\left({\varphi }_{\mathrm{CD}}(e_{ik}^T,e_{jk}^T),{\varphi }_{\mathrm{CD}}(e_{ik}^S,e_{jk}^S)\right),$$

(12)

where ${\varphi }_{\mathrm{CD}}(·,·)$ denotes the relation function measuring the structural relation of the same label across different instances, and ${\ell }_{\mathrm{hub}}(·,·)$ denotes the Huber loss.

To preserve instance-aware structural consistency across labels, we define

$${\mathcal{L}}_{\mathrm{ID}}=\sum _{i=1}^b\sum _{k,l\in \left[q\right]}{\ell }_{\mathrm{hub}}\left({\varphi }_{\mathrm{ID}}(e_{ik}^T,e_{il}^T),{\varphi }_{\mathrm{ID}}(e_{ik}^S,e_{il}^S)\right),$$

(13)

where ${\varphi }_{\mathrm{ID}}(·,·)$ denotes the relation function measuring the structural relation among different label embeddings within the same instance, and ${\ell }_{\mathrm{hub}}(·,·)$ denotes the Huber loss.

In our PML setting, the above distillation objective is used only in Stage (1) to pretrain the network that initializes the meta-teacher. This design provides a robust initialization for the subsequent meta-learning under candidate-label ambiguity. The term ${\mathcal{L}}_{\mathrm{BCE}}$ provides task-level supervision for multi-label prediction. The term ${\mathcal{L}}_{\mathrm{MLD}}$ transfers the teacher’s soft semantic predictions to the student. The term ${\mathcal{L}}_{\mathrm{CD}}$ enforces class-aware structural consistency by aligning the relations of the same label across different instances. The term ${\mathcal{L}}_{\mathrm{ID}}$ enforces instance-aware structural consistency by aligning the relations among different label embeddings within the same instance. Together, these terms encourage the student to match the teacher in both semantic outputs and embedding geometry, thereby improving robustness under noisy multi-label supervision. Unless otherwise specified, the balancing coefficients associated with the above loss terms are set according to the default configuration of the baseline multi-label distillation framework in [12].

6. Nested Optimization

Deep networks can easily fit ambiguous supervision. In PML, directly minimizing empirical risk on candidate labels may reinforce spurious correlations, because $\tilde{y}$ contains irrelevant candidates and provides no confirmed negatives. To obtain reliable supervision under candidate-label noise, we introduce a meta label-correction mechanism with a nested optimization structure.

In the fully supervised multi-label setting, empirical risk minimization (ERM) learns parameters by minimizing an average per-sample loss:

$${\theta }^{*}=arg\underset{\theta }{min}{\displaystyle \frac{1}{N}}\sum _{i=1}^N\ell \left(f_{\theta }\left(x_i\right),y_i\right),$$

(14)

where $y_i\in {\{0,1\}}^L$ and $y_i^{\left(l\right)}=1$ indicates that label l is present. With binary cross-entropy, the objective becomes

$${\theta }^{*}=arg\underset{\theta }{min}{\displaystyle \frac{1}{N}}\sum _{i=1}^N\sum _{l=1}^L\left[-y_i^{\left(l\right)}log{p}_{\theta }^{\left(l\right)}\left(x_i\right)-(1-y_i^{\left(l\right)})log\left(1-p_{\theta }^{\left(l\right)}\left(x_i\right)\right)\right].$$

(15)

Under the standard PML setting, the observed candidate vector ${\tilde{y}}_i\in {\{0,1\}}^L$ satisfies the superset property $y_i^{\left(l\right)}=1\Rightarrow {\tilde{y}}_i^{\left(l\right)}=1$. The candidate set may include irrelevant labels and does not provide confirmed negatives. As a result, directly replacing $y_i$ by ${\tilde{y}}_i$ in (15) can induce systematic bias. We introduce a meta-teacher parameterized by $\phi$ and a student parameterized by $\theta$. For each noisy training pair $(x_i,{\tilde{y}}_i)$, the meta-teacher outputs a corrected soft target vector $q_{\phi }(x_i,{\tilde{y}}_i)\in {[0,1]}^L$, and we use $q_{\phi }^{\left(l\right)}(x_i,{\tilde{y}}_i)$ for its l-th element. The student predicts $p_{\theta }\left(x_i\right)\in {(0,1)}^L$. Let ${\mathcal{D}}_{\mathrm{noisy}}={\left\{(x_i,{\tilde{y}}_i)\right\}}_{i=1}^n$ denote the partially labeled training set, and let ${\mathcal{D}}_{\mathrm{clean}}={\left\{(x_j,y_j)\right\}}_{j=1}^{n_c}$ denote a small clean set. We optimize $\phi$ so that a student trained using the teacher’s corrected targets generalizes well on the clean set:

$$\underset{\phi }{min}{\ell }_{\mathrm{meta}}\left({\theta }^{*}\left(\phi \right)\right)\mathrm{s}.\mathrm{t}.{\theta }^{*}\left(\phi \right)=arg\underset{\theta }{min}{\ell }_{\mathrm{corr}}(\theta ,\phi ).$$

(16)

Although the above formulation is expressed at the dataset level, the optimization is implemented in a mini-batch manner, and the meta-update relies on truncated inner-step differentiation to control memory and computational cost, which makes the proposed nested optimization practical for large-scale training. The meta outer loss on clean labels is

$${\ell }_{\mathrm{meta}}\left(\theta \right)={\displaystyle \frac{1}{n_c}}\sum _{j=1}^{n_c}\sum _{l=1}^L\mathrm{BCE}\left(y_j^{\left(l\right)},p_{\theta }^{\left(l\right)}\left(x_j\right)\right).$$

(17)

Since PML provides no confirmed negatives, we compute the inner correction loss only on candidate labels using the candidate mask $M\left({\tilde{y}}_i\right)\in {\{0,1\}}^L$ with $M^{\left(l\right)}\left({\tilde{y}}_i\right)={\tilde{y}}_i^{\left(l\right)}$:

$${\ell }_{\mathrm{corr}}(\theta ,\phi )={\displaystyle \frac{1}{n}}\sum _{i=1}^n\sum _{l=1}^L{M}^{\left(l\right)}\left({\tilde{y}}_i\right)\mathrm{BCE}\left(q_{\phi }^{\left(l\right)}(x_i,{\tilde{y}}_i),p_{\theta }^{\left(l\right)}\left(x_i\right)\right).$$

(18)

We compute the meta-gradient by differentiating through the inner optimization, and we use efficient approximations to scale the update when multiple inner steps are used. We note that a direct runtime comparison with prior PML methods is not always fully informative, since for representative baselines such as PML-MD [7], the dominant computational cost mainly comes from the shared deep feature extraction backbone rather than the meta-optimization procedure itself. Therefore, end-to-end wall-clock time may largely reflect the cost of ResNet-based [27] feature extraction, instead of the efficiency of the proposed meta-gradient approximation. In our framework, the scalable design mainly aims to reduce the additional overhead introduced by nested optimization through truncated inner-step differentiation and mini-batch-based meta-updates. The next subsection details the proposed scalable meta-gradient computation for noisy multi-label correction.

7. Scalable Meta-Gradient Methods for Noisy Multi-Label Correction

We present a unified meta-gradient computation for the nested formulation in (16). Our derivation provides an exact one-step meta-gradient and a scalable approximation when multiple inner updates are used. We use t to index outer iterations and use $\tau$ to index inner updates. We denote the number of inner-loop steps by K and denote the student learning rate by ${\eta }_{\theta }$.

Proposition 1

(One-Step Meta-Gradient in PML). Assume the student performs one SGD step on the inner correction loss:

$${\theta }^{(t+1)}={\theta }^{\left(t\right)}-{\eta }_{\theta }{\partial }_{\theta }{\ell }_{\mathit{corr}}({\theta }^{\left(t\right)},{\phi }^{\left(t\right)}).$$

(19)

Let

$$g_{\theta }:={\partial }_{\theta }{\ell }_{\mathit{meta}}\left({\theta }^{(t+1)}\right),{\mathcal{H}}_{\theta \phi }^{\left(t\right)}:={\partial }_{\theta \phi }{\ell }_{\mathit{corr}}({\theta }^{\left(t\right)},{\phi }^{\left(t\right)}).$$

(20)

Then, the one-step meta-gradient with respect to ${\phi }^{\left(t\right)}$ is

$${\partial }_{\phi }^{\left(t\right)}:={\displaystyle \frac{d{\ell }_{\mathit{meta}}\left({\theta }^{(t+1)}\right)}{d{\phi }^{\left(t\right)}}}=-{\eta }_{\theta }{\mathcal{H}}_{\theta \phi }^{\left(t\right)\top }{g}_{\theta }.$$

(21)

Proof. 

Differentiate through the SGD update:

$${\displaystyle \frac{d{\theta }^{(t+1)}}{d{\phi }^{\left(t\right)}}}=-{\eta }_{\theta }{\partial }_{\theta \phi }{\ell }_{\mathrm{corr}}({\theta }^{\left(t\right)},{\phi }^{\left(t\right)})=-{\eta }_{\theta }{\mathcal{H}}_{\theta \phi }^{\left(t\right)}.$$

(22)

Applying the chain rule yields

$${\partial }_{\phi }^{\left(t\right)}={\left({\displaystyle \frac{d{\theta }^{(t+1)}}{d{\phi }^{\left(t\right)}}}\right)}^{\top }{\partial }_{\theta }{\ell }_{\mathrm{meta}}\left({\theta }^{(t+1)}\right)=-{\eta }_{\theta }{\mathcal{H}}_{\theta \phi }^{\left(t\right)\top }{g}_{\theta }.$$

(23)

   □

Proposition 2

(Multi-Step Meta-Gradient Approximation in PML). Let $K>1$ denote the number of inner-loop steps, and let $\tau \in [t-K+1,t]$ index these steps. Assume the meta-teacher is fixed within the inner loop, namely ${\phi }^{\left(\tau \right)}={\phi }^{\left(t\right)}$. The student updates are

$${\theta }^{(\tau +1)}={\theta }^{\left(\tau \right)}-{\eta }_{\theta }{\partial }_{\theta }{\ell }_{\mathit{corr}}({\theta }^{\left(\tau \right)},{\phi }^{\left(t\right)}),\tau \in [t-K+1,t].$$

(24)

Define

$${\mathcal{H}}_{\theta \phi }^{\left(\tau \right)}:={\partial }_{\theta \phi }{\ell }_{\mathit{corr}}({\theta }^{\left(\tau \right)},{\phi }^{\left(t\right)}),g_{\theta }:={\partial }_{\theta }{\ell }_{\mathit{meta}}\left({\theta }^{(t+1)}\right).$$

(25)

Then, the multi-step meta-gradient can be approximated by an exponentially weighted aggregation:

$${\partial }_{\phi }^{\left(t\right)}\approx -{\eta }_{\theta }\sum _{\tau =t-K+1}^t{\gamma }_{\theta }^{t-\tau }{\mathcal{H}}_{\theta \phi }^{\left(\tau \right)\top }{g}_{\theta },$$

(26)

where ${\gamma }_{\theta }\in (0,1)$ is a discount factor.

Proof. 

(Sketch). We unroll the K inner updates and recursively differentiate the inner trajectory with respect to $\phi$. We adopt a first-order approximation to avoid explicit second-order terms involving ${\partial }_{\theta \theta }{\ell }_{\mathrm{corr}}$, which yields a discounted accumulation across inner steps.    □

Proposition 3

(Mixed-Derivative Factorization with Candidate Mask). For a noisy PML sample $(x_i,{\tilde{y}}_i)$, let the student logit vector be $s_{\theta }\left(x_i\right)\in {\mathbb{R}}^L$ and the student probability be $p_{\theta }\left(x_i\right)=\sigma \left(s_{\theta }\left(x_i\right)\right)\in {(0,1)}^L$. Let the teacher soft target be $q_{\phi }(x_i,{\tilde{y}}_i)\in {[0,1]}^L$. Let $M\left({\tilde{y}}_i\right)\in {\{0,1\}}^L$ be the candidate mask with $M^{\left(l\right)}\left({\tilde{y}}_i\right)={\tilde{y}}_i^{\left(l\right)}$. Define the per-sample correction loss as

$${\ell }_{\mathit{corr}}^{\left(i\right)}(\theta ,\phi )=\sum _{l=1}^L{M}^{\left(l\right)}\left({\tilde{y}}_i\right)\mathrm{BCE}\left(q_{\phi }^{\left(l\right)}(x_i,{\tilde{y}}_i),p_{\theta }^{\left(l\right)}\left(x_i\right)\right).$$

(27)

Then, the mixed derivative admits the factorization

$${\partial }_{\theta \phi }{\ell }_{\mathit{corr}}^{\left(i\right)}(\theta ,\phi )=-\sum _{l=1}^L{M}^{\left(l\right)}\left({\tilde{y}}_i\right)\underset{{\vartheta }_{\theta }(i,l)}{\underbrace{{\partial }_{\theta }{s}_{\theta }^{\left(l\right)}\left(x_i\right)}}\underset{{\vartheta }_{\phi }{(i,l)}^{\top }}{\underbrace{{\partial }_{\phi }{q}_{\phi }^{\left(l\right)}{(x_i,{\tilde{y}}_i)}^{\top }}}.$$

(28)

Proof. 

For $p_{\theta }^{\left(l\right)}=\sigma \left(s_{\theta }^{\left(l\right)}\right)$, the derivative of BCE with respect to the logit satisfies ${\partial }_s\mathrm{BCE}(q,p)=p-q$. Thus,

$${\partial }_{\theta }\mathrm{BCE}(q_{\phi }^{\left(l\right)},p_{\theta }^{\left(l\right)})=\left(p_{\theta }^{\left(l\right)}-q_{\phi }^{\left(l\right)}\right){\partial }_{\theta }{s}_{\theta }^{\left(l\right)}.$$

(29)

Only $q_{\phi }^{\left(l\right)}$ depends on $\phi$, and the mask term is constant with respect to $(\theta ,\phi )$; so,

$${\partial }_{\theta \phi }\left(M^{\left(l\right)}\mathrm{BCE}(q_{\phi }^{\left(l\right)},p_{\theta }^{\left(l\right)})\right)=-M^{\left(l\right)}{\partial }_{\theta }{s}_{\theta }^{\left(l\right)}{\partial }_{\phi }{q}_{\phi }^{\left(l\right)\top }.$$

(30)

Summing over l yields the stated factorization.    □

Corollary 1

(Jacobian-Product Meta-Gradient with Mask). Let ${\mathcal{B}}_{\mathit{noisy}}$ be a noisy mini-batch used in ${\ell }_{\mathit{corr}}$. For each $i\in {\mathcal{B}}_{\mathit{noisy}}$, define the Jacobians $J_{\theta }\left(i\right):={\partial }_{\theta }{s}_{\theta }\left(x_i\right)\in {\mathbb{R}}^{L\times \left|\theta \right|}$ and $J_{\phi }\left(i\right):={\partial }_{\phi }{q}_{\phi }(x_i,{\tilde{y}}_i)\in {\mathbb{R}}^{L\times \left|\phi \right|}$, and let $M_i:=M\left({\tilde{y}}_i\right)$. Then, the one-step meta-gradient can be computed as

$${\partial }_{\phi }^{\left(t\right)}={\eta }_{\theta }·{\displaystyle \frac{1}{|{\mathcal{B}}_{\mathit{noisy}}|}}\sum _{i\in {\mathcal{B}}_{\mathit{noisy}}}{J}_{\phi }{\left(i\right)}^{\top }\left(M_i\odot \left(J_{\theta }\left(i\right)g_{\theta }\right)\right),$$

(31)

and the multi-step approximation replaces the single-step term by the discounted aggregation across $\tau \in [t-K+1,t]$.Based on the above derivation, the complete procedure for computing the fast PML meta-gradient is summarized in Algorithm 1.

Algorithm 1 Fast PML Meta-Gradient

Input: Student parameter trajectory $\{{\theta }^{(t-K+1)},\dots ,{\theta }^{(t+1)}\}$; fixed teacher parameters ${\phi }^{\left(t\right)}$; noisy batch ${\mathcal{B}}_{\mathrm{noisy}}$; candidate masks ${\left\{M_i\right\}}_{i\in {\mathcal{B}}_{\mathrm{noisy}}}$; discount factor ${\gamma }_{\theta }$; step size ${\eta }_{\theta }$. Output: Approximate meta-gradient ${\partial }_{\phi }^{\left(t\right)}$. Step 1: Compute the meta-gradient with respect to the student parameters $g_{\theta }={\partial }_{\theta }{\ell }_{\mathrm{meta}}\left({\theta }^{(t+1)}\right)$. Initialize ${\partial }_{\phi }^{\left(t\right)}=0$ and set $d_w=1$. Step 2: For $\tau =t$ to $t-K+1$, perform the following steps. Step 3: Initialize the gradient accumulator $G=0$. Step 4: For each sample $i\in {\mathcal{B}}_{\mathrm{noisy}}$: compute the Jacobian–vector product $v_i=J_{\theta }^{\left(\tau \right)}\left(i\right)·g_{\theta }$, apply the candidate mask $v_i=M_i\odot v_i$, and compute the vector–Jacobian product $u_i=J_{\phi }^{\left(\tau \right)}{\left(i\right)}^{\top }·v_i$. Accumulate $G=G+u_i$. Step 5: Update the approximate meta-gradient $${\partial }_{\phi }^{\left(t\right)}\leftarrow {\partial }_{\phi }^{\left(t\right)}+d_w·{\displaystyle \frac{{\eta }_{\theta }}{|{\mathcal{B}}_{\mathrm{noisy}}|}}G.$$ Step 6: Update the discount weight $d_w\leftarrow {\gamma }_{\theta }·d_w$. Step 7: End the loop and return ${\partial }_{\phi }^{\left(t\right)}$.

8. Experiments

8.1. Datasets

To ensure equitable evaluation conditions, our study employs three benchmark datasets: NUS-WIDE [28], MS-COCO2014 [29], and Pascal VOC2007 [30]. NUS-WIDE offers the largest data scale, comprising 161,789 training and 107,859 test images annotated with 81 semantic concepts, averaging 1.9 tags per image. MS-COCO contains 82,081 training and 40,137 test images across 80 object categories, with a higher label density of 2.9 annotations per image. VOC, the smallest of the three, includes 5011 train–val and 4952 test samples labeled with 20 common object classes, averaging 1.6 labels per image. The selected datasets constitute standard benchmarks for deep multi-label classification research. While the existing literature frequently utilizes conventional datasets with pre-extracted feature sets, these legacy collections often demonstrate limited scale and fixed feature representations that prove incompatible with modern end-to-end deep architectures requiring holistic feature learning. To construct partially labeled training sets that are consistent with the PML setting, we adopt an additive candidate-label contamination strategy following [11,31]. Specifically, for each training sample, we preserve all ground-truth positive labels and additionally inject irrelevant labels into the candidate label set, such that the resulting candidate set satisfies the standard PML assumption $y_i\subseteq {\tilde{y}}_i$. The amount of injected noise is scaled according to the label cardinality of each sample. Let $c_i$ denote the number of positive labels in the ground-truth label vector of the i-th sample. Under a contamination ratio $r\in \{30\%,50\%,70\%\}$, we additionally assign approximately $r·c_i$ irrelevant labels from the negative label pool as candidate labels. Therefore, the ratios of 30%, 50%, and 70% in our experiments refer to the candidate-label contamination ratio, rather than a generic label-flipping ratio. To avoid the unrealistic distortion of class occurrence patterns, the additional candidate labels are sampled in a class-aware manner according to the label distribution of the original training set. In this way, the generated candidate sets preserve the inclusion property required by PML while introducing different levels of annotation ambiguity for robustness evaluation.

8.2. Evaluation Metrics

For a comprehensive evaluation of model performance in PML learning scenarios, we employed eight widely-used multi-label evaluation metrics [32], each designed to capture different facets of model capabilities. Specifically, we assessed the classification accuracy through metrics such as the mean Average Precision (mAP), which quantifies the balance between label-wise precision and recall—essential in handling imbalanced PML tasks; Subset Accuracy, measuring exact matches between predicted and true label sets to reflect prediction correctness; and Overall F1 (OF1) and Cumulative F1 (CF1), evaluating harmonic means at instance-level and label-level, respectively. To examine the ranking quality of predicted labels, we utilized Rank Loss, penalizing the incorrect ordering of labels, thus vital for candidate label ranking in PML, and One-error, assessing the failure rate of top predictions in multi-label contexts. Additionally, for error analysis, we adopted Hamming Loss to compute the misclassification ratio per label, effectively capturing sensitivity to noisy supervision, and Average Precision, which extends binary AP to multi-label scenarios and emphasizes the correct ranking of relevant labels. This multi-perspective evaluation framework ensures the rigorous assessment of the capabilities of PML methods in addressing ambiguous supervision, exploiting label correlations, and maintaining robustness against label noise. The selected metrics align explicitly with the unique challenges of PML tasks: partial observability captured by subset accuracy, label interdependence through ranking metrics, and noise tolerance analyzed via Hamming Loss.

8.3. Implementation Setting

Before conducting experiments on the compared methods, we unified the experimental settings as follows. All methods use backbone networks pretrained on ImageNet [33]. For the KD-based method, ResNet101 [27] is adopted as the teacher model and ResNet34 as the student model, while ResNet101 is used as the feature extraction backbone for the compared PML methods. Following the preliminary experiments, the step size for meta-gradient learning is fixed to 3. We resize all images to $224\times 224$ and set the batch size to 64 in all experiments. For each training image, we apply data augmentation including random horizontal flipping and Cutout [34]. The models are trained using the SGD optimizer [35] for 80 epochs. The one-cycle learning rate policy is adopted with a maximum learning rate of 0.0001 and a weight decay of 0.0001 [36]. The balancing hyperparameters in the distillation process for all experiments follow the settings in [12]. The whole training procedure follows the three-stage protocol described in Section 4. Specifically, the teacher network is first initialized by multi-label knowledge distillation pretraining, after which the meta-correction stage is activated to optimize the teacher with clean validation supervision. In this stage, one meta-update is performed for each mini-batch update, where the inner optimization is conducted on the noisy training batch, and the outer objective is computed on the clean validation batch. After meta-optimization converges, the corrected teacher is fixed and used to provide soft supervision for the final student training stage. Unless otherwise stated, all stages are trained within the same 80-epoch budget, and the switching between stages follows the predefined training schedule rather than an adaptive stopping rule. The default values of the key hyperparameters are selected according to the preliminary validation experiments. In particular, the discount factor ${\gamma }_{\theta }$ is set to 0.85, the meta step size ${\eta }_{\theta }$ is set to 3, and the adaptive gate is learned in an instance-wise manner. For the fixed-gate ablation, $\alpha \in \{0.25,0.50,0.75\}$ is evaluated. Unless otherwise specified, ResNet101 is used as the default teacher backbone because it achieves the best overall trade-off between correction quality and robustness in the architecture analysis, while ResNet34 is used as the student backbone to maintain a clear teacher–student capacity gap. The proposed meta-gradient approximation is implemented with a truncated one-step update strategy to reduce memory and computational overhead, making the method practical for large-scale experiments. Each experiment was repeated three times with different random seeds, and the reported results are presented as the mean ± standard deviation. For all compared methods, we use the parameter settings recommended in their original papers, unless otherwise specified. All experiments are conducted on an NVIDIA GeForce RTX 3090 GPU.

8.4. Results

We benchmarked our approach against seven representative PML methods, all implemented using their original literature-recommended parameters. The compared methods mainly follow two lines. The first line tries to make supervision cleaner by identifying unreliable candidate labels and then reducing their training impact, such as PML-NI [3], PML-NLR [8], PML-CD [11], and PML-MD [7]. The second line tries to make learning more stable without explicit removal, and the methods mainly rely on representation correction or training schedules, such as PASE [9], PML-CDCR [10], and UNM [6]. In contrast, MCKD uses a distillation pipeline with a meta-teacher and a student, and MCKD uses a small clean set to guide the teacher update through the outer meta objective. The teacher then outputs refined soft targets, and the student learns from these targets on the full partially labeled training set. All methods are evaluated under three noise ratios $\mathrm{NR}\in \{0.3,0.5,0.7\}$ on NUS-WIDE, MS-COCO, and VOC2007, and we report the mAP, OF1, and CF1 as the primary metrics, with larger values indicating better performance. As shown in Table 1, Table 2 and Table 3, on the NUS-WIDE dataset, MCKD shows strong performance under low and moderate noise conditions. At $\mathrm{NR}=0.3$, MCKD attains 75.60 mAP, 75.70 OF1, and 72.90 CF1, outperforming all competing methods across the three core metrics. When the noise level increases to $\mathrm{NR}=0.5$, MCKD remains competitive, achieving 71.70 mAP, 71.20 OF1, and 68.80 CF1, ranking favorably among the compared methods PASE, NLR, PML-MD, PML-NI, PML-CDCR, UNM, and PML-CD. Under the highest noise level $\mathrm{NR}=0.7$, MCKD remains competitive with 66.40 mAP, 65.80 OF1, and 64.10 CF1, which are close to the best results obtained by PML-CD. Moreover, MCKD exhibits lower Hamming Loss, One-error, and Rank Loss, together with higher Subset Accuracy, indicating improved robustness and more reliable label ranking behavior under noisy supervision. On the MS-COCO dataset, MCKD demonstrates particularly strong performance in terms of the mAP across all noise levels. Specifically, MCKD achieves 88.10 and 85.90 mAP at $\mathrm{NR}=0.3$ and $\mathrm{NR}=0.5$, respectively, ranking first among all compared methods. Even under severe noise at $\mathrm{NR}=0.7$, MCKD reaches 78.00 mAP, which is nearly identical to the best-performing PML-CD. In terms of OF1, MCKD consistently attains the highest or near-highest values across all noise settings, reflecting strong instance-level prediction quality. Although PML-CDCR slightly outperforms MCKD on CF1 under all noise ratios, the performance gaps remain small. In addition, MCKD achieves the lowest or near-lowest Hamming Loss, One-error, and Rank Loss values, as well as the highest Subset Accuracy, indicating competitive robustness on this large-scale dataset. On the VOC2007 dataset, MCKD exhibits a different performance profile. Under low noise conditions ($\mathrm{NR}=0.3$), MCKD achieves the best OF1 and CF1, reaching 57.40 and 54.20, respectively, and also yields the highest Subset Accuracy, indicating effective modeling of instance-level multi-label structures. However, PML-CD consistently outperforms MCKD on mAP across all noise ratios. As the noise level increases, the gap in mAP becomes more pronounced, with PML-CD maintaining higher label-wise precision under heavy noise. Nevertheless, MCKD remains competitive on ranking- and error-based metrics, such as Hamming Loss, One-error, and Rank Loss, where it achieves results comparable to or better than most competing approaches. Overall, these results demonstrate that MCKD delivers strong and stable performance across different datasets and noise conditions. It performs favorably against existing methods on NUS-WIDE and MS-COCO under low and moderate noise, while remaining highly competitive under severe noise. Although PML-CD shows an advantage in mAP on the smaller VOC2007 dataset under heavy noise, MCKD provides a more balanced performance across multiple evaluation criteria, highlighting its effectiveness and robustness for partial multi-label learning with noisy candidate labels.

Table 1. Comparison of our MCKD with PML state-of-the-art methods on the MS-COCO.

Metrics	Noisy Ratio	Ours	PASE	NLR	PML-MD	PML-NI	PML-CDCR	UNM	PML-CD
mAP	0.3	75.60 ± 0.48	74.10 ± 0.58	69.60 ± 0.72	66.50 ± 0.66	61.10 ± 0.80	73.98 ± 0.55	70.10 ± 0.64	74.80 ± 0.60
	0.5	71.70 ± 0.52	69.30 ± 0.60	64.10 ± 0.70	61.00 ± 0.65	55.60 ± 0.82	67.80 ± 0.58	65.10 ± 0.66	70.90 ± 0.62
	0.7	66.40 ± 0.56	63.70 ± 0.70	58.80 ± 0.74	55.30 ± 0.68	49.70 ± 0.86	61.10 ± 0.62	59.20 ± 0.70	66.90 ± 0.65
OF1	0.3	75.70 ± 0.50	73.30 ± 0.62	68.40 ± 0.74	65.20 ± 0.68	59.80 ± 0.83	71.60 ± 0.59	69.40 ± 0.66	74.00 ± 0.63
	0.5	71.20 ± 0.54	68.50 ± 0.65	63.60 ± 0.72	61.10 ± 0.67	56.30 ± 0.85	66.80 ± 0.61	64.20 ± 0.69	70.40 ± 0.64
	0.7	65.80 ± 0.59	62.80 ± 0.70	57.30 ± 0.76	54.80 ± 0.70	50.10 ± 0.88	61.00 ± 0.64	58.70 ± 0.72	66.00 ± 0.66
CF1	0.3	72.90 ± 0.52	70.60 ± 0.64	65.80 ± 0.76	63.50 ± 0.70	58.20 ± 0.86	68.70 ± 0.60	66.40 ± 0.68	71.10 ± 0.65
	0.5	68.80 ± 0.56	65.40 ± 0.67	61.20 ± 0.74	59.30 ± 0.71	54.70 ± 0.87	63.20 ± 0.63	61.00 ± 0.70	67.90 ± 0.66
	0.7	64.10 ± 0.60	60.30 ± 0.72	56.40 ± 0.77	53.70 ± 0.72	48.90 ± 0.89	58.40 ± 0.65	56.70 ± 0.73	64.60 ± 0.67
AvgPrecision	0.3	74.70 ± 0.38	73.35 ± 0.40	69.00 ± 0.46	66.20 ± 0.42	60.30 ± 0.51	71.20 ± 0.43	69.40 ± 0.45	73.90 ± 0.42
	0.5	70.70 ± 0.41	69.30 ± 0.43	64.00 ± 0.49	60.60 ± 0.45	55.20 ± 0.53	67.50 ± 0.46	65.10 ± 0.48	70.80 ± 0.44
	0.7	66.25 ± 0.45	64.20 ± 0.48	58.60 ± 0.52	54.90 ± 0.48	49.70 ± 0.56	61.20 ± 0.49	59.30 ± 0.51	66.40 ± 0.47
HammingLoss	0.3	2.22 ± 0.09	2.35 ± 0.10	2.70 ± 0.12	3.00 ± 0.11	3.30 ± 0.13	2.60 ± 0.11	2.80 ± 0.12	2.40 ± 0.10
	0.5	2.41 ± 0.10	2.50 ± 0.11	3.00 ± 0.13	3.30 ± 0.12	3.70 ± 0.14	2.90 ± 0.12	3.10 ± 0.13	2.61 ± 0.11
	0.7	2.69 ± 0.11	2.80 ± 0.12	3.30 ± 0.14	3.60 ± 0.13	4.00 ± 0.15	3.10 ± 0.13	3.40 ± 0.14	2.68 ± 0.12
OneError	0.3	9.15 ± 0.38	10.20 ± 0.41	12.10 ± 0.46	12.80 ± 0.44	14.90 ± 0.49	11.30 ± 0.42	11.90 ± 0.44	9.70 ± 0.40
	0.5	10.75 ± 0.40	11.70 ± 0.43	13.60 ± 0.47	14.50 ± 0.46	16.20 ± 0.51	12.80 ± 0.45	13.40 ± 0.46	11.10 ± 0.42
	0.7	12.35 ± 0.43	13.10 ± 0.46	15.10 ± 0.49	15.90 ± 0.48	17.80 ± 0.53	14.20 ± 0.47	14.90 ± 0.49	12.30 ± 0.44
RankLoss	0.3	7.55 ± 0.35	8.40 ± 0.39	9.50 ± 0.42	10.10 ± 0.40	11.30 ± 0.45	8.90 ± 0.40	9.40 ± 0.41	8.00 ± 0.38
	0.5	9.09 ± 0.37	10.00 ± 0.40	11.10 ± 0.43	11.90 ± 0.42	13.00 ± 0.46	10.40 ± 0.41	10.80 ± 0.42	9.38 ± 0.39
	0.7	10.79 ± 0.40	11.70 ± 0.43	12.60 ± 0.45	13.30 ± 0.44	14.50 ± 0.48	12.00 ± 0.43	12.20 ± 0.44	10.75 ± 0.41
SubAccuracy	0.3	76.70 ± 0.48	74.50 ± 0.60	68.30 ± 0.75	65.40 ± 0.70	59.00 ± 0.86	71.20 ± 0.62	69.10 ± 0.69	73.90 ± 0.64
	0.5	72.10 ± 0.52	69.80 ± 0.63	63.40 ± 0.73	60.60 ± 0.69	54.20 ± 0.87	66.70 ± 0.64	64.30 ± 0.71	70.60 ± 0.66
	0.7	67.20 ± 0.56	64.20 ± 0.70	58.50 ± 0.76	55.70 ± 0.71	49.30 ± 0.89	61.40 ± 0.66	59.20 ± 0.73	67.30 ± 0.68

Table 2. Comparison of our MCKD with PML state-of-the-art methods on the Pascal VOC2007.

Metrics	Noisy Ratio	Ours	PASE	NLR	PML-MD	PML-NI	PML-CDCR	UNM	PML-CD
mAP	0.3	88.10 ± 0.34	86.40 ± 0.42	82.10 ± 0.50	80.20 ± 0.48	74.10 ± 0.57	87.60 ± 0.37	83.10 ± 0.49	87.50 ± 0.35
	0.5	85.90 ± 0.38	83.40 ± 0.46	79.10 ± 0.53	77.70 ± 0.50	71.80 ± 0.58	85.40 ± 0.40	80.10 ± 0.52	85.50 ± 0.40
	0.7	78.00 ± 0.45	75.20 ± 0.52	71.00 ± 0.58	69.40 ± 0.54	63.60 ± 0.62	77.30 ± 0.46	70.20 ± 0.60	78.20 ± 0.50
OF1	0.3	72.90 ± 0.46	71.10 ± 0.50	67.10 ± 0.55	66.10 ± 0.53	65.55 ± 0.57	72.73 ± 0.49	68.10 ± 0.54	72.40 ± 0.47
	0.5	71.40 ± 0.48	69.60 ± 0.52	65.40 ± 0.57	64.40 ± 0.55	63.10 ± 0.58	71.10 ± 0.50	66.50 ± 0.56	71.60 ± 0.49
	0.7	69.10 ± 0.50	67.30 ± 0.54	63.00 ± 0.59	62.10 ± 0.57	61.00 ± 0.60	69.00 ± 0.52	64.20 ± 0.58	69.30 ± 0.51
CF1	0.3	70.90 ± 0.48	69.20 ± 0.52	65.20 ± 0.56	64.30 ± 0.55	60.97 ± 0.58	70.97 ± 0.51	66.10 ± 0.57	70.60 ± 0.50
	0.5	69.40 ± 0.49	67.70 ± 0.53	63.70 ± 0.57	62.80 ± 0.56	59.10 ± 0.60	69.80 ± 0.52	64.70 ± 0.58	69.60 ± 0.51
	0.7	67.80 ± 0.51	66.00 ± 0.55	62.00 ± 0.59	61.10 ± 0.57	57.20 ± 0.61	68.90 ± 0.53	63.10 ± 0.60	68.10 ± 0.52
AvgPrecision	0.3	88.15 ± 0.38	86.70 ± 0.41	83.50 ± 0.46	82.30 ± 0.44	74.10 ± 0.51	87.60 ± 0.40	84.20 ± 0.48	87.50 ± 0.40
	0.5	85.85 ± 0.40	84.10 ± 0.44	81.30 ± 0.48	80.20 ± 0.46	71.80 ± 0.53	85.40 ± 0.41	82.00 ± 0.50	86.00 ± 0.42
	0.7	78.40 ± 0.45	76.50 ± 0.48	73.00 ± 0.52	71.80 ± 0.49	63.60 ± 0.56	77.70 ± 0.46	72.10 ± 0.52	78.60 ± 0.46
HammingLoss	0.3	1.34 ± 0.06	1.42 ± 0.07	1.58 ± 0.07	1.66 ± 0.07	1.88 ± 0.08	1.40 ± 0.06	1.52 ± 0.07	1.36 ± 0.06
	0.5	1.42 ± 0.07	1.50 ± 0.07	1.65 ± 0.08	1.73 ± 0.08	1.96 ± 0.09	1.48 ± 0.07	1.61 ± 0.08	1.41 ± 0.07
	0.7	1.56 ± 0.07	1.65 ± 0.08	1.80 ± 0.09	1.88 ± 0.09	2.10 ± 0.10	1.62 ± 0.08	1.76 ± 0.09	1.55 ± 0.07
OneError	0.3	6.75 ± 0.31	7.30 ± 0.33	8.25 ± 0.36	8.60 ± 0.35	9.60 ± 0.37	7.00 ± 0.32	7.95 ± 0.35	6.85 ± 0.31
	0.5	7.28 ± 0.32	7.88 ± 0.34	8.85 ± 0.37	9.20 ± 0.36	10.10 ± 0.39	7.45 ± 0.33	8.40 ± 0.36	7.25 ± 0.32
	0.7	8.10 ± 0.34	8.75 ± 0.36	9.70 ± 0.39	10.05 ± 0.38	11.00 ± 0.41	8.30 ± 0.35	9.35 ± 0.40	8.08 ± 0.34
RankLoss	0.3	4.20 ± 0.22	4.68 ± 0.24	5.35 ± 0.26	5.65 ± 0.25	6.50 ± 0.28	4.38 ± 0.23	5.05 ± 0.25	4.25 ± 0.23
	0.5	4.65 ± 0.23	5.09 ± 0.25	5.78 ± 0.27	6.08 ± 0.26	6.90 ± 0.29	4.79 ± 0.24	5.50 ± 0.26	4.62 ± 0.23
	0.7	5.38 ± 0.25	5.89 ± 0.27	6.50 ± 0.29	6.80 ± 0.28	7.60 ± 0.31	5.55 ± 0.26	6.12 ± 0.28	5.36 ± 0.25
SubAccuracy	0.3	77.60 ± 0.44	75.80 ± 0.48	72.10 ± 0.54	71.20 ± 0.52	69.00 ± 0.58	76.10 ± 0.47	73.10 ± 0.55	77.10 ± 0.45
	0.5	76.00 ± 0.45	74.10 ± 0.50	70.40 ± 0.56	69.40 ± 0.53	67.20 ± 0.60	74.90 ± 0.48	71.30 ± 0.57	76.10 ± 0.46
	0.7	73.10 ± 0.47	71.10 ± 0.52	68.00 ± 0.58	67.10 ± 0.55	65.10 ± 0.61	72.50 ± 0.49	69.10 ± 0.59	73.20 ± 0.47

Table 3. Comparison of our MCKD with PML state-of-the-art methods on the NUS-WIDE.

Metrics	Noisy Ratio	Ours	PASE	NLR	PML-MD	PML-NI	PML-CDCR	UNM	PML-CD
mAP	0.3	47.40 ± 0.40	46.00 ± 0.45	42.20 ± 0.50	45.00 ± 0.48	16.39 ± 0.40	39.96 ± 0.46	34.57 ± 0.45	47.62 ± 0.44
	0.5	41.40 ± 0.44	40.10 ± 0.48	36.80 ± 0.53	37.90 ± 0.50	12.02 ± 0.36	31.99 ± 0.49	29.62 ± 0.48	43.20 ± 0.47
	0.7	31.60 ± 0.48	30.10 ± 0.52	27.90 ± 0.56	28.50 ± 0.54	7.89 ± 0.32	22.37 ± 0.51	20.04 ± 0.50	35.73 ± 0.50
OF1	0.3	57.40 ± 0.46	56.10 ± 0.50	53.80 ± 0.54	54.60 ± 0.52	41.30 ± 0.55	55.40 ± 0.50	51.20 ± 0.49	56.10 ± 0.50
	0.5	52.60 ± 0.48	51.10 ± 0.52	48.00 ± 0.56	49.20 ± 0.54	36.20 ± 0.58	49.20 ± 0.52	46.10 ± 0.51	53.10 ± 0.52
	0.7	45.70 ± 0.50	44.30 ± 0.54	41.50 ± 0.58	42.00 ± 0.56	30.40 ± 0.60	43.10 ± 0.54	40.30 ± 0.53	46.00 ± 0.54
CF1	0.3	54.20 ± 0.46	52.80 ± 0.50	50.90 ± 0.54	52.10 ± 0.52	38.60 ± 0.57	52.20 ± 0.50	48.60 ± 0.49	53.60 ± 0.50
	0.5	49.60 ± 0.48	48.30 ± 0.52	45.10 ± 0.56	46.70 ± 0.54	34.10 ± 0.59	47.20 ± 0.52	43.90 ± 0.51	50.20 ± 0.52
	0.7	43.30 ± 0.50	42.00 ± 0.54	39.50 ± 0.58	40.10 ± 0.56	29.10 ± 0.61	41.50 ± 0.54	38.10 ± 0.53	44.00 ± 0.54
AvgPrecision	0.3	61.80 ± 0.40	60.40 ± 0.44	57.60 ± 0.48	59.50 ± 0.46	30.00 ± 0.45	58.80 ± 0.44	55.20 ± 0.45	60.90 ± 0.44
	0.5	56.10 ± 0.44	54.90 ± 0.48	51.50 ± 0.52	53.00 ± 0.50	25.00 ± 0.47	52.00 ± 0.48	49.50 ± 0.49	55.30 ± 0.48
	0.7	49.10 ± 0.48	48.00 ± 0.52	45.20 ± 0.55	46.30 ± 0.53	20.00 ± 0.49	46.00 ± 0.51	43.00 ± 0.52	49.60 ± 0.49
HammingLoss	0.3	2.04 ± 0.09	2.10 ± 0.10	2.28 ± 0.10	2.15 ± 0.10	2.80 ± 0.11	2.10 ± 0.10	2.28 ± 0.10	2.08 ± 0.10
	0.5	2.28 ± 0.10	2.36 ± 0.11	2.55 ± 0.12	2.42 ± 0.11	3.20 ± 0.12	2.48 ± 0.11	2.66 ± 0.11	2.26 ± 0.10
	0.7	2.52 ± 0.11	2.63 ± 0.12	2.85 ± 0.13	2.74 ± 0.12	3.60 ± 0.13	2.88 ± 0.12	3.05 ± 0.12	2.50 ± 0.11
OneError	0.3	13.60 ± 0.41	14.00 ± 0.43	14.60 ± 0.45	14.20 ± 0.44	19.00 ± 0.48	14.20 ± 0.44	15.50 ± 0.45	13.70 ± 0.42
	0.5	15.20 ± 0.43	15.70 ± 0.45	16.30 ± 0.47	15.90 ± 0.46	21.00 ± 0.50	16.40 ± 0.46	17.60 ± 0.47	15.00 ± 0.43
	0.7	17.50 ± 0.45	18.00 ± 0.48	18.90 ± 0.50	18.40 ± 0.49	23.50 ± 0.53	19.00 ± 0.49	20.80 ± 0.50	17.60 ± 0.45
RankLoss	0.3	11.70 ± 0.40	12.00 ± 0.42	12.90 ± 0.44	12.10 ± 0.42	17.00 ± 0.46	12.60 ± 0.43	13.80 ± 0.44	11.80 ± 0.41
	0.5	13.40 ± 0.43	13.80 ± 0.45	14.70 ± 0.47	14.00 ± 0.46	19.00 ± 0.49	14.70 ± 0.47	16.00 ± 0.48	13.20 ± 0.43
	0.7	15.20 ± 0.45	15.70 ± 0.47	16.50 ± 0.49	15.90 ± 0.48	21.50 ± 0.51	17.20 ± 0.49	18.80 ± 0.50	15.30 ± 0.45
SubAccuracy	0.3	60.50 ± 0.46	59.20 ± 0.50	56.80 ± 0.54	58.00 ± 0.52	43.50 ± 0.55	59.80 ± 0.50	56.10 ± 0.49	59.40 ± 0.50
	0.5	55.20 ± 0.48	53.90 ± 0.52	51.40 ± 0.56	52.30 ± 0.54	39.40 ± 0.57	55.10 ± 0.52	51.60 ± 0.51	54.90 ± 0.52
	0.7	48.80 ± 0.50	47.40 ± 0.54	45.00 ± 0.58	46.10 ± 0.56	34.60 ± 0.59	47.70 ± 0.53	43.00 ± 0.53	49.10 ± 0.54

8.5. Ablation Study

To better understand the individual contribution of each module in MCKD, we conduct an ablation study on the NUS-WIDE dataset under a challenging 70% noise ratio. As shown in Table 4, we evaluate four key components in the proposed framework, including teacher meta-learning, student meta-learning, label correction, and corruption-aware candidate labels.

Table 4. Ablation study of different components on NUS-WIDE with 70% NR.

Teacher Meta	Student Meta	Label Correction	Corruption Labels	mAP	OF1	CF1
✗	✗	✗	✗	55.52	76.80	62.23
✗	✗	✗	✓	28.70	41.30	39.20
✗	✗	✓	✓	29.80	42.40	40.10
✓	✗	✓	✓	31.50	45.50	42.90
✗	✓	✓	✓	30.70	43.30	41.40
✓	✓	✓	✓	31.60	45.70	43.30

The first row corresponds to the backbone model without any of the proposed modules. Under this setting, the model achieves 55.52 mAP, 76.80 OF1, and 62.23 CF1. When corruption-aware candidate labels are directly introduced without any correction or meta-optimization mechanism, the performance drops sharply to 28.70 mAP, 41.30 OF1, and 39.20 CF1. This result indicates that directly injecting corrupted supervision into training can severely damage the quality of learned multi-label representations. In other words, although corruption-aware labels provide additional supervision, they also introduce substantial noise interference when no dedicated refinement mechanism is available.

After enabling the label correction module, the performance improves from 28.70 to 29.80 in mAP and from 41.30 to 42.40 in OF1. This observation confirms that explicit candidate-label refinement is beneficial under noisy partial multi-label settings, since it can filter part of the unreliable supervision and recover more informative label cues. On top of label correction, introducing teacher meta-learning further improves mAP to 31.50 and OF1 to 45.50, while introducing student meta-learning yields 30.70 mAP and 43.30 OF1. These results show that both meta-learning branches contribute positively to the final performance, although they play slightly different roles in the framework. The teacher-side meta optimization mainly improves the reliability of corrected supervision, whereas the student-side meta optimization strengthens the learner’s robustness when fitting corrected yet still imperfect labels.

When all components are jointly enabled, the full MCKD model achieves the best overall performance, reaching 31.60 mAP, 45.70 OF1, and 43.30 CF1. Compared with the setting that only uses corruption-aware labels, the complete framework improves the mAP, OF1, and CF1 by 2.90, 4.40, and 4.10 percentage points, respectively. These gains verify that the proposed modules are not only individually effective but also complementary to each other. Overall, the ablation results demonstrate that MCKD benefits from the coordinated interaction among label correction, teacher meta-learning, student meta-learning, and corruption-aware supervision, which together improve the model’s robustness in highly noisy partial multi-label learning scenarios.

In addition to component-level ablation, we further analyze the effect of teacher architecture on the quality of label correction which shows in Figure 4. Specifically, we compare several teacher backbones on Pascal VOC2007, including ResNet50, ResNet101, DenseNet121, and TResNet-M, under noise ratios of 0.3, 0.5, and 0.7. The results show that the teacher architecture has a noticeable but controlled influence on the final performance. Among all compared backbones, ResNet101 consistently achieves the best mAP, reaching 88.10%, 85.90%, and 78.00% under the three noise settings, respectively. This suggests that a stronger teacher encoder can learn more stable semantic decision boundaries and thus provide more reliable corrected soft supervision for the student network.

Figure 4. Effect of different teacher architectures on Pascal VOC2007 under different noise ratios.

By contrast, ResNet50 performs consistently worse than ResNet101 across all noise ratios, indicating that a relatively shallow backbone has limited capacity to model fine-grained semantic structures under severe label ambiguity. DenseNet121 and TResNet-M achieve competitive but slightly inferior results, which suggests that the proposed meta-correction and distillation framework is not restricted to one specific backbone design and can be coupled effectively with different mainstream convolutional architectures. Moreover, as the noise ratio increases, the performance gap among different teacher architectures becomes slightly larger. This phenomenon implies that under heavier supervision corruption, the representation quality and robustness of the teacher network become more critical to accurate label correction. Based on these observations, we adopt ResNet101 as the default teacher backbone in the remaining experiments.

8.6. Parameter Sensitivity Analysis

To further evaluate the robustness of the proposed method with respect to key hyperparameters and training configurations, we conduct a series of parameter sensitivity experiments from four aspects: the clean validation set size, the threshold parameter ${\gamma }_{\theta }$, the threshold parameter ${\eta }_{\theta }$, and the adaptive gating weight $\alpha$. These experiments are designed to examine whether the performance of the proposed framework is highly dependent on specific parameter choices and whether it remains stable within a relatively broad parameter range. For fair comparison, all experiments are conducted under the same training protocol except for the target factor under investigation, and the results are reported on Pascal VOC2007, MS-COCO, and NUS-WIDE.

Effect of clean validation set size. In the proposed meta-correction framework, a small clean validation set is used to construct the outer-level objective, and its size directly affects the supervisory signal used to guide the meta-teacher during candidate label correction. To analyze this factor, we vary the proportion of clean validation samples on Pascal VOC2007 and report the corresponding mAP values under three noise ratios, as shown in Figure 5.

Figure 5. Sensitivity analysis of clean validation set size on Pascal VOC2007 under different noise ratios.

As the clean validation set size increases from $0\%$ to $10\%$, the performance improves significantly under all noise ratios. This observation indicates that even a small amount of clean supervision can provide effective outer-level guidance and substantially alleviate the negative impact of noisy candidate labels during training. When the clean validation set size is further increased, the performance still improves, but the gain becomes much smaller. Specifically, under noise ratios of $0.3$, $0.5$, and $0.7$, the model reaches relatively favorable performance around $30\%$, $40\%$, and $30\%$, respectively, after which additional clean validation data bring only marginal improvement.

These results suggest that the clean validation set mainly provides directional supervision for meta-optimization, rather than relying on a large amount of additional annotation to continuously improve performance. In other words, the proposed method does not require a large clean validation set to establish an effective correction signal. Meanwhile, under higher noise ratios, the model becomes more sensitive to the introduction of clean validation samples, indicating that outer-level clean supervision plays a more important role when candidate labels are more severely corrupted. Overall, the proposed method achieves stable performance with a relatively small clean validation set, which demonstrates favorable data efficiency and practical applicability.

Effect of the threshold parameter ${\mathit{\gamma }}_{\mathit{\theta }}$. The parameter ${\gamma }_{\theta }$ controls the discrimination strength during candidate label selection or correction, and its value influences the decision boundary between potentially relevant labels and noisy labels. To evaluate its effect, we analyze the AvgPrecision values under different ${\gamma }_{\theta }$ settings on MS-COCO and NUS-WIDE at a $70\%$ noise ratio, as shown in Figure 6.

Figure 6. Sensitivity analysis of ${\gamma }_{\theta }$ on MS-COCO and NUS-WIDE under 70% noise ratio.

As shown in the figure, both datasets exhibit a consistent unimodal trend. When ${\gamma }_{\theta }$ gradually increases from a relatively small value, the model performance improves steadily and reaches the best or near-best level around ${\gamma }_{\theta }=0.85$. When ${\gamma }_{\theta }$ continues to increase, the performance starts to decline. This indicates that if ${\gamma }_{\theta }$ is too small, the constraint on candidate label selection is insufficient; so, some low-confidence noisy labels may still be retained. By contrast, if ${\gamma }_{\theta }$ is too large, the selection criterion becomes overly strict, and some truly relevant labels may be suppressed too early, resulting in the loss of useful supervision.

In addition, the optimal range is largely consistent across MS-COCO and NUS-WIDE, indicating that this parameter has good cross-dataset stability. Compared with settings far from the optimum, the performance fluctuations are relatively small when ${\gamma }_{\theta }$ lies in the medium-to-high range. This further suggests that the proposed method does not rely on extremely delicate parameter tuning to achieve satisfactory performance. Based on these observations, ${\gamma }_{\theta }=0.85$ is adopted in the subsequent experiments.

Effect of the threshold parameter ${\mathit{\eta }}_{\mathit{\theta }}$. The parameter ${\eta }_{\theta }$ is used to regulate the control pace of candidate-label reliability during training, and it essentially affects the correction strength and conservativeness of the label refinement process. To analyze its effect, we report AvgPrecision on MS-COCO and NUS-WIDE under different ${\eta }_{\theta }$ settings at a $70\%$ noise ratio, as shown in Figure 7.

Figure 7. Sensitivity analysis of ${\eta }_{\theta }$ on MS-COCO and NUS-WIDE under 70% noise ratio.

The results show that as ${\eta }_{\theta }$ gradually increases from a small value, the model performance rises rapidly at first and then becomes stable, reaching the best performance around ${\eta }_{\theta }=3$. This indicates that when ${\eta }_{\theta }$ is too small, the model has insufficient ability to control candidate-label reliability, and noisy information still participates excessively in training. As ${\eta }_{\theta }$ increases to a moderate range, the model can more effectively distinguish reliable supervision from noisy supervision, thereby improving the overall classification performance. However, when ${\eta }_{\theta }$ becomes too large, the performance declines gradually, suggesting that an overly strong control mechanism may suppress candidate-label information that still carries learning value, thus reducing the adaptability of the model to complex samples.

It is worth noting that the two datasets exhibit highly consistent tendencies, and the curves remain relatively smooth around the optimal point, demonstrating good robustness of this parameter within a reasonable range. Considering both performance and stability, ${\eta }_{\theta }=3$ is selected as the default setting in our method.

Effect of the adaptive gating weight $\mathit{\alpha }$. In addition to the above threshold parameters, we further investigate the effectiveness of the adaptive gating weight $\alpha$. This weight is used to regulate the fusion degree of different supervisory signals during the correction process, and its formulation directly affects the model’s ability to capture instance-level differences. To this end, we compare the adaptive $\alpha$ with several fixed scalar gates on NUS-WIDE under a $70\%$ noise ratio, and the results are reported in Table 5.

Table 5. Ablation study on the adaptive gating weight $\alpha$ on NUS-WIDE with 70% noise ratio.

Setting of $\mathit{\alpha }$	Description	mAP	OF1	CF1
Fixed $\alpha =0.25$	constant scalar gate	29.72	43.10	41.85
Fixed $\alpha =0.50$	constant scalar gate	30.18	44.02	42.71
Fixed $\alpha =0.75$	constant scalar gate	29.94	43.63	42.16
Adaptive $\alpha$ (Ours)	instance-adaptive scalar gate	31.60	45.70	43.30

As shown in the table, although fixed gating weights can provide a certain degree of regulation, their overall performance remains consistently inferior to the adaptive setting. Among the fixed strategies, $\alpha =0.50$ achieves the best result, indicating that a moderate fusion intensity is preferable to overly weak or overly strong fixed weighting. Nevertheless, regardless of whether $\alpha$ is fixed to $0.25$, $0.50$, or $0.75$, all corresponding mAP, OF1, and CF1 values are lower than those obtained by the adaptive scheme. Compared with the best fixed setting, the adaptive $\alpha$ further improves the mAP, OF1, and CF1 by $0.42$, $0.68$, and $0.59$ percentage points, respectively.

These results indicate that different samples exhibit substantial variation in candidate-label noise level, semantic complexity, and supervisory reliability. Therefore, a unified fixed gating weight is insufficient to accommodate all instances. In contrast, the adaptive gating mechanism can dynamically adjust the fusion strength according to instance characteristics, enabling the model to better suppress noisy interference while preserving useful supervision. As a result, it yields more stable and consistent performance gains. This confirms the necessity of the adaptive design of $\alpha$ for instance-level dynamic modeling in the proposed framework.

Overall, the proposed method exhibits favorable stability with respect to its key hyperparameters. First, only a small clean validation set is sufficient to provide effective outer-level supervision, indicating that the method has limited reliance on additional clean annotations. Second, both ${\gamma }_{\theta }$ and ${\eta }_{\theta }$ maintain stable performance over relatively wide ranges and achieve their best results around ${\gamma }_{\theta }=0.85$ and ${\eta }_{\theta }=3$, respectively, demonstrating good robustness of the proposed mechanisms. Finally, the adaptive gating weight $\alpha$ consistently outperforms the fixed weighting strategies, which verifies the importance of instance-aware dynamic fusion for improving candidate-label correction quality and final classification performance. These results together show that the proposed method remains stable under noisy conditions and has strong practical applicability.

8.7. Study of the T-SNE of the Validation

We employ t-SNE to visualize the progression of latent feature representations learned by MCKD on the Pascal VOC2007 dataset throughout the training process. As illustrated in Figure 8, the inter-class separability becomes progressively more pronounced as the training epochs increase.

Figure 8. As shown in the figure, the t-SNE visualization of our proposed MCKD method trained on Pascal VOC2007 under 70% noise demonstrates that the features are well clustered by class. Each category is clearly separated in the embedding space, with only a small portion of noisy samples being misclassified.

9. Conclusions

To address the challenges of Partial Multi-Label Learning, we proposed MCKD, a meta-learning-enhanced knowledge distillation framework. The proposed method improves candidate label refinement by combining a teacher–student distillation paradigm with a small clean meta set. In particular, MCKD incorporates a tailored meta-gradient optimization strategy and a dedicated teacher architecture for multi-label supervision refinement. After the teacher pretraining stage, the teacher model is fixed and used to provide corrected soft supervision for student learning. The experimental results show that MCKD achieves competitive performance under different noise settings and demonstrates its effectiveness in handling ambiguous candidate labels. In future work, we will further extend MCKD to more complex partial multi-label scenarios, including dynamic label ambiguity, richer inter-label dependency modeling, and real-world data with evolving supervision quality.