CoDC: Unified Diffusion and Classification for Enhanced Class-Incremental Learning

Featured Application

Privacy-constrained visual recognition systems and systems with limited storage that must learn new classes continuously without retaining real samples from previous tasks.

Abstract

In class-incremental learning (CIL), a model must learn new classes while retaining previous knowledge without storing all historical data. Generative replay mitigates catastrophic forgetting by synthesizing old class samples, but conventional pipelines usually train separate generative and classification models and can be degraded by generated images of poor quality. This paper proposes the Co-Diffusion Classifier (CoDC), a unified framework based on diffusion that performs image generation and classification in a single network. CoDC attaches a classification branch to the UNet encoder and introduces an exponential noise filtering loss according to diffusion timesteps so that cleaner samples contribute more strongly to representation learning. A base task classification pre-training stage followed by collaborative training with selective parameter freezing reduces conflicts between noise prediction and semantic feature extraction. For rehearsal-free replay, generated samples are selected using confidence and feature consistency filters. Experiments on CIFAR-100, FaceScrub, and Flowers-102 show that CoDC maintains strong incremental accuracy without storing real old class exemplars. Additional comparisons that account for protocol differences with recent exemplar-free and pre-trained model methods clarify the setting in which CoDC is most directly comparable.

1. Introduction

The typical training process for deep neural networks requires a collected dataset, such as large-scale images or text, over which the network iterates multiple times. However, in open-world scenarios, training data often arrives in a streaming format [1]. Due to storage limitations [2] or privacy concerns [3], this streaming data cannot be stored for the long term, requiring the model to be updated incrementally using only new class instances. This need has fostered the development of class-incremental learning (CIL) [4,5,6,7], which aims to build a classifier over all classes observed so far. A critical challenge in CIL is catastrophic forgetting [8], where optimizing a network with new classes erases knowledge of previously learned classes, leading to an irreversible decline in performance. Reducing catastrophic forgetting is therefore central to the design of CIL models. Traditional generative replay approaches employ a cascaded architecture where a generative model synthesizes samples from old classes, which are then used alongside new class samples to train a separate classifier. As illustrated in Figure 1a, this pipeline involves sequential processing through distinct models, resulting in high computational overhead and potential semantic bias from generated samples. In contrast, our proposed Co-Diffusion Classifier (CoDC) unifies these functionalities within a single diffusion model architecture, as shown in Figure 1b. By integrating generation and classification tasks, CoDC reduces computational cost while maintaining performance through joint optimization.

In a typical CIL setup, a model is evaluated on all previously seen classes after learning each new task. An ideal model should perform well on newly learned classes while retaining memory of old ones without forgetting. In practice, training data arrives sequentially at each step, forming a new training dataset (referred to as a “task”), and the model must be updated with these new classes. For instance, a model first learns “bird” and “dog” (Task 1), then “tiger” and “fish” (Task 2), followed by “monkey” and “sheep” (Task 3), and so on. After each task, the model is tested on all seen classes to assess its discriminative ability. A successful CIL model must balance learning features of new classes (plasticity) with preserving patterns of old classes (stability), a trade-off known as the “stability-plasticity dilemma” [8], where stability represents the ability to retain prior knowledge and plasticity represents the ability to adapt to new patterns. Continual and incremental learning were studied before the widespread adoption of deep learning. Early approaches mainly used traditional machine learning models and were often evaluated on a small number of incremental stages. Modern data streams require models that can be updated over longer sequences and larger class sets, making deep neural networks a suitable foundation for CIL. As a result, deep learning-based CIL has become an active research area in machine learning and computer vision.

Despite this progress, a clear research gap remains in strict CIL settings where real samples from previous classes cannot be retained. Memory-assisted methods can preserve old knowledge by storing exemplars, but this assumption is unsuitable for privacy sensitive applications or applications with limited storage. Generative replay avoids storing old images, yet existing replay pipelines are still insufficient in three aspects. First, they usually train a generator and a classifier as separate modules, which increases the model size, training cost, and maintenance complexity. Second, the classifier is trained on generated samples whose semantic quality is not guaranteed, so low fidelity or label ambiguous pseudo-samples can bias the decision boundary. Third, when diffusion-based generators are used, noisy intermediate samples contain different amounts of semantic information, but conventional replay does not explicitly control how much such noisy samples should affect representation learning. CoDC addresses this gap by coupling replay generation and classification in a shared diffusion network: the UNet encoder provides features for the classifier, the exponential timestep weighting reduces the influence of highly noisy samples on the classification loss, and confidence plus feature consistency filtering removes generated replay candidates that are semantically inconsistent with their target classes. Therefore, the central question is whether generation and classification can be coupled in one model while still controlling the noise and semantic mismatch introduced by generated replay.

This work addresses the following research questions:

• RQ1: Can a single diffusion model support both old class image generation and discriminative classification for class-incremental learning?
• RQ2: How can the classifier use noisy diffusion samples without learning noise-contaminated features?
• RQ3: Can generated samples replace real exemplar storage under a strict zero memory protocol?

The main contributions are summarized as follows:

• We propose CoDC, a unified diffusion-classification architecture that attaches a classification branch to the UNet encoder and performs generation and classification within one network.
• We design an exponential noise filtering mechanism that weights the classification loss according to the diffusion timestep, reducing the impact of highly noisy samples.
• We introduce a base task classification pre-training stage followed by collaborative training with parameter freezing to reduce conflicts between noise prediction and semantic feature extraction.
• We develop a rehearsal-free generative replay procedure that selects generated old class samples using confidence and feature consistency before combining them with current new class data.
• We report comparisons under both strict zero memory and mixed memory protocols to clarify when CoDC is directly comparable with existing baselines.

The implementation of CoDC is publicly available at https://github.com/896393962/CoDC-CIL, accessed on 10 June 2026.

2. Related Work

2.1. Class-Incremental Learning

Recent work has advanced this field in machine learning and computer vision. Recent journal surveys have clarified the distinctions among task-incremental, domain-incremental, and class-incremental learning, and have emphasized that CIL is especially challenging because the model must discriminate among all classes seen so far without task identity at inference time [5,6,7,9]. To mitigate catastrophic forgetting in CIL, existing methods can be broadly categorized into parameter regularization, knowledge distillation, data replay, and architectural approaches. Parameter regularization methods constrain important parameters during incremental updates. For instance, EWC [10] was a pioneering work that constrained important parameters using the Fisher information matrix to estimate their importance, thereby reducing forgetting. During the model training phase, the importance of each parameter is calculated by accumulating the sensitivity of the loss function to changes in each parameter. Knowledge distillation methods maintain consistency between the outputs of the old and new models for a given input while learning a new task. The old model is the one that has finished learning the previous task, and the new model inherits the parameters of the old model and can be updated on the new task. This requires introducing an additional distillation loss during network training to reduce the forgetting of old class knowledge. For example, LwF (Learning without Forgetting) [11] introduced a distillation loss that minimizes the KL divergence between the probability distributions of the old and new models, preserving knowledge of old classes. Data replay methods can be divided into two categories: storing a small subset of real samples from past classes (rehearsal) and generating pseudo-samples. Replay remains one of the most effective families of continual learning strategies, but recent journal studies also note that replay becomes problematic when privacy, storage, or long streams prevent retaining historical examples [12,13]. The original teacher–student formulation of knowledge distillation is also relevant to this family of methods [14]. Regardless of whether the samples are real or pseudo, these old class data are used together with new class data to update the current model. Combining old and new class data in this way reduces catastrophic forgetting during the incremental learning process. For example, iCaRL [15] is a prominent rehearsal method that stores a fixed number of exemplars for each old class, selected using a nearest-mean-of-exemplars (NME) strategy. To address privacy concerns, generative replay methods use models such as autoencoders or GANs [13,16,17], including the Closed-loop Memory Replay GAN (CloGAN), to synthesize pseudo-samples of old classes for joint training. More recently, diffusion models [18,19] have also been used. Our work falls into this category but introduces a unified architecture that eliminates the need for a separate generative model and classifier. Architectural methods dynamically adjust the model’s capacity to accommodate new classes. For example, DER (Dynamically Expandable Representation) [20] dynamically expands the network by adding a new feature extractor for each incremental task while freezing the old ones. The features from all extractors are concatenated before being fed to the classifier, preserving old knowledge while providing capacity for new learning. These categories are not mutually exclusive and have no strict boundaries; they provide a practical way to analyze the development of class-incremental learning from different perspectives. Recent exemplar-free CIL methods further focus on compensating feature or semantic drift without retaining old images. Representative examples include FeTrIL [21], SEED [22], and DPCR [23]. These methods are highly relevant to our setting because they avoid exemplar buffers, but their published protocols still differ in task split, backbone, and whether the method is rerun under a small start or big start setting. We therefore report the protocol source explicitly in the experimental comparison instead of mixing all numbers into a single ranking.

In parallel, pre-trained model-based CIL has become an active direction. Methods such as RanPAC [24], EASE [25], TUNA [26], and SplitLoRA [27] use large pre-trained ViT backbones and parameter-efficient adaptation, while DiffClass [18] relies on a pre-trained diffusion model. These approaches provide recent reference results for settings with external pre-training. However, because they exploit large-scale pre-training and different computational assumptions, we separate them from methods with standard backbones and use them only as reference comparisons.

2.2. Diffusion Models

Diffusion models [28,29,30] are a class of generative models that have recently gained prominence. They operate by progressively adding Gaussian noise to data in a “forward process” and then learning to reverse this process to generate data from noise in a “reverse process.” The core of this process lies in learning the denoising process to generate data. The diffusion process is typically divided into multiple timesteps $t\in \{1,\dots ,T\}$, with each timestep corresponding to a different “noisy” state of the data. The objective is to learn how to recover the original data from a state of complete noise. In image diffusion models, a UNet architecture [31] is commonly used as the denoising backbone. The number of timesteps, T, controls the noising and denoising process; a smaller T results in a milder noising process where the data retains more of its original features, but the generated samples may lack detail or appear blurry. A larger T allows the data to approach complete noise, making the denoising process more complex and improving the quality of generated images, but at the cost of increased computation. Subsequent works like DDIM [32] have focused on faster sampling, while Latent Diffusion Models (LDMs) [33] perform the diffusion process in a lower-dimensional latent space to improve efficiency. This approach achieves a good balance between the speed and quality of image generation. Meanwhile, the potential of diffusion models has also been extended to audio processing [34] and text-to-3D generation [35]. Our work leverages the UNet backbone of diffusion models and repurposes its encoder for classification in a multi-task learning framework.

2.3. POD Loss Function

The Predefined Optimal-distribution Loss (POD Loss) [36] aims to maximize inter-class distances while minimizing intra-class distances in the latent feature space. Instead of using a SoftMax layer to constrain posterior probabilities, it directly uses a cosine loss on the features, which has been shown to achieve superior classification performance. The loss is formulated as follows:

$$L_{pod}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left(1-{\cos }_N{\left({\theta }_{y_i}\right)}^2\right)+\lambda \sum _{j\ne i,n_j\ne n_i}{R}_{ij}^2$$

(1)

$${\cos }_N\left({\theta }_{y_i}\right)={\displaystyle \frac{\cos \left({\theta }_{y_i}\right)}{1+\delta e^{-M/|x_i|}}}$$

(2)

$$R={\displaystyle \frac{1}{B-1}}\left(X-X_{pedcc}\right){\left(X-X_{pedcc}\right)}^T$$

(3)

where N is the number of training samples, i indexes a sample, $x_i$ is the feature vector of the i-th sample, $|x_i|$ is its feature magnitude, $y_i$ is its class label, and ${\theta }_{y_i}$ denotes the angle between $x_i$ and the predefined target vector of class $y_i$. The term ${\cos }_N\left({\theta }_{y_i}\right)$ is the magnitude-normalized cosine score used by POD Loss. $\lambda$ controls the strength of the decorrelation term, $R_{ij}$ is the $(i,j)$ entry of the correlation matrix R, and $n_i$ and $n_j$ denote the class indices associated with the corresponding feature outputs when identifying inter-class pairs. The parameter $\delta$ is a scaling factor used to amplify the effect of samples with smaller magnitudes, M is the average feature magnitude in the current training stage, and e denotes the current training epoch. B is the batch size, X is the matrix of network feature outputs in a mini-batch, and $X_{pedcc}$ is the matrix of predefined target feature vectors. POD Loss focuses on the classification accuracy of hard samples via $\delta$ and reduces linear redundancy in the network output through the correlation matrix R.

3. Materials and Methods

To address the challenges of high computational cost and semantic deviation in generative replay, we propose the Co-Diffusion Classifier (CoDC), a unified network for image generation and classification.

3.1. Collaborative Generation and Classification Architecture

In diffusion models, the UNet is used as the core component of the generative network. This paper proposes a method that aims to simultaneously perform feature extraction tasks by leveraging the UNet encoder (the left half of the UNet) during the training process of the diffusion model. Specifically, while training the diffusion model to predict noise, features are extracted from an intermediate layer of the encoder for an image classification task, thereby forming a dual task learning framework, as shown in Figure 2. However, a standard UNet architecture, when compared to a dedicated classification network like ResNet50, often lacks the necessary depth for abstract feature representation. For instance, a ResNet50 adapted for 64 × 64 images contains a deep stack of residual blocks, whereas a typical UNet encoder has fewer convolutional blocks, limiting its feature extraction capacity. To address this, we enhance the UNet encoder by increasing its depth (i.e., the number of residual blocks per stage), enabling it to learn more discriminative representations for classification while retaining its generative function. To achieve dual task learning, a classification branch is introduced into the network, and a composite loss function is designed. Therefore, the total loss function of the network consists of two parts (Equation (4)): the first part is the mean squared error (MSE) loss for the noise prediction task of generation, and the second part is the classification loss for the classification task. We employ POD Loss for $L_{classify}$ due to its effectiveness in creating a geometrically optimal feature space by enforcing orthogonality between class centroids. These two parts work together to constrain the network parameters, theoretically enabling the network to have both generation and classification capabilities.

$$L_{total}={\lambda }_1{L}_{mse}+{\lambda }_2{L}_{classify}$$

(4)

However, in practice, the noisy images can contaminate the feature extraction process, thereby severely degrading classification performance.

3.2. Noise Filtering Mechanism

To address the issue of noise pollution, it is first necessary to understand how diffusion models progressively inject noise into an image. In a diffusion model, as the timestep t increases (from $t=0$ to $t=999$), more noise is added to the image. This is achieved through the mathematical design of the noise schedule parameter ${\beta }_t$. We adopt a linear schedule for ${\beta }_t$:

$${\beta }_t={\beta }_{min}+{\displaystyle \frac{t}{T}}({\beta }_{max}-{\beta }_{min})$$

(5)

where ${\beta }_{min}$ and ${\beta }_{max}$ are the initial and final noise variances of the linear schedule, respectively. For compact notation in the following derivation, we define $\Delta \beta ={\beta }_{max}-{\beta }_{min}$ as the total increase in the noise variance over the diffusion process. The forward noising process of the diffusion model can be defined by the following formulas:

$$x_t=\sqrt{{\overline{\alpha }}_t}{x}_0+\sqrt{1-{\overline{\alpha }}_t}ϵ,ϵ\sim \mathcal{N}(0,I)$$

(6)

where ${\overline{\alpha }}_t={\prod }_{i=1}^t{\alpha }_i$ and ${\alpha }_t=1-{\beta }_t$. Here, ${\overline{\alpha }}_t$ represents the cumulative signal decay coefficient from the initial state $x_0$ to step t. In this subsection, t denotes the diffusion timestep and T is the total number of diffusion steps. For clarity, t is used only for diffusion timesteps, while k denotes the incremental task index in the CIL procedure. As t increases from 0 to T, ${\overline{\alpha }}_t$ monotonically decreases to 0, meaning the image $x_t$ becomes progressively dominated by noise; this signal decay trend motivates the weighting by the timestep introduced below.

Our key insight is that samples with low noise (small t) contain more semantic information and are more valuable for training the classifier. Therefore, we propose a weighted loss strategy where the contribution of $L_{classify}$ for each sample is weighted based on its timestep t. To derive a precise weighting function, we analyzed the relationship between the cumulative signal rate ${\overline{\alpha }}_t$ and the timestep t. The formula for ${\overline{\alpha }}_t$ can be expressed as:

$$\ln {\overline{\alpha }}_t={\displaystyle \sum _{i=0}^t}\ln (1-{\beta }_i)$$

(7)

Given that ${\beta }_i\ll 1$, we can use a Taylor expansion for $\ln (1-{\beta }_i)$ and truncate it to the second order:

$$\ln (1-{\beta }_i)\approx -{\beta }_i-{\displaystyle \frac{{\beta }_i^2}{2}}$$

(8)

This leads to the approximation:

$$\ln {\overline{\alpha }}_t\approx -{\displaystyle \sum _{i=0}^t}{\beta }_i-{\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=0}^t}{\beta }_i^2$$

(9)

From this, we can see that ${\overline{\alpha }}_t$ has an exponential relationship with the powers of t. As shown in Figure 3, the evolution of ${\overline{\alpha }}_t$ is most strongly correlated with the $t^2$ term. This justifies our design of an exponential noise filtering algorithm where the weight is a function of $t^2$, as defined in Equation (11). This ensures that the weights assigned to the classification loss accurately reflect the ratio between signal and noise at each timestep.

$$\begin{array}{cc}{\overline{\alpha }}_t\approx \exp & (-t({\beta }_{min}+{\displaystyle \frac{{\beta }_{min}^2}{2}})-{\displaystyle \frac{t^2}{T}}({\displaystyle \frac{\Delta \beta }{2}}+{\displaystyle \frac{{\beta }_{min}}{2}}\Delta \beta )\\ & -{\displaystyle \frac{t^3}{6T^2}}\Delta {\beta }^2)\end{array}$$

(10)

Equation (10) is used to derive the weighting function for the classification loss. As depicted in Figure 3, an analysis of the different power terms of t in this expression shows a strong correlation between ${\overline{\alpha }}_t$ and the $t^2$ term. Therefore, we design an exponential noise filtering algorithm. After substituting the set values, the weight for a sample at timestep t is defined as:

$$\mathrm{weight}\left(t\right)=\exp \left(-{\displaystyle \frac{t^2}{T}}({\displaystyle \frac{\Delta \beta }{2}}+{\displaystyle \frac{{\beta }_{min}}{2}}\Delta \beta )\right)\approx e^{-t^2/{10}^5}$$

(11)

As shown in Figure 4, this function assigns high weights to samples with low noise (small t) and exponentially decaying weights as t increases, reducing the influence of highly noisy samples on the classification task. This exponential rule is the default noise filtering strategy used by CoDC.

3.3. Two-Step Training with Parameter Freezing

The dual objectives of noise prediction (for generation) and feature extraction (for classification) are inherently contradictory. The former aims to model a noisy distribution, while the latter seeks to filter out noise to capture clean semantics. This conflict can lead to suboptimal performance for both tasks. To address this, we use a two stage initialization and collaborative training scheme in which Stage 1 is applied only to the base task. The rationale for freezing deeper encoder blocks is grounded in the hierarchical nature of feature extraction in deep neural networks. Deeper layers are known to capture abstract semantic information that is critical for discriminating between classes (e.g., the overall shape of an object). In contrast, shallower layers capture more generic visual patterns such as edges and textures. In our collaborative architecture, the generation task, which involves modeling noise, primarily conflicts with the extraction of abstract semantics. By freezing the deep encoder layers after the base task classification pre-training, we protect these semantic representations from being corrupted by the noise prediction objective. This allows the shallower layers to remain plastic and adapt to the dual task loss, balancing the need for stable semantic features for classification with the flexible feature adjustments required for high quality image generation. This targeted freezing strategy is key to mitigating task conflict and preventing the degradation of the model’s classification capabilities. Stage 1: Base Task Classification Pre-training. Stage 1 is performed only once on the base task. We first train the CoDC network using only the classification loss ($L_{classify}$) and the real samples from the base classes. This allows the UNet encoder to develop stable and discriminative semantic features without interference from the noise prediction task before incremental learning begins. Stage 2: Collaborative Training with Parameter Freezing. After this base task pre-training stage, we freeze the parameters of the deeper blocks of the UNet encoder. These deep layers capture abstract, task-specific semantic information required for classification. By freezing them, we preserve the stability of the learned feature space. The shallower layers, which extract more general visual features, remain trainable, along with the decoder and classification head. The model is then trained on the full collaborative loss ($L_{total}$). This collaborative stage is used for the base task after Stage 1 and for all subsequent incremental tasks; Stage 1 is not repeated for later tasks. This approach allows the network to adapt its early feature extraction and generative capabilities while protecting the core semantic representations from being corrupted by the noise prediction objective.

3.4. Rehearsal-Free Generative Replay

With the collaborative network, we can implement a generative replay strategy for CIL that requires no storage of real data. The complete streaming procedure is illustrated in Figure 5 and is summarized as follows. At incremental step k, the training data contain only the real samples of the newly arriving classes. Here, k indexes the incremental learning task and is distinct from the diffusion timestep t used in the noise-filtering derivation. Before updating the model on these new classes, the CoDC model learned after step $k-1$ generates pseudo-samples for previously seen classes. These generated old class samples are screened by the confidence and feature consistency criteria described below. The retained generated samples are then merged with the real samples from the current step, and the combined set is used to update CoDC with the collaborative objective. In these later incremental steps, the base task classification pre-training stage is not repeated. After training at step k, the real samples from the current step are discarded; only the model parameters and lightweight feature statistics used for filtering are kept. Thus, the replay set used during incremental learning contains generated images rather than stored real old class exemplars.

Although diffusion models can synthesize visually realistic images, the discrepancy between the distribution of generated images and the original training distribution remains an underexplored issue. For instance, a diffusion model trained on a certain dataset to generate 100 images of class A might occasionally produce a few images that resemble other classes (negative samples). This discrepancy can stem from training data bias or sampling stochasticity. Because generated images are used as replay data in CIL, it is important to select generated samples that reduce the distribution gap between generated and real data. Therefore, to address this issue, we propose a two-step filtering strategy to select only generated samples of high quality, as illustrated in Figure 6. (1) Confidence-Based Filtering: The core idea is to filter out generated images that do not match their class labels based on classification confidence. That is, during the image generation process, specifically at the last denoising step, we use the output probability from the UNet’s classification head to screen for samples with high confidence and filter out ambiguous samples with low confidence. This method naturally leverages the advantage of the collaborative network. The confidence is calculated as follows:

$$F_1=\max p(y=y_i|x_i)>T_1$$

(12)

We set a threshold $T_1$ (e.g., 0.80). If the confidence is greater than $T_1$, the image is retained as replay data. Otherwise, it is discarded. (2) Feature Consistency Filtering: Using only filtering based on confidence might still allow some “deceptive” samples to pass through. To mitigate this, we introduce filtering based on feature space consistency. First, when a class is learned, we compute and retain its mean feature vector ${\mu }_c$ from encoder features and normalize it using L2 normalization. This stores a compact feature statistic rather than retaining the original images. Then, during image generation, at the final denoising step, we extract the feature vector $f_{x_{gen}}$ of the generated image, also perform L2 normalization, and finally calculate the cosine similarity with the class mean.

$$\cos \left(\theta \right)={\displaystyle \frac{f_{x_{gen}}·{\mu }_c}{\parallel f_{x_{gen}}\parallel \parallel {\mu }_c\parallel }}>T_2$$

(13)

A sample is accepted only if its similarity is above threshold $T_2$ (e.g., 0.75). This ensures that the generated exemplars not only have high confidence but also cluster tightly around the class centroid in the feature space. In our experiments, we first apply the confidence threshold, followed by the feature cosine similarity, to select the most suitable generated images for data replay.

3.5. Experimental Protocol

Datasets: We evaluate our method on three benchmark datasets: CIFAR-100 [37], a 100-class dataset of 32 × 32 images; FaceScrub [38], a fine-grained face dataset with 100 identities and 64 × 64 images; and Flowers-102 [39], a fine-grained flower dataset where we use the first 100 classes with 64 × 64 images. For Flowers-102, each class has 30 training samples and 7 test samples. Incremental Settings: We use two task protocols. In Scenario 1, the model starts from 20 base classes and then learns four increments of 20 classes. In Scenario 2, the model starts from 50 base classes and then learns five increments of 10 classes. Unless otherwise stated, the class order is fixed across methods, and the random seed is set to 1993 in the reproduced baseline runs. Evaluation Metrics: In the context of CIL, researchers commonly use incremental accuracy and forgetting rate as performance evaluation tools.

• Incremental Accuracy (A): The average classification accuracy across all incremental tasks at the end of the entire learning process. Given K tasks, let $A_k$ be the accuracy on all seen classes after task k. The final average accuracy is:

  $$A={\displaystyle \frac{1}{K}}{\displaystyle \sum _{k=1}^K}{A}_k$$

  (14)
• Average Forgetting (F): This metric quantifies the model’s ability to retain knowledge of previously learned classes. For any task $i<k$, the degree of forgetting for task i after completing task k, $f_{k,i}$, is the difference between the highest accuracy ever achieved for task i and the current accuracy: $f_{k,i}={\max }_{m\in \{1,\dots ,k-1\}}{a}_{m,i}-a_{k,i}$, where $a_{m,n}$ is the accuracy on task n after learning task m. The average incremental forgetting rate across the entire process is:

  $$F={\displaystyle \frac{1}{K}}{\displaystyle \sum _{k=1}^K}{F}_k\mathrm{where}{F}_k={\displaystyle \frac{1}{k-1}}{\displaystyle \sum _{i=1}^{k-1}}{f}_{k,i}$$

  (15)
• Fréchet Inception Distance (FID) [40]: A common method for evaluating the quality of generated images. A lower FID score indicates higher quality.

Implementation Details: The CoDC implementation is based on Python 3.11, PyTorch 2.5.1, and torchvision 0.20.1. Experiments were conducted on a workstation running Ubuntu 22.04 LTS with an Intel Core i7-13700K CPU, 32 GB system RAM, and an NVIDIA GeForce RTX 3090 GPU with 24 GB memory; the CUDA version was 12.4. The diffusion model uses $T=1000$ denoising steps, a linear noise schedule from ${\beta }_1={10}^{-4}$ to ${\beta }_T=0.02$, UNet channel width 128, channel multipliers [1, 2, 4, 4], three residual blocks per stage, dropout 0.15, learning rate ${10}^{-4}$, gradient clipping 1.0, and the default exponential noise filtering rule in Equation (11). Generated replay images are sampled with DDIM using 250 steps. The public source package retains the training scripts and configuration logs used for the reported baseline experiments. Baselines: We compare CoDC with representative CIL methods, including rehearsal-based methods (iCaRL [15], DER [20], FOSTER [41], MEMO [42]), and a knowledge distillation method (LwF [11]).

4. Results

4.1. Analysis of the Collaborative Network

We first validate the effectiveness of our collaborative CoDC network. We retain the original purpose of the diffusion model’s training noise, which is to use the MSE loss function to constrain the input and output of the UNet network for noise training, while blocking the update of decoder parameters from the classification loss during backpropagation to reduce degradation in generated image quality.

Table 1. Classification and generation performance of CoDC.

Metric	Model	CIFAR-100	FaceScrub	Flowers
ACC (%)	Specialist	74.60	94.01	63.57
	CoDC (Ours, exponential filter)	74.87	92.95	64.66
FID	Specialist	5.32	5.27	24.70
	CoDC (Ours)	5.48	5.10	25.10

compares the classification accuracy (ACC) and generation quality (FID) of our joint model (CoDC) against specialized models (a UNet trained only for classification and a diffusion model trained only for generation). Generated images are sampled using DDIM with 250 steps, and the CoDC accuracy is reported under the default exponential noise filtering configuration used throughout this paper. The results show that CoDC achieves comparable performance on both tasks simultaneously. For instance, on CIFAR-100, CoDC reaches 74.87% classification accuracy, slightly exceeding the specialist classifier by 0.27%, while the FID score increases by only 0.16. These results indicate that the unified architecture integrates both functions without a substantial loss in either task.

To further investigate the impact of the collaborative network on the generated data distribution, we analyzed the classification performance of images generated by both the original generative network and our collaborative network. We used the same specialist UNet classifier to evaluate both sets of generated images. As shown in

Table 2. Classification performance of data generated by original and collaborative networks.

Generated Data Source	CIFAR-100	FaceScrub	Flowers
Collaborative Network	66.43	86.58	54.10
Original Network	67.88	85.43	55.20

, the classification accuracies are very close, indicating that while the collaborative training creates a slight divergence, the generated images still retain most of the original dataset’s features.

4.2. Comparison with Existing Methods

To avoid overclaiming across different memory assumptions, we report two types of comparisons.

Table 3. Mixed-protocol reference comparison: final accuracy (%) with 20 base classes and 20-class steps (baselines use 4% real-sample memory; CoDC uses 0 memory).

Method	CIFAR-100	FaceScrub	Flowers
ICaRL(NME) [15]	54.28	80.39	27.30
Foster(CNN) [41]	59.44	66.30	29.10
DER(CNN) [20]	63.92	90.43	34.80
MEMO(CNN) [42]	62.29	80.98	32.00
CoDC (Ours, 0 memory)	65.97	80.50	55.30

and

Table 4. Mixed-protocol reference comparison: final accuracy (%) with 50 base classes and 10-class steps (baselines use 4% real-sample memory; CoDC uses 0 memory).

Method	CIFAR-100	FaceScrub	Flowers
ICaRL(NME) [15]	51.57	79.49	33.10
Foster(CNN) [41]	58.77	76.46	34.50
DER(NME) [20]	63.46	90.20	46.90
MEMO(NME) [42]	58.74	60.65	28.10
CoDC (Ours, 0 memory)	65.50	80.16	53.73

are mixed-protocol reference comparisons in which CoDC stores zero real old samples whereas rehearsal-based baselines use a 4% exemplar buffer. These tables show how CoDC relates to commonly used memory-assisted settings, but they are not treated as strictly equivalent evaluations.

Table 5. Rehearsal-free setting (0 memory) comparison—Scenario 1 (20 base, 20-class steps).

	CIFAR-100		FaceScrub		Flowers
Method	Accuracy (%)	Forgetting (%)	Accuracy (%)	Forgetting (%)	Accuracy (%)	Forgetting (%)
ICaRL(CNN) [15]	17.53	82.17	18.63	93.05	16.30	76.20
DER(CNN) [20]	15.92	82.25	19.17	94.67	16.40	74.53
MEMO(CNN) [42]	17.01	84.07	18.84	92.95	17.30	75.91
LwF(CNN) [11]	35.20	46.56	40.86	53.14	33.60	39.99
Ours	65.97	10.11	80.50	13.97	55.30	22.86

and

Table 6. Rehearsal-free setting (0 memory) comparison—Scenario 2 (50 base, 10-class steps).

	CIFAR-100		FaceScrub		Flowers
Method	Accuracy (%)	Forgetting (%)	Accuracy (%)	Forgetting (%)	Accuracy (%)	Forgetting (%)
ICaRL(CNN) [15]	8.94	83.18	9.34	95.45	7.60	76.40
DER(CNN) [20]	8.57	80.65	9.78	95.58	8.90	75.93
MEMO(CNN) [42]	8.49	83.25	9.55	93.16	8.50	78.81
LwF(CNN) [11]	24.12	37.83	27.57	39.83	18.40	34.21
Ours	65.50	8.78	80.16	10.02	53.73	18.58

provide the direct rehearsal-free comparison, where all methods use zero real-sample memory. We evaluate CoDC in two CIL scenarios. In the first scenario, the model starts with 20 base classes, and 20 new classes are added at each incremental step. Baseline methods are allowed to store a small buffer of real samples (4% exemplars), while our method stores none. Because the memory assumptions differ, Table 3 is intended as a mixed-protocol reference rather than a strictly equivalent ranking. Under this reference view, CoDC obtains the best results on CIFAR-100 and Flowers despite using zero real exemplars. For example, on CIFAR-100, CoDC achieves 65.97% accuracy, which is 2.05 percentage points higher than DER in this mixed-protocol reference setting.

In the second scenario, simulating a warm start, the model begins with 50 base classes and adds 10 classes per step. Table 4 provides the same mixed-protocol reference comparison for this scenario. CoDC is 2.04 percentage points higher than DER on CIFAR-100 under this reference setting, while direct equivalence is assessed in the zero-memory tables below.

DER with memory obtains the highest FaceScrub accuracy in Table 3. This result is reasonable because FaceScrub is an identity-recognition dataset where a small number of stored real exemplars can strongly preserve class-specific facial features. In contrast, CoDC stores no real samples and relies only on generated replay. When the same baselines are evaluated without memory, DER drops sharply on FaceScrub, whereas CoDC maintains 80.50% accuracy. Therefore, Table 3 and Table 4 should be interpreted as mixed memory reference comparisons, while Table 5 and Table 6 provide the strict rehearsal-free comparison.

More importantly, we conduct a direct comparison in a strictly rehearsal-free setting where no method is allowed to store real samples. As shown in Table 5 and Table 6, without exemplars, the performance of all baseline methods plummets due to severe catastrophic forgetting. In contrast, our generative replay strategy allows CoDC to maintain high accuracy and low forgetting rates under the same zero memory constraint. On CIFAR-100 (Scenario 1), CoDC achieves 65.97% accuracy with only 10.11% forgetting, while the next best method, LwF, only reaches 35.20% accuracy with 46.56% forgetting. These zero-memory results are the basis for direct claims of baseline superiority under equivalent memory conditions and show the advantage of replacing real exemplar storage with filtered generated replay in privacy sensitive CIL scenarios and settings with limited storage.

To respond to recent progress while keeping the comparison fair,

Table 7. Protocol aware comparison with recent rehearsal-free and pre-trained model-based CIL methods on CIFAR-100. Direct comparison is intended only within Group A. Group B is included to position CoDC relative to recent large pre-trained model results, not to rank methods under the same computational setting.

Method	Venue	Setting/Source	Backbone or Pre-Training	Avg. Acc. (%)	Final Acc. (%)
Group A: Standard methods without large external pre-training (direct comparison)
LwF [11]	TPAMI’17	Ours, 20-class steps	ResNet-32	35.20	–
ABD (reported in [18])	ICCV’21	DiffClass, N = 5, memory = 0	ResNet-18	60.78	44.69
PASS (reported in [18])	CVPR’21	DiffClass, N = 5, memory = 0	ResNet-18	63.31	49.27
IL2A (reported in [18])	NeurIPS’21	DiffClass, N = 5, memory = 0	ResNet-18	58.67	44.96
R-DFCIL (reported in [18])	ECCV’22	DiffClass, N = 5, memory = 0	ResNet-18	64.67	49.48
SSRE [43]	CVPR’22	DiffClass, N = 5, memory = 0	ResNet-18	56.96	39.89
FeTrIL [21]	WACV’23	DiffClass, N = 5, memory = 0	ResNet-18	58.68	42.67
SEED [22]	ICLR’24	DiffClass, N = 5, memory = 0	ResNet-18	63.05	52.14
DPCR [23]	ICML’25	Authors’ CIFAR-100, 10 tasks	ResNet-18	63.21	50.24
CoDC (Ours)	–	Ours, 20-class steps	UNet	65.97	57.21
Group B: Pre-trained large-model methods (reference only)
DiffClass [18]	ECCV’24	DiffClass, N = 5, memory = 0	Stable Diffusion v1.5 + LoRA	69.77	62.21
L2P (reported in [25])	CVPR’22	EASE, B0 Inc5 PTM setting	ViT-B/16-IN21K	85.94	79.93
DualPrompt (reported in [25])	ECCV’22	EASE, B0 Inc5 PTM setting	ViT-B/16-IN21K	87.87	81.15
CODA-Prompt (reported in [25])	CVPR’23	EASE, B0 Inc5 PTM setting	ViT-B/16-IN21K	89.11	81.96
SimpleCIL (reported in [25])	CVPR’24	EASE, B0 Inc5 PTM setting	ViT-B/16-IN21K	87.57	81.26
ADAM+Adapter (reported in [25])	CVPR’24	EASE, B0 Inc5 PTM setting	ViT-B/16-IN21K	90.65	85.15
EASE [25]	CVPR’24	EASE, B0 Inc5 PTM setting	ViT-B/16-IN21K	91.51	85.80
TUNA [26]	ICCV’25	Authors’ B0 Inc5 PTM setting	ViT-B/16-IN21K	94.44	90.74
SplitLoRA [27]	ICLR’26	Authors’ B0 Inc5 PTM setting	ViT-B/16-IN21K	93.11	90.84

Avg. Acc. denotes average incremental accuracy. Final Acc. denotes the last-stage accuracy when reported. Rows marked “reported in” reuse published comparison tables rather than rerunning the methods. Because task splits, initialization, and backbones are not identical across sources, Group A is used for protocol aware comparison and Group B is reference only.

separates published CIFAR-100 results into two groups. Group A contains standard or exemplar-free CIL methods that do not rely on large externally pre-trained foundation models. This is the group used for direct comparison with CoDC. Group B contains pre-trained model-based methods that use large ViT or diffusion backbones and are therefore reported as reference numbers only.

Under this protocol aware view, CoDC remains competitive among standard methods, reaching 65.97% average incremental accuracy without storing real old class exemplars. Its accuracy is higher than several recent exemplar-free results reported under the closest five task CIFAR-100 setting, including FeTrIL [21] and SEED [22], and is comparable to the strongest results obtained with standard backbones. The gap between average incremental accuracy and final accuracy reflects the accumulated difficulty of retaining early classes after all increments are learned. For CoDC, the final accuracy is lower than the average accuracy, which is expected in long horizon rehearsal-free CIL because no real old class exemplars are stored. Therefore, both metrics are reported to distinguish overall incremental performance from retention ability at the final stage. We avoid claiming direct superiority over Group B because those methods use external pre-training and substantially different model capacity.

4.3. Data Augmentation with Generated Images

Data augmentation is a key technique to improve model generalization. Research has shown that using generated data for augmentation can enhance performance, but the relationship between the quantity of generated images and accuracy is not extensively discussed. We analyze the effectiveness of using varying quantities of generated images for data augmentation and compare it with the mixup method. As shown in

Table 8. Data augmentation effects of generated images and mixup (accuracy %).

Method	CIFAR10	CIFAR100	FaceScrub	Flowers
base	93.97	74.60	94.01	63.57
mixup	95.30	77.36	95.17	65.20
ddim250 × 1	91.03	67.88	85.43	55.20
ddim250 × 3	93.70	74.03	88.73	63.86
ddim250 × 5	94.26	75.85	89.84	66.83
base+ddim250 × 1	94.50	75.91	94.46	68.56
base+ddim250 × 3	95.17	77.27	95.19	68.92
base+ddim250 × 5	95.32	77.47	94.48	70.29
base+ddim250 × 7	95.31	77.39	94.37	70.51

and Figure 7, adding multiples of generated data to the original training set can further boost the classification network’s performance, often achieving superior results to mixup. However, the performance curves show that classification performance does not increase indefinitely; inflection points appear at different multiples for different datasets, indicating that an excessive number of generated images can have adverse effects, such as overfitting.

4.4. Ablation Studies

Effectiveness of Noise Filtering: To validate our noise filtering mechanism, we compare the classification performance of CoDC with exponential filtering (our default method), a linear filtering baseline, and no filtering. As shown in

Table 9. Ablation study on noise filtering mechanism (accuracy %).

Filter Type	CIFAR-100	FaceScrub	Flowers
No Filter	61.14	82.10	49.30
Linear Filter	73.61	92.47	63.31
Exponential (Ours)	74.87	92.95	64.66

and Figure 8 and Figure 9, without any filtering, the noise severely pollutes the features, leading to poor accuracy (e.g., 61.14% on CIFAR-100). Linear filtering improves the results, but our proposed exponential filtering, which more accurately models the noise schedule, yields the best performance. This behavior is consistent with the diffusion formulation in Section 3.2: the useful semantic signal retained in $x_t$ is controlled by ${\overline{\alpha }}_t$, which decays multiplicatively rather than linearly as the timestep increases. Without filtering, all timesteps contribute equally to $L_{classify}$, so heavily corrupted samples produce classification gradients that are weakly related to object semantics and can force the classifier branch to learn noise-dominated representations. Linear filtering reduces this problem, but its slow decay still assigns relatively large weights to medium and high noise timesteps where class evidence has already been substantially degraded. By contrast, the exponential rule follows the rapid decay pattern of ${\overline{\alpha }}_t$ more closely: it preserves strong supervision from low-noise samples, sharply suppresses semantically ambiguous high-noise samples, and therefore better matches the signal-to-noise ratio seen by the shared UNet encoder. The empirical gaps support this interpretation. Compared with no filtering, exponential filtering improves accuracy by 13.73, 10.85, and 15.36 percentage points on CIFAR-100, FaceScrub, and Flowers, respectively. Compared with linear filtering, it further improves accuracy by 1.26, 0.48, and 1.35 percentage points, showing that the gain is not only due to down-weighting noise in general but also to matching the nonlinear noise accumulation process.

Ablation on Model Depth and Parameter Freezing: We analyze two key design choices: the depth of the UNet encoder and our two-step parameter freezing strategy.

Table 10. Ablation study on UNet encoder depth (accuracy %).

Blocks per Stage	CIFAR-100	FaceScrub	Flowers
2	74.60	94.01	63.57
3 (default)	75.15	94.20	65.25
4	76.52	94.21	65.74

shows that increasing the number of residual blocks in the UNet encoder from 2 to 3 improves classification performance across datasets, justifying our architectural enhancement. A fourth residual block gives additional gains on CIFAR-100 and Flowers and a marginal gain on FaceScrub, but the default model uses three residual blocks per stage to balance accuracy and model complexity.

Table 11. Ablation study on parameter freezing (accuracy %).

Strategy	CIFAR-100	FaceScrub
No Freezing	65.50	80.16
With Freezing (Ours)	66.28	80.25

demonstrates the effectiveness of parameter freezing. In the collaborative training setting, freezing the deep encoder layers prevents the degradation of classification ability, improving accuracy by 0.78% on CIFAR-100. This result supports our strategy for mitigating the conflict between the generation and classification tasks.

Effectiveness of Sample Selection: We also analyze the impact of our two-step sample selection strategy.

Table 12. Ablation study on generated sample selection (accuracy %).

Selection Strategy	CIFAR-100	FaceScrub	Flowers
No Selection	63.29	79.56	51.87
Confidence Only	65.66	80.29	54.86
Confidence + Feature	65.97	80.50	55.30

shows the results. Without any filtering, performance is limited by the quality of the generated data. Applying only filtering based on confidence improves accuracy by removing semantically ambiguous samples. The full strategy, combining both confidence and feature consistency filtering, yields the best results, improving accuracy by 2.68%, 0.94%, and 3.43% on CIFAR-100, FaceScrub, and Flowers respectively, compared to no selection. This result supports the effectiveness of filtering out deceptive samples with high confidence.

5. Discussion

The results show that CoDC is most appropriate for rehearsal-free CIL settings where storing real old class images is undesirable because of privacy, storage, or deployment constraints. Under the strict rehearsal-free comparison, the method improves over zero-memory baselines without pre-training by replacing real exemplar storage with generated replay and by filtering generated samples before they are reused for incremental learning.

The comparison with methods that use memory should be interpreted carefully. In Table 3 and Table 4, several baselines use a 4% real sample memory buffer, whereas CoDC stores no real old class samples. These tables therefore provide a mixed memory reference comparison rather than a strictly identical protocol. The strict rehearsal-free comparison is reported in Table 5 and Table 6, where all methods use zero memory. Recent large pre-trained model methods in Table 7 are also separated into Group B because they rely on external pre-training and substantially larger backbone capacity.

There are still limitations. First, CoDC depends on the quality and class consistency of generated replay samples, so classes with subtle visual differences may require stronger filtering or better generative modeling. Second, diffusion sampling is more expensive than storing a small memory buffer, even though it avoids retaining private historical images. Third, the reported comparison with recent published baselines is protocol aware but not fully unified across all task splits and backbones. Future work will focus on accelerating replay sampling, improving sample selection, and rerunning more recent baselines under exactly matched class order, initialization, and backbone settings.

6. Conclusions

In this paper, we introduced the Co-Diffusion Classifier (CoDC), a framework for class-incremental learning that unifies image generation and classification within a single diffusion model. The collaborative network does not require an additional classifier, reducing model redundancy and training overhead. By using a dual task architecture with exponential noise filtering and a base task classification pre-training stage followed by collaborative training with parameter freezing, CoDC performs both tasks while mitigating task conflict and noise contamination. We also investigated the use of generated images for data augmentation and observed that an optimal quantity exists because excessive synthetic data can reduce accuracy. In incremental learning, our rehearsal-free generative replay strategy, enhanced by a two-step sample selection process, mitigates catastrophic forgetting without storing any real data from past tasks. Under the strict zero memory setting, CoDC outperforms the compared baselines without pre-training by a large margin. For example, in Scenario 1 on CIFAR-100, CoDC improves accuracy from 35.20% for LwF to 65.97%, corresponding to a 30.77 percentage point gain, and reduces forgetting from 46.56% to 10.11%. In the mixed-protocol reference comparison, CoDC is also 2.05 percentage points higher than the strongest conventional memory-based baseline on CIFAR-100 despite storing no real old class images. In Scenario 2, CoDC reaches 65.50% accuracy on CIFAR-100, compared with 24.12% for LwF, yielding a 41.38 percentage point improvement under the zero memory protocol. In the corresponding mixed-protocol reference table, CoDC is 2.04 percentage points higher than the strongest memory-based baseline on CIFAR-100. These results indicate that filtered generated replay can be a practical alternative to real exemplar storage when privacy or storage constraints prevent rehearsal.