Countering Model Collapse in Iterative Self-Training via Dynamic Center-Edge Sampling

Abstract

Iterative self-training of language models presents a promising avenue for realizing self-improving Artificial Intelligence systems; however, this process is often hindered by the fundamental challenge of “Model Collapse.” Existing research indicates that models undergo catastrophic performance degradation and diversity collapse when recursively trained on their own increasingly homogenized synthetic data. Although some data selection-based approaches attempt to mitigate this issue by enhancing diversity, they predominantly rely on static strategies, lacking a feedback mechanism capable of adapting in real-time to the dynamic evolution of the model state and data distribution. To address this limitation, we propose a dynamic data selection framework titled “DCES” (dynamic center-edge sampling). We conducted extensive experiments on iterative self-training tasks across multiple model architectures. The results demonstrate that our system significantly outperforms baselines in terms of Perplexity (PPL) and loss across various models and test sets. Simultaneously, the framework effectively mitigates the degradation of Expected Calibration Error (ECE) and entropy metrics, successfully preventing mode collapse. Our findings highlight that an adaptive system capable of intelligent data curation based on training feedback is pivotal for maintaining the dynamic balance of data distributions and achieving sustainable AI self-evolution. This work provides a systematic methodology for realizing this goal.

1. Introduction

The unprecedented capabilities of Large Language Models (LLMs) have been fundamentally driven by the scaling laws, which mandate a continuous intake of massive, high-quality human corpora to minimize loss. Yet, this trajectory is approaching a formidable ceiling: the exhaustion of public human text data is projected to occur within the near future [1]. Consequently, the paradigm of training models on synthetic data—content generated by the models themselves—has emerged as a critical strategy to bypass this bottleneck. By leveraging iterative self-training, the AI community envisions a future where models can continuously self-improve, transitioning from mere imitation of human patterns to autonomous knowledge discovery.

However, establishing such a self-sustaining evolutionary loop proves to be theoretically precarious. Unlike the diverse, noisy, and complex nature of human-generated text, synthetic data inherently exhibits lower variance and tends to gravitate towards high-probability regions of the latent space. When a model is recursively trained on this increasingly homogenized output, a degenerative feedback loop is established. In this closed cycle, the model systematically discards rare, “long-tail” knowledge while amplifying its own biases and hallucinations. This process results in an irreversible detachment from the true data distribution and a catastrophic drop in generation quality, a phenomenon now widely recognized as “Model Collapse” [2].

To address this limitation and establish a sustainable self-evolution loop, we propose a dynamic data selection framework titled “DCES” (dynamic center-edge sampling). At its core, the framework employs a novel clustering-based dynamic center-edge sampling strategy. By monitoring the intra-cluster dispersion of each semantic cluster, it adaptively adjusts the sampling ratios between the semantic center and edge samples. Furthermore, it incorporates token-level editing for center samples to inject diversity and prevent unbounded error accumulation. To ensure sample quality, the framework further integrates an edge sample quality filter based on a frozen anchor model, alongside an adaptive cluster re-weighting mechanism designed to maintain global distributional balance.

In this work, we narrow our scope to the data-centric question of what synthetic samples to train on at each iteration. Accordingly, DCES is positioned as a data selection strategy that operates on the generated pool via selection and lightweight post-processing, while keeping the underlying model architecture and fine-tuning objective unchanged.

Unlike static strategies, DCES employs a “model-aware” feedback mechanism to actively maintain the distributional health of training data. It reduces reliance on external supervision by avoiding continual injection of new human data and does not require a separate verifier model; instead, it uses the frozen initial model as an anchor and a fixed encoder for clustering. In our experiments, DCES improves robustness to model collapse, though some performance trade-offs remain.

While several individual components in DCES (e.g., K-means, EMA smoothing, probability-based filtering) are based on standard techniques, our core novelty lies in their system-level coupling into a model-aware, dynamic closed-loop data selection mechanism: training-time signals (intra-cluster dispersion) are fed back to continuously rebalance data selection, enabling self-correction under a drifting synthetic data distribution.

The main contributions of this paper are summarized as follows:

• We propose DCES, a model-aware closed-loop data selection framework that uses intra-cluster dispersion as a real-time probe to dynamically rebalance both inter-cluster sampling weights and intra-cluster center-edge ratios during iterative self-training.
• We introduce a coupled center/edge processing mechanism that combines diversity injection (token-level editing for center samples) with noise control (anchor-based quality filtering for edge samples), so that the feedback-driven sampling remains effective under noisy synthetic generation.
• Extensive evaluations across multiple model architectures show that DCES improves robustness to collapse compared to naive self-training baselines (e.g., slower PPL/loss degradation and mitigated entropy collapse), while revealing calibration and semantic-fidelity trade-offs in some settings.

2. Related Work

2.1. Iterative Self-Training and Reasoning

Iterative self-training has emerged as a dominant paradigm for scaling synthetic data. Pioneering works like Self-Instruct [3] demonstrated that LLMs could improve their instruction-following capabilities by generating their own training data from a small set of seed tasks. Building on this, STaR [4] introduced a “bootstrapping” mechanism for reasoning tasks, where the model generates rationales and filters them based on final answer correctness. These methods rely on the premise that the model’s high-confidence outputs can serve as a valid training signal. However, these approaches typically assume a static or improving data distribution and often overlook the cumulative distribution shift that occurs over multiple iterations of recursive training.

2.2. The Mechanics of Model Collapse

Recent research has formally identified the causes of performance degradation in recursive self-training. Shumailov et al. [2] analyzed this degradation, showing that training on generated data causes the model’s output distribution to approximate a Dirac delta function, resulting in a significant reduction in variance. Dohmatob et al. [5] provided a theoretical analysis indicating that scaling laws fail under synthetic data because models tend to forget low-probability events (the “tails” of the distribution) while reinforcing high-probability modes. This phenomenon leads to the homogenization of generated content, a problem further categorized by Holtzman et al. [6] as neural text degeneration. Additionally, Zhu et al. [7] established theoretical error bounds for this process, demonstrating that without intervention to bound sampling error, the divergence between the synthetic distribution and the true distribution increases indefinitely.

2.3. Data Selection and Mitigation Strategies

Existing methods to prevent this degradation generally fall into three categories:

Integration of Human Data: Gerstgrasser et al. [8] empirically showed that maintaining a fixed proportion of original human data during training can delay performance degradation. However, Villalobos et al. [1] project that the supply of high-quality public human text data will be exhausted by 2026, limiting the long-term sustainability of methods that rely on continuous fresh human data injection.

External Verification: Approaches such as Self-Critiquing [9] utilize external reward models or “verifiers” to evaluate and filter generated data. While effective, this approach introduces a dependency on a superior supervisor model, which increases computational costs and is limited by the capability of the verifier model itself.

Static Data Filtering: Methods like AlpaGasus [10] and SemDeDup [11] apply static metrics, such as perplexity or semantic embedding similarity, to filter or deduplicate datasets prior to training. While SemDeDup effectively reduces redundancy, these methods lack “model-awareness”—they do not adapt to the changing uncertainty of the model during the training process. Unlike our proposed DCES, which dynamically adjusts sampling weights based on real-time intra-cluster dispersion, static methods cannot detect or correct the specific distributional shifts occurring in the latent space during iterations.

3. Methodology

3.1. The DCES Framework

To counteract the degradation effects observed in the baseline method, we introduce the DCES framework, a closed-loop, adaptive data curation system. The complete workflow of the framework is illustrated in Figure 1; its core components include semantic clustering, adaptive weight adjustment, dynamic center-edge sampling, and differentiated sample processing. To formalize the aforementioned process, we summarize the complete iterative self-training procedure of DCES in Algorithm 1. This algorithm outlines the closed-loop cycle starting from an initial model and proceeding through multiple rounds of data generation, clustering analysis, dynamic sampling, and fine-tuning.

Figure 1. DCES framework.

Figure 1. DCES framework.

Algorithm 1 Dynamic Center-Edge Sampling (DCES)

Require: $\tau$: Center sampling ratio vector; $\alpha$: EMA smoothing factor; $DispH$: Dispersion history; B: Iteration budget; $\lambda$: Edge filtering threshold; $p_h, p_l$: Thresholds for TokenEdit   1:  ize: $M_{cur}\leftarrow M_0$, ${\left\{W_{k,0}\right\}}_{k=1}^K$, $\tau \leftarrow {\left\{0.35\right\}}^K$, $DispH\leftarrow \varnothing$   2:  for $n=1$ to $N_{iter}$ do   3:        $prompts$ ← SamplePrompts($D_{human}$)   4:        $D_{pool}$ ← GenerateData($M_{cur}, prompts$)   5:        Cluster $D_{pool}$ into ${\{C_k^{\left(n\right)}, Ce{n}_k^{\left(n\right)}\}}_{k=1}^K$   6:        for $k=1$ to K do   7:              $d_{mean}\leftarrow {\displaystyle \frac{1}{|C_k^{\left(n\right)}|}}{\sum }_{s\in C_k^{\left(n\right)}}\left(1-cos(\mathrm{emb}\left(s\right),Ce{n}_k^{\left(n\right)})\right)$   8:              $Dis{p}^{\left(n\right)}\left[k\right]\leftarrow d_{mean}$   9:        end for 10:        Append $Dis{p}^{\left(n\right)}$ to $DispH$ 11:        $\left\{W_{k,n}\right\}$ ← UpdateWeights($\left\{\right|C_k^{\left(n\right)}\left|\right\},\left\{W_{k,n-1}\right\},\alpha$)          ▹ See Algorithm  2 12:        $\tau$ ← UpdateRatio($DispH,\tau ,\alpha$)                 ▹ See Algorithm 3 13:        $D_{select}\leftarrow \varnothing$ 14:        for $k=1$ to K do 15:              $N_{tot}\leftarrow \lfloor W_{k,n}·B\rfloor$ 16:              $N_{cen}\leftarrow \lfloor N_{tot}·\tau \left[k\right]\rfloor$ 17:              $N_{edge}\leftarrow N_{tot}-N_{cen}$ 18:              Partition $C_k^{\left(n\right)}$ into pools using threshold $Dis{p}^{\left(n\right)}\left[k\right]$: 19:                   $D_{pool\_c}=\{s\mid (1-cos(s,Ce{n}_k^{\left(n\right)}))\le Dis{p}^{\left(n\right)}\left[k\right]\}$ 20:                   $D_{pool\_e}=\{s\mid (1-cos(s,Ce{n}_k^{\left(n\right)}))>Dis{p}^{\left(n\right)}\left[k\right]\}$ 21:              $D_{cen}$ ← RandomSample($D_{pool\_c}, N_{cen}$) 22:              $D_{edge}$ ← RandomSample($D_{pool\_e}, N_{edge}$) 23:              $D_c^{\prime }$ ← TokenEdit($D_{cen}, M_0, p_h, p_l$)             ▹ See Algorithm 4 24:              $D_e^{\prime }$ ← FilterFallback($D_{edge}, D_{pool\_e}, N_{edge}, M_0, D_{pool},\lambda$)       ▹ See Algorithm 5 25:              $D_{select}\leftarrow D_{select}\cup D_c^{\prime }\cup D_e^{\prime }$ 26:        end for 27:        $M_{cur}\leftarrow FineTune(M_{cur}, D_{select})$ 28:  end for 29:  return $M_{cur}$

3.2. Semantic Clustering and Adaptive Cluster Weighting

Our methodology originates from an analysis of the distributional discrepancies between AI-generated synthetic data and original human data. We observe that purely synthetic data exhibits a significantly more concentrated structure, characterized by markedly smaller intra-cluster distances and narrower lexical features compared to human data.

Upon clustering (see Figure 2 and Figure 3), the synthetic data presents a distinctly more ‘compact’ structure. As corroborated by the quantitative metrics in Table 1, the AI-generated corpus exhibits significantly higher Silhouette coefficients and lower Davies–Bouldin Indices compared to human-authored text, revealing a latent space organized into highly discrete, thematically crystallized regions rather than the diffuse, continuous manifold typical of natural language. Samples within these central clusters typically exhibit lower internal variance and lexical homogeneity, reflecting the generative model’s tendency to repetitively sample high-probability, formulaic linguistic expressions. This pronounced structural stratification—specifically the clear delineation between dense semantic cores and sparse peripheries—serves as the empirical foundation for our DCES framework. It allows us to operationalize intra-cluster dispersion as a reliable proxy for distinguishing between ‘safe’ homogeneous patterns and ‘risky’ tail diversity, thereby motivating our design of the dynamic center-edge sampling mechanism.

Figure 2. K-means clustering results on AI-generated synthetic data.

Figure 3. K-means clustering results on human-authored data.

Table 1. Clustering metrics comparison between AI-generated and human-authored data.

Metric	AI-Generated Data	Human-Authored Data
Silhouette (cosine)	0.2642	0.0577
DBI	2.2835	4.6034
CH	2690.5063	777.3969
Max cluster ratio	0.1054	0.1858
Dispersion mean (1-cos, raw)	0.4872	0.7702
Dispersion mean (1-cos, unit)	0.2844	0.5284

To address this, we propose the DCES framework. This framework performs selective dynamic sampling on purely synthetic data, aiming to anchor the iterative training process to the feature distribution of authentic human corpora. At the beginning of each iteration, we first perform semantic characterization on the newly generated synthetic dataset. We utilize a pre-trained all-MiniLM-L6-v2 [12] model to encode each data sample into a vector. Subsequently, we employ the K-means algorithm [13] to perform clustering on these embeddings, partitioning the dataset into K semantically cohesive clusters. The objective of this step is to transform unstructured text data into an analyzable, structured view of the semantic distribution.

We acknowledge that clustering in a pre-trained embedding space (via a sentence encoder and K-means) is a strong engineering assumption; in DCES, we primarily use it as a stable representation across iterations. DCES does not require perfect clustering and only needs a coarse separation between dense cores and sparse edges; we set $K=8$ based on validation PPL, while larger K can increase computation and the risk of small/empty clusters.

To counteract the inherent tendency of traditional random sampling to over-represent common semantic themes while neglecting rare ones, we designed an adaptive cluster sampling weight adjustment mechanism. Its core philosophy lies in enabling the model to prioritize clusters with smaller sample sizes during the early stages of training, thereby preventing the loss of semantic distribution coverage.

The specific implementation of this mechanism is outlined in Algorithm 2. Formally, for each semantic cluster $C_k$ at iteration n, we first calculate weight $raw$, that is inversely proportional to its current cluster size $S_k$. This ensures that under-represented “long-tail” clusters receive higher sampling probabilities. To further stabilize the training process and reduce sensitivity to clustering instability and generation noise, we employ an Exponential Moving Average (EMA) strategy to update the final sampling weights $W_{k,n}$, smoothing the transition based on historical distributions $W_{k,n-1}$. This design choice is also supported by prior findings that adaptive algorithms can dynamically optimize task planning under changing conditions and demands in contemporary computing surroundings [14].

Algorithm 2 UpdateWeights: EMA-based Adaptation

Require: Cluster sizes $\left\{S_k\right\}$, previous weights $W_{old}$, smoothing factor $\alpha$   1:   $w_{raw}\leftarrow {\{1/S_k\}}_{k=1}^K$   2:   $w_{norm}\leftarrow w_{raw}/\sum w_{raw}$   3:   $W_{k,n}\leftarrow \alpha ·w_{norm}+(1-\alpha )·W_{k,n-1}$   4:   return $W_{k,n}$

Since clustering is re-estimated on newly generated pools at each iteration, cluster statistics (e.g., sizes and dispersion) can fluctuate due to stochastic generation and the local optimization behavior of K-means. EMA therefore functions as a low-pass filter, attenuating transient fluctuations so that the feedback signal is more likely to reflect persistent semantic drift rather than random variation. Overall, these design choices help make DCES less dependent on perfect or fully stable clustering, as long as the embedding space provides a reasonable separation between dense cores and sparse edges.

3.3. Dynamic Center-Edge Sampling

The dynamic adjustment logic for the sampling boundary is detailed in Algorithm 3. Instead of using a fixed threshold to separate center and edge samples, we employ a dispersion-guided feedback loop. At each iteration, we calculate the relative rate of change $\delta$, in the intra-cluster dispersion for each semantic cluster (Line 6). This metric serves as a real-time indicator of the cluster’s stability:

Algorithm 3 UpdateRatio: Adaptive Dispersion-Guided Adjustment

Require: $DispH$, $\tau$, smoothing factor $\alpha$   1:   Initialize: step bounds $[{\Delta }_{min}, {\Delta }_{max}]$, scale factor S, bounds $b_{min}, b_{max}$   2:   if $\left|DispH\right|<2$ then return $\tau$   3:   end if   4:   $D_{prev},D_{cur}\leftarrow$ last two entries of $DispH$   5:   for $k=1$ to K do   6:         $\delta \leftarrow (D_{cur}\left[k\right]-D_{prev}\left[k\right])/\left(D_{prev}\left[k\right]\right)$   7:         $step\leftarrow \mathrm{Clip}\left(\right|\delta |·S,{\Delta }_{min},{\Delta }_{max})$   8:         if $\delta >0$ then   9:               ${\tau }_{target}\leftarrow min(b_{max},\tau \left[k\right]+step)$ 10:         else 11:               ${\tau }_{target}\leftarrow max(b_{min},\tau \left[k\right]-step)$ 12:         end if 13:         $\tau \left[k\right]\leftarrow \alpha ·{\tau }_{target}+(1-\alpha )·\tau \left[k\right]$ 14:   end for 15:   return $\tau$

When dispersion increases ($\delta >0$), this indicates that the generated samples are becoming more divergent or unstable. To counteract this, the algorithm raises the percentile threshold $\tau$ (Line 9). This action effectively increases the sampling proportion of ‘Center Samples,’ prioritizing high-confidence, representative data to re-stabilize the semantic core.

When dispersion decreases ($\delta \le 0$), this suggests the cluster is becoming overly concentrated, potentially signaling the onset of mode collapse. In response, the algorithm lowers $\tau$ (Line 11), thereby increasing the proportion of “Edge Samples.” This forces the model to learn from lower-probability tail data, injecting necessary variance back into the training process.

To ensure smooth transitions and prevent the system from overreacting to transient noise, we calculate a dynamic step size based on the magnitude of change (clipped within $[{\Delta }_{min}, {\Delta }_{max}]$) and apply an EMA update to the final threshold (Line 13).

3.4. Differentiated Sample Processing

3.4.1. Center Samples: Token-Level Editing

Our approach to processing center samples is grounded in recent theoretical insights regarding the mechanics of model collapse. As highlighted in recent studies on synthetic data synthesis [7], iterative training on self-generated data leads to a phenomenon of “feature over-concentration.” Specifically, while center samples exhibit high semantic consistency, they often suffer from coverage collapse: the model increasingly exploits a narrow set of high-probability patterns and under-explores medium-probability alternatives. In practice, this manifests as a polarized (“U-shaped”) token probability distribution, where token probabilities concentrate near the extremes ($p\approx 1$ or $p\approx 0$), with relatively fewer tokens occupying intermediate probability values that reflect the nuanced lexical choices in human-authored text. Such polarization drives lexical homogenization and reduced semantic coverage, creating a self-reinforcing loop where the model becomes increasingly confident in a narrowing set of formulaic expressions.

The specific implementation of this token-level intervention is detailed in Algorithm 4. We intentionally use the frozen initial model $M_0$ as the anchor for token probability estimation. Since $M_0$ is typically pre-trained on large-scale human corpora, it serves as a practical proxy for that original distribution. Using the current model $M_{cur}$ as the reference would instead evaluate samples in an already-shifted coordinate system, potentially reinforcing accumulated biases (a “blind leading the blind” failure mode). To execute this, we utilize the anchor model $M_0$ to calculate the conditional probability of each token within the selected center samples. Based on this probability, we apply a dual-threshold filtering mechanism to rectify the data distribution:

Algorithm 4 TokenEdit (Center Sample Editing)

Require: Selected Samples $D_{in}$, Anchor Model $M_0$, thresholds $p_h,p_l$   1:   $D^{\prime }\leftarrow \varnothing$   2:   for each sample $s\in D_{in}$ do   3:         for each token $t_i$ in s do   4:               $p\leftarrow P(t_i\mid t_{<i},M_0)$   5:               if $p>p_h$ then   6:                     Replace $t_i$ with sampled token   7:               else if $p<p_l$ then   8:                     Remove $t_i$   9:               end if 10:         end for 11:         Add modified s to $D^{\prime }$ 12:   end for 13:   return $D^{\prime }$

Handling “Homogenization” ($p>p_h$): Tokens with excessively high probability typically represent the model’s tendency to overfit to high-frequency patterns or repeat rigid, formulaic phrases. In the polarized “U-shaped” distribution, these tokens correspond to the right-side peak ($p\approx 1$). We therefore set $p_h$ close to 1 (e.g., $p_h=0.98$) so that only extremely predictable positions are edited. As shown in Line 6, we identify such tokens and replace them by resampling from the conditional distribution. Although the resampling is drawn from the same model distribution, applying it only at highly peaked positions injects controlled stochasticity and increases the chance of selecting medium-probability alternatives, thereby flattening the right peak and reducing exact lexical repetition in center samples.

Handling “Hallucination” ($p<p_l$): Conversely, tokens with extremely low probability often indicate logical discontinuities, noise, or model hallucinations; under the “U-shaped” view, they lie in the far-left tail ($p\approx 0$). As shown in Line 8, these tokens are excised to trim the leftmost noise mass. To reduce the risk of removing informative “long-tail” knowledge or damaging grammatical structure/long-range dependencies, we set $p_l$ conservatively (e.g., $p_l=0.001$) so that deletions are sparse and localized. Moreover, TokenEdit is applied only to center samples, while long-tail semantics are primarily preserved via edge sampling (Algorithm 5).

By selectively reconstructing the “head” (high-confidence) and trimming the “tail” (low-confidence noise) of the token distribution, Algorithm 4 acts as a form of Manifold Regularization, ensuring the model learns from cleaner, more diverse syntactic structures.

Algorithm 5 FilterFallback (Edge Sample Control)

Require: $D_{edge}, D_{pool\_e}$, budget $N_{edge}$, anchor $M_0$, global pool $D_{pool}$, $\lambda$   1:   $D^{\prime }\leftarrow \varnothing$   2:   for each $s\in D_{edge}$ do   3:         if $\mathrm{LogProb}(s\mid M_0)<\lambda$ then   4:               remove s   5:         end if   6:   end for   7:   $D^{\prime }\leftarrow D_{edge}$   8:   while $|D^{\prime }|<N_{edge}$ do   9:         $D^{\prime }\leftarrow D^{\prime }$ ∪ RandomSample($D_{pool\_e}$) 10:   end while 11:   while $|D^{\prime }|<N_{edge}$ do 12:         $D^{\prime }\leftarrow D^{\prime }$ ∪ RandomSample($D_{pool}$) 13:   end while 14:   return $D^{\prime }$

3.4.2. Edge Samples: Quality Filtering and Refill

Unlike center samples, which undergo token-level restructuring, edge samples are handled with a “preservation-first” philosophy to protect their unique, long-tail semantic features. The detailed procedure is outlined in Algorithm 5.

Since edge samples reside on the periphery of the distribution, they inherently carry a higher risk of being degenerated noise or hallucinations rather than valid diverse data. To distinguish between “valuable rarity” and “meaningless noise,” we introduce an anchor-based quality filter (lines 2–6). After a substantial distribution shift, $M_{cur}$ can become over-confident on its own artifacts and less reliable for detecting implausible generations. A fixed anchor provides a more stable prior for filtering samples that are highly unlikely under the original distribution, which helps limit self-reinforcing error accumulation in recursive training [2]. We utilize the frozen initial model $M_0$ as a reference anchor to compute the log-probability of each edge sample. Samples falling below a strict threshold $\lambda$ are deemed low-quality noise and are discarded.

Crucially, simply removing samples would disrupt the cluster re-weighting balance established in Algorithm 2. Therefore, we implement a Hierarchical Refill Mechanism to maintain the target data budget $N_{edge}$:

Priority 1: Intra-cluster Refill (lines 8–10): We first attempt to replenish the discarded quota by sampling from the specific edge pool $D_{pool\_e}$ of the same semantic cluster. This preserves the semantic integrity of the cluster.

Priority 2: Global Fallback (lines 11–13): If the specific cluster pool is exhausted, the algorithm falls back to the global pool $D_{pool}$ to ensure the training volume is met.

This mechanism ensures that the model continues to learn from diverse, high-quality tail data without being polluted by generative artifacts.

4. Results

4.1. Implementation Details

All experiments were implemented using the PyTorch (version 2.5.1) framework [15] and the Hugging Face Transformers library (version 4.57.1) [16]. We utilized the Alpaca dataset [17] as the source of seed prompts for synthetic data generation. Specifically, prompts were sampled by directly extracting the original prompt text (instruction and optional input) from Alpaca instances, without using the reference outputs. Unless otherwise specified, we fixed the generation hyperparameters across all iterations (top-p = 0.95, temperature = 1.0) and used an identical random seed for all compared methods to control stochasticity and ensure reproducibility. Model fine-tuning was performed for 1 epoch per iteration round, with a learning rate of 5 × 10⁻⁵ and a batch size of 8. A total of 10 rounds of iterative self-training were conducted.

For evaluation, we assessed in-distribution performance by measuring Expected Calibration Error (ECE), BERTScore, and entropy on the Alpaca test set. Additionally, to evaluate the model’s generalization capability on out-of-distribution text, we computed Perplexity (PPL) on the WikiText-2 dataset [18].

Regarding specific hyperparameters: in the clustering phase, we set the number of clusters to $K=8$. The initial center sampling rate was initialized at 35%. For token-level editing, the confidence thresholds were configured as $p_{high}=0.98$ and $p_{low}=0.001$ [7], while the log-probability threshold for edge sample filtering was set to −1.5. Unless otherwise specified, all computational experiments were executed on a single NVIDIA RTX 4090 GPU; the OPT-6.7B experiment was trained on 2× NVIDIA A800 GPUs.

4.2. PPL and Loss on Test-Set

To validate the effectiveness of the DCES framework in mitigating model collapse, we conducted a comprehensive comparison between the DCES model (after 10 rounds of iterative self-training) and a baseline model employing a naive self-training strategy. Additionally, we included the initial model (Round 0)—prior to any iterative updates—to serve as a reference for the performance upper bound.

To examine the generality and robustness of the DCES framework across varying model architectures and parameter scales, we extended our comparative evaluation to include OPT-125M [19], GPT-2 124M [20], Qwen3-0.6B [21], as well as a substantially larger OPT-6.7B model [19]. Figure 4, Figure 5, Figure 6 and Figure 7 illustrate the Perplexity (PPL) trajectories on the test sets for these models; the corresponding loss curves follow the same trend. Across all evaluated settings, the baseline methods exhibit a clear degradation trend, with PPL/loss increasing markedly as iterations progress—a behavior consistent with the onset of model collapse. In contrast, DCES slows this degradation and yields more stable trajectories. This relative advantage is also observed when scaling model size (e.g., from OPT-125M to OPT-6.7B; see Figure 7), indicating that the effect is not limited to a single parameter regime.

Figure 4. OPT-125M PPL on test sets.

Figure 5. GPT2-124M PPL on test sets.

Figure 6. Qwen3-0.6B PPL on test sets.

Figure 7. OPT-6.7B PPL on test sets.

Specifically, on the OPT-125M model, DCES constrained the total PPL increase to less than 60% of the baseline’s growth. This advantage is also apparent in the GPT-2 and Qwen3 experiments. For GPT-2, the final PPL growth magnitude under DCES was 36.9% of that observed in the baseline, while the Qwen3-0.6B results show a similar suppression of divergence. Overall, these results suggest that combining token-level perturbation for ‘Center Samples’ with quality filtering for ‘Edge Samples’ can reduce overfitting to homogenized output patterns and delay collapse. Across the evaluated architectures, DCES helps maintain a better balance between coherence and diversity during iterative self-training, although it does not fully eliminate degradation relative to the initial model.

4.2.1. Detailed Metric Analysis (Case Study: OPT-125M)

We utilize the OPT-125M model as a primary case study to dissect the specific contributions of DCES across four key dimensions, as detailed in Table 2. To contextualize the effectiveness of DCES, we also compare against common data-filtering strategies:

(1)

Random: uniformly samples instances from the generated pool at each iteration.

(2)

PPL filtering: computes a per-sample perplexity score and prioritizes lower-PPL (higher-likelihood) samples under a fixed training budget.

(3)

SemDeDup: filters the generated pool by semantic deduplication, removing near-duplicate samples based on embedding similarity to reduce redundancy [11].

Table 2. Comparison of model performance on OPT-125M after 10 rounds of iterative self-training.

TEST	Initial Model	DCES	Random	PPL Filtering	SemDeDup
PPL (alpaca)	10.4749	13.6425	14.7834	19.8921	13.4902
PPL (wiki-text)	23.5618	29.7028	33.141	32.9096	30.4964
ECE	0.0379	0.0447	0.0557	0.1965	0.0531
BERTScore	50.61%	46.98%	45.74%	54.22%	59.41%
Entropy	3.238	2.686	2.6151	1.9957	2.4244

Table 2. Comparison of model performance on OPT-125M after 10 rounds of iterative self-training.

TEST	Initial Model	DCES	Random	PPL Filtering	SemDeDup
PPL (alpaca)	10.4749	13.6425	14.7834	19.8921	13.4902
PPL (wiki-text)	23.5618	29.7028	33.141	32.9096	30.4964
ECE	0.0379	0.0447	0.0557	0.1965	0.0531
BERTScore	50.61%	46.98%	45.74%	54.22%	59.41%
Entropy	3.238	2.686	2.6151	1.9957	2.4244

Mitigation of Perplexity Degradation: Perplexity serves as a fundamental metric for assessing the goodness of fit of a language model’s probability distribution. Table 2 shows that naive random sampling (random) leads to pronounced degradation after 10 rounds. On the out-of-distribution WikiText-2 dataset [18], PPL increases from 23.56 (initial) to 33.14 (random). DCES reduces this degradation (29.70), and SemDeDup also improves over random (30.50) while yielding slightly better in-distribution PPL on Alpaca (13.49) than DCES (13.64). In contrast, PPL filtering performs poorly in-domain (19.89) and provides limited gains out-of-domain (32.91), suggesting that likelihood-only selection does not reliably mitigate distribution shift under recursive training.

Improvement in Model Calibration: A hallmark pathological characteristic of model collapse is ‘blind confidence,’ wherein the model assigns excessive confidence to incorrect predictions. We employ the Expected Calibration Error (ECE) [22] to quantify this phenomenon. In Table 2, random increases ECE from 0.0379 (initial) to 0.0557. DCES achieves the lowest ECE among the compared selection methods (0.0447), while SemDeDup yields a modest improvement over random (0.0531). Notably, PPL filtering results in a large ECE increase (0.1965), indicating that selecting only low-PPL samples can produce severe miscalibration in this setting.

Preservation of Semantic Quality: BERTScore [23] leverages contextual embeddings to quantify the semantic similarity between generated text and reference text. In Table 2, DCES yields a slightly higher BERTScore than random (46.98% vs. 45.74%), but remains below the initial model (50.61%). Interestingly, the static filtering methods achieve higher BERTScore (54.22% for PPL filtering and 59.41% for SemDeDup). However, these gains coincide with lower entropy (especially for PPL filtering) and, for PPL filtering, substantially worse calibration, suggesting that higher semantic similarity alone does not guarantee healthier distributional diversity or calibration under iterative self-training.

Maintenance of Distributional Diversity: Average entropy serves as a proxy for the richness of the model’s output probability distribution. Table 2 shows that random reduces entropy from 3.238 (initial) to 2.6151, signaling the onset of distributional concentration [6]. DCES retains higher entropy (2.6860), while SemDeDup reduces entropy to 2.4244, and PPL filtering leads to the lowest entropy (1.9957). This comparison suggests that static filtering—particularly likelihood-based selection—can exacerbate concentration, whereas DCES better maintains distributional breadth by explicitly rebalancing center-edge sampling.

4.2.2. Cross-Architecture Generalization

To confirm that the improvements observed in OPT-125M are not anecdotal, we further evaluate DCES on GPT-2 (Table 3), Qwen3-0.6B (Table 4) and OPT-6.7B (Table 5). The results demonstrate that the efficacy of our framework is consistent across different model sizes and families.

Table 3. Comparison of model performance on GPT2 after 10 rounds.

TEST	Initial Model	DCES	Random	PPL Filtering	SemDeDup
PPL (alpaca)	11.3806	11.0158	11.6865	11.6927	10.6612
PPL (wiki-text)	24.8463	27.1640	29.4228	28.0517	27.4203
ECE	0.0865	0.0454	0.0635	0.0404	0.0321
BERTScore	50.52%	51.28%	50.75%	55.85%	61.07%
Entropy	9.9412	9.7593	9.9434	9.3270	9.6170

Table 4. Comparison of model performance on Qwen after 10 rounds.

TEST	Initial Model	DCES	Random	PPL Filtering	SemDeDup
PPL (alpaca)	6.3140	8.1571	9.2625	20.7562	8.2395
PPL (wiki-text)	17.7195	26.8662	28.6802	38.2676	22.7640
ECE	0.0476	0.1058	0.1200	0.2481	0.0959
BERTScore	50.69%	46.94%	46.18%	51.32%	61.59%
Entropy	2.0002	1.7904	1.7154	1.1923	1.5354

Table 5. Comparison of model performance on OPT-6.7B after 10 rounds.

TEST	Initial Model	DCES	Random
PPL (alpaca)	4.6713	7.6090	8.3730
PPL (wiki-text)	9.3872	17.9020	18.4600
ECE	0.0473	0.0560	0.0680
BERTScore	61.82%	56.41%	55.14%
Entropy	2.0514	1.8770	2.0470

Consistency in PPL Suppression: Across GPT-2 and Qwen, DCES improves over random sampling in terms of PPL, while the static filters exhibit distinct behaviors. On GPT-2 (Table 3), DCES improves in-distribution PPL over random and is comparable to PPL filtering, while SemDeDup achieves the lowest Alpaca PPL (10.66) with out-of-distribution PPL close to DCES. On Qwen (Table 4), SemDeDup yields a substantially lower WikiText PPL (22.76) than both DCES (26.87) and random (28.68), whereas PPL filtering performs poorly (38.27), suggesting that likelihood-only filtering can be unstable under distribution shift for some model families.

Universal Calibration and Diversity Benefits: Calibration and diversity metrics further differentiate the selection strategies. On Qwen (Table 4), DCES improves ECE over random (0.1058 vs. 0.1200) and preserves the highest entropy among the compared synthetic-data methods (1.7904), whereas PPL filtering substantially worsens both ECE (0.2481) and entropy (1.1923). SemDeDup reduces ECE (0.0959) and improves PPL, but at the cost of lower entropy (1.5354) than DCES. On GPT-2 (Table 3), both PPL filtering and SemDeDup achieve lower ECE than DCES, but they also reduce entropy relative to random/initial, indicating a trade-off between confidence-based selection, redundancy reduction, and distributional diversity. We also observe that PPL filtering and SemDeDup often yield higher BERTScore, yet these gains can coincide with reduced entropy, underscoring the need to consider semantic fidelity and diversity jointly.

In summary, Table 2, Table 3, Table 4 and Table 5 suggest that DCES consistently improves over random sampling and provides a strong balance between coherence and diversity. Compared with the static filters, PPL filtering exhibits highly variable behavior across model families (e.g., severe miscalibration and entropy collapse on OPT/Qwen, but lower ECE on GPT-2), while SemDeDup can be competitive in PPL and BERTScore but consistently yields lower entropy than DCES. These comparisons highlight the complementary roles of redundancy reduction and feedback-driven sampling for maintaining distributional health under iterative self-training.

4.3. Qualitative Analysis

To vividly illustrate the divergence in model behaviors after iterative self-training, Table 6 presents a representative case study from the 10th round.

Table 6. Qualitative comparison of generated samples from the baseline and DCES models at Round 10.

Input Prompt	Baseline (Round 10)	DCES (Round 10)
“Name a sport that requires good hand-eye coordination”	Name a sport that requires good hand-eye coordination 1.1.1.1.1.1.1.1.1.1. 1.1.1.1.1.1. 1.1.1.1.1.1. 1 1 [1]1.1 (Model degenerates into non-linguistic numerical repetition)	Name a sport that requires good hand-eye coordination in order to get the ball right. For example, tennis matches are a sport with great hand-eye coordination, but in some cases… Here is a list of what to consider during the day: 1. Soccer in…

Catastrophic Collapse in Baseline: The output from the baseline model exemplifies an extreme form of “distributional collapse.” The model exhibits catastrophic degeneration, failing to produce meaningful linguistic tokens. Instead, it gets trapped in a meaningless numerical loop (repeating “1.1.1…”). This phenomenon occurs because the model’s variance has been reduced to near zero, causing it to blindly repeat a single high-probability token pattern without any semantic grounding. Knowledge Retention in DCES: In stark contrast, the DCES model shows greater resilience in this example. Despite the challenging synthetic training environment, it retrieves relevant entities (e.g., “Tennis”, “Soccer”) and preserves basic syntactic coherence. While the DCES output still exhibits coherence drift typical of synthetic data training, it avoids the severe non-linguistic degeneration observed in the baseline, suggesting that the DCES framework can help protect the model’s linguistic manifold during recursive self-training.

4.4. Ablation Studies Results

To validate the individual contributions of the core components within the DCES framework, we devised the following two sets of ablation experiments.The detailed performance comparison is summarized in Table 7:

• DCES w/o Token Editing: This variant removes the token-level editing module for center samples. The complete dynamic sampling logic is retained; however, the selected center samples are utilized directly for training without undergoing any modification.
• DCES w/o Dynamic Sampling: In this configuration, we retain the token editing capability but exclude the dynamic center-edge sampling mechanism. Samples are randomly drawn from the clusters irrespective of their intra-cluster positions, and token editing is applied to all sampled instances.

Table 7. Ablation studies based on OPT-125M.

Ablation Studies	PPL (Alpaca)	PPL (Wiki-Text)	ECE	BERTScore	Entropy
Baseline	14.7834	33.141	0.0557	45.74%	2.6151
DCES w/o Token Editing	14.2091	31.5922	0.0577	45.63%	2.6434
DCES w/o Dynamic Sampling	13.8794	32.4577	0.0447	46.94%	2.7040
DCES	13.6425	29.7028	0.0447	46.98%	2.6860

Note: Bold values indicate the best-performing results for each metric.

Table 7. Ablation studies based on OPT-125M.

Ablation Studies	PPL (Alpaca)	PPL (Wiki-Text)	ECE	BERTScore	Entropy
Baseline	14.7834	33.141	0.0557	45.74%	2.6151
DCES w/o Token Editing	14.2091	31.5922	0.0577	45.63%	2.6434
DCES w/o Dynamic Sampling	13.8794	32.4577	0.0447	46.94%	2.7040
DCES	13.6425	29.7028	0.0447	46.98%	2.6860

Note: Bold values indicate the best-performing results for each metric.

5. Conclusions and Limitations

Our findings suggest that within a synthetic data-dominated training paradigm, maintaining a dynamic balance of the data distribution is important for mitigating model collapse. By exploiting the non-uniformity of data within the semantic space, DCES constructs an automated ‘bias-correction system’ via dynamic inter-cluster sampling, dynamic intra-cluster center-edge sampling, and differentiated post-processing for center and edge samples. This mechanism can help preserve generalization capabilities and reduce distributional drift over iterations.

Within DCES, our center sample editing strategy (Algorithm 4) can be interpreted as a form of Manifold Regularization [24]. It retains the syntactic skeleton of the data while selectively restructuring local, high-confidence ‘homogenized tokens’, which may help bound the accumulation of sampling errors [7].

Balanced Interpretation and Trade-offs: While DCES improves robustness to collapse relative to naive self-training (e.g., lower PPL and mitigated entropy collapse), it does not fully match the initial model on all metrics. In particular, BERTScore often remains below the starting point, and calibration error (ECE) can still increase relative to the initial model for some settings (e.g., Qwen and OPT). This reflects an inherent trade-off in our design: to prevent mode collapse, we intentionally inject controlled perturbations (token editing and increased edge sampling) to maintain diversity, which can partially sacrifice fine-grained semantic matching and may affect probability calibration. In practice, DCES is most beneficial when iterative self-training exhibits early signs of collapse (rapid PPL growth or entropy collapse), whereas applications requiring maximal semantic fidelity or strict calibration may benefit from more conservative perturbation schedules or additional calibration-aware selection.

Limitations and Future Directions: Despite improving robustness to collapse in our experiments, DCES is subject to notable limitations. (1) Parameter Sensitivity: The framework’s effectiveness is contingent upon hyperparameter configurations. Suboptimal settings for the cluster count K in K-means, the smoothing factor $\alpha$ in EMA, or extreme thresholds for token editing and edge quality filtering could compromise performance. (2) Dependency on Clustering Quality: The efficacy of DCES is also influenced by the intrinsic distribution of the raw data and the resulting clustering quality. Disordered clustering may cause the system to degenerate into a mere combination of token editing and quality filtering, potentially rendering its distribution correction mechanism ineffective. (3) Metric Trade-offs: The diversity-preserving perturbations that improve stability can reduce semantic matching (e.g., BERTScore) and lead to non-monotonic calibration behavior relative to the initial model. Future work could incorporate semantic-preservation constraints in token editing, calibration-aware sampling objectives, or adaptive perturbation schedules that adjust exploration strength based on model uncertainty.