Convergence Guarantees for Time-Inhomogeneous Uniform-Rate Discrete Diffusion Models

Abstract

Discrete diffusion models have become an important class of generative models for categorical data, yet their theoretical understanding remains largely limited to time-homogeneous noise schedules. In this work, we study uniform-rate discrete diffusion models with time-inhomogeneous continuous-time Markov chain forward processes. We establish convergence guarantees for practical reverse-time samplers by directly controlling the total variation distance, avoiding the indirect route of first bounding KL divergence and then applying Pinsker’s inequality. Our analysis decomposes the sampling error into initialization, score-estimation, discretization, and early-stopping errors, and explicitly characterizes how each term depends on the accumulated noise, the local noise rate, and the smoothness of the noise schedule. Under suitable regularity conditions on the noise schedule, we further derive step-complexity guarantees that match the order of existing results for homogeneous samplers.

1. Introduction

Generative modeling aims to produce samples that approximate the training data distribution. Diffusion models [1,2,3] have become a leading approach, with strong performance in image, video, audio, and text generation [4,5,6,7,8]. They generate data by learning to reverse a forward noising process; for discrete data, this process is naturally modeled as a CTMC on a finite state space.

A growing line of work studies the theoretical convergence of discrete diffusion samplers. And a growing body of work suggests that for discrete data such as natural language and graphs, discrete diffusion models offer greater advantages and more flexibility than their continuous counterparts [9,10]. For uniform-rate discrete diffusion models, existing guarantees have primarily been developed for time-homogeneous CTMCs. Early work analyzed $\tau$-leaping under total variation distance [11], while subsequent studies established convergence guarantees through score-entropy control for uniformization, exact-step, and $\tau$-leaping-type samplers [12,13,14,15,16]. More recently, sharper analyses have been obtained for practical samplers such as the Euler method and Tweedie $\tau$-leaping [17,18]. However, these results largely focus on homogeneous noise schedules, leaving open the question of how non-constant noise schedules affect the sampling error.

This limitation is important because time-inhomogeneous schedules are widely used in practice and can substantially affect generative performance. For continuous diffusion models, it is empirically showed in [19] that noise scheduling is a crucial design choice: the optimal schedule can depend on the task, and higher-resolution generation may benefit from schedules that place more mass on noisier regimes. These observations suggest that the noise schedule is not merely a technical detail but rather a central component controlling the trade-off between data corruption, score estimation, and reverse-time discretization. Motivated by this, we develop a convergence analysis for non-homogeneous uniform-rate discrete diffusion models. Our results quantify how the accumulated noise, the local rate ${\beta }_t$, and the smoothness of the schedule jointly determine initialization, discretization, and early-stopping errors.

2. Related Work

Theoretical studies of uniform-rate discrete diffusion models have so far largely considered time-homogeneous CTMCs. We have provided a summary in

Table 1. Summary of results for regular uniform-rate discrete diffusion samplers in terms of the number of steps needed to achieve $\sqrt{\epsilon }$-accuracy in $\mathrm{TV}(q_{\delta },p_{T-\delta })$. Only the best result for each sampler is shown. All log-dependencies are omitted. In the table, d is the dimension, S is the vocabulary size, and M is an upper bound of the score estimates. In this work, we provide the first analysis under non-homogeneous noise schedule and $L_1$ estimation error (i.e., in total-variation metric), having a convergence rate comparable to the state-of-the-art results.

Sampler	Assumption	Time-Homo-Geneous?	Results: Num of Steps	Reference
Kolmogorov	Score entropy, bounded score	Yes	$\mathcal{O}\left({\displaystyle \frac{\sqrt{dM}S}{\sqrt{\epsilon }}}\right)$	[13]
DMPM	Score entropy	Yes	$\mathcal{O}\left({\displaystyle \frac{d{S}^2}{{\epsilon }^2}}\right)$	[14,20]
$\tau$-leaping	$L_{\infty }$ error	Yes	$\mathcal{O}\left({\displaystyle \frac{d^4{S}^4}{\sqrt{\epsilon }}}\right)$	[11]
$\tau$-leaping	Score entropy, bounded score	Yes	$\mathcal{O}\left({\displaystyle \frac{d^2{S}^2}{\epsilon }}\right)$	[16,17]
$\tau$-leaping	Score entropy	Yes	$\mathcal{O}\left({\displaystyle \frac{d}{\epsilon }}\right)$	[15]
Euler method, Tweedie $\tau$-leaping	Score entropy, bounded score	Yes	$\mathcal{O}\left({\displaystyle \frac{d^{2}S}{\epsilon }}\right)$	[17]
Euler method, Tweedie $\tau$-leaping	$L_1$ error, slow-varying noise	No	$\mathcal{O}\left({\displaystyle \frac{d^{2}S}{\sqrt{\epsilon }}}\right)$	Theorem 2

. An early result by [11] analyzed $\tau$-leaping in total variation distance but required strong assumptions on the estimated reverse rates and led to suboptimal parameter dependence. Subsequent works focused on score-entropy-based guarantees for random-step samplers: ref. [12] studied a uniformization sampler on the hypercube, and ref. [16] extended the analysis to general product spaces ${\left[S\right]}^d$.

Other recent works analyze deterministic-step samplers. Some assume an exact per-step solver [13] or a specially designed sampler [14,20], while others study existing $\tau$-leaping-type samplers used in practice [15,16,17]. Beyond $\tau$-leaping, ref. [17] developed a sharp analysis for the Euler method and Tweedie $\tau$-leaping [8], avoiding a Girsanov change-of-measure argument. Besides regular samplers, refs. [21,22] investigated possible accelerations to these vanilla samplers. Most of these results are stated in KL divergence and then converted to total variation distance, which can introduce looseness. Also, these works are focused on time-homogeneous noise schedules, which are almost never used in practice. In contrast, our work directly analyzes total variation error for time-inhomogeneous uniform-rate processes, thereby capturing the effect of non-constant schedules such as the geometric schedule used in empirical works (e.g., [8,23]).

There is a concurrent work in [24] that derives an S-independent upper bound on the convergence error also for the time-inhomogeneous process. Differently, their work assumes exact simulation of the continuous-time sampling process, and the step complexity when there is discretization remains unclear.

A parallel line of work studies masked, or absorbing-rate, discrete diffusion models. Convergence guarantees have been established in [15,18,20,25,26].

Our Contributions

Our main contributions are summarized as follows:

• We establish convergence guarantees for discrete diffusion models with time-inhomogeneous uniform-rate generators. This extends prior analyses, which primarily focus on homogeneous noise schedules.
• We identify regularity conditions on the noise schedule under which explicit convergence rates can be obtained. Under these conditions, the resulting rates match state-of-the-art guarantees for homogeneous discrete diffusion samplers.

3. Preliminaries of Discrete Diffusion Samplers

In this section, we provide the background of the continuous-time discrete-space diffusion sampler.

3.1. Continuous-Time Forward Dynamics on Discrete State Spaces

We consider discrete data represented as a length-d sequence

$$x_0=(x_0^1,\dots ,x_0^d)\in {\left[S\right]}^d,$$

where each coordinate $x_0^i$ takes values in a finite alphabet $\left[S\right]$ of size S. Let $q_0^i$ denote the marginal probability mass function of the i-th token, and let $q_0$ denote the joint distribution of the complete data vector $x_0$.

The forward noising process is modeled as a time-inhomogeneous continuous-time Markov chain (CTMC) on the finite state space ${\left[S\right]}^d$. This process is specified by the initial law $q_0$ and a time-dependent transition-rate matrix $R_t\in {\mathbb{R}}^{S^d\times S^d}$. Under the convention that $R_t(x,y)$ denotes the instantaneous rate of jumping from state x to state y, the infinitesimal transition probability satisfies, for sufficiently small $\Delta t$,

$$q_{t+\Delta t\mid t}(y\mid x)=\mathbb{1}\left\{y=x\right\}+R_t(x,y)\Delta t+o(\Delta t),x,y\in {\left[S\right]}^d.$$

(1)

Equivalently, the marginal law $q_t$ evolves according to the Kolmogorov forward equation

$${\displaystyle \frac{\mathrm{d}}{\mathrm{d}t}}{q}_t\left(y\right)=\sum _{x\in {\left[S\right]}^d}{q}_t\left(x\right)R_t(x,y),y\in {\left[S\right]}^d.$$

For $R_t$ to define a valid CTMC generator, it must satisfy the standard conditions

$$R_t(x,y)\ge 0\mathrm{for}x\ne y,R_t(x,x)\le 0,\sum _{y\in {\left[S\right]}^d}{R}_t(x,y)=0.$$

Directly specifying a generator over the full space ${\left[S\right]}^d$ is generally infeasible when either S or d is large. A common simplification is therefore to impose coordinate-wise independent corruption so that only one token changes at any infinitesimal transition [8,11]. In this case, for $x\ne y$, the generator takes the form [11] (Prop. 3)

$$R_t(x,y)=\left\{\begin{array}{cc}{R}_t^{\mathrm{tok}}(x^i,y^i),\hfill & \mathrm{if}\mathrm{Ham}\left(x,y\right)=1\mathrm{and}{x}^i\ne y^i,\hfill \\ 0,\hfill & \mathrm{if}\mathrm{Ham}\left(x,y\right)>1,\hfill \end{array}\right.$$

(2)

where $R_t^{\mathrm{tok}}\in {\mathbb{R}}^{S\times S}$ is the token-level transition-rate matrix and $\mathrm{Ham}\left(x,y\right)$ denotes the Hamming distance between x and y. The diagonal entries of $R_t$ are then determined by the zero-row-sum constraint.

Following [11], we parameterize the token-level generator as

$$R_t^{\mathrm{tok}}={\beta }_t{R}_{\mathrm{base}},$$

where ${\beta }_t>0$ is a noise schedule and $R_{\mathrm{base}}$ is a fixed base generator. Prior work often studies the homogeneous case ${\beta }_t\equiv 1$; see, for example, refs. [13,16]. We define the accumulated noise level by

$${\overline{\beta }}_t:={\int }_0^t{\beta }_s\mathrm{d}s<\infty .$$

Since ${\beta }_t>0$, the map $t\mapsto {\overline{\beta }}_t$ is increasing. We denote the inverse of ${\overline{\beta }}_t$ as ${\overline{\beta }}^{-1}$, so that

$${\overline{\beta }}^{-1}\left({\overline{\beta }}_t\right)=t.$$

Throughout, we assume that ${\beta }_t$ is smooth on $[0,T]$.

The coordinate-wise construction yields a tractable closed-form transition kernel. In particular, the token-level transition matrix from time 0 to time t is

$$Q_{t\mid 0}^{\mathrm{tok}}=exp\left({\overline{\beta }}_t{R}_{\mathrm{base}}\right),$$

and the full transition kernel factorizes across coordinates as

$$q_{t\mid 0}(x_t\mid x_0)={\displaystyle \prod _{i=1}^d}{Q}_{t\mid 0}^{\mathrm{tok}}(x_0^i,x_t^i).$$

This closed form is especially useful for training discrete diffusion models.

In this work, we focus on the commonly used uniform-rate discrete diffusion model, with

$$R_{\mathrm{base}}:={\displaystyle \frac{1}{S}}{\mathbf{1}}_S{\mathbf{1}}_S^{⊺}-I_S,$$

which appears in several prior works [3,8,11]. For this choice, each token is driven toward the uniform distribution over $\left[S\right]$. Moreover, the diagonal entry of the full-state generator is

$$R_t(x,x)=-\sum _{y:y\ne x}{R}_t(x,y)=-{\beta }_t{\displaystyle \frac{S-1}{S}}d,x\in {\left[S\right]}^d.$$

With such an $R_t$, as T becomes large, the terminal distribution induced by this forward process approaches the uniform distribution on ${\left[S\right]}^d$.

3.2. Reverse Dynamics and Discrete-Time Sampling

The CTMC forward process induces a time-reversed CTMC whose marginals coincide with those of the forward chain run backward in time [27]. More precisely, by [11] (Prop. 1), the reverse process starts from

$${{\displaystyle \overleftarrow{{\displaystyle q}}}}_0:=q_T$$

and evolves with transition-rate matrix ${\overleftarrow{R}}_t$ given, for $x\ne y$, by

$${{\displaystyle \overleftarrow{{\displaystyle R}}}}_t(x,y)=R_{T-t}(y,x){\displaystyle \frac{q_{T-t}\left(y\right)}{q_{T-t}\left(x\right)}},x,y\in {\left[S\right]}^d,$$

(3)

with diagonal entries determined by

$${{\displaystyle \overleftarrow{{\displaystyle R}}}}_t(x,x)=-\sum _{y:y\ne x}{{\displaystyle \overleftarrow{{\displaystyle R}}}}_t(x,y).$$

With this setup, the marginal distribution of this reverse chain satisfies

$${{\displaystyle \overleftarrow{{\displaystyle q}}}}_t=q_{T-t}.$$

Under the coordinate-wise forward generator in (2), the reverse generator inherits the same sparsity pattern: if $\mathrm{Ham}\left(x,y\right)\ge 2$, then

$${{\displaystyle \overleftarrow{{\displaystyle R}}}}_t(x,y)=0.$$

Thus, both the forward and reverse processes only allow infinitesimal transitions that modify a single token. In practice, the reverse process is stopped at time $T-\delta$ for a small $\delta >0$. This early-stopping convention avoids the numerical instability that may arise as the forward time approaches $0^{+}$, where the corresponding score ratios can become irregular (as shown in [17] (Lemma 2)).

The exact reverse CTMC is not directly implementable, and the sampling procedure therefore introduces several approximations. Let $p_t$ denote the marginal distribution of the practical sampler at reverse time $t\in [0,T-\delta ]$. First, the exact initial law $q_T$ is typically unavailable. Since the forward process with the uniformizing base generator converges to the uniform distribution, the sampler is initialized as

$$p_0:=\mathrm{Uniform}\left({\left[S\right]}^d\right).$$

Second, the likelihood ratio appearing in (3) is unknown. For reverse time t, the required ratio ${\displaystyle \frac{q_{T-t}\left(y\right)}{q_{T-t}\left(x\right)}}$ is approximated by a learned concrete score function

$$s_{T-t}(y,x)\approx {\displaystyle \frac{q_{T-t}\left(y\right)}{q_{T-t}\left(x\right)}}.$$

Consequently, on a discrete sampling grid, the reverse generator is replaced by an estimated generator.

Beyond estimation error, we also account for the error introduced by approximate sampling procedures. Let

$$0=t_0<t_1<\cdots <t_N=T-\delta$$

be the reverse-time discretization grid. At grid point $t_k$, the off-diagonal entries of the estimated reverse generator are

$${\widehat{R}}_{t_k}(x,y):=R_{T-t_k}(y,x)s_{T-t_k}(y,x),x\ne y,$$

(4)

with

$${\widehat{R}}_{t_k}(x,x)=-\sum _{y:y\ne x}{\widehat{R}}_{t_k}(x,y).$$

This approximation replaces the exact reverse rate in (3) by a score-based estimate.

Following [8,23], one example is to use the Euler method that freezes the estimated generator over each interval $[t_k,t_{k+1})$. Since the generator only permits single-token transitions, the update can be written in a token-wise form. Given the current state $x_{t_k}$, define, for each coordinate $i\in \left[d\right]$ and candidate token $a\in \left[S\right]$ with $a\ne x_{t_k}^i$,

$${\widehat{R}}_k^i(x_{t_k}^i,a):={\widehat{R}}_{t_k}\left(x_{t_k},x_{t_k}^{-i}{\oplus }_{i}a\right),$$

(5)

where $x_{t_k}^{-i}{\oplus }_{i}a$ denotes the vector obtained from $x_{t_k}$ by replacing its i-th token with a. The corresponding diagonal token rate is

$${\widehat{R}}_k^i(x_{t_k}^i,x_{t_k}^i):=-\sum _{a:a\ne x_{t_k}^i}{\widehat{R}}_k^i(x_{t_k}^i,a).$$

The Euler transition for the i-th token is then

$$\mathbb{P}\left(x_{t_{k+1}}^i=a\mid x_{t_k}\right)=\left\{\begin{array}{cc}(t_{k+1}-t_k){\widehat{R}}_k^i(x_{t_k}^i,a),\hfill & a\ne x_{t_k}^i,\hfill \\ 1+(t_{k+1}-t_k){\widehat{R}}_k^i(x_{t_k}^i,x_{t_k}^i),\hfill & a=x_{t_k}^i.\hfill \end{array}\right.$$

(6)

Equivalently, each token either remains unchanged or jumps to another symbol according to the local estimated reverse rates. The step size is typically small, so that the probabilities in (6) remain nonnegative. The final output of the sampler is the state at reverse time $T-\delta$, whose law is intended to approximate the early-stopped data distribution $q_{\delta }$.

3.3. Notations

Let $x^i(1\le i\le d)$ denote the i-th element of a vector $x\in {\left[S\right]}^d$ and $x^{-i}\in {\left[S\right]}^{d-1}$ denote the i-th element removed. Define $\mathrm{Ham}\left(x,y\right)$ as the Hamming distance between two vectors x and y. For a positive integer n, $\left[n\right]:=\{1,\dots ,n\}$. Write ${\mathbf{1}}_S$ as a vector of length S that contains all 1’s, and $I_S$ as an identity matrix of size $S\times S$.

4. Convergence Under General Non-Homogeneous Noise Schedule

In this section, we establish improved upper error bounds and convergence rates for the uniform-rate matrix. Following the approach of [18], our results are stated directly in terms of total variation distance rather than first deriving a KL-divergence bound and then applying Pinsker’s inequality. Our analysis relies on the following estimation-error condition.

Assumption 1.

The score estimation error satisfies that ${\mathcal{L}}_{TV}\left(s\right)\le \sqrt{{\epsilon }_{\mathrm{TV}}}$, where

$${\mathcal{L}}_{TV}\left(s\right):={\displaystyle \sum _{k=0}^{N-1}}(t_{k+1}-t_k){\mathbb{E}}_{x_k\sim q_{T-t_k}}\left[\sum _{y:y\ne x_k}{R}_{T-t_k}(y,x_k)\left|s_{T-t_k}(y,x_k)-{\displaystyle \frac{q_{T-t_k}\left(y\right)}{q_{T-t_k}\left(x_k\right)}}\right|\right].$$

Assumption 1 requires the learned score to be accurate at $L_1$ distance, measured as the expected sum of absolute differences between the exact and estimated rate matrices. This condition differs from the commonly used score-entropy loss [16,17] but is particularly convenient for our direct total-variation-based analysis. For masked diffusion models, ref. [18] shows that ${\mathcal{L}}_{TV}$ can be upper-bounded by the score-entropy loss. We have provided a similar justification in Appendix A.

With this condition in place, the following theorem provides an upper bound on the sampling error in total variation distance.

Theorem 1.

Suppose that Assumption 1 holds. As long as $t_{k+1}-t_k$ vanishes at small ε and that each Euler step is valid, we have

$$\begin{array}{cc}\mathrm{TV}(q_0,p_{T-\delta })\lesssim min\{d,\hfill & \sqrt{dS}\}{e}^{-{\overline{\beta }}_T}+\sqrt{{\epsilon }_{\mathrm{TV}}}\hfill \\ & \hfill +dS{\displaystyle \sum _{k=0}^{N-1}}max\{1,{\overline{\beta }}_{T-t_{k+1}}^{-2}\}\left(\right|{\beta }_{T-t_k}^{\prime }|+d{\beta }_{T-t_k}^2){(t_{k+1}-t_k)}^2+d{\overline{\beta }}_{\delta }.\end{array}$$

Theorem 1 decomposes the total sampling error into four distinct contributions. The first term, $min\{d,\sqrt{dS}\}{e}^{-{\overline{\beta }}_T}$, corresponds to the initialization error, which quantifies the discrepancy between the true terminal distribution $q_T$ and the uniform distribution used to initialize the sampler. This error decays exponentially fast with the accumulated noise level ${\overline{\beta }}_T$. The second term, $\sqrt{{\epsilon }_{\mathrm{TV}}}$, captures the error due to imperfect score estimation under Assumption 1. The third term represents the discretization error incurred by replacing the continuous-time reverse CTMC with a discrete-time approximation on the grid ${\left\{t_k\right\}}_{k=0}^N$. Finally, the last term, $d{\overline{\beta }}_{\delta }$, is the early-stopping error, which arises because the reverse process is terminated at time $T-\delta$ rather than being run all the way to the data distribution.

Remark 1

(On the time steps $t_{k+1}-t_k$). In order to guarantee the validity of the Euler steps, it is necessary to have that

$$|{\widehat{R}}_k^i(x^i,x^i)|=\sum _{a:a\ne x^i}{\widehat{R}}_k^i(x^i,a)<1.$$

Assuming that the score functions are accurately estimated, from Lemma A2 we have $|{\widehat{R}}_k^i(x^i,x^i)|\lesssim S$, necessarily yielding $t_{k+1}-t_k=o(1/S)$. Note that this condition is satisfied for the step size in Theorem 2.

Remark 2

(On the early-stopping parameter $\delta$). For time-homogeneous models, to achieve $\mathcal{O}\left(\sqrt{\epsilon }\right)$ error, δ can be easily set to be $\sqrt{\epsilon }/d$ (see Corollary 1). For non-homogeneous schedules, however, such a dependence on d becomes implicit, and one should set δ such that ${\overline{\beta }}^{-1}(\sqrt{\epsilon }/d)$, which depends on the choice of the actual schedule.

For comparison, we provide the following corollary for the special case of a time-homogeneous CTMC, which is immediate upon plugging in ${\beta }_t\equiv 1$.

Corollary 1.

Under the setting of Theorem 1, for time-homogeneous CTMC, i.e., when ${\beta }_t\equiv 1$, we have

$$\begin{array}{cc}\mathrm{TV}(q_0,p_{T-\delta })\lesssim min\left\{d,\sqrt{dS}\right\}{e}^{-T}+\hfill & \sqrt{{\epsilon }_{\mathrm{TV}}}\hfill \\ & +d^{2}S{\displaystyle \sum _{k=0}^{N-1}}max\{1,{\left(T-t_{k+1}\right)}^{-2}\}{(t_{k+1}-t_k)}^2+d\delta .\hfill \end{array}$$

Compared with the time-homogeneous CTMC setting, our error bound in Theorem 1 exhibits two key differences. First, the initialization and early-stopping errors are no longer determined solely by the diffusion times T and $\delta$; they also depend on the prescribed noise schedule through the accumulated noise levels ${\overline{\beta }}_T$ and ${\overline{\beta }}_{\delta }$. Second, the discretization error is not controlled only by the selected time steps $(t_{k+1}-t_k)$. Instead, it additionally depends on the smoothness of the noise schedule through $|{\beta }_{T-t_k}^{\prime }|$ and the magnitude of the transition rates through ${\beta }_{T-t_k}^2$, which is, effectively speaking, the size of each discretization interval as measured by the amount of noise injected over that interval. We have provided a proof sketch in Section 5.

To further determine the number of steps to reach $\mathcal{O}\left(\sqrt{\epsilon }\right)$ error, we consider those ${\beta }_t$’s that satisfy the following regularity condition.

Assumption 2

(Slow-varying noise). Suppose that noise schedule ${\beta }_t$ satisfies the following regularity conditions:

1. 

The schedule is uniformly bounded away from zero on the relevant low-noise interval, namely,

$${\beta }^{\ast }:=\underset{z\in [\delta ,{\overline{\beta }}^{-1}\left(1\right)]}{min}{\beta }_z\gtrsim 1.$$

2. 

The schedule is asymptotically constant (ignoring logarithms):

$${\beta }_t,\left|{\beta }_t^{\prime }\right|=\tilde{O}\left(1\right),\forall t\in [\delta ,T].$$

3. 

The inverse accumulated-noise map grows at most polynomially:

$${\overline{\beta }}^{-1}\left(x\right)=\mathrm{poly}\left(x\right).$$

On a high level, Assumption 2 assumes that the noise schedule is slow-varying. This assumption is used only to determine a suitable choice of the number of sampling steps. In the time-homogeneous CTMC setting, where ${\beta }_t\equiv 1$, we have ${\beta }^{*}=1$ and ${\beta }_t^{\prime }\equiv 0$. Therefore, the homogeneous case automatically satisfies Assumption 2. Another example of noise schedule that satisfies Assumption 2 is the polynomial schedule where ${\overline{\beta }}_t=t^p$, where $p>0$. With this choice, ${\beta }_t=p{t}^{p-1}$ and ${\beta }^{*}=p{\delta }^{p-1}$ if $p\ge 1$ and p otherwise. ${\overline{\beta }}^{-1}\left(x\right)=x^{1/p}=\mathrm{poly}\left(x\right)$. Further, for $t\in [\delta ,T]$, ${\beta }_t=p{t}^{p-1}\le p{T}^{p-1}$ and $|{\beta }_t^{\prime }|\le |p(p-1)T^{p-2}|$, for which we note that $T=\mathrm{poly}(log(d/\sqrt{\epsilon }))$ is asymptotically a constant. In general, both the initial ${\beta }_0$ and the growth of ${\beta }_t$ w.r.t. t need to be mild to satisfy Assumption 2.

Beyond polynomial schedule, the exponential (also known as geometric) noise schedule does not satisfy this assumption. This will be our main focus in Section 6.

The first condition in Assumption 2 imposes a constant lower bound on ${\beta }_t$ over the low-noise regime. Intuitively, this prevents the noise schedule from becoming too small near the data distribution, ensuring that the score function does not blow up over the entire reverse process. The second condition controls the cumulative magnitude and variation of the schedule along the discretization grid. This requirement ensures that the transition rates do not depend on the system parameters, which is a common practice, so that the discrete-time approximation remains stable and reliable. Finally, the third condition requires the inverse accumulated-noise map ${\overline{\beta }}^{-1}$ to grow at most polynomially. Equivalently, this guarantees that one can reach a sufficiently large terminal accumulated noise level ${\overline{\beta }}_T$ within a polynomial diffusion time, thereby ensuring that enough noise is injected by the terminal time.

With Assumption 2, the following theorem characterizes the number of sampling steps required to achieve $\mathcal{O}\left(\sqrt{\epsilon }\right)$ error in the time-inhomogeneous setting.

Theorem 2.

Under the setting of Theorem 1, suppose further that Assumption 2 holds. Let $t_{k+1}-t_k=\kappa min\left\{1,{\overline{\beta }}_{T-t_k}\right\}$. Then, in order to achieve $\mathcal{O}\left(\sqrt{\epsilon }\right)$ TV-error, one can choose $T={\overline{\beta }}^{-1}(log(d/\sqrt{\epsilon }))$, $\kappa \asymp \sqrt{\epsilon }/\left(d^{2}S\right)$, and thus $N=\tilde{O}\left(d^{2}S/\sqrt{\epsilon }\right)$.

The proof of Theorem 2 is in Appendix C. This theorem shows that under the stated regularity conditions on the noise schedule, the time-inhomogeneous sampler requires the same order of sampling steps as in the time-homogeneous setting; see, e.g., refs. [17,18]. The main difference is that in the inhomogeneous case, both the discretization step sizes and the terminal diffusion time must be chosen according to the prescribed noise schedule.

5. Proof Sketch of Theorem 1

We now provide a proof sketch of Theorem 1. The full proof is in Appendix B.

Proof sketch of Theorem 1.

We analyze the Euler sampler through the truncated $\tau$-leaping sampler, which is asymptotically equivalent to Euler by [17] (Lemma 8). The key property is that the estimated rate is piecewise constant on each interval $[t_k,t_{k+1})$:

$${\widehat{R}}_t(x_{t_k},·)={\widehat{R}}_{t_k}(x_{t_k},·),t\in [t_k,t_{k+1}).$$

Let ${\overleftarrow{q}}_t=q_{T-t}$ denote the exact reverse marginal. By the TV perturbation bound of [18] (Theorem 1),

$$\mathrm{TV}({{\displaystyle \overleftarrow{{\displaystyle q}}}}_{T-\delta },p_{T-\delta })\le \mathrm{TV}({{\displaystyle \overleftarrow{{\displaystyle q}}}}_0,p_0)+{\displaystyle \sum _{k=0}^{N-1}}{\int }_{t_k}^{t_{k+1}}{\mathbb{E}}_{x_t\sim {\overleftarrow{q}}_t}\sum _{y:y\ne x_t}\left|{\widehat{R}}_t(x_t,y)-{{\displaystyle \overleftarrow{{\displaystyle R}}}}_t(x_t,y)\right|\mathrm{d}t.$$

Writing

$$h_t\left(x\right):=\sum _{y:y\ne x}\left|{\widehat{R}}_t(x,y)-{{\displaystyle \overleftarrow{{\displaystyle R}}}}_t(x,y)\right|,$$

and adding and subtracting $h_{t_k}\left(x_{t_k}\right)$, the error decomposes into initialization, score-estimation, and discretization terms. By Assumption 1, the score-estimation term is bounded by

$${\displaystyle \sum _{k=0}^{N-1}}(t_{k+1}-t_k){\mathbb{E}}_{x_{t_k}\sim {\overleftarrow{q}}_{t_k}}{h}_{t_k}\left(x_{t_k}\right)\le \sqrt{{\epsilon }_{\mathrm{TV}}}.$$

The initialization error is controlled by the convergence of the forward chain to the uniform distribution. Since ${\overleftarrow{q}}_0=q_T$ and $p_0=\mathrm{Uniform}\left({\left[S\right]}^d\right)$, we have

$$\mathrm{TV}({{\displaystyle \overleftarrow{{\displaystyle q}}}}_0,p_0)=\mathrm{TV}(q_T,p_0)\lesssim min\{d,\sqrt{dS}\}{e}^{-{\overline{\beta }}_T}.$$

It remains to bound the discretization error. The part caused by the change from $x_{t_k}$ to $x_t$ is lower-order for sufficiently small step size, since the total outgoing reverse rate is of order $d{\beta }_{T-t}$. The dominant contribution comes from the time variation of the reverse rate. Using the piecewise-constant property of ${\widehat{R}}_t$,

$${\mathbb{E}}_{x_{t_k}\sim {\overleftarrow{q}}_{t_k}}\left[h_t\left(x_{t_k}\right)-h_{t_k}\left(x_{t_k}\right)\right]\lesssim {\mathbb{E}}_{x_{t_k}\sim {\overleftarrow{q}}_{t_k}}\sum _{y:y\ne x_{t_k}}\left|{{\displaystyle \overleftarrow{{\displaystyle R}}}}_t(x_{t_k},y)-{{\displaystyle \overleftarrow{{\displaystyle R}}}}_{t_k}(x_{t_k},y)\right|.$$

By the reverse-rate formula in (3) and its coordinate-wise structure, only states satisfying $\mathrm{Ham}\left(x,y\right)=1$ contribute. The difference is then split into the change in the likelihood ratio and the change in the rate matrix. The likelihood-ratio term is bounded using the non-homogeneous score-ratio estimate

$${\displaystyle \frac{q_s\left(y\right)}{q_s\left(x\right)}}\lesssim Smax\{1,{\overline{\beta }}_s^{-1}\},\forall (x,y):\mathrm{Ham}\left(x,y\right)=1,$$

together with the corresponding likelihood-ratio difference bound. The rate-matrix term follows from smoothness of ${\beta }_t$:

$$|R_{T-t_k}(y,x)-R_{T-t}(y,x)|\lesssim {\displaystyle \frac{1}{S}}\left|{\beta }_{T-t_k}^{\prime }\right|(t-t_k).$$

Combining these bounds yields

$${\mathcal{D}}_{\mathrm{disc}}\lesssim dS{\displaystyle \sum _{k=0}^{N-1}}max\{1,{\overline{\beta }}_{T-t_{k+1}}^{-2}\}\left(|{\beta }_{T-t_k}^{\prime }|+d{\beta }_{T-t_k}^2\right){(t_{k+1}-t_k)}^2,$$

where ${\mathcal{D}}_{\mathrm{disc}}$ represents the discretization error.

Finally, because the reverse process is stopped at $T-\delta$, its target marginal is $q_{\delta }$ rather than $q_0$. The early-stopping error satisfies

$$\mathrm{TV}(q_0,q_{\delta })\lesssim d{\overline{\beta }}_{\delta }.$$

Using the triangle inequality,

$$\mathrm{TV}(q_0,p_{T-\delta })\le \mathrm{TV}(q_0,q_{\delta })+\mathrm{TV}({{\displaystyle \overleftarrow{{\displaystyle q}}}}_{T-\delta },p_{T-\delta }),$$

and combining the four bounds gives the desired result. □

6. Geometric Noise Schedule

While the geometric schedule [8] does not directly satisfy Assumption 2, the preceding time-inhomogeneous framework enables us to analyze such a noise schedule. Different from the common scenario where T diverges, in practice one typically sets $T=1$ without diverging with vanishing $\epsilon$. This way, the geometric noise schedule is defined as follows.

$${\overline{\beta }}_t:={\beta }_{\min }^{1-t}{\beta }_{\max }^t,{\beta }_t={\beta }_{\min }^{1-t}{\beta }_{\max }^{t}log({\beta }_{\max }/{\beta }_{\min })={\overline{\beta }}_{t}log({\beta }_{\max }/{\beta }_{\min }),t\in [0,1],$$

(7)

where $0<{\beta }_{\min }<1<{\beta }_{\max }$ are prescribed constants. The default choice in [8] is ${\beta }_{\min }={10}^{-4}$ and ${\beta }_{\max }=20$. Here, both ${\overline{\beta }}_t$ and ${\beta }_t$ are increasing functions on $[0,1]$. Moreover, differentiating ${\beta }_t$ gives

$${\beta }_t^{\prime }={\beta }_{\min }^{1-t}{\beta }_{\max }^t{log}^2({\beta }_{\max }/{\beta }_{\min })={\beta }_{t}log({\beta }_{\max }/{\beta }_{\min }).$$

We first follow ref. [8] and use a constant step size $\kappa$, so that the total number of discretization steps is $N={\kappa }^{-1}$. The following theorem specializes our general time-inhomogeneous result to the geometric noise schedule.

Theorem 3.

Suppose that Assumption 1 holds. With the geometric noise schedule defined in (7), choosing $t_{k+1}-t_k=\kappa$ where $t_0=0$ and $t_N=1-\delta$, we have

$$\begin{array}{cc}\underset{\delta \to 0^{+}}{lim}\mathrm{TV}(q_0,p_{1-\delta })\lesssim min\hfill & \left\{d,\sqrt{dS}\right\}{e}^{-{\beta }_{\mathit{max}}}+\sqrt{{\epsilon }_{\mathrm{TV}}}\hfill \\ & \hfill +\kappa d^{2}Slog({\beta }_{\mathit{max}}/{\beta }_{\mathit{min}})\left({({\beta }_{\mathit{min}}^{-1}-1)}^2+{({\beta }_{\mathit{max}}-1)}^2\right)+d{\beta }_{\mathit{min}}.\end{array}$$

Thus, in order to achieve $\mathcal{O}\left(\sqrt{\epsilon }\right)$ TV error, we can choose

$${\beta }_{\mathit{min}}\asymp {\displaystyle \frac{\sqrt{\epsilon }}{d}},{\beta }_{\mathit{max}}\asymp log{\displaystyle \frac{min\left\{d,\sqrt{dS}\right\}}{\sqrt{\epsilon }}},\mathit{and}\kappa \asymp {\displaystyle \frac{{\epsilon }^{3/2}}{d^{4}Slog(d/\epsilon )}},$$

we have $N=\tilde{\mathcal{O}}(d^{4}S/{\epsilon }^{3/2})$.

Theorem 3 highlights an interesting regime in which a finite error upper bound can be obtained even as $\delta \to 0^{+}$. This contrasts with prior bounds whose dependence on $log{\delta }^{-1}$ diverges in the vanishing early-stopping limit [15,16,17,18]. While the initialization, score-estimation, and early-stopping terms retain the same interpretation as in the general time-inhomogeneous result, the discretization error admits an explicit expression under the geometric noise schedule. In particular, up to logarithmic factors, this term scales quadratically with the prescribed schedule parameters ${\beta }_{\min }^{-1}$ and ${\beta }_{\max }$.

Our result illustrates the trade-off governed by the schedule parameters ${\beta }_{\min }$ and ${\beta }_{\max }$, when the step sizes are constant, as is typical in many empirical studies [8,23]. A smaller ${\beta }_{\min }$ reduces the early-stopping error but increases the accumulated discretization error. Similarly, a larger ${\beta }_{\max }$ decreases the initialization error while again amplifying the discretization error.

Note that the choice of the sampling steps is not optimized here (i.e., constant) and therefore may lead to a suboptimal overall step complexity. As in Theorem 2, we can similarly employ a constant-then-decreasing step size towards the end to obtain an improved convergence result, given as follows.

Theorem 4.

With the same setting as in Theorem 3, but choosing $t_{k+1}-t_k=\kappa min\left\{1,{\beta }_{\mathit{min}}^{t_k}{\beta }_{\mathit{max}}^{1-t_k}\right\}$, we have

$$\begin{array}{cc}\underset{\delta \to 0^{+}}{lim}\mathrm{TV}(q_0,p_{1-\delta })\lesssim min\left\{d,\sqrt{dS}\right\}\hfill & e^{-{\beta }_{\mathit{max}}}+\sqrt{{\epsilon }_{\mathrm{TV}}}\hfill \\ & \hfill +\kappa d^{2}Slog({\beta }_{\mathit{max}}/{\beta }_{\mathit{min}})\left({({\beta }_{\mathit{max}}-1)}^2+1\right)+d{\beta }_{\mathit{min}}.\end{array}$$

Meanwhile, the number of steps satisfies that

$$N\lesssim {\displaystyle \frac{1}{\kappa }}\left(1+{\displaystyle \frac{{\beta }_{\mathit{min}}^{-1}-1}{log({\beta }_{\mathit{max}}/{\beta }_{\mathit{min}})}}\right).$$

Thus, choosing

$${\beta }_{\mathit{min}}\asymp {\displaystyle \frac{\sqrt{\epsilon }}{d}},{\beta }_{\mathit{max}}\asymp log{\displaystyle \frac{min\left\{d,\sqrt{dS}\right\}}{\sqrt{\epsilon }}},\mathit{and}\kappa \asymp {\displaystyle \frac{\sqrt{\epsilon }}{d^{2}Slog(d/\epsilon )}},$$

we have $N=\tilde{\mathcal{O}}(d^{3}S/\epsilon )$.

Compared to Theorem 3, Theorem 4 improves the step complexity by a factor of order $\mathcal{O}(d/\sqrt{\epsilon })$. The key observation is that under the geometric noise schedule, the effective noise level decays exponentially along the reverse-time trajectory. The exponential-then-constant step size adapts to this behavior: it uses relatively large constant steps in the high-noise regime, where the reverse dynamics are more stable, and automatically takes smaller steps near the low-noise regime, where the discretization error is most sensitive. This refinement removes the unfavorable dependence on ${\beta }_{\min }^{-2}$ that appears under a constant step size while increasing the number of steps only in the region where finer discretization is necessary. As a result, the adaptive scheme achieves a sharper overall sampling complexity.

As a future direction, it might be interesting to investigate why using such a geometric noise schedule would yield worsened results than using polynomial ones.

7. Conclusions

In conclusion, this work provides a theoretical foundation for discrete diffusion models driven by time-inhomogeneous uniform-rate generators. By moving beyond the homogeneous noise schedules considered in much of the prior literature, our analysis broadens the class of discrete diffusion processes for which convergence can be rigorously understood. We further identify regularity conditions on the noise schedule that enable explicit convergence-rate estimates. Under these conditions, the resulting guarantees match state-of-the-art rates known for homogeneous discrete diffusion samplers, demonstrating that carefully controlled time-inhomogeneous schedules can retain the same theoretical efficiency while offering greater modeling flexibility.