Support Size of ε-Capacity-Achieving Inputs for the Amplitude-Constrained AWGN Channel

Abstract

We study the discrete-time amplitude-constrained additive white Gaussian noise (AWGN) channel from the perspective of near-optimal input distributions in the high-SNR, or equivalently large-amplitude, regime. While it is known that the capacity-achieving input is discrete with finitely many mass points, the precise scaling of its support size as a function of the amplitude constraint remains an open problem. In this work, we instead consider the minimal support size required to achieve capacity up to an $\epsilon$-gap. We introduce the quantity $K_{\epsilon }\left(A\right)$, defined as the smallest support size among discrete inputs supported on $[-A,A]$ that achieves mutual information within $\epsilon$ of capacity. We show that this relaxed formulation is significantly more tractable and admits sharp characterizations in several vanishing-gap regimes. In particular, for polynomially decaying gaps, $\epsilon =A^{-\beta }$ with $\beta \ge 1$, we establish that $K_{\epsilon }\left(A\right)=\Theta \left(A\sqrt{logA}\right)$ as $A\to \infty$. For exponentially small gaps, we obtain bounds of order between $A\sqrt{logA}$ and $A^{3/2}$. Our approach combines approximation-theoretic bounds for Gaussian mixtures with information-theoretic control of entropy via ${\chi }^2$-divergence, together with a wrapping argument that relates the problem to approximating the uniform distribution on a circle. Beyond the technical results, our framework provides a conceptual explanation for the variety of scaling laws observed in prior numerical studies, suggesting that these may correspond to different regimes of $\epsilon$-optimality rather than intrinsic properties of the exact optimizer.

1. Introduction

We consider a discrete-time additive real-valued white Gaussian noise (AWGN) channel subject to a peak-power constraint. The channel output is given by

$$Y=X+Z,$$

(1)

where the input random variable X satisfies the constraint $\left|X\right|\le A$ almost surely (a.s.), and Z is a standard normal random variable independent of X. The capacity under the amplitude constraint is

$$C\left(A\right):=\underset{P_X:\mathsf{supp}\left(P_X\right)\subset [-A,A]}{sup}I(X;Y).$$

(2)

We denote by $X^{\ast }$ a capacity-achieving input random variable and by $Y^{\ast }$ the corresponding induced output random variable. In general, both the exact value of the capacity $C\left(A\right)$ and the precise structure of the capacity-achieving distribution $X^{\ast }$ remain unknown.

1.1. Problem Formulation

In contrast to prior work focusing on exact optimality, we study the minimal support size required to achieve capacity up to an $\epsilon$-gap. Specifically, for $\epsilon >0$, define

$$K_{\epsilon }\left(A\right):=min\left\{|\mathsf{supp}\left(P_X\right)|:\mathsf{supp}\left(P_X\right)\subset [-A,A],P_X\mathrm{discrete},I(X;Y)\ge C\left(A\right)-\epsilon \right\}.$$

(3)

While the precise scaling of $\left|\mathsf{supp}\right(P_{X^{\ast }}\left)\right|$ remains unknown, this relaxed formulation turns out to be significantly more tractable. In particular, we are able to characterize the exact scaling of $K_{\epsilon }\left(A\right)$ for a range of regimes. For example, when the gap decays polynomially with A, i.e., $\epsilon =A^{-\beta }$ for some $\beta \ge 1$, we obtain a sharp characterization of $K_{\epsilon }\left(A\right)$.

1.2. Literature Review

The literature on the amplitude constraint channel is large and we do not try to survey it fully and only mention key relevant results. For a comprehensive review, interest readers are referred to [1,2,3] and references therein.

Studying the capacity of the amplitude-constrained AWGN channel is a classical problem in information theory, originating in the work of Shannon [4]. A fundamental result by Smith [5,6] shows that the capacity-achieving input distribution is discrete with finitely many mass points, in contrast to the average-power-constrained setting where the capacity-achieving distribution is Gaussian. Subsequent work further characterized the structure of the optimal input, including transition thresholds where binary and ternary constellations are optimal [7]. However, the precise scaling of the support size with the amplitude constraint remains unresolved: the best-known non-asymptotic bounds, [8] and [1], place it between $A\sqrt{logA}$ and $A^2$, respectively.

Somewhat intriguingly, numerical investigations have suggested a range of alternative asymptotic behaviors. In particular, Mattingly et al. [9] reported an empirical scaling of order $A^{4/3}$ as $A\to \infty$, based on experiments involving not only the AWGN channel but also several non-Gaussian models, including the binomial channel and certain two-dimensional channels. A follow-up work by Abbott and Machta [10] provided a heuristic, physics-inspired justification for this scaling. These findings coexist with Zhang’s spacing-based heuristic, which suggests growth of order $A\sqrt{logA}$ ([11] pp. 91, 95, 96), while earlier work conjectured linear scaling [1], a claim that has since been disproved [8].

We argue that this apparent diversity of scaling laws can be naturally interpreted through the lens of $\epsilon$-optimality. Indeed, all numerical procedures implicitly operate with a finite tolerance, effectively computing inputs that are only $\epsilon$-capacity-achieving for some algorithm-dependent $\epsilon$. From this perspective, different observed scalings correspond to different regimes of $\epsilon =\epsilon \left(A\right)$, rather than intrinsic properties of the exact optimizer. This viewpoint also explains the well-known numerical sensitivity of the problem: in the large-A regime, the optimal output distribution becomes nearly uniform in the interior, and deviations that distinguish competing inputs occur at a scale comparable to numerical precision. Consequently, implementations based on, e.g., the Blahut–Arimoto algorithm [12,13], which involve repeated numerical integrations of log-densities, are particularly susceptible to bias and instability. As a result, different numerical tolerances and methodologies can lead to markedly different empirical scaling laws.

In parallel, a large body of work has developed capacity bounds via entropy methods, duality, and estimation-theoretic representations; see [14,15,16] and references therein. There is also a large body of work that focuses on showing discreteness of capacity-achieving inputs for non-Gaussian channels [17,18,19,20,21,22,23,24,25].

1.3. Outline and Contributions

Section 2 collects the main technical tools used throughout the paper, including stability bounds for entropy, approximation results for Gaussian mixtures, and properties of the wrapping operation. These ingredients form the backbone of both the achievability and converse arguments.

Section 3 presents the main results together with their derivations. In particular, we obtain sharp bounds on $K_{\epsilon }\left(A\right)$ and provide a complete characterization in the regime where $\epsilon$ decays at most polynomially in A. We also discuss the behavior in the exponential regime and highlight the transition in scaling.

Section 4 concludes the paper with a summary of the main findings and a discussion of open problems.

We conclude this section by presenting relevant notation.

1.4. Notation

Throughout the paper, the deterministic scalar quantities are denoted by lowercase letters and random variables are denoted by uppercase letters.

We denote the distribution of a random variable X by $P_X$. The support set of $P_X$ is denoted and defined as

$$\mathsf{supp}\left(P_X\right)=\left\{x:\mathrm{for}\mathrm{every}\mathrm{open}\mathrm{set}\mathcal{D}\ni x\mathrm{we}\mathrm{have}\mathrm{that}{P}_X\left(\mathcal{D}\right)>0\right\}.$$

(4)

The notation $|·|$, depending on the context, denotes either absolute value or cardinality of the set. For example, $\left|\mathsf{supp}\right(P_X\left)\right|$ denotes the size of the support of $P_X$. All logarithms are taken with base $\mathrm{e}$. The density of a standard normal will be denoted by $\phi$.

We denote the differential entropy of a continuous random variable X by $h\left(X\right)$. Given two probability distributions P and Q with probability densities functions (pdfs) p and q, respectively, we will require the following distances,

$$\begin{array}{cccc}\hfill & \mathrm{Total}\mathrm{Variation}:\hfill & \hfill \mathsf{TV}(P\parallel Q)& ={\displaystyle \frac{1}{2}}\int |p\left(x\right)-q\left(x\right)|\mathrm{d}x,\hfill \end{array}$$

(5)

$$\begin{array}{cccc}\hfill & \mathrm{Relative}\mathrm{Entropy}:\hfill & \hfill \mathsf{D}(P\parallel Q)& =\int p\left(x\right)log{\displaystyle \frac{p\left(x\right)}{q\left(x\right)}}\mathrm{d}x,\hfill \end{array}$$

(6)

$$\begin{array}{cccc}\hfill & {\chi }^2\mathrm{Divergence}:\hfill & \hfill {\chi }^2(P\parallel Q)& =\int {\displaystyle \frac{{(p\left(x\right)-q\left(x\right))}^2}{q\left(x\right)}}\mathrm{d}x,\hfill \end{array}$$

(7)

with the understanding that the relative entropy and ${\chi }^2$ are equal to infinity if P is not absolutely continuous with respect to Q. Finally, ${log}^{+}\left(x\right)=max\{log\left(x\right),0\}$.

2. Tools and Preliminaries

In this section, we collect several technical ingredients that underlie our analysis. At a high level, our approach proceeds by approximating the output distribution induced by the capacity-achieving input using finite Gaussian mixtures, and then quantifying the resulting loss in mutual information. This leads to three main components: (i) a stability bound for entropy in terms of ${\chi }^2$-divergence, (ii) approximation guarantees for Gaussian mixtures, and (iii) a wrapping argument that allows us to compare distributions to the uniform law on a circle.

2.1. Entropy Loss via ${\chi }^2$

The first ingredient is a quantitative stability bound for differential entropy under ${\chi }^2$-perturbations. This allows us to convert approximation guarantees at the level of densities into bounds on mutual information.

Lemma 1 

(Entropy loss controlled by ${\chi }^2$). Let $f,g$ be densities on $\mathbb{R}$ such that ${\chi }^2(g\parallel f)<\infty$ and $\int {(logf)}^{2}f<\infty$. Then,

$$h\left(f\right)-h\left(g\right)\le {\parallel logf\parallel }_{L^2\left(f\right)}\sqrt{{\chi }^2(g\parallel f)}+{\chi }^2(g\parallel f),$$

(8)

where ${\parallel logf\parallel }_{L^2\left(f\right)}:={\left(\int {(logf)}^{2}f\right)}^{1/2}$.

Proof. 

Write

$$u(x):={\displaystyle \frac{g(x)}{f(x)}}-1,$$

(9)

so that $g(x)=(1+u(x))f(x)$ and

$$\int u(x)f(x)dx=\int g(x)dx-\int f(x)dx=0.$$

(10)

Moreover,

$${\chi }^2(g\parallel f)=\int u{(x)}^{2}f(x)dx.$$

(11)

Then

$$h(g)=-\int g(x)logg(x)dx$$

(12)

$$\begin{array}{cc}\hfill & =-\int (1+u(x))f(x)log\left((1+u(x))f(x)\right)dx\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill & =-\int (1+u(x))f(x)logf(x)dx-\int (1+u(x))f(x)log(1+u(x))dx.\hfill \end{array}$$

(14)

Therefore,

$$h(f)-h(g)=-\int f(x)logf(x)dx+\int (1+u(x))f(x)logf(x)dx$$

$$\begin{array}{cc}\hfill & +\int (1+u(x))f(x)log(1+u(x))dx\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & =\int u(x)f(x)logf(x)dx+\int (1+u(x))f(x)log(1+u(x))dx.\hfill \end{array}$$

(16)

By Cauchy–Schwarz,

$$\begin{array}{cc}\hfill \left|\int u(x)f(x)logf(x)dx\right|& \le {\left(\int u{(x)}^{2}f(x)dx\right)}^{1/2}{\left(\int {(logf(x))}^{2}f(x)dx\right)}^{1/2}\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill & ={\parallel logf\parallel }_{L^2(f)}\sqrt{{\chi }^2(g\parallel f)}.\hfill \end{array}$$

(18)

For the second term, using $log(1+u)\le u$ for all $u>-1$, we get

$$(1+u(x))log(1+u(x))\le (1+u(x))u(x)=u(x)+u{(x)}^2.$$

(19)

Hence,

$$\begin{array}{cc}\hfill \int (1+u(x))f(x)log(1+u(x))dx& \le \int \left(u(x)+u{(x)}^2\right)f(x)dx\hfill \\ \hfill & =\int u(x)f(x)dx+\int u{(x)}^{2}f(x)dx\hfill \\ \hfill & ={\chi }^2(g\parallel f).\hfill \end{array}$$

Combining the two bounds gives

$$h(f)-h(g)\le {\parallel logf\parallel }_{L^2(f)}\sqrt{{\chi }^2(g\parallel f)}+{\chi }^2(g\parallel f),$$

(20)

which proves the claim. □

The key feature of Lemma 1 is that it provides control of entropy loss in terms of ${\chi }^2$-divergence. In particular, small ${\chi }^2$-error directly translates into a small loss in mutual information, up to a multiplicative factor depending on ${\parallel logf\parallel }_{L^2\left(f\right)}$.

We will apply this lemma in a setting where f corresponds to the output density induced by the capacity-achieving input, and g corresponds to an approximating Gaussian mixture. The next result provides a uniform bound on the prefactor ${\parallel logf\parallel }_{L^2\left(f\right)}$ in this setting.

Lemma 2. 

Fix $A>0$. Let $Y=X+Z$ where $Z\sim \mathcal{N}(0,1)$ and $X\sim P_X$ be supported on $[-A,A]$. Then,

$$\parallel log{f}_Y{\parallel }_{L^2\left(f_Y\right)}\le \sqrt{10}(1+A^2).$$

(21)

Proof. 

Fix $y\in \mathbb{R}$. First, notice that

$$\begin{array}{c}\hfill f_Y\left(y\right)=\mathbb{E}\left[\phi (y-X)\right]\le {\displaystyle \frac{1}{\sqrt{2\pi }}}<1\end{array}$$

(22)

since $\phi \left(y\right)\le {\displaystyle \frac{1}{\sqrt{2\pi }}}$. Since $\phi \left(x\right)={\left(2\pi \right)}^{-1/2}{e}^{-x^2/2}$ is strictly decreasing in $\left|x\right|$, for $\theta \in [-A,A]$ we have $|y-\theta |\le \left|y\right|+A$; hence $\phi (y-\theta )\ge \phi \left(\right|y|+A)$. Averaging gives

$$f_Y\left(y\right)=\int \phi (y-\theta )\mathrm{d}{P}_X\left(\theta \right)\ge \phi \left(\right|y|+A)={\left(2\pi \right)}^{-1/2}exp\left(-{\displaystyle \frac{{\left(\right|y|+A)}^2}{2}}\right).$$

(23)

Thus,

$$-log{f}_Y\left(y\right)\le {\displaystyle \frac{{\left(\right|y|+A)}^2}{2}}+{\displaystyle \frac{1}{2}}log\left(2\pi \right)=:{\displaystyle \frac{{\left(\right|y|+A)}^2}{2}}+c_0.$$

(24)

Consequently,

$$\begin{array}{cc}\hfill \parallel log{f}_Y{\parallel }_{L^2\left(f_Y\right)}^2& =\mathbb{E}\left[{(-log{f}_Y\left(Y\right))}^2\right]\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill & \le \mathbb{E}\left[{\left({\displaystyle \frac{{\left(\right|X+Z|+A)}^2}{2}}+c_0\right)}^2\right]\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill & \le 8{(1+A^2)}^2+2c_0^2\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill & \le 10{(1+A^2)}^2\hfill \end{array}$$

(28)

where (26) follows from the bound in (24) and from $-log{f}_Y\left(y\right)>0$ thanks to (22), and (27) follows from sequential application of the inequality ${(a+b)}^2\le 2(a^2+b^2)$, the bound $\left|X\right|\le A$, and the evaluation of the moments of Z. □

Lemma 2 shows that the entropy sensitivity grows at most quadratically in A. Combined with Lemma 1, this implies that achieving a small ${\chi }^2$-approximation error is sufficient to ensure near optimality in mutual information.

2.2. Finite-Mixture Approximations

The second ingredient is a sharp approximation result for Gaussian mixtures. Recall that for any probability measure P supported on $[-A,A]$, the induced output density takes the form

$$f_P\left(y\right):={\int }_{-A}^A\phi (y-\theta )\mathrm{d}P\left(\theta \right),$$

(29)

i.e., a Gaussian location mixture.

Our goal is to approximate such densities using mixtures supported on finitely many points. The following result, by Ma, Wu, and Yang, provides near-optimal bounds on this approximation error.

Lemma 3 

(Ma–Wu–Yang approximation theorem [26]). Let $A>0$ and let ${\mathcal{P}}_A^{\mathrm{Bdd}}$ denote the set of probability measures supported on $[-A,A]$. Let ${\mathcal{P}}_m$ denote the set of probability measures supported on at most m points. Define the worst-case best approximation error

$$E^{\star }(m,{\mathcal{P}}_A^{\mathrm{Bdd}},{\chi }^2):=\underset{P\in {\mathcal{P}}_A^{\mathrm{Bdd}}}{sup}\underset{Q\in {\mathcal{P}}_A^{\mathrm{Bdd}}\cap {\mathcal{P}}_m}{inf}{\chi }^2(f_Q\parallel f_P),$$

where $f_P$ is the mixture density (29). Then there exists a universal constant $\kappa \ge 16e^3$ such that for all $m\in \mathbb{N}$ and $A>0$,

$$E^{\star }(m,{\mathcal{P}}_A^{\mathrm{Bdd}},{\chi }^2)\le \left\{\begin{array}{cc}exp\left(-{\displaystyle \frac{mlogm}{A^2}}\right),\hfill & m\ge \kappa A^2,\hfill \\ exp\left(-{\displaystyle \frac{log\kappa }{4\kappa }}{\displaystyle \frac{m^2}{A^2}}\right),\hfill & 3\sqrt{\kappa A}\le m\le \kappa A^2.\hfill \end{array}\right.$$

(30)

The key takeaway from Lemma 3 is that Gaussian mixtures supported on m points can approximate arbitrary mixtures supported on $[-A,A]$ with exponentially small ${\chi }^2$-error. Moreover, the approximation exhibits two distinct regimes:

• A quadratic regime, where the error behaves like $exp(-m^2/A^2)$;
• A large-m regime, where the error behaves like $exp(-mlogm/A^2)$.

The dichotomy in Lemma 3 comes from the two approximation mechanisms used in the proof of [26]. When m is large compared to $A^2$, one can approximate the mixing distribution globally by matching a large number of moments, for example via Gauss quadrature. This gives the large-m exponent $exp(-mlogm/A^2)$. In contrast, when m is below the $A^2$ scale, global moment matching is no longer efficient over the whole interval $[-A,A]$. The remedy is to partition $[-A,A]$ into smaller intervals and apply moment matching locally to the conditional distribution on each subinterval. Optimizing the number of intervals and the number of atoms per interval leads to the quadratic exponent $exp(-c{m}^2/A^2)$. Thus, the two rates reflect the transition between a global moment-matching regime and a local moment-matching regime.

This dichotomy will directly translate into the two regimes in our main results. In particular, it is precisely this approximation behavior that determines the scaling of $K_{\epsilon }\left(A\right)$.

2.3. Wrapped Random Variables

The final ingredient is a wrapping argument, which was introduced in [8], that allows us to compare output distributions to the uniform distribution on a circle. This plays a key role in the converse, where we lower bound the support size by quantifying how far the induced output can be from uniformity.

For $B>0$, define the wrapping map ${\langle ·\rangle }_B:\mathbb{R}\to [-\pi ,\pi )$ by

$${\langle W\rangle }_B:={\displaystyle \frac{\pi }{B}}(Wmod2B),$$

(31)

where $W\mathrm{mod}2B:=W-2B⌊{\displaystyle \frac{W+B}{2B}}⌋$.

The wrapping operation has several useful properties summarized below.

Proposition 1. 

1.

(Wrapped density formula) If W has density $f_W$, then ${\langle W\rangle }_B$ has density

$$f_{{\langle W\rangle }_B}\left(\theta \right)={\displaystyle \frac{B}{\pi }}\sum _{k\in \mathbb{Z}}{f}_W\left({\displaystyle \frac{B}{\pi }}(\theta +2\pi k)\right),\theta \in (-\pi ,\pi ).$$

(32)

2.

(Uniformity after wrapping) Let $U\sim \mathrm{Unif}\left(\right[-B,B\left]\right)$ be independent of $Z\sim \mathcal{N}(0,1)$. Then ${\langle U+Z\rangle }_B$ is uniform on $(-\pi ,\pi )$:

$$f_{{\langle U+Z\rangle }_B}\left(\theta \right)={\displaystyle \frac{1}{2\pi }},\theta \in (-\pi ,\pi ).$$

(33)

3.

(Wrapped-mixture lower bound) Let $X\in [-A,A]$ be discrete with $K:=\left|\mathsf{supp}\right(P_X\left)\right|\ge 2$, and let $U\sim \mathrm{Unif}\left(\right[-A,A\left]\right)$. Then

$${\chi }^2\left(P_{{\langle X+Z\rangle }_A}\parallel P_{{\langle U+Z\rangle }_A}\right)\ge {\displaystyle \frac{1}{2}}exp\left(-4{\pi }^2{\displaystyle \frac{K^2}{A^2}}\right).$$

(34)

4.

(Uniform $L^{\infty }$ bound for the wrapped density) Let $Y=X+Z$ and assume $A\ge 1$ and $D(P_Y\parallel P_{Y^{\star }})\le \epsilon$ with $\epsilon \le 1/A$. Then there is an explicit absolute constant $M_0$ such that

$${\parallel f_{{\langle X+Z\rangle }_A}\parallel }_{\infty }\le M_0.$$

(35)

One may take

$$M_0:=\left({\displaystyle \frac{3}{\pi }}+{\displaystyle \frac{1}{\sqrt{2\pi }}}\right){\displaystyle \frac{2e^2}{{\alpha }_0}}\pi e^2,{\alpha }_0={\int }_{-1}^1{e}^{-t^2/2}dt.$$

(36)

5.

Let $Y=X+Z$ and assume $A\ge 1$ and $D(P_Y\parallel P_{Y^{\star }})\le \epsilon$ with $\epsilon \le 1/A$. Then,

$$C_0{\chi }^2\left(P_{{\langle X+Z\rangle }_A}\parallel P_{{\langle U+Z\rangle }_A}\right)\le \mathsf{TV}(P_{{\langle X+Z\rangle }_A},P_{{\langle U+Z\rangle }_A})$$

(37)

where $C_0={\displaystyle \frac{1}{4(2\pi M_0+1)}}$.

Proof. 

The proof of the first three statements can be found in [8]. The last two statements are shown in Appendix A. □

Conceptually, the wrapping argument allows us to reduce the problem to approximating the uniform distribution on a circle using wrapped Gaussian mixtures. Since uniformity is highly structured, this provides a robust way to obtain lower bounds on the number of support points.

3. Main Result

3.1. Some Basic Properties of $K_{\epsilon }\left(A\right)$

In this section, we collect several basic but important structural properties of $K_{\epsilon }\left(A\right)$. In particular, we show that $K_{\epsilon }\left(A\right)$ behaves monotonically in $\epsilon$ and recovers the support size of the capacity-achieving input in the limit $\epsilon \to 0$.

Theorem 1 

(Basic properties of $K_{\epsilon }\left(A\right)$). Fix $A>0$. Then the following statements hold.

1.

For every $\epsilon >0$, the quantity $K_{\epsilon }\left(A\right)$ is well-defined and finite. In particular,

$$1\le K_{\epsilon }\left(A\right)\le \left|\mathsf{supp}\left(P_{X^{\ast }}\right)\right|<\infty ,$$

where $P_{X^{\ast }}$ is the capacity-achieving input distribution.

2.

The map $\epsilon \mapsto K_{\epsilon }\left(A\right)$ is non-increasing on $(0,\infty )$.

3.

Choose an integer $1\le m<\left|\mathsf{supp}\right(P_{X^{\ast }}\left)\right|$ and let

$$C_{-m}\left(A\right):=sup\left\{I(X;Y):\mathsf{supp}\left(P_X\right)\subset [-A,A],P_X\mathit{discrete},|\mathsf{supp}\left(P_X\right)|\le |\mathsf{supp}\left(P_{X^{\ast }}\right)|-m\right\},$$

(38)

and let

$${\delta }_m\left(A\right)=C\left(A\right)-C_{-m}\left(A\right).$$

(39)

Then, for every $0<\epsilon <{\delta }_m\left(A\right)$

$$K_{\epsilon }\left(A\right)\ge \left|\mathsf{supp}\left(P_{X^{\ast }}\right)\right|-m+1,.$$

(40)

Consequently, by choosing $m=1$

$$\begin{array}{c}\hfill \underset{ϵ\to 0}{lim}{K}_{\epsilon }\left(A\right)=\left|\mathsf{supp}\left(P_{X^{\ast }}\right)\right|.\end{array}$$

(41)

Proof. 

We only show the last statement. Let $K^{\ast }\left(A\right)=\left|\mathsf{supp}\left(P_{X^{\ast }}\right)\right|$. By definition of $K^{\ast }\left(A\right)$, no input supported on fewer than $K^{\ast }\left(A\right)$ points can achieve capacity. In particular, no input supported on at most $K^{\ast }\left(A\right)-m$ points can achieve capacity. Hence

$$C_{-m}\left(A\right)<C\left(A\right).$$

(42)

Set

$${\delta }_m\left(A\right):=C\left(A\right)-C_{-m}\left(A\right)>0.$$

(43)

Therefore, for any $0<\epsilon <{\delta }_m\left(A\right)$, we have

$$C\left(A\right)-\epsilon >C\left(A\right)-{\delta }_m\left(A\right)=C_{-m}\left(A\right).$$

(44)

On the other hand, by definition of $C_{-m}\left(A\right)$, every discrete input $P_X$ with

$$|\mathsf{supp}\left(P_X\right)|\le K^{\ast }\left(A\right)-m$$

(45)

satisfies

$$I(X;Y)\le C_{-m}\left(A\right)<C\left(A\right)-\epsilon .$$

(46)

Thus, no such input can be $\epsilon$-capacity-achieving. It follows that any $\epsilon$-capacity-achieving input must satisfy

$$|\mathsf{supp}\left(P_X\right)|\ge K^{\ast }\left(A\right)-m+1.$$

(47)

By the definition of $K_{\epsilon }\left(A\right)$, this implies that

$$K_{\epsilon }\left(A\right)\ge K^{\ast }\left(A\right)-m+1=\left|\mathsf{supp}\left(P_{X^{\ast }}\right)\right|-m+1.$$

(48)

This proves that for every $0<\epsilon <{\delta }_m\left(A\right)$,

$$K_{\epsilon }\left(A\right)\ge \left|\mathsf{supp}\left(P_{X^{\ast }}\right)\right|-m+1.$$

(49)

This concludes the proof. □

Remark 1. 

Theorem 1 shows that $K_{\epsilon }\left(A\right)$ interpolates between two regimes. For large ε, small support sizes suffice, while as $\epsilon \downarrow 0$, the quantity $K_{\epsilon }\left(A\right)$ recovers the exact support size of the capacity-achieving input. Moreover, the quantity ${\delta }_m\left(A\right)$ quantifies how much capacity is lost when restricting to inputs with fewer than $\left|\mathsf{supp}\right(P_{X^{\ast }}\left)\right|-m$ points.

3.2. Bounds

We now state the main results of this work, which characterize the scaling of $K_{\epsilon }\left(A\right)$ in different regimes of the capacity gap.

Theorem 2. 

Suppose that $A>1600$. Then the following bounds hold.

• Polynomial capacity gap. For $\beta \ge 1$,

  $${\displaystyle \frac{1}{2\sqrt{2}\pi }}A\sqrt{{log}^{+}\left(c_{1}A\right)}\le K_{A^{-\beta }}\left(A\right)\le 32\mathrm{e}A\sqrt{c_{2}log\left(A\right)}.$$

  (50)
• Exponential capacity gap.

  $${\displaystyle \frac{1}{2\sqrt{2}\pi }}A\sqrt{{log}^{+}\left(c_{1}A\right)}\le K_{{\mathrm{e}}^{-A}}\left(A\right)\le c_3{A}^{3/2}.$$

  (51)

The constants are given by

$$c_1={\displaystyle \frac{1}{8\left(1+\sqrt{\frac{\pi \mathrm{e}}{2}}\right){\left(\left(6+\sqrt{2\pi }\right)\frac{2\pi {\mathrm{e}}^4}{{\alpha }_0}+1\right)}^2}},{\alpha }_0={\int }_{-1}^1{\mathrm{e}}^{-t^2/2}dt,$$

(52)

$$c_2={\displaystyle \frac{(2\beta +5)\mathrm{e}}{4log\left(2\right)+3}},c_3=8\mathrm{e}\sqrt{{\displaystyle \frac{2\mathrm{e}}{4log\left(2\right)+3}}}.$$

(53)

We make the following remarks:

• For polynomially decaying gaps, the upper and lower bounds match up to constants, yielding the characterization

  $$K_{A^{-\beta }}\left(A\right)=\Theta \left(A\sqrt{logA}\right).$$
  In particular, the scaling is independent of $\beta \ge 1$ at the level of first-order asymptotics.
• The lower bound is universal across both regimes and reflects an intrinsic limitation: even moderately accurate approximation of the optimal output requires at least $A\sqrt{logA}$ mass points.
• In contrast, the upper bounds reveal a phase transition: while polynomial accuracy can be achieved with $A\sqrt{logA}$ points, exponentially small gaps might require significantly larger support, up to order $A^{3/2}$.
• Taken together, these results suggest that different apparent scaling laws may reflect different accuracy regimes. We emphasize that the bounds proved in this paper are asymptotic and apply in the large-amplitude regime. Thus, they should not be interpreted as a direct finite-A explanation of the numerical observations in [9,10,11]. Rather, they provide an asymptotic mechanism by which different scalings can emerge from different implicit choices of the accuracy level $\epsilon =\epsilon \left(A\right)$.
• Note that our analysis has focused primarily on the regime in which $\epsilon$ decays with A. For many practical purposes, however, it is also natural to consider the case where $\epsilon$ is fixed. Obtaining sharp bounds in this regime for arbitrary choices of $\epsilon$ appears to be more delicate. Nevertheless, one can obtain a simple consequence from an Ozarow–Wyner-type bound [27,28,29]. In particular, if $X_D$ is a PAM input with the number of mass points chosen proportional to A, then, as $A\to \infty$,

  $$C\left(A\right)-I(X_D;Y)\le {\displaystyle \frac{1}{2}}log\left({\displaystyle \frac{\pi \mathrm{e}}{3}}\right).$$

  (54)
  Consequently, this implies the fixed-gap upper bound

  $$K_{{\displaystyle \frac{1}{2}}log\left({\displaystyle \frac{\pi \mathrm{e}}{3}}\right)}\left(A\right)=O\left(A\right).$$

  (55)

3.3. Achievability

In this section, we demonstrate the achievability part of the main result. We begin by presenting a general achievability bound.

Theorem 3 

(Achievability bound). Fix $A\ge 1$ and $0<\epsilon \le 1$. Then,

$$K_{ϵ}\left(A\right)\le m$$

(56)

where

$$m:=\left\{\begin{array}{cc}{m}_1,\hfill & m_1\le \kappa A^2,\hfill \\ m_2,\hfill & m_1>\kappa A^2.\hfill \end{array}\right.$$

(57)

and

$$\begin{array}{cc}\hfill m_1& =⌈3\sqrt{\kappa A}+A\sqrt{{\displaystyle \frac{1}{c}}log\left({\displaystyle \frac{1}{{\delta }_A}}\right)}⌉,\hfill \end{array}$$

(58)

$$\begin{array}{cc}\hfill m_2& =⌈max\{3,\kappa A^2\}+A^{2}log\left({\displaystyle \frac{1}{{\delta }_A}}\right)⌉,\hfill \end{array}$$

(59)

$$\begin{array}{cc}\hfill {\delta }_A& =min\left\{{\displaystyle \frac{\epsilon }{2}},{\displaystyle \frac{{\epsilon }^2}{40{(1+A^2)}^2}}\right\}.\hfill \end{array}$$

(60)

where $c={\displaystyle \frac{log\kappa }{4\kappa }}$ and κ is defined in Lemma 3.

Proof. 

Let $P^{\star }$ be capacity-achieving on $[-A,A]$ and write $f^{\star }:=f_{P^{\star }}$. We seek to apply Lemma 3, targeting $P=P^{\star }$ and m chosen as in (57).

Case $m_1\le \kappa A^2$: From (57), $m=m_1$. Moreover, from (58), we have that $m=m_1\ge 3\sqrt{\kappa A}$, and Lemma 3 (quadratic regime) yields a Q supported on at most m points with

$${\chi }^2(f_Q\parallel f^{\star })\le exp\left(-c{\displaystyle \frac{m^2}{A^2}}\right).$$

(61)

Since by (58) we also have $m=m_1\ge A\sqrt{{\displaystyle \frac{1}{c}}log(1/{\delta }_A)}$, we get that

$${\chi }^2(f_Q\parallel f^{\star })\le exp\left(-c{\displaystyle \frac{m_1^2}{A^2}}\right)\le {\delta }_A.$$

(62)

Case $m_1>\kappa A^2$: From (57), $m=m_2$. Moreover, from (59), we have that $m=m_2\ge max\{3,\kappa A^2\}$; in particular $m\ge 3$ and therefore $logm\ge log3\ge 1$. Lemma 3 (large-m regime) yields a Q supported on at most m points such that

$${\chi }^2(f_Q\parallel f^{\star })\le exp\left(-{\displaystyle \frac{mlogm}{A^2}}\right)\le exp\left(-{\displaystyle \frac{m}{A^2}}\right).$$

Since, by (59) $m=m_2\ge A^{2}log(1/{\delta }_A)$, we have that

$${\chi }^2(f_Q\parallel f^{\star })\le exp\left(-{\displaystyle \frac{m}{A^2}}\right)\le {\delta }_A.$$

(63)

In both cases, we have produced a discrete Q supported on at most m points such that

$${\chi }^2(f_Q\parallel f^{\star })\le {\delta }_A.$$

(64)

To couple ${\delta }_A$ and $\epsilon$, let $X_Q$ be the distribution achieving the bound in (64). Also, let $Y_Q=X_Q+Z$, with density $f_Q$, and note that

$$\begin{array}{cc}\hfill C\left(A\right)-I(X_Q;Y_Q)& =h\left(f^{\star }\right)-h\left(f_Q\right)\hfill \end{array}$$

(65)

$$\begin{array}{cc}\hfill & \le \parallel log{f}^{\star }{\parallel }_{L^2\left(f^{\star }\right)}\sqrt{{\chi }^2(f_Q\parallel f^{\star })}+{\chi }^2(f_Q\parallel f^{\star })\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill & \le \sqrt{10}(1+A^2)\sqrt{{\chi }^2(f_Q\parallel f^{\star })}+{\chi }^2(f_Q\parallel f^{\star })\hfill \end{array}$$

(67)

$$\begin{array}{cc}\hfill & \le \sqrt{10}(1+A^2)\sqrt{{\delta }_A}+{\delta }_A\hfill \end{array}$$

(68)

$$\begin{array}{cc}\hfill & \le {\displaystyle \frac{\epsilon }{2}}+{\displaystyle \frac{\epsilon }{2}}=\epsilon ,\hfill \end{array}$$

(69)

where (66) follows from Lemma 1; (67) follows from Lemma 2; and (69) follows from the choice of ${\delta }_A$ in (60).

Therefore, $X_Q$ is feasible in (3) with $\left|\mathsf{supp}\right(X_Q\left)\right|\le m$. This concludes the proof. □

With Theorem 3 at our disposal, we now show the two regimes of Theorem 2.

• Achievability for $K_{{\displaystyle \frac{1}{A^{\beta }}}}\left(A\right)$:

In Theorem 3, let $\epsilon ={\displaystyle \frac{1}{A^{\beta }}}$ and note that for $A\ge 1$

$${\displaystyle \frac{1}{{\delta }_A}}=40A^{2\beta }{(1+A^2)}^2\le 40A^{2\beta }{\left(2A^2\right)}^2=160A^{2\beta +4},$$

(70)

Hence $log(1/{\delta }_A)\le log160+(2\beta +4)logA$. Substituting into the expression for $m_1$ in (58) and using $1/c=4\kappa /log\kappa$

$$\begin{array}{cc}\hfill m_1& \le 3\sqrt{\kappa A}+A\sqrt{{\displaystyle \frac{4\kappa }{log\kappa }}\left(log160+(2\beta +4)logA\right)}+1\hfill \end{array}$$

(71)

$$\begin{array}{cc}\hfill & \le CA\sqrt{logA},\hfill \end{array}$$

(72)

where in the last inequality we have used that $log160+(2\beta +4)logA\le (2\beta +5)logA$ for all $A\ge 160$ and let $C:=4\sqrt{{\displaystyle \frac{4(2\beta +5)\kappa }{log\kappa }}}$ (the last inequality holds because $A\sqrt{logA}$ dominates $\sqrt{A}$ and 1 for $A\ge \mathrm{e}$). The proof is concluded by noting that $CA\sqrt{log\left(A\right)}\le k{A}^2$, which from Theorem 3 implies that $K_{{\displaystyle \frac{1}{A^{\beta }}}}\left(A\right)\le m_1\le CA\sqrt{log\left(A\right)}$.

• Achievability for $K_{{\mathrm{e}}^{-A}}\left(A\right)$:

In Theorem 3 let $\epsilon ={\mathrm{e}}^{-A}$. To bound $log(1/{\delta }_A)$, use $1+A^2\le 2A^2$ for $A\ge 1$:

$${\displaystyle \frac{1}{{\delta }_A}}=40{\mathrm{e}}^{2A}{(1+A^2)}^2\le 40{\mathrm{e}}^{2A}{\left(2A^2\right)}^2=160{\mathrm{e}}^{2A}{A}^4,$$

(73)

Hence $log(1/{\delta }_A)\le log160+4logA+2A$. Substituting into the expression for $m_1$ in (58) and using $1/c=4\kappa /log\kappa$

$$\begin{array}{cc}\hfill m_1& \le 3\sqrt{\kappa A}+A\sqrt{{\displaystyle \frac{4\kappa }{log\kappa }}\left(log160+4logA+2A\right)}+1\hfill \end{array}$$

(74)

$$\begin{array}{cc}\hfill & \le C{A}^{3/2}\hfill \end{array}$$

(75)

where in the last inequality we have used that $log160+4logA+2A\le 5logA+2A$ for all $A\ge 160$ with $C:=2\sqrt{{\displaystyle \frac{2\kappa }{log\kappa }}}$.

3.4. Converse

We now show the converse bound.

Theorem 4. 

Let $A\ge 1$ and let $0<\epsilon \le 1/A$. Then

$$K_{ϵ}\left(A\right)\ge {\displaystyle \frac{A}{2\pi }}\sqrt{{log}^{+}\left({\displaystyle \frac{c_L}{\sqrt{\epsilon /2}+\sqrt{b_0/\left(2A\right)}}}\right)},$$

(76)

where $b_0=\sqrt{{\displaystyle \frac{\pi \mathrm{e}}{2}}}$, $M_0$ is defined in (36) and

$$c_L:={\displaystyle \frac{1}{8(2\pi M_0+1)}}.$$

(77)

Proof. 

Assume X is such that $C\left(A\right)-I(X;Y)\le ϵ$, which by using [30] implies that

$$\mathsf{D}(P_Y\parallel P_{Y^{\star }})\le C\left(A\right)-I(X;Y)\le ϵ.$$

(78)

We also need the following bound [8]:

$$\mathsf{D}(P_{{\langle U+Z\rangle }_A}\parallel P_{{\langle Y^{\star }\rangle }_A})\le {\displaystyle \frac{b_0}{A}}$$

(79)

Now,

$$\begin{array}{cc}\hfill {\displaystyle \frac{C_0}{2}}exp\left(-4{\pi }^2{\displaystyle \frac{K^2}{A^2}}\right)& \le C_0{\chi }^2\left(P_{{\langle X+Z\rangle }_A}\parallel P_{{\langle U+Z\rangle }_A}\right)\hfill \end{array}$$

(80)

$$\begin{array}{cc}\hfill & \le \mathsf{TV}(P_{{\langle X+Z\rangle }_A},P_{{\langle U+Z\rangle }_A})\hfill \end{array}$$

(81)

$$\begin{array}{cc}\hfill & \le \mathsf{TV}(P_{{\langle X+Z\rangle }_A},P_{{\langle Y^{\star }\rangle }_A})+\mathsf{TV}(P_{{\langle U+Z\rangle }_A},P_{{\langle Y^{\star }\rangle }_A})\hfill \end{array}$$

(82)

$$\begin{array}{cc}\hfill & \le \sqrt{{\displaystyle \frac{1}{2}}\mathsf{D}(P_{{\langle X+Z\rangle }_A}\parallel P_{{\langle Y^{\star }\rangle }_A})}+\sqrt{{\displaystyle \frac{1}{2}}\mathsf{D}(P_{{\langle U+Z\rangle }_A}\parallel P_{{\langle Y^{\star }\rangle }_A})}\hfill \end{array}$$

(83)

$$\begin{array}{cc}\hfill & \le \sqrt{{\displaystyle \frac{1}{2}}\mathsf{D}(P_Y\parallel P_{Y^{\star }})}+\sqrt{{\displaystyle \frac{1}{2}}\mathsf{D}(P_{U+Z}\parallel P_{Y^{\star }})}\hfill \end{array}$$

(84)

$$\begin{array}{cc}\hfill & \le \sqrt{{\displaystyle \frac{ϵ}{2}}}+\sqrt{{\displaystyle \frac{b_0}{2A}}},\hfill \end{array}$$

(85)

where (80) and (81) follow from Proposition 1; (82) follows from the triangular inequality; (83) follows from Pinsker’s inequality; (84) follows from the data processing inequality; and the first bound in (85) follows from (78) and the second bound follows from (79). Rearrange and take logarithms of (85) (using ${log}^{+}\left(x\right):=max\{logx,0\}$):

$$4{\pi }^2{\displaystyle \frac{K^2}{A^2}}\ge {log}^{+}\left({\displaystyle \frac{c_L}{\sqrt{\epsilon /2}+\sqrt{b_0/\left(2A\right)}}}\right).$$

(86)

By rearranging the terms, we conclude the proof. □

As a consequence of Theorem 2 note that since $\epsilon \le 1/A$, $\sqrt{\epsilon /2}\le 1/\sqrt{2A}$ and hence $\sqrt{\epsilon /2}+\sqrt{b_0/\left(2A\right)}\le \sqrt{(1+b_0)/\left(2A\right)}$. Plugging this bound into (86) yields

$$K\ge {\displaystyle \frac{A}{2\pi }}\sqrt{{log}^{+}\left(c_L\sqrt{{\displaystyle \frac{2A}{1+b_0}}}\right)}={\displaystyle \frac{A}{2\sqrt{2}\pi }}\sqrt{{log}^{+}\left({\displaystyle \frac{2c_L^2}{1+b_0}}A\right)},$$

which gives the explicit scaling form

$$K\ge {\displaystyle \frac{A}{2\sqrt{2}\pi }}\sqrt{{log}^{+}\left(cA\right)},c:={\displaystyle \frac{2c_L^2}{1+b_0}}.$$

(87)

Consequently, for all A large enough (so that $cA>1$),

$$K_{\epsilon }\left(A\right)\ge {\displaystyle \frac{A}{2\sqrt{2}\pi }}\sqrt{log\left(cA\right)}.$$

4. Conclusions

In this work, we studied the amplitude-constrained AWGN channel from the perspective of near-optimal input distributions. Rather than focusing on the exact capacity-achieving input, whose support size remains poorly understood, we introduced the quantity $K_{\epsilon }\left(A\right)$, which captures the minimal support size required to achieve capacity up to an $\epsilon$-gap.

We showed that this relaxed formulation is significantly more tractable and admits sharp characterizations across different regimes. In particular, for polynomially decaying gaps, we established that $K_{\epsilon }\left(A\right)=\Theta \left(A\sqrt{logA}\right)$, while for exponentially small gaps, the required support size increases to at most order $A^{3/2}$.

Beyond the technical results, our approach provides a conceptual explanation for the variety of scaling laws observed in prior numerical studies. Namely, different empirical scalings can be interpreted as arising from different implicit choices of $\epsilon$.

Several open problems remain. In particular, it would be of interest to obtain tighter bounds in the exponential regime, as well as to better understand the behavior of the exact optimizer and its relation to $K_{\epsilon }\left(A\right)$ as $\epsilon \to 0$. More broadly, the $\epsilon$-capacity perspective may prove useful in other settings where exact structural characterization is difficult but near-optimal behavior is more accessible.