The Listsize Capacity of the Gaussian Channel with Decoder Assistance

Abstract

The listsize capacity is computed for the Gaussian channel with a helper that—cognizant of the channel-noise sequence but not of the transmitted message—provides the decoder with a rate-limited description of said sequence. This capacity is shown to equal the sum of the cutoff rate of the Gaussian channel without help and the rate of help. In particular, zero-rate help raises the listsize capacity from zero to the cutoff rate. This is achieved by having the helper provide the decoder with a sufficiently fine quantization of the normalized squared Euclidean norm of the noise sequence.

1. Introduction

The order-$\rho$ listsize capacity $C_{\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{t}}^{\left(\rho \right)}$ of a channel is the supremum of the coding rates for which there exist codes guaranteeing the large-blocklength convergence to one of the $\rho$-th moment of the cardinality of the list of messages that, given the received output sequence, have positive a posteriori probability. It is zero for the Gaussian channel because, on this channel, no codeword is ruled out by any received sequence so said list contains all the messages. Here we derive this capacity for the Gaussian channel with a helper that observes the noise sequence and describes it to the decoder using a rate-limited noise-free bit pipe; see Figure 1.

We show that the listsize capacity $C_{\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{t}}^{\left(\rho \right)}\left(R_{\mathrm{h}}\right)$ is then the sum of bit-pipe’s rate $R_{\mathrm{h}}$ and the order-$\rho$ cutoff rate $R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)$ of the Gaussian channel without a helper

$$\begin{array}{ccc}\hfill C_{\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{t}}^{\left(\rho \right)}\left(R_{\mathrm{h}}\right)& =\hfill & R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)+R_{\mathrm{h}}.\end{array}$$

(1)

The latter’s definition is similar to that of the listsize capacity, but with the list now comprising only those messages that are a posteriori at least as likely as the transmitted one. As we shall see, for the Gaussian channel with average power $\mathsf{P}$, noise-variance $\mathsf{N}$, and corresponding signal-to-noise ratio (SNR) $\mathsf{A}\triangleq \mathsf{P}/\mathsf{N}$,

$$\begin{array}{ccc}\hfill R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)& =\hfill & R_0\left(\rho \right),\end{array}$$

(2)

where

$$\begin{array}{ccc}\hfill R_0\left(\rho \right)& =\hfill & \frac{1}{2}\ln \frac{1}{2}\left(1+\frac{\mathsf{A}}{1+\rho }+\sqrt{{\left(1-\frac{\mathsf{A}}{1+\rho }\right)}^2+\frac{4\mathsf{A}}{{(1+\rho )}^2}}\right)\\ & & +\frac{1+\rho }{2\rho }\frac{1}{2}\left(1+\frac{\mathsf{A}}{1+\rho }-\sqrt{{\left(1-\frac{\mathsf{A}}{1+\rho }\right)}^2+\frac{4\mathsf{A}}{{(1+\rho )}^2}}\right)\\ & & +\frac{1}{2\rho }\ln \frac{1}{2}\left(1-\frac{\mathsf{A}}{1+\rho }+\sqrt{{\left(1-\frac{\mathsf{A}}{1+\rho }\right)}^2+\frac{4\mathsf{A}}{{(1+\rho )}^2}}\right)\end{array}$$

(3)

(in nats) is a function that plays a prominent role in the analysis of the Reliability Function of said channel (Section 7.4 in [1]), [2]. That analysis does not, however, carry over directly to our setting because it deals with error exponents and not lists.

It is interesting to note that (1) also holds when the help rate $R_{\mathrm{h}}$ is zero: the number of help bits required to increase the listsize capacity from zero to $R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)$ is sublinear in the blocklength. In fact, as we shall see, all it takes is a sufficiently fine quantization of the normalized squared Euclidean norm of the noise sequence.

The relation (1) is reminiscent of the analogous result on the erasures-only capacity $C_{\text{e-o}}\left(R_{\mathrm{h}}\right)$ of the Gaussian channel with a rate-$R_{\mathrm{h}}$ helper (Remark 10 in [3]), namely, that

$$\begin{array}{ccc}\hfill C_{\text{e-o}}\left(R_{\mathrm{h}}\right)& =\hfill & C+R_{\mathrm{h}},\end{array}$$

(4)

where C denotes the Shannon capacity of the Gaussian channel (without help) (Theorem 9.1.1 in [4]), and $C_{\text{e-o}}\left(R_{\mathrm{h}}\right)$ is the erasures-only capacity, which is defined like $C_{\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{t}}^{\left(\rho \right)}\left(R_{\mathrm{h}}\right)$ but with the requirement on the $\rho$-th moment of the list replaced by the requirement that the list be of size 1 with probability tending to one. (The Gaussian erasures-only capacity with a helper is given by the RHS of (4) irrespective of whether the assistance is provided to the encoder or decoder.) The latter result in turn is reminiscent of the analogous result on the Shannon capacity with a helper $C\left(R_{\mathrm{h}}\right)$ [5,6,7,8]

$$\begin{array}{ccc}\hfill C\left(R_{\mathrm{h}}\right)& =\hfill & C+R_{\mathrm{h}}.\end{array}$$

(5)

In proving (1), we shall focus on the “direct part,” i.e., that the right-hand side (RHS) of (1) is achievable. The “converse,” that no rate exceeding the RHS of (1) is achievable, is omitted because it follows directly from (Remark 4 in [3]): There it is shown that this is true even if, given the received sequence and the provided help, the list contains only a subset of the messages that are of positive a posteriori probability, namely, those that are a posteriori at least as likely as the transmitted message.

The listsize capacity is relevant, for example, when the message set corresponds to tasks [9] and the transmitted message corresponds to one that must be performed by the decoder with absolute certainty. To ensure this, the decoder must perform all the tasks in the list of tasks that are not ruled out by the received sequence. (In addition to the transmitted task, other tasks need not but may be performed.) The $\rho$-th moment of the list’s size then measures the receiver’s average effort.

Results on the listsize capacity and the erasures-only capacity of general discrete memoryless channels (DMCs) in the absence of help are scarce. Noteworthy exceptions are the results of Pinsker and Sheverdjaev [10], Csiszár and Narayan [11], and Telatar [12], that provide sufficient conditions for the erasures-only capacity to equal the Shannon capacity and for the listsize capacity to equal the cutoff rate. Asymptotic results on the erasures-only capacity in the low-noise regime can be found in [13,14]. Once noiseless feedback is introduced, the problems become more tractable [15,16,17].

The rest of the paper is organized as follows. Section 2 describes our set-up and presents the main result. Section 3 contains some classical and some new observations regarding Gallager’s $E_0$ function and its modification. Section 4 derives the cutoff rate of the Gaussian channel without help and proves (2). Section 5 describes and analyzes a coding scheme that proves the direct part of (1).

2. The Main Result

A power-$\mathsf{P}$ blocklength-n encoder $f^{\left(n\right)}$ for a message set $\mathcal{M}$ is a mapping

$$f^{\left(n\right)}:\mathcal{M}\to {\mathbb{R}}^n$$

(6)

that maps each message $m\in \mathcal{M}$ to an n-tuple $f^{\left(n\right)}\left(m\right)$ whose Euclidean norm $\parallel f^{\left(n\right)}\left(m\right)\parallel$ satisfies

$$\parallel f^{\left(n\right)}{\left(m\right)\parallel }^2\le n\mathsf{P},m\in \mathcal{M}.$$

(7)

We sometimes use ${\mathbf{x}}_m$ to denote $f^{\left(n\right)}\left(m\right)$, and $x_{m,k}$ to denote the k-th component of ${\mathbf{x}}_m$, so

$$f^{\left(n\right)}\left(m\right)={\mathbf{x}}_m=(x_{m,1},\dots ,x_{m,n}).$$

(8)

The encoder is said to be of rate R if the cardinality of $\mathcal{M}$ is $e^{nR}$, in which case we often assume that $\mathcal{M}=\{1,\cdots ,e^{nR}\}$. (We ignore the fact that $e^{nR}$ need not be an integer; this issue washes out in the large-n asymptotics we study.)

When a message $m\in \mathcal{M}$ is sent over the discrete-time additive Gaussian noise channel using the encoder $f^{\left(n\right)}$, the channel produces the random vector $\mathbf{Y}\in {\mathbb{R}}^n$ whose k-th component $Y_k$ is

$$Y_k=x_{m,k}+Z_k,k=1,\dots ,n,$$

(9)

where $\left\{Z_k\right\}$ are independent and identically distributed (IID) zero-mean Gaussians of variance $\mathsf{N}$. We assume that $\mathsf{N}$ is positive and use $w\left(y\right|x)$ to denote the density of the channel’s output when its input is x, i.e., the mean-x variance-$\mathsf{N}$ Gaussian density

$$w\left(y\right|x)=\frac{1}{\sqrt{2\pi \mathsf{N}}}{e}^{-\frac{{(y-x)}^2}{2\mathsf{N}}},x,y\in \mathbb{R},$$

(10)

which we extend to n-tuples in a memoryless fashion:

$$w\left(\mathbf{y}\right|\mathbf{x})={\displaystyle \prod _{k=1}^n}w\left(y_k|x_k\right),\mathbf{x},\mathbf{y}\in {\mathbb{R}}^n.$$

(11)

For convenience, we define

$$\mathsf{A}=\frac{\mathsf{P}}{\mathsf{N}}.$$

(12)

Given an output sequence $\mathbf{y}$ and a message m, we define the “at-least-as-likely list”

$$\mathcal{L}(m,\mathbf{y})=\left\{m^{\prime }\in \mathcal{M}:w\left(\mathbf{y}\right|{\mathbf{x}}_{m^{\prime }})\ge w\left(\mathbf{y}\right|{\mathbf{x}}_m)\right\}.$$

(13)

Assuming, as we do, that the messages are a priori equally likely, this list comprises the messages that, given the output sequence $\mathbf{y}$, are a posteriori at least as likely as m.

If a message M, drawn equiprobably from $\mathcal{M}$, is transmitted over the channel with a resulting received sequence $\mathbf{Y}$, then the cardinality of the at-least-as-likely list is a random positive integer, and we denote its $\rho$-th moment $\mathbb{E}\left[{\left|\mathcal{L}(M,\mathbf{Y})\right|}^{\rho }\right]$:

$$\mathbb{E}\left[{\left|\mathcal{L}(M,\mathbf{Y})\right|}^{\rho }\right]=\frac{1}{\left|\mathcal{M}\right|}\sum _{m\in \mathcal{M}}\int w\left(\mathbf{y}\right|{\mathbf{x}}_m{\left)\right|\mathcal{L}(m,\mathbf{y})|}^{\rho }\mathrm{d}\nu \left(\mathbf{y}\right),$$

(14)

where $\nu (·)$ denotes the Lebesgue measure on ${\mathbb{R}}^n$.

For a given $\rho >0$, we define the order-$\rho$ cutoff rate $R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)$ as the supremum of the rates R for which there exists a sequence of rate-R power-$\mathsf{P}$ blocklength-n encoders $\left\{f^{\left(n\right)}\right\}$ satisfying

$$\underset{n\to \infty }{lim}\mathbb{E}\left[{\left|\mathcal{L}(M,\mathbf{Y})\right|}^{\rho }\right]=1.$$

(15)

Theorem 1.

The order-ρ cutoff rate $R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)$ of the additive Gaussian noise channel equals $R_0\left(\rho \right)$ of (3).

Proof. 

See Section 4. □

A ${\mathcal{T}}_n$-valued description of the noise sequence $\mathbf{Z}=(Z_1,\dots ,Z_n)$ is a mapping

$${\varphi }^{\left(n\right)}:{\mathbb{R}}^n\to {\mathcal{T}}_n$$

(16)

with the understanding that ${\varphi }^{\left(n\right)}\left(\mathbf{Z}\right)$, which we denote T, is the description of $\mathbf{Z}$. We say that a sequence $\left\{{\varphi }^{\left(n\right)}\right\}$ of descriptions is of rate $R_{\mathrm{h}}$ (nats) if

$$\underset{n\to \infty }{lim}\frac{1}{n}\ln \left|{\mathcal{T}}_n\right|=R_{\mathrm{h}}.$$

(17)

Suppose now that, in addition to the received sequence $\mathbf{Y}$, the receiver is also presented with the description $T={\varphi }^{\left(n\right)}\left(\mathbf{Z}\right)$ of the noise, and that, based on the two, it forms the “remotely-plausible list” $\mathcal{L}(\mathbf{Y},T)$ comprising the messages that have positive a posteriori probability given the two:

$$\mathcal{L}(\mathbf{y},t)=\left\{m\in \mathcal{M}:{\varphi }^{\left(n\right)}(\mathbf{y}-{\mathbf{x}}_m)=t\right\}.$$

(18)

Given $\rho >0$, the listsize capacity $C_{\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{t}}^{\left(\rho \right)}\left(R_{\mathrm{h}}\right)$ with rate-$R_{\mathrm{h}}$ decoder assistance is the supremum of the rates R for which there exists a sequence of rate-R power-$\mathsf{P}$ blocklength-n encoders $\left\{f^{\left(n\right)}\right\}$ and a sequence $\left\{{\varphi }^{\left(n\right)}\right\}$ of descriptions of rate $R_{\mathrm{h}}$ such that

$$\underset{n\to \infty }{lim}\mathbb{E}\left[{\left|\mathcal{L}\left(\mathbf{Y},{\varphi }^{\left(n\right)}\left(\mathbf{Z}\right)\right)\right|}^{\rho }\right]=1.$$

(19)

Theorem 2.

On the Gaussian channel, the listsize capacity with rate-$R_{\mathrm{h}}$ decoder assistance $C_{\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{t}}^{\left(\rho \right)}\left(R_{\mathrm{h}}\right)$ is given by

$$\begin{array}{ccc}\hfill C_{\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{t}}^{\left(\rho \right)}\left(R_{\mathrm{h}}\right)& =& R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)+R_{\mathrm{h}}\hfill \end{array}$$

(20)

where $R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)$ is the order-ρ cutoff rate of the channel (without assistance) as given in (2) and (3).

Proof. 

The “converse,” that (19) cannot be achieved when the rate exceeds the RHS of (20), follows from (Remark 4 in [3]). The “direct part,” describing a coding scheme that achieves (19) with rates approaching the RHS of (20), is proved in Section 5. □

3. Preliminaries

Given $\rho \ge 0$ and any probability measure Q on $\mathbb{R}$, Gallager’s $E_0$ function for our channel is defined as [1]

$$E_0(\rho ,Q)=-\ln {\int }_{y\in \mathbb{R}}{\left({\int }_{x\in \mathbb{R}}w{\left(y\right|x)}^{\frac{1}{1+\rho }}\mathrm{d}Q\left(x\right)\right)}^{1+\rho }\mathrm{d}\nu \left(y\right),$$

(21)

where $\nu (·)$ is now the Lebesgue measure on $\mathbb{R}$. The result of maximizing $E_0(\rho ,Q)$ over all Q under which $\mathbb{E}\left[X^2\right]\le \mathsf{P}$, is denoted $E_0^{*}\left(\rho \right)$:

$$\begin{array}{ccc}\hfill E_0^{*}\left(\rho \right)& =\hfill & \underset{Q:\int x^2\mathrm{d}Q\left(x\right)\le \mathsf{P}}{\sup }{E}_0(\rho ,Q).\end{array}$$

(22)

The multi-letter extension of $E_0$ is

$$E_0^{\left(n\right)}\left(\rho ,Q^{\left(n\right)}\right)=-\frac{1}{n}\ln {\int }_{\mathbf{y}\in {\mathbb{R}}^n}{\left({\int }_{\mathbf{x}\in {\mathbb{R}}^n}w{\left(\mathbf{y}\right|\mathbf{x})}^{\frac{1}{1+\rho }}\mathrm{d}{Q}^{\left(n\right)}\left(\mathbf{x}\right)\right)}^{1+\rho }\mathrm{d}\nu \left(\mathbf{y}\right),$$

(23)

where $Q^{\left(n\right)}$ is a probability measure on ${\mathbb{R}}^n$; the integrals are over ${\mathbb{R}}^n$; the channel $w\left(\mathbf{y}\right|\mathbf{x})$ is defined in (11). Similarly,

$$\begin{array}{ccc}\hfill E_0^{\left(n\right),*}\left[n\right]\left(\rho \right)& =\hfill & \underset{Q^{\left(n\right)}:\int {\parallel \mathbf{x}\parallel }^2\mathrm{d}{Q}^{\left(n\right)}\left(\mathbf{x}\right)\le n\mathsf{P}}{\sup }{E}_0^{\left(n\right)}(\rho ,Q^{\left(n\right)}).\end{array}$$

(24)

Given probability measures $Q^{\left(m\right)}$ on ${\mathbb{R}}^m$ and $Q^{\left(n\right)}$ on ${\mathbb{R}}^n$ that satisfy the power constraints $\mathbb{E}[\parallel \mathbf{X}{\parallel }^2]\le m\mathsf{P}$ and $\mathbb{E}[\parallel \mathbf{X}{\parallel }^2]\le n\mathsf{P}$ respectively, the product measure $Q^{\left(m\right)}\times Q^{\left(n\right)}$ on ${\mathbb{R}}^{m+n}$ satisfies the power constraint $\mathbb{E}[\parallel \mathbf{X}{\parallel }^2]\le (m+n)\mathsf{P}$ and

$$(m+n)E_0^{(m+n)}\left(\rho ,Q^{\left(m\right)}\times Q^{\left(n\right)}\right)=m{E}_0^{\left(m\right)}\left(\rho ,Q^{\left(m\right)}\right)+n{E}_0^{\left(n\right)}\left(\rho ,Q^{\left(n\right)}\right)$$

(25)

because

$$\begin{array}{l}(m+n)E_0^{(m+n)}\left(\rho ,Q^{\left(m\right)}\times Q^{\left(n\right)}\right)\\ (26)& =-\ln {\int }_{\mathbf{y}\in {\mathbb{R}}^{m+n}}{\left({\int }_{\mathbf{x}\in {\mathbb{R}}^{m+n}}w{\left(\mathbf{y}\right|\mathbf{x})}^{\frac{1}{1+\rho }}\mathrm{d}\left(Q^{\left(m\right)}\times Q^{\left(n\right)}\right)\left(\mathbf{x}\right)\right)}^{1+\rho }\mathrm{d}\nu \left(\mathbf{y}\right)(27)& =m{E}_0^{\left(m\right)}(\rho ,Q^{\left(m\right)})+n{E}_0^{\left(n\right)}(\rho ,Q^{\left(n\right)}).\end{array}$$

The sequence $\left\{n{E}_0^{\left(n\right),*}\left(\rho \right)\right\}$ is thus superadditive, and Feket’s Subadditive lemma implies that $\left\{E_0^{\left(n\right),*}\left(\rho \right)\right\}$ converges to its supremum:

$$\underset{n\to \infty }{lim}{E}_0^{\left(n\right),*}\left(\rho \right)=\underset{n}{\sup }{E}_0^{\left(n\right),*}\left(\rho \right).$$

(28)

We shall later see (cf. (55) ahead) that

$$\frac{1}{\rho }\underset{n}{\sup }{E}_0^{\left(n\right),*}\left(\rho \right)=R_0\left(\rho \right),$$

(29)

where $R_0\left(\rho \right)$ is defined in (3).

We shall also need Gallager’s modified $E_0$ function. To highlight its relation to the unmodified function, which is quite general, we shall use $g\left(x\right)$ for $x^2$ and $g\left(\mathbf{x}\right)$ for ${\parallel \mathbf{x}\parallel }^2$. We shall also replace $\mathsf{P}$ with $\mathrm{\Gamma }$.

Given some $\rho \ge 0$, some probability distribution Q on $\mathbb{R}$ under which $\mathbb{E}\left[g\right(X\left)\right]\le \mathrm{\Gamma }$, and some $r\ge 0$, the modified Gallager’s $E_0$ function $E_{0,\mathrm{m}}(\rho ,Q,r)$ is defined as

$$E_{0,\mathrm{m}}(\rho ,Q,r)=-\ln {\int }_{y\in \mathbb{R}}{\left({\int }_{x\in \mathbb{R}}{e}^{r\left(g\right(x)-\mathrm{\Gamma })}w{\left(y\right|x)}^{\frac{1}{1+\rho }}\mathrm{d}Q\left(x\right)\right)}^{1+\rho }\mathrm{d}\nu \left(y\right).$$

(30)

We shall also be interested in the maximum of $E_{0,\mathrm{m}}(\rho ,Q,r)$ over both Q and r. We distinguish between two cases depending on whether $\mathbb{E}\left[g\right(X\left)\right]\le \mathrm{\Gamma }$ holds strictly or not. In the former case we only allow r to be zero, whereas in the latter case it can be any non-negative number. We thus define

$$E_{0,\mathrm{m}}^{*}(\rho ,Q)=\left\{\begin{array}{cc}{\sup }_{r\ge 0}{E}_{0,\mathrm{m}}(\rho ,Q,r),\hfill & \mathrm{if}\int g\left(x\right)\mathrm{d}Q\left(x\right)=\mathrm{\Gamma },\hfill \\ E_0(\rho ,Q),\hfill & \mathrm{if}\int g\left(x\right)\mathrm{d}Q\left(x\right)<\mathrm{\Gamma },\hfill \end{array}\right.$$

(31)

and

$$E_{0,\mathrm{m}}^{**}\left(\rho \right)=\underset{Q:\int g\left(x\right)\mathrm{d}Q\left(x\right)\le \mathrm{\Gamma }}{\sup }{E}_{0,\mathrm{m}}^{*}(\rho ,Q).$$

(32)

The next proposition provides a lower bound on $lim{E}_0^{\left(n\right),*}\left(\rho \right)$.

Proposition 1.

Any probability distribution Q on $\mathbb{R}$ under which $g\left(X\right)$ is of finite second moment and of expectation Γ provides the lower bound

$$\underset{n\to \infty }{lim}{E}_0^{\left(n\right),*}\left(\rho \right)\ge E_{0,\mathrm{m}}^{*}(\rho ,Q).$$

(33)

Proof. 

Let Q be any input distributions Q under which $g\left(X\right)$ is of finite second moment and $\mathbb{E}\left[g\right(X\left)\right]=\mathrm{\Gamma }$. For each $n\in \mathbb{N}$, let $Q^{\left(n\right)}$ be the conditional distribution of the n-fold product distribution $Q^{\times n}$ given the event $\{\mathbf{X}\in {\mathcal{A}}_n\}$, where

$${\mathcal{A}}_n=\left\{\mathbf{x}\in {\mathbb{R}}^n:n\mathrm{\Gamma }-\delta <g\left(\mathbf{x}\right)\le n\mathrm{\Gamma }\right\}$$

(34)

where $\delta >0$ is some positive constant. Thus, for every Borel measurable subset $\mathcal{B}$ of ${\mathbb{R}}^n$,

$$Q^{\left(n\right)}\left(\mathcal{B}\right)=\frac{1}{\mu }{Q}^{\times n}(\mathcal{B}\cap {\mathcal{A}}_n),\mathcal{B}\in \mathcal{B}\left({\mathbb{R}}^n\right)$$

(35)

with

$$\begin{array}{c}\hfill \mu =Q^{\times n}\left({\mathcal{A}}_n\right).\end{array}$$

(36)

For any $r\ge 0$, we can upper-bound the Radon–Nykodim derivative of $Q^{\left(n\right)}$ with respect to product distribution $Q^{\times n}$ as follows:

$$\begin{array}{}(37)& \frac{\mathrm{d}{Q}^{\left(n\right)}}{\mathrm{d}{Q}^{\times n}}& =\frac{1}{\mu }\mathrm{I}\{\mathbf{x}\in {\mathcal{A}}_n\}\hfill (38)& & \le \frac{1}{\mu }{e}^{r\left(g\right(\mathbf{x})-n\mathrm{\Gamma }+\delta )}\hfill (39)& & =\frac{1}{\mu }{e}^{r\delta }{e}^{r\left(g\right(\mathbf{x})-n\mathrm{\Gamma })}\hfill \end{array}$$

where $\mathrm{I}\left\{\mathrm{statement}\right\}$ equals 1 if the statement is true and else 0. Using this bound on the Radon–Nykodim derivative we obtain:

$$\begin{array}{}(40)& E_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})& =\hfill & -\frac{1}{n}\ln {\int }_{\mathbf{y}\in {\mathbb{R}}^n}{\left({\int }_{\mathbf{x}\in {\mathbb{R}}^n}w{\left(\mathbf{y}\right|\mathbf{x})}^{\frac{1}{1+\rho }}d{Q}^{\left(n\right)}\left(\mathbf{x}\right)\right)}^{1+\rho }\mathrm{d}\nu \left(\mathbf{y}\right)(41)& & \ge \hfill & -\frac{1}{n}\ln {\int }_{\mathbf{y}\in {\mathbb{R}}^n}{\left({\int }_{\mathbf{x}\in {\mathbb{R}}^n}w{\left(\mathbf{y}\right|\mathbf{x})}^{\frac{1}{1+\rho }}·\frac{1}{\mu }{e}^{r\delta }{e}^{r\left(g\right(\mathbf{x})-n\mathrm{\Gamma })}\mathrm{d}{Q}^{\times n}\left(\mathbf{x}\right)\right)}^{1+\rho }\mathrm{d}\nu \left(\mathbf{y}\right)(42)& & =\hfill & -\frac{1+\rho }{n}\ln \frac{e^{r\delta }}{\mu }-\ln {\int }_{y\in \mathbb{R}}{\left({\int }_{x\in \mathbb{R}}{e}^{r\left(g\right(x)-\mathrm{\Gamma })}w{\left(y\right|x)}^{\frac{1}{1+\rho }}dQ\left(x\right)\right)}^{1+\rho }\mathrm{d}\nu \left(y\right)(43)& & =\hfill & -\frac{1+\rho }{n}\ln \frac{e^{r\delta }}{\mu }+E_{0,\mathrm{m}}(\rho ,Q,r).\end{array}$$

By the Central Limit Theorem, $\mu$ tends to 1/2 as n tends to infinity, so (43) implies that

$$\begin{array}{c}\hfill {\displaystyle \underset{n\to \infty }{lim\; inf}}\frac{1}{n}{E}_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})\ge E_{0,\mathrm{m}}(\rho ,Q,r).\end{array}$$

(44)

Taking the supremum of the RHS over all $r\ge 0$, establishes that

$$\begin{array}{c}\hfill {\displaystyle \underset{n\to \infty }{lim\; inf}}\frac{1}{n}{E}_0^{\left(n\right)}\left(\rho ,Q^{\left(n\right)}\right)\ge E_{0,\mathrm{m}}^{*}(\rho ,Q)\end{array}$$

(45)

and hence, by (24), proves (33). □

We next turn to upper-bounding $lim{E}_0^{\left(n\right),*}\left(\rho \right)$.

Proposition 2.

If the probability distribution $Q^{\left(n\right)}$ on ${\mathbb{R}}^n$ is such that $\mathbb{E}\left[g\right(\mathbf{X}\left)\right]\le n\mathrm{\Gamma }$, and if $f_{\mathrm{R}}$ is any density on $\mathbb{R}$, then

$$\begin{array}{ccc}\hfill E_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})& \le & \underset{P:\int g\left(x\right)\mathrm{d}P\left(x\right)\le \mathrm{\Gamma }}{\sup }-(1+\rho ){\int }_{x\in \mathbb{R}}\ln \left({\int }_{y\in \mathbb{R}}w{\left(y\right|x)}^{\frac{1}{1+\rho }}{f}_R{\left(y\right)}^{\frac{\rho }{1+\rho }}\mathrm{d}y\right)\mathrm{d}P\left(x\right)\hfill \end{array}$$

(46)

and, consequently,

$$\begin{array}{ccc}\hfill \underset{n\to \infty }{lim}{E}_0^{\left(n\right),*}\left(\rho \right)& \le & \underset{P:\int g\left(x\right)\mathrm{d}P\left(x\right)\le \mathrm{\Gamma }}{\sup }-(1+\rho ){\int }_{x\in \mathbb{R}}\ln \left({\int }_{y\in \mathbb{R}}w{\left(y\right|x)}^{\frac{1}{1+\rho }}{f}_R{\left(y\right)}^{\frac{\rho }{1+\rho }}\mathrm{d}y\right)\mathrm{d}P\left(x\right).\hfill \end{array}$$

(47)

Proof. 

The proof is based on Proposition 2 in [18], which implies that for every density $f_{\mathrm{R}}^{\left(n\right)}$ on ${\mathbb{R}}^n$ and any probability measure $Q^{\left(n\right)}$ on ${\mathbb{R}}^n$,

$$n{E}_0^{\left(n\right)}\left(\rho ,Q^{\left(n\right)}\right)\le -(1+\rho ){\int }_{\mathbf{x}\in {\mathbb{R}}^n}\ln \left({\int }_{\mathbf{y}\in {\mathbb{R}}^n}w{\left(\mathbf{y}\right|\mathbf{x})}^{\frac{1}{1+\rho }}{f}_R^{\left(n\right)}{\left(\mathbf{y}\right)}^{\frac{\rho }{1+\rho }}\mathrm{d}\mathbf{y}\right)\mathrm{d}{Q}^{\left(n\right)}\left(\mathbf{x}\right).$$

(48)

Applying this inequality to the product density

$$f_{\mathrm{R}}^{\left(n\right)}\left(\mathbf{y}\right)={\displaystyle \prod _{i=1}^n}{f}_{\mathrm{R}}\left(y_i\right),$$

(49)

where $f_{\mathrm{R}}$ is a density on $\mathbb{R}$, and using the product form of the channel (11), we obtain that for any density $f_{\mathrm{R}}$ on $\mathbb{R}$

$$\begin{array}{}(50)& E_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})& \le \hfill & -\frac{1}{n}(1+\rho ){\displaystyle \sum _{i=1}^n}{\int }_{x_i\in \mathbb{R}}\ln \left({\int }_{y\in \mathbb{R}}w{\left(y\right|x_i)}^{\frac{1}{1+\rho }}{f}_R{\left(y\right)}^{\frac{\rho }{1+\rho }}\mathrm{d}y\right)\mathrm{d}{Q}_i^{\left(n\right)}\left(x_i\right)(51)& & =\hfill & -(1+\rho ){\int }_{x\in \mathbb{R}}\ln \left({\int }_{y\in \mathbb{R}}w{\left(y\right|x)}^{\frac{1}{1+\rho }}{f}_R{\left(y\right)}^{\frac{\rho }{1+\rho }}\mathrm{d}y\right)\mathrm{d}\overline{Q}\left(x\right),\end{array}$$

where $Q_i^{\left(n\right)}$ is the i-th marginal of $Q^{\left(n\right)}$, and $\overline{Q}$ is the probability measure on $\mathbb{R}$ defined by

$$\overline{Q}=\frac{1}{n}{\displaystyle \sum _{i=1}^n}{Q}_i^{\left(n\right)}.$$

(52)

Observe that if $\mathbb{E}\left[g\right(\mathbf{X}\left)\right]\le n\mathrm{\Gamma }$ under $Q^{\left(n\right)}$, then $\mathbb{E}\left[g\right(X\left)\right]\le \mathrm{\Gamma }$ under $\overline{Q}$. This observation and (51) establish (46). Since (46) holds for all n, (47) must also hold. □

4. The Cutoff Rate of the Gaussian Channel

In this section, we prove Theorem 1. Since scaling the output does not change the cutoff rate, we will assume WLOG that the noise variance is 1 and the transmit power is $\mathsf{A}$; see (12). Thus,

$$w\left(y\right|x)=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{{(y-x)}^2}{2}},x,y\in \mathbb{R},$$

(53)

and each codeword ${\mathbf{x}}_m$ satisfies

$$\begin{array}{c}\hfill \parallel {\mathbf{x}}_m{\parallel }^2\le n\mathsf{A}.\end{array}$$

(54)

4.1. Computing $lim{E}_0^{\left(n\right),*}\left(\rho \right)$

Here we shall establish that on the Gaussian channel (53)

$$\underset{n\to \infty }{lim}{E}_0^{\left(n\right),*}\left(\rho \right)=\rho R_0\left(\rho \right)=E_{0,\mathrm{m}}^{*}(\rho ,Q_G),$$

(55)

where $R_0\left(\rho \right)$ is defined in (3), and $Q_G$ is the zero-mean variance-$\mathsf{A}$ Gaussian distribution. To this end, we shall derive matching upper and lower bounds on the limit. We begin with the former.

4.1.1. Upper-Bounding $lim{E}_0^{\left(n\right),*}\left(\rho \right)$

We show that on the channel (10)

$$\underset{n\to \infty }{lim}{E}_0^{\left(n\right),*}\left(\rho \right)\le \rho R_0\left(\rho \right).$$

(56)

The proof is based on Proposition 2 with the density $f_R$ corresponding to a centered Gaussian of variance ${\sigma }^2$, where

$$\begin{array}{c}\hfill {\sigma }^2=\frac{\mathsf{A}}{(1+\rho )\beta }+1\end{array}$$

(57)

and

$$\begin{array}{c}\hfill \beta =\frac{1}{2}\left(1-\frac{\mathsf{A}}{1+\rho }+\sqrt{{\left(1-\frac{\mathsf{A}}{1+\rho }\right)}^2+\frac{4\mathsf{A}}{{(1+\rho )}^2}}\right).\end{array}$$

(58)

Evaluating the RHS of (47) for this density, we obtain

$$\begin{array}{}(59)& \hspace{1em}\underset{P:\mathbb{E}\left[X^2\right]\le \mathsf{A}}{\sup }-(1+\rho ){\int }_{x\in \mathbb{R}}\ln {\int }_{y\in \mathbb{R}}w{\left(y\right|x)}^{\frac{1}{1+\rho }}{f}_R{\left(y\right)}^{\frac{\rho }{1+\rho }}dydP\left(x\right)\hfill (60)& =\underset{P:\mathbb{E}\left[X^2\right]\le \mathsf{A}}{\sup }-(1+\rho ){\int }_{x\in \mathbb{R}}\ln {\int }_{y\in \mathbb{R}}\frac{1}{{\left(\sqrt{2\pi }\right)}^{\frac{1}{1+\rho }}}{e}^{-\frac{{(y-x)}^2}{2(1+\rho )}}\frac{1}{{\left(\sqrt{2\pi {\sigma }^2}\right)}^{\frac{\rho }{1+\rho }}}{e}^{-\frac{y^2\rho }{2{\sigma }^2(1+\rho )}}dydP\left(x\right)(61)& =\underset{P:\mathbb{E}\left[X^2\right]\le \mathsf{A}}{\sup }-(1+\rho ){\int }_{x\in \mathbb{R}}\ln \left(\sqrt{\frac{2\pi {(1+\rho )}^2}{\rho }}{\sqrt{{\sigma }^2}}^{\frac{1}{1+\rho }}\right)\frac{1}{\sqrt{2\pi {\sigma }_1^2}}{e}^{-\frac{x^2}{2{\sigma }_1^2}}dP\left(x\right)(62)& =\underset{P:\mathbb{E}\left[X^2\right]\le \mathsf{A}}{\sup }-(1+\rho )\ln \left(\sqrt{\frac{{(1+\rho )}^2}{\rho }}{\sqrt{{\sigma }^2}}^{\frac{1}{1+\rho }}\frac{1}{\sqrt{{\sigma }_1^2}}\right)+(1+\rho ){\int }_{x\in \mathbb{R}}\frac{x^2}{2{\sigma }_1^2}dP\left(x\right)(63)& =-(1+\rho )\ln \left(\sqrt{\frac{{(1+\rho )}^2}{\rho }}{\sqrt{{\sigma }^2}}^{\frac{1}{1+\rho }}\frac{1}{\sqrt{{\sigma }_1^2}}\right)+(1+\rho )\frac{\mathsf{A}}{2{\sigma }_1^2}(64)& =\frac{1+\rho }{2}\frac{\mathsf{A}}{{\sigma }_1^2}+\frac{1+\rho }{2}\ln {\sigma }_1^2-\frac{1}{2}\ln {\sigma }^2-\frac{1+\rho }{2}\ln \frac{{(1+\rho )}^2}{\rho }\end{array}$$

where in (61) we defined

$$\begin{array}{}(65)& \hfill {\sigma }_1^2& \triangleq \hfill & 1+\rho +\frac{{\sigma }^2(1+\rho )}{\rho }(66)& & =\hfill & \frac{\mathsf{A}}{\rho \beta }+\frac{{(1+\rho )}^2}{\rho }.\end{array}$$

To conclude the proof, it remains to show that the RHS of (64) coincides with $\rho R_0\left(\rho \right)$. To this end, observe that some basic algebra reveals that

$$\begin{array}{ccc}\hfill \beta \left(\beta -1+\frac{\mathsf{A}}{1+\rho }\right)& =\hfill & \frac{\mathsf{A}}{{(1+\rho )}^2}\end{array}$$

(67)

and

$$\begin{array}{ccc}\hfill (\beta +\frac{\mathsf{A}}{1+\rho })(1-\beta )& =\hfill & \frac{\mathsf{A}\rho }{{(1+\rho )}^2}.\end{array}$$

(68)

Therefore, the first term in (64) can be rewritten as

$$\begin{array}{}(69)& \frac{1+\rho }{2}\frac{\mathsf{A}}{{\sigma }_1^2}& =\hfill & \frac{1+\rho }{2}\frac{\mathsf{A}}{\frac{\mathsf{A}}{\rho \beta }+\frac{{(1+\rho )}^2}{\rho }}=\frac{1+\rho }{2}\frac{\mathsf{A}\rho }{{(1+\rho )}^2}\frac{\beta }{\frac{\mathsf{A}}{{(1+\rho )}^2}+\beta }(70)& & =\hfill & \frac{\mathsf{A}\rho }{2(1+\rho )}\frac{1}{\beta +\frac{\mathsf{A}}{1+\rho }}=\frac{(1+\rho )(1-\beta )}{2},\end{array}$$

and the remaining terms rewritten as

$$\begin{array}{l}\frac{1+\rho }{2}\ln {\sigma }_1^2-\frac{1}{2}\ln {\sigma }^2-\frac{1+\rho }{2}\ln \frac{{(1+\rho )}^2}{\rho }\\ (71)& \hspace{1em}=\frac{1+\rho }{2}\ln \left(\frac{\mathsf{A}}{\rho \beta }+\frac{{(1+\rho )}^2}{\rho }\right)-\frac{1}{2}\ln \left(\frac{\mathsf{A}}{(1+\rho )\beta }+1\right)-\frac{1+\rho }{2}\ln \frac{{(1+\rho )}^2}{\rho }(72)& \hspace{1em}=\frac{1+\rho }{2}\ln \left(\frac{\mathsf{A}}{{(1+\rho )}^2\beta }+1\right)-\frac{1}{2}\ln \left(\frac{\mathsf{A}}{(1+\rho )\beta }+1\right)(73)& \hspace{1em}=\frac{\rho }{2}\ln \left(\beta +\frac{\mathsf{A}}{1+\rho }\right)+\frac{1}{2}\ln \beta .\end{array}$$

The sum equals to $\rho R_0\left(\rho \right)$.

4.1.2. Lower-Bounding $lim{E}_0^{\left(n\right),*}\left(\rho \right)$

To lower-bound $lim{E}_0^{\left(n\right),*}\left(\rho \right)$, we shall use Proposition 1 with Q chosen as a centered variance-$\mathsf{A}$ Gaussian distribution $Q_G$. For this probability distribution Gallager calculated $E_{0,\mathrm{m}}^{*}(\rho ,Q_G)$ (Section 7.4 in [1]). He showed that for any $\rho >0$,

$$E_{0,\mathrm{m}}^{*}(\rho ,Q_G)=\rho R_0\left(\rho \right),$$

(74)

where $R_0\left(\rho \right)$ is defined in (3). Using this result and Proposition 1 we obtain

$$\begin{array}{}(75)& \hfill {\displaystyle \underset{n\to \infty }{lim}}{E}_0^{\left(n\right),*}\left(\rho \right)& \ge \hfill & E_{0,\mathrm{m}}^{*}(\rho ,Q_G)(76)& & =\hfill & \rho R_0\left(\rho \right).\end{array}$$

4.2. The Mapping $\rho \mapsto R_0\left(\rho \right)$ Is Monotonically Decreasing

For the purpose of proving the achievability of $R_0\left(\rho \right)$, we will need the fact that it is monotonically decreasing in $\rho$. In view of (55), it suffices to show that, for every $n\in \mathbb{N}$, the mapping $\rho \mapsto {\rho }^{-1}{E}_0^{\left(n\right),*}\left(\rho \right)$ is monotonically decreasing. In view of (24), the latter will follow once we establish the monotonicity of $\rho \mapsto {\rho }^{-1}{E}_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})$ for any fixed $Q^{\left(n\right)}$. Since $E_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})$ evaluates to zero at $\rho =0$, this monotonicity can be established by showing that the mapping $\rho \mapsto E_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})$ is concave. This is established in (Appendix 5.B in [1]). (That appendix deals with finite alphabets, but the proof goes through also to our case.)

4.3. Achievability of $R_0\left(\rho \right)$

The achievability of $R_0\left(\rho \right)$ will be proved using a random-coding argument. Let Q be the zero-mean variance-$\mathsf{A}$ Gaussian distribution, let $\delta >0$ be a positive constant, and let $Q^{\left(n\right)}$ be the distribution on ${\mathbb{R}}^n$ defined in (35) and (36). Draw the codewords ${\left\{{\mathbf{X}}_m\right\}}_{m=1,\cdots ,e^{nR}}$ of a blocklength-n random codebook independently, each according to $Q^{\left(n\right)}$, so $\parallel {\mathbf{X}}_m{\parallel }^2\le n\mathsf{A}$ with probability 1 for every $m\in \mathcal{M}$. By symmetry, $\mathbb{E}\left[{\left|\mathcal{L}(m,\mathbf{Y})\right|}^{\rho }\right]$ (where the expectation is over the random choice of codebook and on the channel behavior) does not depend on m. Consequently,

$$\mathbb{E}\left[e^{-nR}\sum _{m\in \mathcal{M}}{\left|\mathcal{L}(m,\mathbf{Y})\right|}^{\rho }\right]=\mathbb{E}\left[{\left|\mathcal{L}(1,\mathbf{Y})\right|}^{\rho }\right],$$

(77)

and if we establish that $\mathbb{E}\left[{\left|\mathcal{L}(1,\mathbf{Y})\right|}^{\rho }\right]$ tends to 1, it will follow by the random-coding argument that there exists a codebook for which the LHS of (77)—with the expectation now over the channel behavior only—tends to 1.

Defining

$$\begin{array}{c}\hfill B_m({\mathbf{x}}_1,\mathbf{y})=\mathrm{𝟙}\left\{w\left(\mathbf{y}\right|{\mathbf{X}}_m)\ge w\left(\mathbf{y}\right|{\mathbf{x}}_1)\right\},{\mathbf{x}}_1,\mathbf{y}\in {\mathbb{R}}^n,\end{array}$$

(78)

we can express the RHS of (77) as

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|\mathcal{L}(1,\mathbf{Y})\right|}^{\rho }\right]& =\hfill & \mathbb{E}\left[{\left(1+{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{X}}_1,\mathbf{Y})\right)}^{\rho }\right],\end{array}$$

(79)

and we seek to show that

$$\begin{array}{c}\hfill {\displaystyle \underset{n\to \infty }{lim}}\mathbb{E}\left[{\left(1+{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{X}}_1,\mathbf{Y})\right)}^{\rho }\right]=1.\end{array}$$

(80)

To this end, we shall need the following lemma.

Lemma 1.

Let $\left\{Z_n\right\}$ be a sequence of random variables taking values in $\mathbb{N}$, and let $\rho >0$ be fixed. The following two conditions are then equivalent:

(i) 

$\mathbb{E}\left[{(1+Z_n)}^{\rho }\right]=1+o\left(1\right)$

(ii) 

$\mathbb{E}\left[Z_n^{\rho }\right]=o\left(1\right)$

where $o\left(1\right)$ tends to zero as n tends to infinity. Thus

$$\begin{array}{c}\hfill \left(\underset{n\to \infty }{lim}\mathbb{E}\left[{(1+Z_n)}^{\rho }\right]=1\right)\iff \left(\underset{n\to \infty }{lim}\mathbb{E}\left[Z_n^{\rho }\right]=0\right).\end{array}$$

(81)

Proof. 

The implication (ii) ⟹ (i) follows by noting for any $z\in \mathbb{N}$ and $\rho >0$

$$\begin{array}{c}\hfill {(1+z)}^{\rho }\le 1+2^{\rho }{z}^{\rho },\end{array}$$

(82)

so

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{(1+Z_n)}^{\rho }\right]& \le \hfill & 1+2^{\rho }\mathbb{E}\left[Z_n^{\rho }\right].\end{array}$$

(83)

As for the implication (i) ⟹ (ii), note that any $y\in \mathbb{N}$ and $\rho >0$

$$\begin{array}{c}\hfill {(1+y)}^{\rho }\ge y^{\rho }+\mathrm{𝟙}\{y=0\},\end{array}$$

(84)

so

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{(1+Z_n)}^{\rho }\right]& \ge \hfill & \mathbb{E}\left[Z_n^{\rho }\right]+\mathrm{Pr}[Z_n=0].\end{array}$$

(85)

The implication is now established by noting that (i) implies that $\mathrm{Pr}[Z_n=0]\to 1$ because, by Markov’s inequality (and the strict positivity of $\rho$),

$$\begin{array}{}(86)& \hfill \mathrm{Pr}[Z_n\ne 0]& =\hfill & \mathrm{Pr}[{(1+Z_n)}^{\rho }-1\ge 2^{\rho }-1](87)& & \le \hfill & \frac{\mathbb{E}\left[{(1+Z_n)}^{\rho }\right]-1}{2^{\rho }-1}.\end{array}$$

□

In light of the above lemma, to establish (80) it suffices to show that

$$\begin{array}{c}\hfill {\displaystyle \underset{n\to \infty }{lim}}\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{X}}_1,\mathbf{Y})\right)}^{\rho }\right]=0,\end{array}$$

(88)

i.e., that

$$\begin{array}{c}\hfill {\displaystyle \underset{n\to \infty }{lim}}\mathbb{E}\left[\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})\right)}^{\rho }|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]\right]=0,\end{array}$$

(89)

where the outer expectation is over ${\mathbf{X}}_1$ and $\mathbf{Y}$.

A related expectation—but one where it is the conditional expectation that is raised to the $\rho$-th power—is studied in the following lemma:

Lemma 2.

If $\rho >0$ and $R<R_0\left(\rho \right)$, then

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\mathbb{E}\left[\mathbb{E}{\left[\sum _{m\ne 1}{B}_m({\mathbf{x}}_1,\mathbf{y})|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]}^{\rho }\right]=0.\end{array}$$

(90)

Proof. 

See Appendix A. □

To establish (88) using this lemma, we distinguish between two cases depending on whether $0<\rho \le 1$ or $\rho >1$. In the former case $x\mapsto x^{\rho }$ is concave, so Jensen’s inequality implies that

$$\begin{array}{c}\hfill \mathbb{E}\left[\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})\right)}^{\rho }|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]\right]\le \mathbb{E}\left[\mathbb{E}{\left[{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]}^{\rho }\right],\end{array}$$

(91)

which, together with Lemma 2, implies (88) whenever $R<R_0\left(\rho \right)$.

Suppose now that $\rho >1$. Conditional on the transmitted codeword ${\mathbf{x}}_1$ and the output $\mathbf{y}$, the random variables ${\left\{B_m\right\}}_{m\ne 1}$ are IID Bernoulli, with $B_m$ determined by ${\mathbf{X}}_m$. We can thus use Rosenthal’s technique (Lemma 5.10 in [19]), [20] to obtain

$$\begin{array}{l}\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})\right)}^{\rho }|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]\\ (92)& \hspace{1em}\le 2^{{\rho }^2}max\left\{\mathbb{E}{\left[{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]}^{\rho },\mathbb{E}\left[{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]\right\}(93)& \hspace{1em}\le 2^{{\rho }^2}\left(\mathbb{E}{\left[{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]}^{\rho }+\mathbb{E}\left[{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]\right).\end{array}$$

Taking the expectation over ${\mathbf{X}}_1$ and $\mathbf{Y}$ yields

$$\begin{array}{}(94)& \mathbb{E}\left[\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})\right)}^{\rho }|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]\right](95)& \hspace{1em}\le 2^{{\rho }^2}\mathbb{E}\left[\mathbb{E}{\left[{\displaystyle \sum _{m\ne 1}}{B}_m\left(\mathbf{y}\right)|\mathbf{Y}=\mathbf{y}\right]}^{\rho }\right]+2^{{\rho }^2}\mathbb{E}\left[\mathbb{E}\left[{\displaystyle \sum _{m\ne 1}}{B}_m\left(\mathbf{y}\right)|\mathbf{Y}=\mathbf{y}\right]\right](96)& \hspace{1em}\le 2^{{\rho }^2}\mathbb{E}\left[\mathbb{E}{\left[{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]}^{\rho }\right]+2^{{\rho }^2}\mathbb{E}\left[\mathbb{E}\left[{\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{x}}_1,\mathbf{y})|{\mathbf{X}}_1={\mathbf{x}}_1,\mathbf{Y}=\mathbf{y}\right]\right].\end{array}$$

The first term on the RHS can be treated using the lemma. The second—but for the $2^{{\rho }^2}$ constant—is the one encountered when $\rho$ is 1. Since by Section 4.2, $R_0\left(\rho \right)\le R_0\left(1\right)$ (because $\rho >1$ for the case at hand), it too tends to zero when $R<R_0\left(\rho \right)$.

4.4. No Rate Exceeding $R_0\left(\rho \right)$ Is Achievable

To show the converse, we need Arıkan’s lower bound on guessing [21].

Fix any sequence of rate-R blocklength-n codebooks $\left\{{\mathcal{C}}_n\right\}$ satisfying the cost constraint. For any $n\in \mathbb{N}$, let

$$\begin{array}{c}\hfill Q^{\left(n\right)}\left(\mathbf{x}\right)=\left\{\begin{array}{cc}\frac{1}{|{\mathcal{C}}_n|}\hfill & \mathrm{if}\mathbf{x}\in {\mathcal{C}}_n,\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(97)

be the induced probability distribution on ${\mathbb{R}}^n$. Since the codebook satisfies the cost constraint, $\mathbb{E}[\parallel \mathbf{X}{\parallel }^2]\le n\mathsf{A}$ under $Q^{\left(n\right)}$.

Given $\mathbf{y}$, list the messages $m\in \mathcal{M}$ in decreasing order of likelihood $w\left(\mathbf{y}\right|{\mathbf{x}}_m)$ (resolving ties arbitrarily, e.g., ranking low numerical values of m higher), and let $G\left(m\right|\mathbf{y})$ denote the ranking of the message m in this list. Note that

$$\begin{array}{c}\hfill \left|\mathcal{L}\right(m,\mathbf{y}\left)\right|\ge G\left(m\right|\mathbf{y}),\end{array}$$

(98)

where the inequality can be strict because there may be messages that are in $\mathcal{L}(m,\mathbf{y})$ because they have the same likelihood as m, and that are yet ranked lower than m by $G(·|\mathbf{y})$ because of the way ties are resolved. It follows from this inequality that the $\rho$-th moment of $\left|\mathcal{L}\right(M,\mathbf{Y}\left)\right|$ cannot tend to one unless the $\rho$-th moment of $G\left(M\right|\mathbf{Y})$ does. By Arıkan’s guessing inequality [21],

$$\begin{array}{ccc}\hfill \mathbb{E}\left[G{\left(M\right|\mathbf{Y})}^{\rho }\right]& \ge \hfill & {(1+nR)}^{-\rho }·exp\left(n\rho R-n{E}_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})\right),\end{array}$$

(99)

so the $\rho$-th moment of $G\left(M\right|\mathbf{Y})$ can tend to one only if

$$\begin{array}{ccc}\hfill \rho R& \le \hfill & {\displaystyle \underset{n\to \infty }{lim\; inf}}{E}_0^{\left(n\right)}(\rho ,Q^{\left(n\right)}).\end{array}$$

(100)

From this, the converse now follows using (24) and (55) because

$$\begin{array}{}(101)& {\displaystyle \underset{n\to \infty }{lim\; inf}}{E}_0^{\left(n\right)}(\rho ,Q^{\left(n\right)})& \le \hfill & {\displaystyle \underset{n\to \infty }{lim}}{E}_0^{\left(n\right),*}\left(\rho \right)(102)& & =\hfill & \rho R_0\left(\rho \right).\end{array}$$

5. The Direct Part of Theorem 2

In this section we prove the direct part of Theorem 2: when the decoder can be provided with a rate-$R_{\mathrm{h}}$ description of the noise, the convergence (19) can be achieved at all transmission rates below $R_0\left(\rho \right)+R_{\mathrm{h}}$. As noted earlier, the converse follows directly from (Remark 4 in [3]).

Our proof treats the cases $R_{\mathrm{h}}=0$ and $R_{\mathrm{h}}>0$ separately. As in Section 4, we assume that the channel is normalized to having noise variance 1 and transmit power $\mathsf{A}$.

5.1. Case 1: $R_{\mathrm{h}}=0$

The analogous result for the modulo-additive channel was proved in [3] by having the helper provide the decoder with a lossless description of the type of the noise sequence. Since this type fully specifies the a posteriori probability of the transmitted message, the decoder’s remotely-plausible-with-this-help list $\mathcal{L}(\mathbf{Y},T)$ contains only messages whose a posteriori probability is equal to that of the correct message. It is therefore a subset of the at-least-as-likely list $\mathcal{L}(M,\mathbf{Y})$ (without help) and hence of smaller-or-equal $\rho$-th moment. Consequently, any rate that allows the latter to tend to one, also allows the former to tend to one.

On the Gaussian channel the likelihood $w\left(\mathbf{y}\right|{\mathbf{x}}_m)$ is specified by the normalized squared Euclidean norm of the noise sequence ${\parallel \mathbf{z}\parallel }^2/n$. The latter, however, cannot be described at zero rate with infinite precision. This motivates us to quantize it and have the quantized version be the zero-rate help. The result will then follow by considering the high-resolution limit of the achievable rates. For this purpose, a uniform quantizer will do.

Given some large $M>0$ (which determines the overload region) and some large K (corresponding to the number of quantization cells), we partition the interval $[0,M]$ into K subintervals, each of length $\Delta =M/K$. The helper, upon observing the noise sequence $\mathbf{Z}$, produces

$$\begin{array}{c}\hfill T={\varphi }^{\left(n\right)}\left(\mathbf{Z}\right)=\left\{\begin{array}{cc}⌊{\parallel \mathbf{Z}\parallel }^2/(n\Delta )⌋\hfill & {\mathrm{if}\parallel \mathbf{Z}\parallel }^2/n<M,\hfill \\ K\hfill & \mathrm{otherwise}\hfill \end{array}\right..\end{array}$$

(103)

The constant M, which does not depend on the blocklength n, is chosen large enough to guarantee that the large-deviation probability of overload $\mathrm{Pr}[\parallel \mathbf{Z}{\parallel }^2/n\ge M]$ decay sufficiently fast in n so that the contribution of the overload to the $\rho$-th moment of the list be negligible, even if an overload results in the list containing all $e^{nR}$ codewords:

$$\begin{array}{ccc}\hfill {\displaystyle \underset{n\to \infty }{lim}}{e}^{n\rho R}·\mathrm{Pr}\left[n^{-1}{\parallel \mathbf{Z}\parallel }^2\ge M\right]& =\hfill & 0.\end{array}$$

(104)

(Upper bounds on the tail of the ${\chi }^2$ distribution show, for example, that for $R<R_0\left(\rho \right)$, the choice $M=max\{2,20\rho R_0\left(\rho \right)\}$ will do.) Since the help takes values in the finite set ${\mathcal{T}}_n=\{0,1,\cdots ,K\}$, where K does not depend on the blocklength, it is of zero rate.

As in Section 4.3, we consider a random codebook ${\left\{{\mathbf{X}}_m\right\}}_{m=1,\cdots ,e^{nR}}$ whose codewords are drawn independently from the conditional Gaussian distribution, i.e., from $Q^{\left(n\right)}$ defined in (35) and (36) with Q being $Q_G$, the centered variance-$\mathsf{A}$ Gaussian distribution. Using the same symmetry arguments, we also assume that the transmitted message is $m=1$ and study the $\rho$-th moment of the list under this assumption. Defining

$$\begin{array}{ccc}\hfill V_m({\mathbf{x}}_1,\mathbf{y})& =\hfill & \mathrm{𝟙}\left\{{\varphi }^{\left(n\right)}(\mathbf{y}-{\mathbf{X}}_m)={\varphi }^{\left(n\right)}(\mathbf{y}-{\mathbf{x}}_1)\right\},{\mathbf{x}}_1,\mathbf{y}\in {\mathbb{R}}^n,\end{array}$$

(105)

we can express the $\rho$-th moment of the remotely-plausible list when $m=1$ as

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|\mathcal{L}(\mathbf{Y},T)\right|}^{\rho }\right]& =\hfill & \mathbb{E}\left[{\left(1+{\displaystyle \sum _{m\ne 1}}{V}_m({\mathbf{X}}_1,\mathbf{Y})\right)}^{\rho }\right].\end{array}$$

(106)

In view of Lemma 1, we need to prove that

$$\begin{array}{c}\hfill {\displaystyle \underset{n\to \infty }{lim}}\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{V}_m({\mathbf{X}}_1,\mathbf{Y})\right)}^{\rho }\right]=0,\end{array}$$

(107)

where the expectation is over both the random choice of the codebook and the channel behavior.

To analyze the LHS of (107), we define for every ${\mathbf{x}}_1,\mathbf{y}\in {\mathbb{R}}^n$ and every message $m\ne 1$ the binary random variable

$$\begin{array}{ccc}\hfill B_m({\mathbf{x}}_1,\mathbf{y};\Delta )& =\hfill & \mathrm{𝟙}\left\{w\left(\mathbf{y}\right|{\mathbf{X}}_m)\ge w\left(\mathbf{y}\right|{\mathbf{x}}_1)·e^{-\frac{n\Delta }{2}}\right\}.\end{array}$$

(108)

Our analysis of $V_m({\mathbf{x}}_1,\mathbf{y})$ depends on whether ${\varphi }^{\left(n\right)}(\mathbf{y}-{\mathbf{x}}_1)$ differs from K (no overload) or equals K (corresponding to quantizer overload). In the former case, the random variable $V_m({\mathbf{x}}_1,\mathbf{y})$ can be upper bounded by $B_m({\mathbf{x}}_1,\mathbf{y};\Delta )$ because

$$\begin{array}{}(109)& V_m({\mathbf{x}}_1,\mathbf{y})& =\hfill & \mathrm{𝟙}\left\{{\varphi }^{\left(n\right)}(\mathbf{y}-{\mathbf{X}}_m)={\varphi }^{\left(n\right)}(\mathbf{y}-{\mathbf{x}}_1)\right\}(110)& & \le \hfill & \mathrm{𝟙}\left\{|\parallel \mathbf{y}-{\mathbf{X}}_m{\parallel }^2-{\parallel \mathbf{y}-{\mathbf{x}}_1\parallel }^2|<n\Delta \right\}(111)& & \le \hfill & \mathrm{𝟙}\left\{\parallel \mathbf{y}-{\mathbf{X}}_m{\parallel }^2\le {\parallel \mathbf{y}-{\mathbf{x}}_1\parallel }^2+n\Delta \right\}(112)& & =\hfill & \mathrm{𝟙}\left\{e^{-\frac{\parallel \mathbf{y}-{\mathbf{X}}_m{\parallel }^2}{2}}\ge e^{-\frac{\parallel \mathbf{y}-{\mathbf{x}}_1{\parallel }^2+n\Delta }{2}}\right\}(113)& & =\hfill & B_m({\mathbf{x}}_1,\mathbf{y};\Delta ),\end{array}$$

where (110) holds because, for the case at hand, the equality of helper’s description implies that $\parallel \mathbf{y}-{\mathbf{X}}_m{\parallel }^2$ and $\parallel \mathbf{y}-{\mathbf{x}}_1{\parallel }^2$ lie in a same interval of length $n\Delta$. In the latter case—which is exponentially rare when M exceeds the noise variance—we simply upper bound $V_m({\mathbf{x}}_1,\mathbf{y})$ by 1.

The $\rho$-th moment of the list can now be expressed using the law of total expectation as

$$\begin{array}{l}\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{V}_m({\mathbf{X}}_1,\mathbf{Y})\right)}^{\rho }\right]\\ (114)& \hspace{1em}=\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{V}_m({\mathbf{X}}_1,\mathbf{Y})\right)}^{\rho }|T\ne K\right]\mathrm{Pr}[T\ne K]+\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{V}_m({\mathbf{X}}_1,\mathbf{Y})\right)}^{\rho }|T=K\right]\mathrm{Pr}[T=K](115)& \hspace{1em}\le \mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{X}}_1,\mathbf{Y};\Delta )\right)}^{\rho }|T\ne K\right]\mathrm{Pr}[T\ne K]+e^{n\rho R}\mathrm{Pr}[T=K](116)& \hspace{1em}\le \mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{X}}_1,\mathbf{Y};\Delta )\right)}^{\rho }\right]+e^{n\rho R}\mathrm{Pr}[T=K].\end{array}$$

The second term on the RHS of (116) tends to zero by (104). The first term is studied in the following lemma:

Lemma 3.

If $\rho >0$, $\Delta >0$, and $R<R_0\left(\rho \right)-\Delta$, then

$$\begin{array}{c}\hfill {\displaystyle \underset{n\to \infty }{lim}}\mathbb{E}\left[{\left({\displaystyle \sum _{m\ne 1}}{B}_m({\mathbf{X}}_1,\mathbf{Y};\Delta )\right)}^{\rho }\right]=0.\end{array}$$

(117)

Proof. 

See Appendix B. □

For a given $R<R_0\left(\rho \right)$, achievability is thus established using this lemma and (116) by picking M sufficiently large for (104) to hold, and then picking K large enough to guarantee that $R<R_0\left(\rho \right)-M/K$ so that, by Lemma 3, the first term on the RHS of (116) will also tend to zero.

5.2. Case 2: $R_{\mathrm{h}}>0$

The key to proving the achievability of $R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)+R_{\mathrm{h}}$ is in showing that rate-$R_{\mathrm{h}}$ help can be utilized to increase the data rate by $R_{\mathrm{h}}$, and that this can be done losslessly, with arbitrarily small (positive) power, and in one channel use. To show how this can be done, we show that—by using the channel once to send a single input that is bounded by $\sqrt{\mathsf{A}}$ (with $\mathsf{A}$ any prespecified positive number) and using help taking values in the set $\mathcal{T}=\{0,\dots ,\kappa -1\}$—we can send error-free a message taking values in said set. To transmit $m\in \{0,\dots ,\kappa -1\}$, the encoder sends

$$\begin{array}{c}\hfill x=m·\frac{\sqrt{\mathsf{A}}}{\kappa },\end{array}$$

(118)

which is upper-bounded by $\sqrt{\mathsf{A}}$. Upon observing the noise Z, the helper produces the description T by quantizing the normalized noise and taking modulo, i.e.,

$$\begin{array}{c}\hfill T=⌊Z·\frac{\kappa }{\sqrt{\mathsf{A}}}⌋\mathrm{mod}\kappa ,\end{array}$$

(119)

which is an element of $\{0,\dots ,\kappa -1\}$. Based on Y and T, the decoder can calculate

$$\begin{array}{c}\hfill \widehat{m}=⌊Y·\frac{\kappa }{\sqrt{\mathsf{A}}}-T⌋\mathrm{mod}\kappa ,\end{array}$$

(120)

which equals m, because

$$\begin{array}{}(121)& \hfill \widehat{m}& =\hfill & ⌊\left(x+Z\right)·\frac{\kappa }{\sqrt{\mathsf{A}}}-T⌋\mathrm{mod}\kappa (122)& & =\hfill & ⌊m+Z·\frac{\kappa }{\sqrt{\mathsf{A}}}-T⌋\mathrm{mod}\kappa (123)& & =\hfill & \left(m+⌊Z·\frac{\kappa }{\sqrt{\mathsf{A}}}⌋-T\right)\mathrm{mod}\kappa (124)& & =\hfill & m,\end{array}$$

where (123) holds because m and T are both integers.

Using this building-block, we can now prove the achievability of $R_{\mathrm{c}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{f}}\left(\rho \right)+R_{\mathrm{h}}$ by employing two-phase time sharing. Specifically, we propose the following blocklength-$(n+1)$ scheme. In the first n channel uses, the helper operates at rate zero as in Section 5.1. By the achievability result proved in Section 5.1, for any $R<R_0\left(\rho \right)$, there exists a sequence of blocklength-n rate-R codebooks ${\left\{{\mathbf{x}}_m\right\}}_{m=1,\cdots ,e^{nR}}$, with $\parallel {\mathbf{x}}_m{\parallel }^2\le (n-1)\mathsf{A}$ for every m, and zero-rate helpers $\varphi \left(Z^n\right)$, such that the remotely-plausible-list $\mathcal{L}(Y^n,\varphi \left(Z^n\right))$ satisfies

$$\begin{array}{c}\hfill {\displaystyle \underset{n\to \infty }{lim}}\mathbb{E}\left[|\mathcal{L}(Y^n,\varphi \left(Z^n\right)){|}^{\rho }\right]=1.\end{array}$$

(125)

In the $(n+1)$-th channel-use we use the aforementioned coding scheme with $\kappa$ being $\lceil e^{n{R}_{\mathrm{h}}}\rceil$. Since that scheme is error-free, the overall remotely-plausible-list for the two phases has the same cardinality as that of the first phase, namely $\left|\mathcal{L}\right(Y^n,\varphi \left(Z^n\right)\left)\right|$, and hence, its $\rho$-th moment tends to 1 by (125).

The achievability now follows by verifying that, the power of the transmitted input sequence $\mathbf{x}$ satisfies

$$\begin{array}{c}\hfill {\parallel \mathbf{x}\parallel }^2=\parallel x^n{\parallel }^2+{\parallel x_{n+1}\parallel }^2\le n\mathsf{A}+\mathsf{A}=(n+1)\mathsf{A};\end{array}$$

(126)

the rate of the helper is

$$\begin{array}{c}\hfill \frac{1}{n+1}\left(0+n{R}_{\mathrm{h}}\right)\end{array}$$

(127)

and the rate achieved by the scheme is

$$\begin{array}{c}\hfill \frac{1}{n+1}\left(n{R}_0\left(\rho \right)+n{R}_{\mathrm{h}}\right)\end{array}$$

(128)

which tend to $R_{\mathrm{h}}$ and $R_0\left(\rho \right)+R_{\mathrm{h}}$, respectively, as n tends to infinity.