Smoothing of Binary Codes, Uniform Distributions, and Applications

Abstract

The action of a noise operator on a code transforms it into a distribution on the respective space. Some common examples from information theory include Bernoulli noise acting on a code in the Hamming space and Gaussian noise acting on a lattice in the Euclidean space. We aim to characterize the cases when the output distribution is close to the uniform distribution on the space, as measured by the Rényi divergence of order $\alpha \in (1,\infty ]$. A version of this question is known as the channel resolvability problem in information theory, and it has implications for security guarantees in wiretap channels, error correction, discrepancy, worst-to-average case complexity reductions, and many other problems. Our work quantifies the requirements for asymptotic uniformity (perfect smoothing) and identifies explicit code families that achieve it under the action of the Bernoulli and ball noise operators on the code. We derive expressions for the minimum rate of codes required to attain asymptotically perfect smoothing. In proving our results, we leverage recent results from harmonic analysis of functions on the Hamming space. Another result pertains to the use of code families in Wyner’s transmission scheme on the binary wiretap channel. We identify explicit families that guarantee strong secrecy when applied in this scheme, showing that nested Reed–Muller codes can transmit messages reliably and securely over a binary symmetric wiretap channel with a positive rate. Finally, we establish a connection between smoothing and error correction in the binary symmetric channel.

1. Introduction

Many problems of information theory involve the action of a noise operator on a code distribution, transforming it into some other distribution. For instance, one can think of Bernoulli noise acting on a code in the Hamming space or Gaussian noise acting on a lattice in the Euclidean space. We are interested in characterizing the cases when the output distribution is close to the uniform distribution on the space. Versions of this problem have been considered under different names, including resolvability [1,2,3], smoothing [4,5], discrepancy [6,7], and the entropy of noisy functions [8,9,10]. Direct applications of smoothing include secrecy guarantees in both the binary symmetric wiretap channel [2,3,11] and the Gaussian wiretap channel [12,13], error correction in the binary symmetric channel (BSC) [14,15], converse coding theorems of information theory [1,16,17,18], strong coordination [11,19,20,21,22], secret key generation [13,23], and worst-to-average case reductions in cryptography [5,24]. Some aspects of this problem also touch upon approximation problems in statistics and machine learning [25,26,27].

Our main results are formulated for the smoothing in the binary Hamming space ${\mathcal{H}}_n$. For $r:{\mathcal{H}}_n\to {\mathbb{R}}_0^{+}$, and $f:{\mathcal{H}}_n\to \mathbb{R}$ define

$$T_{r}f\left(x\right)=(r\ast f)\left(x\right):=\sum _{z\in {\mathcal{H}}_n}r\left(z\right)f(x-z)$$

as the action of r on the functions on the space. We set r to be a probability mass function (pmf) and call the function $T_{r}f$ the noisy version of f with respect to r, and refer to r and $T_r$ as a noise kernel and a noise operator, respectively. By smoothing f with respect to r, we mean applying the noise kernel r to f. We often assume that $r\left(x\right)$ is a radial kernel, i.e., its value on the argument $x\in {\mathcal{H}}_n$ depends only on the Hamming weight of x.

There are several ways to view the smoothing operation. Interpreting it as a shift-invariant linear operator, we note that, from Young’s inequality, $\parallel T_r{f\parallel }_{\alpha }={\parallel f\ast r\parallel }_{\alpha }\le {\parallel f\parallel }_{\alpha },1\le \alpha \le \infty ,$ so smoothing contracts the $\alpha$-norm. Upon applying $T_r$, the noisy version of f becomes “flatter”; hence, the designation “smoothing”. Note that if f is a pmf, then $T_{r}f$ is also a pmf, and so this view allows us to model the effect of communication channels with additive noise.

The class of functions that we consider are (normalized) indicators of subsets (codes) in ${\mathcal{H}}_n$. A code $\mathcal{C}\subset {\mathcal{H}}_n$ defines a pmf $f_{\mathcal{C}}=\frac{{\mathbb{1}}_{\mathcal{C}}}{\left|\mathcal{C}\right|},$ and, thus, $T_r{f}_{\mathcal{C}}$ can be viewed as a noisy version of the code (we also sometimes call it a noisy distribution) with respect to the kernel r. The main question of interest for us is the proximity of this distribution to $U_n,$ or the “smoothness” of the noisy code distributions. To quantify closeness to $U_n$, we use the Kullback–Leibler (KL) and Rényi divergences (equivalently, $L_{\alpha }$ norms), and the smoothness measured in $D_{\alpha }(·\parallel ·)$ is termed the $D_{\alpha }$-smoothness ($L_{\alpha }$-smoothness).

We say that a code is perfectly smoothable with respect to the noise kernel r if the resultant noisy distribution becomes uniform. Our main emphasis is on the asymptotic version of perfect smoothing and its implications for some of the basic information-theoretic problems. A sequence of codes ${\left({\mathcal{C}}_n\right)}_n$ is asymptotically smoothed by the kernel sequence $r_n$ if the distance between $\left(T_{r_n}{f}_{{\mathcal{C}}_n}\right)$ and $U_n$ approaches 0 as n increases. This property is closely related to the more general problem of channel resolvability introduced by Han and Verdú in [1]. Given a discrete memoryless channel $\mathcal{W}\left(Y\right|X)$ and a distribution $P_X$, we observe a distribution $P_Y$ on the output of the channel. The task of channel resolvability is to find $P_X$ supported on a subset $\mathcal{C}\subset {\mathcal{H}}_n$ that approximates $P_Y$ with respect to the KL divergence. As shown in [1], there exists a threshold value of the rate such that it is impossible to approximate $P_Y$ using codes of lower rate, while any output process can be approximated by a well-chosen code of a rate larger than the threshold. Other proximity measures between distributions were considered for this problem in [3,28,29]. Following the setting in [3], we consider Rényi divergences for measuring the closeness to uniformity. We call the minimum rate required to achieve perfect asymptotic smoothing the $D_{\alpha }$-smoothing capacity of the noise kernels ${\left(r_n\right)}_n$, where the proximity to uniformity is measured by the $\alpha$-Rényi divergence. In this work, we characterize the $D_{\alpha }$-smoothing capacity of the sequence ${\left(r_n\right)}_n$ using its Rényi entropy rate.

Asymptotic smoothing. We will limit ourselves to studying smoothing bounds under the action of the Bernoulli noise or ball noise kernels, defined formally below. A common approach to deriving bounds on the norm of a noisy function is through hypercontractivity inequalities [30,31,32]. In its basic version, given a code $\mathcal{C}$ of size M, it yields the estimate

$$\begin{array}{c}\hfill \parallel T_{\delta }{f}_{\mathcal{C}}{\parallel }_{\alpha }\le {\parallel f_{\mathcal{C}}\parallel }_{{\alpha }^{\prime }}=M^{\frac{1-{\alpha }^{\prime }}{{\alpha }^{\prime }}}{2}^{-\frac{n}{{\alpha }^{\prime }}},\end{array}$$

where $T_{\delta }$ is the Bernoulli kernel (see Section 2 for formal definitions) and ${\alpha }^{\prime }=1+{(1-2\delta )}^2(\alpha -1)$. This upper bound does not differentiate codes yielding higher or lower smoothness, which in many situations may not be sufficiently informative. Note that other tools, such as “Mrs. Gerber’s lemma” [30,33] or strong data-processing inequalities, also suffer from the same limitation.

A new perspective of the bounds for smoothing has recently been introduced in the works of Samorodnitsky [8,9,10]. Essentially, his results imply that codes satisfying certain regularity conditions have good smoothing properties. Their efficiency is highlighted in recent papers [14,34], which leveraged results for code performance on the binary erasure channel (BEC) to prove strong claims about the error correction capabilities of the codes when used on the BSC. Using Samorodnitsky’s inequalities, we show that the duals of some BEC capacity-achieving codes achieve $D_{\alpha }$-smoothing capacity for $\alpha \in \{2,3,\dots ,\infty \}$ with respect to the Bernoulli noise. This includes the duals of polar codes and doubly transitive codes, such as the Reed–Muller (RM) codes.

Smoothing and the wiretap channel. Wyner’s wiretap channel [35] models communication in the presence of an eavesdropper. Code design for this channel pursues reliable communication between the legitimate parties, while at the same time leaking as little information as possible about the transmitted messages to the eavesdropper. The connection between secrecy in wiretap channels and resolvability was first mentioned by Csiszár [36] and later developed by Hayashi [2]. It rests on the observation that to achieve secrecy it suffices to make the distribution of an eavesdropper’s observations conditioned on the transmitted message nearly independent of the message. The idea of characterizing secrecy based on smoothness works irrespective of the measure of secrecy [2,3,11], and it was also employed for nested lattice codes used over the Gaussian wiretap channel in [12].

Secrecy on the wiretap channel can be defined in two ways, measured by the information gained by the eavesdropper, and it depends on whether this quantity is normalized to the number of channel uses (weak secrecy) or not (strong secrecy). This distinction was first highlighted by Maurer [37], and it has been adopted widely in the recent literature. Early papers devoted to code design for the wiretap channel relied on random codes, but, for simple channel models such as BSC or BEC, this has changed with the advent of explicit capacity-approaching code families. Weak secrecy results based on LDPC codes were presented in [38], but initial attempts to attain strong secrecy encountered some obstacles. To circumvent this, the first works on code construction [39,40] had to assume that the main channel is noiseless. The problem of combining strong secrecy and reliability for general wiretap channels was resolved in [41], but that work had to assume that the two communicating parties share a small number of random bits unavailable to the eavesdropper. Apart from the polar coding scheme of [41], explicit code families that support reliable communication with positive rate and strong secrecy have not previously appeared in the literature. In this work, we show that nested RM codes perform well in binary symmetric wiretap channels based on their smoothing properties. While our work falls short of proving that nested RM codes achieve capacity, we show that they can transmit messages reliably and secretly at rates close to capacity.

Ball noise and decoding error. Ball-noise smoothing provides a tool for estimating the error probability of decoding on the BSC. We derive impossibility and achievability bounds for the $D_{\alpha }$-smoothness of noisy distributions with respect to the ball noise. Smoothing of a code with respect to the $L_2$ norm plays a special role because, in this case, the second norm (the variance) of the resulting distribution can be expressed via the pairwise distance between codewords, enabling one to rely on tools from Fourier analysis. The recent paper by Debris-Alazard et al. [4] established universal bounds for the smoothing of codes or lattices, with cryptographic reductions in mind. The paper by Sprumont and Rao [15] addressed bounds for error probability of list decoding at rates above BSC capacity. A paper by one of the present authors [42] studied the variance of the number of codewords in balls of different radii (a quantity known as the quadratic discrepancy [43,44]).

The main contributions of this paper are the following:

• Characterizing the $D_{\alpha }$-smoothing capacities of noise operators on the Hamming space for $\alpha \in (1,\infty ]$.
• Identifying some explicit code families that attain a smoothing capacity of the Bernoulli noise for $\alpha \in \{2,3,\dots ,\infty \}$;
• Obtaining rate estimates for the RM codes used on the BSC wiretap channel under the strong secrecy condition;
• Showing that codes possessing sufficiently good smoothing properties are suitable for error correction.

In Section 2, we set up the notation and introduce the relevant basic concepts. Then, in Section 3, we derive expressions for the $D_{\alpha }$-smoothing capacities for $\alpha \in (1,\infty ]$, and in Section 4, we use these results to analyze the smoothing of code families under the action of the Bernoulli noise. Section 5 is devoted to the application of these results for the binary symmetric wiretap channel. In particular, we show that RM codes can achieve rates close to the capacity of the BSC wiretap channel, while at the same time guaranteeing strong secrecy. In Section 6, we establish threshold rates for smoothing under ball noise, and derive bounds for the error probability of decoding on the BSC, including the list case, based on the distance distribution. Concluding the paper, Section 7 briefly points out that the well-known class of uniformly packed codes are perfectly smoothable with respect to “small” noise kernels.

2. Preliminaries

2.1. Notation

Throughout this paper, ${\mathcal{H}}_n$ is the binary n-dimensional Hamming space

Balls and spheres. Denote by $B(x,t):=\{y\in {\mathcal{H}}_n:|y-x|\le t\}$ the metric ball of radius t in ${\mathcal{H}}_n$ with center at x, and denote by $S(x,t):=\{y\in {\mathcal{H}}_n:|y-x|=t\}$ the sphere of radius t. Let $V_t=\left|B(x,t)\right|$ be the volume of the ball, and let ${\mu }_t\left(i\right)$ be the intersection volume of two balls of radius t whose centers are distance i apart:

$$\begin{array}{c}\hfill {\mu }_t\left(i\right)=|B(0,t)\cap B(x,t)|,\mathrm{where}\left|x\right|=i.\end{array}$$

(1)

Codes and distributions. A code $\mathcal{C}$ is a subset in ${\mathcal{H}}_n$. The rate and distance of the code are denoted by $R\left(\mathcal{C}\right):=log\left|\mathcal{C}\right|/n$ and $d\left(\mathcal{C}\right)$, respectively. Let

$$A_i=\frac{1}{\left|\mathcal{C}\right|}\left|\{(x,y)\in {\mathcal{C}}^2:d(x,y)=i\}\right|$$

(2)

and let $(A_i,i=0,\dots ,n)$ be the distance distribution of the code. If the code $\mathcal{C}$ forms an ${\mathbb{F}}_2$-linear subspace in ${\mathcal{H}}_n,$ we denote by ${\mathcal{C}}^{\perp }:=\{y\in {\mathcal{H}}_n:{\sum }_i{x}_i{y}_i=0\mathrm{for}\mathrm{all}x\in \mathcal{C}\}$ its dual code.

The function ${\mathbb{1}}_{\mathcal{C}}$ denotes the indicator of a subset $\mathcal{C}\subset {\mathcal{H}}_n,$ and $f_{\mathcal{C}}=\frac{{\mathbb{1}}_{\mathcal{C}}}{\left|\mathcal{C}\right|}$ is the corresponding pmf denoting the uniform distribution over the set, calling it a code distribution. Let $b_t$ denote the uniform distribution on the ball $B_{0,t}$, given by $b_t\left(x\right)=\frac{{\mathbb{1}}_{B_0,t}}{V_t}$. In the context of noise operators, we refer to $T_{b_t}$ as the ball noise. Finally, ${\beta }_{\delta }$ is the binomial distribution on ${\mathcal{H}}_n,$ given by

$${\beta }_{\delta }\left(x\right)={\beta }_{\delta }^{\left(n\right)}\left(x\right)={\delta }^{\left|x\right|}{(1-\delta )}^{n-\left|x\right|},$$

(3)

and $U_n$ is the uniform distribution, given by $U_n\left(x\right)=2^{-n}$ for all $x.$

Entropies and norms. For a function $f:{\mathcal{H}}_n\to \mathbb{R}$, we define its $\alpha$-norm as follows.

$$\begin{array}{cc}\hfill {\parallel f\parallel }_{\alpha }& ={\left(\frac{1}{2^n}\sum _{x\in {\mathcal{H}}_n}{\left|f\right(x\left)\right|}^{\alpha }\right)}^{1/\alpha }\mathrm{for}\alpha \in (0,\infty )\hfill \\ \hfill {\parallel f\parallel }_{\infty }& =\underset{x\in {\mathcal{H}}_n}{max}\left|f\right(x\left)\right|.\hfill \end{array}$$

Given a pmf P, let

$$\begin{array}{cc}\hfill H\left(P\right)& =-\sum _i{P}_{i}log{P}_i,\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill H_{\alpha }\left(P\right)& =\frac{1}{1-\alpha }log\left(\sum _i{P}_i^{\alpha }\right)\hfill \end{array}$$

(5)

denote its Shannon entropy and Rényi entropy of order $\alpha ,$ respectively. If P is supported on two points, we write $h\left(P\right)$ and $h_{\alpha }\left(P\right)$ instead (all logarithms are to the base 2). The limiting cases of $\alpha =0,1,\infty$ are well-defined; in particular, for $\alpha =1$, $H_{\alpha }\left(P\right)$ reduces to $H\left(P\right).$

For two discrete probability distributions P and Q, the $\alpha$-Rényi divergence (or simply the $\alpha$-divergence) is defined as follows:

$$D_{\alpha }(P\parallel Q)=\left\{\begin{array}{cc}-logQ\left(\{i:P_i>0\}\right)\hfill & \mathrm{if} \alpha =0\hfill \\ \frac{1}{\alpha -1}log\sum _i{P}_i^{\alpha }{Q}_i^{-(\alpha -1)}\hfill & \mathrm{if} \alpha \in (0,1)\cup (1,\infty )\hfill \\ \sum _i{P}_{i}log\frac{P_i}{Q_i}\hfill & \mathrm{if} \alpha =1\hfill \\ \underset{i}{max}log\frac{P_i}{Q_i}\hfill & \mathrm{if} \alpha =\infty \hfill \end{array}\right..$$

(6)

The divergence $D_{\alpha }(P\parallel Q)$ is a continuous function of $\alpha$ for $\alpha \in [0,\infty ]$. For a pmf f on ${\mathcal{H}}_n$

$$\begin{array}{cc}\hfill D_{\alpha }(f\parallel U_n)& =\frac{\alpha }{\alpha -1}log{\parallel 2^{n}f\parallel }_{\alpha },\alpha \in (0,1)\cap (1,\infty )\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill D_{\infty }(f\parallel U_n)& =log\parallel 2^n{f\parallel }_{\infty }.\hfill \end{array}$$

(8)

Note that $D_{\alpha }(f\parallel U_n)=n-H_{\alpha }\left(f\right)$ for all $0\le \alpha \le \infty .$

Channels. In this paper, a channel is a conditional probability distribution $\mathcal{W}:\{0,1\}\to \mathcal{Y},$ where $\mathcal{Y}$ is a finite set, so that $\mathcal{W}\left(y\right|x)$ is the conditional probability of the output y for the input x. We frequently consider the binary symmetric channel with crossover probability $\delta$ and the binary erasure channel with erasure probability $\lambda ,$ abbreviating them as $\mathrm{BSC}\left(\delta \right)$ and $\mathrm{BEC}\left(\lambda \right),$ respectively. We are often interested in the n-fold channel ${\mathcal{W}}^{\left(n\right)},$ i.e., the conditional probability distribution corresponding to n-uses of the channel. For the input X, let $Y_{(X,\mathcal{W})}$ be the random output of the channel ${\mathcal{W}}^{\left(n\right)}$. If the input sequences are chosen from a uniform distribution on a code $\mathcal{C}$, we denote the input by $X_{\mathcal{C}}.$ Since the number of uses of the channel is usually clear from the context, we suppress the dependency on n from the notation for channels and sequences.

Let $\mathcal{C}$ be a code of length n. For a channel $\mathcal{W}$ and input $X_{\mathcal{C}}$, the block-MAP decoder is defined as

$$\begin{array}{c}\hfill \widehat{x}\left(y\right)=\underset{x\in \mathcal{C}}{error}Pr\left(x\right|y).\end{array}$$

For a given code and channel, denote the error probability of the block-MAP decoding by

$$\begin{array}{c}\hfill P_B(\mathcal{W},\mathcal{C})=Pr(X_{\mathcal{C}}\ne \widehat{X}\left(Y_{(X_{\mathcal{C}},\mathcal{W})}\right).\end{array}$$

2.2. $D_{\alpha }$- and $L_{\alpha }$-Smoothness

Recall that in the introduction, we expressed the smoothness of a distribution as its proximity to uniformity. Here, we formalize this notion based on two (equivalent) proximity measures.

Let g be a pmf on ${\mathcal{H}}_n$. A natural measure of the uniformity of g is $D_{\alpha }(g\parallel U_n)$ ($\alpha \in [0,\infty ]$). We call this the $D_{\alpha }$-smoothness of g. Observe that

$$\begin{array}{cc}\hfill \parallel 2^n{g\parallel }_{\alpha }& =\frac{{\parallel g\parallel }_{\alpha }}{{\parallel g\parallel }_1}\ge 1\mathrm{for}\alpha \in (1,\infty ],\mathrm{and}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill \parallel 2^n{g\parallel }_{\alpha }& =\frac{{\parallel g\parallel }_{\alpha }}{{\parallel g\parallel }_1}\le 1\mathrm{for}\alpha \in (0,1)\hfill \end{array}$$

(10)

with equality iff $g=U_n$. Thus, the better the pmf g approximates uniformity, the closer is $\parallel 2^n{g\parallel }_{\alpha }$ to 1 (the denominator is simply a normalization quantity that allows dimension-agnostic analysis). Therefore, $\parallel 2^n{g\parallel }_{\alpha }$ ($\alpha \in (0,1)\cup (1,\infty ]$) can be considered as another measure of proximity. We call $\parallel 2^n{g\parallel }_{\alpha }$ the $L_{\alpha }$-smoothness of g. From (7) and (8), it follows that the $D_{\alpha }$-smoothness and $L_{\alpha }$-smoothness are equivalent.

Remark 1. 

It is easily seen that $D_{\alpha }(g\parallel U_n)=n-H_{\alpha }\left(g\right)$; hence, $D_{\alpha }(g\parallel U_n)$ is an increasing function of α.

Recall that for a given code $\mathcal{C}$, and a noise kernel r, $T_r{f}_{\mathcal{C}}=r\ast f_{\mathcal{C}}$ is the noisy distribution of code $\mathcal{C}$ with respect to r. We intend to study the smoothing properties of such noisy distributions of codes. In particular, we characterize the necessary conditions for $D_{\alpha }(T_r{f}_{\mathcal{C}}\parallel U_n)$ to be close to zero (equivalently, for $\parallel 2^n{T}_r{f}_{\mathcal{C}}{\parallel }_{\alpha }$ close to one). In Section 3, we quantify these requirements in the asymptotic setting.

2.3. Resolvability

The problem of channel resolvability was introduced by Han and Verdú [1] under the name of approximating the output statistics of the channel. The objective of channel resolvability is to approximate the output distribution of a given input by the output distribution of a code with a smaller support size. In this work, we are interested in code families whose noisy distributions approximate uniformity. Resolvability characterizes the necessary conditions for this to happen in terms of the rate of the code.

Let $\mathcal{W}$ be a (discrete memoryless) channel whose input alphabet is $\mathcal{X}$ and whose output alphabet is $\mathcal{Y}$. Let $\mathit{X}={\left\{X_n\right\}}_{n=1}^{\infty }$ be a discrete-time random process where the RVs $X_n$ take values in $\mathcal{X}$. Denote by $Y_n$ the random output of $\mathcal{W}$ with input $X_n$ and let $\mathit{Y}={\left\{Y_n\right\}}_{n=1}^{\infty }.$ Denote by $P_Y$ the distribution of $\mathit{Y}$ and let $P_{Y^{\left(n\right)}}$ be the pmf of the n-tuple $Y^{\left(n\right)}:=\{Y_1,Y_2,\dots ,Y_n\}$.

For a legitimate (realizable) output process $\mathit{Y}$, define

$$\begin{array}{c}\hfill J^{\left(\mathsf{\Delta }\right)}(\mathcal{W},P_{\mathit{Y}})=\underset{{\mathcal{C}}_n\subset {\mathcal{X}}^n}{inf}\{\underset{n}{lim\; inf}R\left({\mathcal{C}}_n\right):\mathsf{\Delta }(f_{{\mathcal{C}}_n},P_{Y^{\left(n\right)}})\to 0\},\end{array}$$

(11)

where $\mathsf{\Delta }$ is a measure of closeness of a pair of probability distributions. In words, we look for sequences of distributions ${\left(f_{{\mathcal{C}}_n}\right)}_n$ of the smallest possible rate that approximates $P_Y$ on the output of $\mathcal{W}.$

The original problem as formulated by Han and Verdú in [1] seeks to find the resolvability of the channel, defined as

$$\begin{array}{c}\hfill C_r^{\left(\mathsf{\Delta }\right)}\left(\mathcal{W}\right)=\underset{P_Y}{inf}\{J^{\left(\mathsf{\Delta }\right)}(\mathcal{W},P_{\mathit{Y}}):\mathit{Y}\mathrm{is}\mathrm{an}\mathrm{output}\mathrm{process}\mathrm{over}\mathcal{W}\}.\end{array}$$

(12)

where $\mathsf{\Delta }$ is either the variational distance or the normalized KL divergence $\frac{1}{n}D(·\parallel ·).$ Hayashi [2] considered the same problem where the proximity was measured by the unnormalized KL divergence. In each case, the resolvability equals the Shannon capacity of the channel $\mathcal{W}.$

Theorem 1 

([1,2]). Let $\mathcal{W}$ be a discrete memoryless channel. Suppose that Δ is either the KL divergence (normalized or not) or the variational distance; then, the resolvability is given by

$$\begin{array}{c}\hfill C_r^{\left(\mathsf{\Delta }\right)}\left(\mathcal{W}\right)=C\left(\mathcal{W}\right),\end{array}$$

where $C\left(\mathcal{W}\right)$ is the Shannon capacity of the channel.

The authors of [1] proved this result under the additional assumption that the channel $\mathcal{W}$ satisfies strong converse, and Hayashi [2] later showed that this assumption is unessential.

In addition to the proximity measures considered in Theorem 1, the papers [3,28,29] considered other possibilities. In particular, Yu and Tan [3] studied the resolvability problem for a specific target distribution $P_{\mathit{Y}}$ and for the Rényi divergence $\mathsf{\Delta }=D_{\alpha }$ (6). Their main result is as follows.

Theorem 2 

([3], Theorem 2). Let $\mathcal{W}$ be a channel and $P_{\mathit{Y}}$ be an output distribution. then

$$\begin{array}{c}\hfill J^{\left(D_{\alpha }\right)}(\mathcal{W},P_{\mathit{Y}})=\left\{\begin{array}{cc}\underset{P_X\in \mathcal{P}(\mathcal{W},P_{\mathit{Y}})}{min}\sum _x{P}_X\left(x\right)D_{\alpha }(\mathcal{W}(·|x)\parallel P_{\mathit{Y}})\hfill & if\alpha \in (1,2]\cup \{\infty \}\hfill \\ \underset{P_X\in \mathcal{P}(\mathcal{W},P_{\mathit{Y}})}{min}D(\mathcal{W}\parallel P_{\mathit{Y}}\left|P_X\right)\hfill & if\alpha \in (0,1]\hfill \\ 0\hfill & if\alpha =0,\hfill \end{array}\right.\end{array}$$

where $\mathcal{P}(\mathcal{W},P_{\mathit{Y}})$ is the set of distributions $P_{\mathit{X}}$ consistent with the output $P_{\mathit{Y}}$.

A direct corollary of Theorem 2 is the following:

Corollary 1 

([3], Equation (55)). Let ${\mathit{Y}}^{\ast }$ be the output process where for each n, $Y_n^{\ast }\sim \mathrm{Ber}(1/2)$. Then,

$$\begin{array}{c}\hfill J^{\left(D_{\alpha }\right)}(\mathrm{BSC}\left(\delta \right),P_{{\mathit{Y}}^{\ast }})=\left\{\begin{array}{cc}1-h_{\alpha }\left(\delta \right)\hfill & if\alpha \in (1,2]\cup \{\infty \}\hfill \\ 1-h\left(\delta \right)\hfill & if\alpha \in (0,1]\hfill \\ 0\hfill & if\alpha =0.\hfill \end{array}\right.\end{array}$$

This corollary gives necessary conditions for the rate of codes that can approximate the uniform distribution via smoothing. We will connect this result to the problem of finding smoothing thresholds in Section 4.

3. Perfect Smoothing—The Asymptotic Case

For a given family of noise kernels ${\left(T_{r_n}\right)}_n$, there exists a threshold rate such that it is impossible to approximate uniformity with codes of rate below the threshold irrespective of the chosen code, while at the same time, there exist families of codes with a rate above the threshold that allows perfect approximation in the limit of infinite length. For instance, for the Bernoulli$\left(\delta \right)$ noise applied to a code $\mathcal{C}$, the smoothed distribution is nonuniform unless $\mathcal{C}={\mathcal{H}}_n$ or $\delta =1/2$. At the same time, it is possible to approach the uniform distribution asymptotically for large n once the code sequence satisfies certain conditions. Intuitively, it is clear that, for a fixed noise kernel, it is easier to approximate uniformity if the code rate is sufficiently high. In this section, we characterize the threshold rate for (asymptotically) perfect smoothing. Of course, the threshold also depends on the proximity measure $\mathsf{\Delta }$ that we are using. In this section, we use perfect smoothing to mean “asymptotically perfect”. If the proximity measure $\mathsf{\Delta }$ for smoothing is not specified, this means that we are using the KL divergence. We obtain the threshold rates for perfect smoothing measured with respect to the $\alpha$-divergence for several values of $\alpha$. In the subsequent sections, we work out the details for the Bernoulli and ball noise operators, which also have some implications for communication problems.

Definition 1. 

Let ${\left({\mathcal{C}}_n\right)}_n$ be a sequence of codes of increasing length n and let $0\le \alpha \le \infty .$ We say that the sequence ${\mathcal{C}}_n$ is asymptotically perfectly $D_{\alpha }$-smoothable with respect to the noise kernels $r_n$ if

$$\underset{n\to \infty }{lim}{D}_{\alpha }(T_{r_n}{f}_{{\mathcal{C}}_n}\parallel U_n)=0,$$

or equivalently (7) and (8) if

$$\underset{n\to \infty }{lim}{\parallel 2^n{T}_{r_n}{f}_{{\mathcal{C}}_n}\parallel }_{\alpha }=1(\alpha \ne 0,1).$$

One can also define a dimensionless measure for perfect asymptotic smoothing by considering the limiting process

$$\frac{\parallel T_{r_n}{f}_{{\mathcal{C}}_n}-U_n{\parallel }_{\alpha }}{\parallel T_{r_n}{f}_{{\mathcal{C}}_n}{\parallel }_1}=2^n{\parallel T_{r_n}{f}_{{\mathcal{C}}_n}-U_n\parallel }_{\alpha }\to 0.$$

(13)

Proposition 1. 

Convergence in (13) implies perfect smoothing for all $1<\alpha \le \infty$ and is equivalent to it for $\alpha \ne \infty .$

Proof. 

Let $\mathcal{C}={\mathcal{C}}_n\subset {\mathcal{H}}_n$ for some fixed n. Since by the triangle inequality,

$$2^n\parallel T_r{f}_{\mathcal{C}}{\parallel }_{\alpha }-1\le 2^n{\parallel T_r{f}_{\mathcal{C}}-U_n\parallel }_{\alpha },$$

(13) is not weaker than the mode of convergence in Definition 1 for all $\alpha \in [1,\infty ]$. For $\alpha \ne 1,\infty$, we use Clarkson’s inequalities ([45], p. 388). Their form depends on $\alpha$; namely, for $2\le \alpha <\infty$, we have

$$\begin{array}{cc}\hfill 1+{∥\frac{2^n{T}_r{f}_{\mathcal{C}}-1}{2}∥}_{\alpha }^{\alpha }& \le {∥\frac{2^n{T}_r{f}_{\mathcal{C}}+1}{2}∥}_{\alpha }^{\alpha }+{∥\frac{2^n{T}_r{f}_{\mathcal{C}}-1}{2}∥}_{\alpha }^{\alpha }\hfill \\ \hfill & \le \frac{1}{2}(\parallel 2^n{T}_r{f}_{\mathcal{C}}{\parallel }_{\alpha }^{\alpha }+1).\hfill \end{array}$$

For $1<\alpha <2$, the inequality has the form

$$\begin{array}{cc}\hfill 1+{∥\frac{2^n{T}_r{f}_{\mathcal{C}}-1}{2}∥}_{\alpha }^{{\alpha }^{\prime }}& \le {∥\frac{2^n{T}_r{f}_{\mathcal{C}}+1}{2}∥}_{\alpha }^{{\alpha }^{\prime }}+{∥\frac{2^n{T}_r{f}_{\mathcal{C}}-1}{2}∥}_{\alpha }^{{\alpha }^{\prime }}\hfill \\ \hfill & \le {\left(\frac{1}{2}(\parallel 2^n{T}_r{f}_{\mathcal{C}}{\parallel }_{\alpha }^{\alpha }+1)\right)}^{{\alpha }^{\prime }/\alpha },\hfill \end{array}$$

where ${\alpha }^{\prime }=\frac{\alpha }{\alpha -1}$ is the Hölder conjugate. These equations show that, for $\alpha \in (1,\infty )$, $\parallel 2^n{T}_{r_n}{f}_{{\mathcal{C}}_n}{\parallel }_{\alpha }\to 1$ implies $\parallel 2^n{T}_{r_n}{f}_{{\mathcal{C}}_n}{-1\parallel }_{\alpha }\to 0,$ establishing the claimed equivalence. □

Definition 2. 

Let ${\left(r_n\right)}_n$ be a sequence of noise kernels. We say that the rate R is achievable for perfect $D_{\alpha }$-smoothing if there exists a sequence of codes ${\left({\mathcal{C}}_n\right)}_n$ such that $R\left({\mathcal{C}}_n\right)\to R$ as $n\to \infty$ and ${\left({\mathcal{C}}_n\right)}_n$ is perfectly $D_{\alpha }$-smoothable.

Note that if $R_1$ is achievable, then any rate $1\ge R_2>R_1$ is also achievable. Indeed, consider a (linear) code ${\mathcal{C}}_1$ of rate $R_1$ that has good smoothing properties. Construct ${\mathcal{C}}_2$ by taking the union of $2^{n(R_2-R_1)}$ non-overlapping shifts of ${\mathcal{C}}_1$. Then the rate of ${\mathcal{C}}_2$ is $R_2$, and since each shift has good smoothing properties, the same is true for ${\mathcal{C}}_2$. Therefore, let us define the main concept of this section.

Definition 3. 

Given a sequence of kernels $r={\left(r_n\right)}_n$, define the $D_{\alpha }$-smoothing capacity as

$$S_{\alpha }^r:=\underset{{\left({\mathcal{C}}_n\right)}_n}{inf}\{\underset{n\to \infty }{lim\; inf}R\left({\mathcal{C}}_n\right):\underset{n\to \infty }{lim}{D}_{\alpha }(T_{r_n}{f}_{{\mathcal{C}}_n}\parallel U_n)=0\}.$$

(14)

Note that this quantity is closely related to the resolvability: if, rather than optimizing on the output process in (12), we set the output distribution to uniform and take $\mathsf{\Delta }=D_{\alpha },$ then $S_{\alpha }^r$ equals $J^{\left(D_{\alpha }\right)}(\mathcal{W},P_{\mathcal{Y}})$ for the channel $\mathcal{W}$ given by the noise kernel $r.$ To avoid future confusion, we refer to the capacity of reliable transmission as Shannon’s capacity.

The following lemma provides a lower bound for $D_{\alpha }$-smoothness. It follows from Lemma 2 in [3], and we give a direct proof for completeness.

Lemma 1. 

Let $\mathcal{C}\subset {\mathcal{H}}_n$ be a code of size $M=2^{nR}$ and let r be a noise kernel. Then, for $\alpha \in [0,\infty ]$

$$D_{\alpha }(T_r{f}_{\mathcal{C}}\parallel U_n)\ge n(1-R)-H_{\alpha }\left(r\right).$$

Proof. 

We will first prove that $\parallel 2^n{T}_r{f}_{\mathcal{C}}{\parallel }_{\alpha }^{\alpha }\ge 2^{(\alpha -1)[n(1-R)-H_{\alpha }\left(r\right)]}$ for $\alpha \in (1,\infty )$:

$$\begin{array}{cc}\hfill \parallel 2^n{T}_r{f}_{\mathcal{C}}{\parallel }_{\alpha }^{\alpha }& =\frac{2^{n\alpha }}{2^n}\sum _{x\in {\mathcal{H}}_n}{T}_r{f}_{\mathcal{C}}{\left(x\right)}^{\alpha }\hfill \\ \hfill & =2^{n(\alpha -1)}\sum _{x\in {\mathcal{H}}_n}{\left[\sum _{y\in {\mathcal{H}}_n}r\left(y\right)f_{\mathcal{C}}(x-y)\right]}^{\alpha }\hfill \\ \hfill & \ge 2^{n(\alpha -1)}\sum _{x\in {\mathcal{H}}_n}\sum _{y\in {\mathcal{H}}_n}{\left[r\left(y\right)f_{\mathcal{C}}(x-y)\right]}^{\alpha }\hfill \\ \hfill & =2^{n(\alpha -1)}\sum _{y\in {\mathcal{H}}_n}r{\left(y\right)}^{\alpha }\sum _{x\in {\mathcal{H}}_n}{f}_{\mathcal{C}}{(x-y)}^{\alpha }\hfill \\ \hfill & =\frac{2^{n\alpha }}{{\left|\mathcal{C}\right|}^{(\alpha -1)}}{\parallel r\parallel }_{\alpha }^{\alpha }\hfill \\ \hfill & =2^{(\alpha -1)[n(1-R)-H_{\alpha }\left(r\right)]}.\hfill \end{array}$$

Together with (7), this implies that the claimed inequality holds for $\alpha \in (1,\infty )$.

A similar calculation shows that for $\alpha \in (0,1)$, $\parallel 2^n{T}_r{f}_{\mathcal{C}}{\parallel }_{\alpha }^{\alpha }\le 2^{(\alpha -1)[n(1-R)-H_{\alpha }\left(r\right)]},$ yielding the claim for $\alpha \in (0,1)$. The limiting cases $\alpha =0$, $\alpha =1$, and $\alpha =\infty$ follow by continuity of $D_{\alpha }$ and $H_{\alpha }$ for all $\alpha \ge 0.$ □

Define

$$\pi \left(\alpha \right)=\underset{n\to \infty }{lim\; inf}\frac{H_{\alpha }\left(r_n\right)}{n}$$

(15)

Lemma 1 shows that it is impossible to achieve perfect $D_{\alpha }$-smoothing if $R<1-\pi \left(\alpha \right).$ A question of interest is whether there exist sequences of codes of $R>1-\pi \left(\alpha \right)$ that achieve perfect $D_{\alpha }$-smoothing. The next theorem shows that this is the case for $\alpha \in (1,\infty ]$.

Theorem 3. 

Let $r={\left(r_n\right)}_n$ be a sequence of noise kernels and let $\alpha \in (1,\infty ]$. Then,

$$\begin{array}{c}\hfill S_{\alpha }^r=1-\pi \left(\alpha \right).\end{array}$$

(16)

The proof relies on a random coding argument and is given in Appendix B. This result will be used below to characterize the smoothing capacity of the Bernoulli and ball noise operators.

Remark 2. 

Equality (16) does not hold in the case $\alpha \in [0,1]$. From Theorem 4 below, the Bernoulli noise does not satisfy (16) for $\alpha \in [0,1)$. To construct a counterexample for $\alpha =1$, consider the noise kernel that is almost uniform except for one distinguished point, for instance, $r_n\left(x\right)=2^{-(n+1)}$ for $x\ne 0$ and $r_n\left(0\right)=\frac{1}{2}+\frac{1}{2^{n+1}}.$ Performing the calculations, we then obtain that $S_1^r=1$ while $\pi \left(1\right)=\frac{1}{2}.$

Remark 3. 

It is worth noting that $\pi \left(\alpha \right)$ is a decreasing function of α for $0\le \alpha \le \infty .$

4. Bernoulli Noise

In this section, we characterize the value $S_{\alpha }^{{\beta }_{\delta }}$ for a range of values of $\alpha$. Then, we provide explicit code families that attain the $D_{\alpha }$-smoothing capacities.

As already mentioned, the resolvability for ${\beta }_{\delta }$ with respect to $\alpha$-divergence was considered by Yu and Tan [3]. Their results, stated in Corollary 1, yield an expression for $S_{\alpha }^{{\beta }_{\delta }}$ for $\alpha \in [0,2]\cup \{\infty \}$. The next theorem summarizes the current knowledge about $S_{\alpha }^{{\beta }_{\delta }},$ where the claims for $2<\alpha <\infty$ form new results.

Theorem 4. 

$$\begin{array}{c}\hfill S_{\alpha }^{{\beta }_{\delta }}=\left\{\begin{array}{cc}0\hfill & if\alpha =0\hfill \\ 1-h\left(\delta \right)\hfill & if\alpha \in (0,1]\hfill \\ 1-h_{\alpha }\left(\delta \right)\hfill & if\alpha \in (1,\infty ].\hfill \end{array}\right.\end{array}$$

(17)

Proof. 

The claims for $\alpha \in [0,1]$ follow from Corollary 1. The results for $\alpha =(1,\infty ]$ follow from Theorem 3 since $\frac{H_{\alpha }\left({\beta }_{\delta }\right)}{n}=h_{\alpha }\left(\delta \right)$. □

Having quantified the smoothing capacities, let us examine the code families with strong smoothing properties. Since the $D_1$-smoothing capacity and the Shannon capacity coincide, it is natural to speculate that codes that achieve the Shannon capacity when used on the BSC$\left(\delta \right)$ would also attain the $D_1$-smoothing capacity. However, the following result demonstrates that the capacity-achieving codes do not yield perfect smoothing. For typographical reasons, we abbreviate $T_{{\beta }_{\delta }}$ by $T_{\delta }$ from this section onward.

Proposition 2. 

Let ${\mathcal{C}}_n$ be a sequence of codes achieving a capacity of $\mathrm{BSC}\left(\delta \right)$. Then,

$$D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)\to \infty ,D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)=o\left(n\right).$$

Proof. 

The second part of the statement is Theorem 2 in [46]. The first part is obtained as follows: Let ${\mathcal{C}}_n$ be a capacity-achieving sequence of codes in $\mathrm{BSC}\left(\delta \right)$. Then, from [47] (Theorem 49), there exists a constant $K>0$ such that $nR\left({\mathcal{C}}_n\right)\le n(1-h\left(\delta \right))-K\sqrt{n}$ for large n. Therefore,

$$\begin{array}{c}\hfill 0\le H\left(X_{{\mathcal{C}}_n}\right|Y_{\mathrm{BSC}\left(\delta \right),X})=n(R\left({\mathcal{C}}_n\right)+h\left(\delta \right)-1)+D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n),\end{array}$$

which implies $D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)\ge K\sqrt{n}$. □

Apart from random codes, only polar codes are known to achieve $D_1$-smoothing capacity. Before stating the formal result, recall that polar codes are formed by applying several iterations of a linear transformation to the input, which results in creating virtual channels for individual bits with Shannon’s capacity close to zero or to one, plus a vanishing proportion of intermediate-capacity channels. While by Proposition 2, that polar codes that achieve the BSC capacity cannot achieve $D_1$-smoothing capacity, adding some intermediate-bit channels to the set of data bits makes this possible. This idea was first introduced in [39] and expressed in terms of resolvability in [48].

Theorem 5 

([48], Proposition 1). Let $\mathcal{W}$ be the $\mathrm{BSC}\left(\delta \right)$ channel and ${\mathcal{W}}_n^{\left(i\right)}$ be the virtual channels formed after applying n steps of the polarization procedure. For $\gamma \in (0,1/2)$, define ${\mathcal{G}}_n=\{i\in \{1,\dots ,n\}:C\left({\mathcal{W}}_n^{\left(i\right)}\right)\ge 2^{-n^{\gamma }}\}$. Let ${\mathcal{C}}_n$ be the polar code corresponding to the virtual channels ${\mathcal{G}}_n$. Then, $D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)\to 0$.

Note that ${lim}_{n\to \infty }R\left({\mathcal{C}}_n\right)={lim}_{n\to \infty }\frac{|{\mathcal{G}}_n|}{n}=1-h\left(\delta \right)$. Hence, the polar code construction presented above achieves the perfect smoothing threshold with respect to the KL divergence. Furthermore, since the convergence in the $\alpha$ divergence for $\alpha <1$ is weaker than the convergence in $\alpha =1$, the same polar code sequence is perfectly $D_{\alpha }$-smoothable for $\alpha <1$. Noting that the smoothing threshold for $\alpha <1$ is $1-h\left(\delta \right)$ by Theorem 4, we conclude that the above polar code sequence achieves smoothing capacity in $\alpha$-divergence for $\alpha <1$.

As mentioned earlier, the smoothing properties of code families other than random codes and polar codes have not been extensively studied. We show that the duals of capacity-achieving codes in the BEC exhibit good smoothing properties using the tools developed in [10]. As the first step, we establish a connection between the smoothing of a generic linear code and the erasure correction performance of its dual code.

Lemma 2. 

Let $\mathcal{C}$ be a linear code and let $X_{{\mathcal{C}}^{\perp }}$ be a random uniform codeword of ${\mathcal{C}}^{\perp }$. Let $Y_{X_{{\mathcal{C}}^{\perp }},\mathrm{BEC}\left(\lambda \right)}$ be the output of the erasure channel $\mathrm{BEC}\left(\lambda \right)$ for the input $X_{{\mathcal{C}}^{\perp }}$. Then,

$$\begin{array}{c}\hfill D_{\alpha }(T_{\delta }{f}_{\mathcal{C}}\parallel U_n)\le H\left(X_{{\mathcal{C}}^{\perp }}\right|Y_{X_{{\mathcal{C}}^{\perp }},\mathrm{BEC}\left(\lambda \right)}),\end{array}$$

(18)

where $\lambda ={(1-2\delta )}^2$ for $\alpha =1$ and $\lambda =1-h_{\alpha }\left(\delta \right)$ for $\alpha \in \{2,3,\dots ,\infty \}$.

The proof is given in Appendix D.

Using this lemma, we show that the duals of the BEC capacity-achieving codes (with growing distance) exhibit good smoothing properties. In particular, they achieve $D_{\alpha }$-smoothing capacities for $\alpha \in \{2,3,\dots ,\infty \}$.

Theorem 6. 

Let ${\left({\mathcal{C}}_n\right)}_n$ be a sequence of linear codes with rate $R_n\to R$. Suppose that the dual sequence ${\left({\mathcal{C}}_n^{\perp }\right)}_n$ achieves Shannon’s capacity of the BEC$\left(\lambda \right)$ with $\lambda =R$, and assume that $d\left({\mathcal{C}}_n^{\perp }\right)=\omega (logn).$ If $R>{(1-2\delta )}^2$, then,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)=0.\end{array}$$

Additionally, for $\alpha \in \{2,3,\dots ,\infty \}$, if $R>1-h_{\alpha }\left(\delta \right)$, then,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{D}_{\alpha }(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)=0.\end{array}$$

In particular, the sequence ${\mathcal{C}}_n$ achieves $D_{\alpha }$-smoothing capacity $S_{\alpha }^{{\beta }_{\delta }}$ for $\alpha \in \{2,3,\dots ,\infty \}$.

Proof. 

Since the dual codes achieve the capacity of the BEC, it follows from ([49], Theorem 5.2) that, if their distance grows with n, then their decoding error probability vanishes. In particular, if $d\left({\mathcal{C}}_n^{\perp }\right)=\omega (log\left(n\right)),$ then, $P_B(\mathrm{BEC}(R-ϵ),{\mathcal{C}}_n^{\perp })=o\left(\frac{1}{n}\right)$ for all $ϵ\in (0,R].$ Hence, from Fano’s inequality,

$$\underset{n\to \infty }{lim}H\left(X_{{\mathcal{C}}_n^{\perp }}\right|Y_{X_{{\mathcal{C}}_n^{\perp }},\mathrm{BEC}(R-ϵ)})=0.$$

Now, if $R>{(1-2\delta )}^2$, then there exists ${ϵ}_0$ such that $R-{ϵ}_0={(1-2\delta )}^2$. Therefore, from Lemma 2,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)\le \underset{n\to \infty }{lim}H\left(X_{{\mathcal{C}}_n^{\perp }}\right|Y_{X_{{\mathcal{C}}_n^{\perp }},\mathrm{BEC}(R-{ϵ}_0)})=0.\end{array}$$

Similarly, $\mathrm{if}R>1-h_{\alpha }\left(\delta \right)\mathrm{for}\alpha \in \{2,3,\dots ,\infty \},\mathrm{then},{lim}_{n\to \infty }{D}_{\alpha }(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)=0.$

Together with Theorem 4, we have now proved the final claim. □

The known code families that achieve the capacity of the BEC include polar codes, LDPC codes, and doubly transitive codes, such as constant-rate RM codes. LDPC codes do not fit the assumptions because of low dual distance, but the other codes do. This yields explicit families of codes that achieve the $D_{\alpha }$-smoothing capacity.

We illustrate the results of this section in Figure 1, where the curves show the achievability and impossibility rates for perfect smoothing with respect to the Bernoulli noise. Given a code (sequence) of rate R, putting it through a noise ${\beta }_{\delta }$ below the Shannon capacity cannot achieve perfect smoothing. The sequence of polar codes from [39], cited in Theorem 5, is smoothable at rates equal to the Shannon capacity, although these codes do not provide a decoding guarantee at that noise level. At the second curve from the bottom, the duals of the codes that achieve Shannon’s capacity in BEC achieve perfect $D_1$-smoothing; at the third (fourth) curve, these codes are perfectly $D_2$- (or $D_{\infty }$-) smoothable, and they achieve the corresponding smoothing capacity.

Remark 4. 

Observe that the strong converse of the channel coding theorem does not imply perfect smoothing. To give a quick example, consider a code ${\mathcal{C}}_n=B(0,{\delta }^{\prime }n)$ formed of all the vectors in the ball. Let $0<\delta <1/2$ and let us use this code on a BSC$\left(\delta \right)$, where $h\left(\delta \right)+h\left({\delta }^{\prime }\right)>1$ and $\delta <1/2$. From the choice of the parameters, the rate of ${\mathcal{C}}_n$ is above capacity, and, therefore, $P_B(\mathrm{BSC}\left(\delta \right),{\mathcal{C}}_n)\approx 1$ from the strong converse. At the same time,

$$\begin{array}{cc}\hfill D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)& =n-H(b_{{\delta }^{\prime }n}\ast {\beta }_{\delta })=n-H({\beta }_{{\delta }^{\prime }}\ast {\beta }_{\delta })+O\left(\sqrt{n}\right)\hfill \\ \hfill & =n(1-h({\delta }^{\prime }(1-\delta )+\delta (1-{\delta }^{\prime })))+O\left(\sqrt{n}\right).\hfill \end{array}$$

where the transition from the ball noise to the Bernoulli noise (the second equality) is shown in [30]. Since ${\delta }^{\prime }(1-\delta )+\delta (1-{\delta }^{\prime }))<1/2$ for all $\delta <1/2,{\delta }^{\prime }<1/2,$ we conclude that $D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)\nrightarrow 0.$

Remark 5. 

In this paper, we mostly study the trade-off between the rate of codes and the level of the noise needed to achieve perfect smoothing. A recent work of Debris-Alazard et al. [4] considered guarantees for smoothing derived from the distance distribution of codes and their dual distance (earlier, similar calculations were performed in [42,50]). Our approach enables us to find the conditions for perfect smoothing similar to [4] but relying on fewer assumptions.

Proposition 3. 

Let ${\mathcal{C}}_n$ be a sequence of codes whose dual distance $d\left({\mathcal{C}}_n^{\perp }\right)\ge {\partial }^{\perp }n$ where ${\partial }^{\perp }\in (0,1)$. If ${\partial }^{\perp }>{(1-2\delta )}^2$, then,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}D(T_{\delta }{f}_{{\mathcal{C}}_n}\parallel U_n)=0.\end{array}$$

Proof. 

Notice that ${lim}_{n\to \infty }H\left(X_{{\mathcal{C}}_n^{\perp }}\right|Y_{X_{{\mathcal{C}}^{\perp }},\mathrm{BEC}\left(\lambda \right)})=0$ if ${\partial }^{\perp }>\lambda$. With this, the proof is a straightforward application of Lemma 2. □

Compared to [4], this claim removes the restrictions on the support of the dual distance distribution of the codes ${\mathcal{C}}_n.$

5. Binary Symmetric Wiretap Channels

In this section, we discuss applications of perfect smoothing to the BSC wiretap channel. Wyner’s wiretap channel model $\mathcal{V}$ [35] for the case of BSCs is defined as follows: The system is formed of three terminals, $A,B,$ and E. Terminal A communicates with B by sending messages M chosen from a finite set $\mathcal{M}$. Communication from A to B occurs over a BSC ${\mathcal{W}}_b$ with crossover probability ${\delta }_b$, and it is observed by the eavesdropper E via another BSC ${\mathcal{W}}_e$ with crossover probability ${\delta }_e>{\delta }_b.$ A message $M\in \mathcal{M}$ is encoded into a bit sequence $X\in {\mathcal{H}}_n$ and sent from A to B in n uses of the channel ${\mathcal{W}}_b.$ Terminal B observes the sequence $Y=X+W_b,$ where $W_b\sim \mathrm{Bin}(n,{\delta }_b)$ is the noise vector, while terminal E observes the sequence $Z=X+W_e$ with $W_e\sim \mathrm{Bin}(n,{\delta }_e)$. We assume that the messages are encoded into a subset of ${\mathcal{H}}_n,$ which imposes some probability distribution on the input of the channels. The goal of the encoding is to ensure reliability and secrecy of communication. The reliability requirement amounts to the condition $Pr(M\ne \widehat{M})\to 0$ as $n\to \infty$, where $\widehat{M}$ is the estimate of M made by B. To ensure secrecy, we require the strong secrecy condition $I(M;Z)\to 0$. This is in contrast to the condition $\frac{1}{n}I(M;Z)\to 0$ studied in the early works on the wiretap channel, which is now called weak secrecy. Denote by $R=\frac{1}{n}log\left|\mathcal{M}\right|$ the transmission rate. The secrecy capacity$C_s\left(\mathcal{V}\right)$ is defined as the supremum of the rates that permit reliable transmission, which also conforms to the secrecy condition.

The nested coding scheme, proposed by Wyner [35], has been the principal tool of constructing well-performing transmission protocols for the wiretap channel [38,39,41]. To describe it, let ${\mathcal{C}}_e$ and ${\mathcal{C}}_b$ be two linear codes such that ${\mathcal{C}}_e\subset {\mathcal{C}}_b$ and $\left|\mathcal{M}\right|=\frac{|{\mathcal{C}}_b|}{|{\mathcal{C}}_e|}$. We assign each message m to a unique coset of ${\mathcal{C}}_e$ in ${\mathcal{C}}_b$. The sequence transmitted by A is a uniform random vector from the coset. As long as the rate of the code ${\mathcal{C}}_b$ is below the capacity of ${\mathcal{W}}_b$, we can ensure the reliability of communication from A to B.

Strong secrecy can be achieved relying on perfect smoothing. Denote by $c_m$ a leader of the coset that corresponds to the message m. The basic idea is that if $P_{Z|M=m}=\left(T_{\delta }{f}_{{\mathcal{C}}_e}\right)(·+c_m)$ is close to a uniform distribution $U_n$ for all m, these conditional pmfs are almost indistinguishable from each other, and terminal E has no means of inferring the transmitted message from the observed bit string Z.

As mentioned earlier, the weak secrecy results for the wiretap channel based on LDPC codes and on polar codes were presented in [38,39], respectively. The problem that these schemes faced, highlighted in Theorems 2 and 5, is that code sequences that achieve BSC capacity have a rate gap of at least $1/\sqrt{n}$ to the capacity value. At the same time, the rate of perfectly smoothable codes must exceed the capacity by a similar quantity [51]. For this reason, the authors of [39] included the intermediate virtual channels in their polar coding scheme, which gave them strong secrecy, but interfered with transmission reliability. A similar general issue arose earlier in attempting to use LDPC codes for the wiretap channel [40].

Contributing to the line of work connecting smoothing and thewiretap channel [2,3,11], we show that nested coding schemes ${\mathcal{C}}_e\subset {\mathcal{C}}_b,$ where the code ${\mathcal{C}}_b$ is good for error correction in $\mathrm{BSC}\left({\delta }_b\right)$ and ${\mathcal{C}}_e$ is perfectly smoothable with respect to ${\beta }_{{\delta }_b}$, attain strong secrecy and reliability for a BSC wiretap channel $({\delta }_b,{\delta }_e)$. As observed in Lemma 2, the duals of the good erasure-correcting codes are perfectly smoothable for certain noise levels and, hence, they form a good choice for ${\mathcal{C}}_e$ in this scenario.

The following lemma establishes a connection between the smoothness of a noisy distribution of a code and strong secrecy.

Lemma 3. 

Consider the nested coding scheme for the BSC wiretap channel introduced above. If $D(T_{{\delta }_e}{f}_{{\mathcal{C}}_e}\parallel U_n)<ϵ,$ then $I(M;Z)<ϵ.$

Proof. 

We have

$$\begin{array}{cc}\hfill D(P_{Z|M}\parallel U_n|P_M)& =\sum _{\begin{array}{c}m\in \mathcal{M}\\ z\in {\mathcal{H}}_n\end{array}}{P}_{MZ}(m,z)log\frac{P_{Z|M}\left(z\right|m)}{U_n\left(z\right)}\hfill \\ \hfill & =I(M;Z)+D(P_Z\parallel U_n).\hfill \end{array}$$

Now, note that $P_{Z|M=m}\left(z\right)=\left(T_{{\delta }_e}{f}_{{\mathcal{C}}_e}\right)(z+c_m)=P_{Z|M=m^{\prime }}(z+c_{m^{\prime }}+c_m),$ so $D(P_{Z|M=m}\parallel U_n)$ is independent of m. Therefore, for all $m\in \mathcal{M}$

$$\begin{array}{cc}\hfill D(P_{Z|M=m}\parallel U_n)& =D(P_{Z|M}\parallel U_n|P_M)\hfill \\ \hfill & =I(M;Z)+D(P_Z\parallel U_n)\ge I(M;Z).\hspace{1em}\hspace{1em}\square \hfill \end{array}$$

This lemma enables us to formulate conditions for reliable communication while guaranteeing the strong secrecy condition. Namely, it suffices to take a pair (a sequence of pairs) of nested codes ${\mathcal{C}}_e\subset {\mathcal{C}}_b$ such that $D(T_{{\delta }_e}{f}_{{\mathcal{C}}_e}\parallel U_n)\to 0$ as $n\to \infty .$ If at the same time the code ${\mathcal{C}}_b$ corrects errors on a BSC$\left({\delta }_b\right),$ then the scheme fulfills both the reliability and strong secrecy requirements under noise levels ${\delta }_b$ and ${\delta }_e$ for channels ${\mathcal{W}}_b$ and ${\mathcal{W}}_e$, respectively, supporting transmission from A to B at rate $R_b-R_e$. Together with the results established earlier, we can now make this claim more specific.

Theorem 7. 

Let ${\left({\left({\mathcal{C}}_e^n\right)}^{\perp }\right)}_n$ and ${\left({\mathcal{C}}_b^n\right)}_n$ be sequences of linear codes that achieve the capacity of the BEC for their respective rates. Suppose that ${\mathcal{C}}_e^n\subset {\mathcal{C}}_b^n$ and

1 

$d\left({\left({\mathcal{C}}_e^n\right)}^{\perp }\right)=\omega (logn),R\left({\mathcal{C}}_e^n\right)\to R_e$;

2 

$d\left({\mathcal{C}}_b^n\right)=\omega (logn),R\left({\mathcal{C}}_b^n\right)\to R_b$.

If $R_b<1-log(1+2\sqrt{{\delta }_b(1-{\delta }_b)})$ and $R_e>4{\delta }_e(1-{\delta }_e)$, then the nested coding scheme based on ${\mathcal{C}}_e^n$ and ${\mathcal{C}}_b^n$ can transmit messages with rate $R_b-R_e$ from A to B, satisfying the reliability and strong secrecy conditions.

Proof. 

From Corollary A1, the conditions $d\left({\mathcal{C}}_b^{\left(n\right)}\right)=\omega (logn)$ and $R_b<1-log(1+2\sqrt{{\delta }_b(1-{\delta }_b)})$ guarantee transmission reliability. Furthermore, by Theorem 6, the conditions $d\left({\left({\mathcal{C}}_e^n\right)}^{\perp }\right)=\omega (logn)$ and $R_e>4{\delta }_e(1-{\delta }_e)$ imply that $D(T_{{\delta }_e}{f}_{{\mathcal{C}}_e}\parallel U_n)\to 0$, which in its turn implies strong secrecy by Lemma 3. □

To give an example of a code family that satisfies the assumptions of this theorem, consider the RM codes of constant rate. Namely, let ${\mathcal{C}}_e^n\subset {\mathcal{C}}_b^n$ be two sequences of RM codes whose rates converge to $R_e$ and $R_b$, respectively. Note that the duals of the RM codes are themselves RM codes. By a well-known result [52], the RM codes achieve the capacity of the BEC, and for any sequence of constant-rate RM codes, the distance scales as $2^{\Theta \left(\sqrt{n}\right)}$. Therefore, the RM codes satisfy the assumptions of Theorem 7.

Note that for the RM codes, we can obtain a stronger result, based on their error correction properties on the BSC. Involving this additional argument brings them closer to the secrecy capacity under the strong secrecy assumption.

Theorem 8. 

Let ${\mathcal{C}}_e^n$ and ${\mathcal{C}}_b^n$ be two sequences of RM codes satisfying ${\mathcal{C}}_e^n\subset {\mathcal{C}}_b^n$ whose rates approach $R_e>0$ and $R_b>0,$ respectively. If $R_b<1-h\left({\delta }_b\right)$ and $R_e>4{\delta }_e(1-{\delta }_e)$, then the nested coding scheme based on ${\mathcal{C}}_e^n$ and ${\mathcal{C}}_b^n$ supports transmission on a BSC wiretap channel $({\delta }_b,{\delta }_e)$ with rate $R_b-R_e,$ guaranteeing communication reliability and strong secrecy.

Proof. 

Very recently, Abbe and Sandon [53], building upon the work of Reeves and Pfister [54], proved that RM codes achieve capacity in symmetric channels. Therefore, the condition $R_b<1-h\left({\delta }_b\right)$ guarantees reliability. The rest of the proof is similar to that of Theorem 7. □

Theorems 7 and 8 stop short of constructing codes that attain the secrecy capacity of the channel (this is similar to the results of [14] for the transmission problem over the BSC). To quantify the gap to capacity, we plot the smoothing and decodability rate bounds in Figure 2.

As an example, let us set the noise parameters ${\delta }_b=0.05$ and ${\delta }_e=0.3$ and denote the corresponding secrecy capacity by $C_s$. Suppose that we use a BEC capacity-achieving code as code ${\mathcal{C}}_b$ and a dual of a BEC capacity-achieving code as code ${\mathcal{C}}_e$ in the nested scheme. The value $R^{\prime }$ is the largest rate at which we can guarantee both reliability and strong secrecy. In the example in Figure 2, $C_s=R_b^{\left(1\right)}-R_e^{\left(1\right)}=0.5949$ and $R^{\prime }=R_b^{\left(2\right)}-R_e^{\left(2\right)}=0.3181$. The only assumption required here is that the codes ${\mathcal{C}}_e^{\perp }$ and ${\mathcal{C}}_b$ have good erasure correction properties.

As noted, generally, the RM codes support a higher communication rate than the $R^{\prime }$. Let $R^{\prime \prime }$ be their achievable rate. For the same noise parameters as above, we obtain $R^{\prime \prime }=R_b^{\left(1\right)}-R_e^{\left(2\right)}=0.5536,$ which is closer to $C_s$ than $R^{\prime }$.

Remark 6. 

The fact that the RM codes achieve capacity in symmetric channels immediately implies that nested RM codes achieve the secrecy capacity in the BSC wiretap channel under weak secrecy. While it is tempting to assume that, coupled with the channel duality theorems of [55,56], this result also implies that RM codes fulfil the strong secrecy requirement on the BSC wiretap channel, an immediate proof looks out of reach [57].

Secrecy from $\alpha$-Divergence

Classically, the (strong) secrecy in the wiretap channel is measured by $I(M,Z)$. In [11], slightly weaker secrecy measures were considered besides the mutual information. However, more stringent secrecy measures may be required in certain scenarios; $\alpha$-divergence-based secrecy measures were introduced by Yu and Tan [3] as a solution to this problem.

Observe that the secrecy measured by $D_{\alpha }(P_{Z|M}\parallel U_n\left|M\right)$ for $\alpha \ge 1$ is stronger than the mutual-information-based secrecy. This is because for $\alpha \ge 1$

$$\begin{array}{c}\hfill I(M;Z)\le D(P_{Z|M}\parallel U_n|P_M)\le D_{\alpha }(P_{Z|M}\parallel U_n\left|P_M\right).\end{array}$$

Given a wiretap channel with an encoding-decoding scheme, we say the $\alpha$-secrecy is satisfied if

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{D}_{\alpha }(P_{Z|M}\parallel U_n\left|P_M\right)=0.\end{array}$$

The following theorem establishes that it is possible to achieve the rate $C\left({\delta }_b\right)-S_{\alpha }^{{\beta }_{{\delta }_e}}=h_{\alpha }\left({\delta }_e\right)-h\left({\delta }_b\right)$ with RM codes for $\alpha \in \{2,3,\dots ,\infty \}$.

Theorem 9. 

Let $\alpha \in \{2,3,\dots ,\infty \}$. Let ${\mathcal{C}}_e^n$ and ${\mathcal{C}}_b^n$ be two sequences of RM codes satisfying ${\mathcal{C}}_e^n\subset {\mathcal{C}}_b^n$ whose rates approach $R_e>0$ and $R_b>0,$ respectively. If $R_b<1-h\left({\delta }_b\right)$ and $R_e>1-h_{\alpha }\left({\delta }_e\right)$, then the nested coding scheme based on ${\mathcal{C}}_e^n$ and ${\mathcal{C}}_b^n$ supports transmission on a BSC wiretap channel $({\delta }_b,{\delta }_e)$ guaranteeing α-secrecy with rate $R_b-R_e,$ provided that $h_{\alpha }\left({\delta }_e\right)-h\left({\delta }_b\right)>0$.

Evidently, to achieve a stringent version of secrecy, it is necessary to reduce the rate of the message. The capacity of the $({\delta }_b,{\delta }_e)$-wiretap channel is $h\left({\delta }_e\right)-h\left({\delta }_b\right)$, while the known highest rate that assures $\alpha$-secrecy and reliability is $h_{\alpha }\left({\delta }_e\right)-h\left({\delta }_b\right)$. Hence, to achieve $\alpha$-secrecy, we must give up $h\left({\delta }_e\right)-h_{\alpha }\left({\delta }_e\right)$ of the attainable rate.

6. Ball Noise and Error Probability of Decoding

This section focuses on achieving the best possible smoothing with respect to the ball noise. As an application, we show that codes that possess good smoothing properties with respect to the ball noise are suitable for error correction in the BSC.

6.1. Ball Noise

Recall that the perfect smoothing of a sequence of codes is only possible if the rate is greater than the corresponding $D_{\alpha }$-smoothing capacity. In addition to characterizing the $D_{\alpha }$-smoothing capacities of the ball noise, we quantify the best smoothing one can expect with rates below the $D_{\alpha }$-smoothing capacity. We will use these results in the upcoming subsection when we derive upper bounds for the decoding error probability on a BSC. The next theorem summarizes our main result on smoothing with respect to the ball noise.

Theorem 10. 

Let ${\left(b_{\delta n}\right)}_n$ be the sequence of ball noise operators, where $\delta n$ is the radius of the ball. Let $\delta \in [0,1/2],\alpha \in [0,\infty ].$ Let ${\mathcal{C}}_n$ be a code of length n and rate $R_n.$ Then, we have the following bounds:

$$\begin{array}{cc}\hfill D_{\alpha }(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)& \ge 0\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \frac{1}{n}{D}_{\alpha }(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)& \ge 1-R_n-h\left(\delta \right).\hfill \end{array}$$

(20)

There exist sequences of codes of rate $R_n\to R$ that achieve asymptotic equality in (19) for all $R>1-h\left(\delta \right).$ At the same time, if $R<1-h\left(\delta \right)$, then there exist sequences of codes achieving asymptotic equality in (20).

Proof. 

The inequality in (19) is trivial. Let us prove that asymptotically it can be achieved with equality. From Theorem 3, there exists a sequence of codes ${\left({\mathcal{C}}_n\right)}_n$ such that $D_{\infty }(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)=$$o\left(1\right)$ given that $R>1-h\left(\delta \right)$. Hence, for $\alpha \in [0,\infty ]$

$$\begin{array}{c}\hfill 0\le D_{\alpha }(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)\le D_{\infty }(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)=o\left(1\right).\end{array}$$

Hence, the equality case in (19) is achievable for all $\alpha \in [0,\infty ]$.

Let us prove (20). From Lemma 1, we have

$$\begin{array}{cc}\hfill D_{\alpha }(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)& \ge n(1-R_n)-H_{\alpha }\left(b_{\delta n}\right)\ge n(1-R_n-h\left(\delta \right))\hfill \end{array}$$

because $\frac{1}{n}{H}_{\alpha }\left(b_{\delta n}\right)=\frac{1}{n}log{V}_{\delta n}\le h\left(\delta \right)$.

We are left to show that for $R<1-h\left(\delta \right),$ (20) can be achieved with equality in the limit of large n. We use a random coding argument to prove this. Let ${\mathcal{C}}_n$ be an $(n,2^{n{R}_n})$ code whose codewords are chosen independently and uniformly. In Equation (A6), Appendix B, we define the expected norm of the noisy function. Here, we use this quantity for the ball noise kernel. For $\alpha \in [0,\infty )$, define

$$Q_n\left(\alpha \right)={\mathbb{E}}_{{\mathcal{C}}_n}{2}^{(\alpha -1)D_{\alpha }(T_{b_{\delta n}}\parallel U_n)}.$$

From Lemma A2, for any rational $\alpha \ge 1$,

$$\begin{array}{cc}\hfill Q_n\left(\alpha \right)& \le \sum _{k=0}^p\left(\genfrac{}{}{0pt}{}{p}{k}\right)2^{\frac{nk}{q}(1-R_n-\frac{log{V}_{\delta n}}{n})}{Q}_n\left(\frac{p-k}{q}\right),\hfill \end{array}$$

(21)

for $p,q\in {\mathbb{Z}}_0^{+}$ such that $\alpha =1+\frac{p}{q}$.

Assume that $R<1-h\left(\delta \right)$. Let us prove that $Q_n\left(\alpha \right)\le 2^{n(\alpha -1)(1-R-h(\delta )+o(1\left)\right)}$ for rational values of $\alpha$ using induction. Let $\alpha \in [1,2]$ be rational and note that $p\le q$. Since $Q_n(·)\le 1$ when the argument is less than 1, we can write (21) as follows:

$$\begin{array}{cc}\hfill Q_n\left(\alpha \right)& \le \sum _{k=0}^p\left(\genfrac{}{}{0pt}{}{p}{k}\right)2^{\frac{nk}{q}(1-R_n-\frac{log{V}_{\delta n}}{n})}=2^{n(\alpha -1)(1-R-h(\delta )+o(1\left)\right)}.\hfill \end{array}$$

Now, assume that (21) holds for all rational $\alpha \in [1,m]$ for some integer $m\ge 2$ and prove that, in this case, it holds also for $\alpha \in (m,m+1].$ By the induction hypothesis,

$$\begin{array}{cc}\hfill Q_n\left(\alpha \right)& \le \sum _{0\le k\le p-q}\left(\genfrac{}{}{0pt}{}{p}{k}\right)2^{\frac{nk}{q}(1-R_n-\frac{log{V}_{\delta n}}{n})}{2}^{n\frac{p-k-q}{q}(1-R-h\left(\delta \right)+o\left(1\right))}+\sum _{k=p-q}^p\left(\genfrac{}{}{0pt}{}{p}{k}\right)2^{\frac{nk}{q}(1-R_n-\frac{log{V}_{\delta n}}{n})}\hfill \\ \hfill & \le \sum _{0\le k\le p-q}\left(\genfrac{}{}{0pt}{}{p}{k}\right)2^{n(\alpha -2)(1-R-h(\delta )+o(1\left)\right)}+\sum _{k=p-q}^p\left(\genfrac{}{}{0pt}{}{p}{k}\right)2^{n(\alpha -1)(1-R-h(\delta )+o(1\left)\right)}\hfill \\ \hfill & =2^{n(\alpha -1)(1-R-h(\delta )+o(1\left)\right)}.\hfill \end{array}$$

Therefore, for every rational $\alpha \in (1,\infty )$, there exists a sequence of codes satisfying

$$\begin{array}{c}\hfill D_{\alpha }(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)=n(1-R-h\left(\delta \right)+o\left(1\right)),\end{array}$$

(22)

which is equivalent to the equality in (20).

Let us extend this result to non-negative reals. Let $\alpha \in [0,\infty )$ and let us choose a rational ${\alpha }^{\prime }\in (1,\infty )$ such that $\alpha <{\alpha }^{\prime }$. We know that there exists a sequence of codes satisfying

$$\begin{array}{c}\hfill D_{{\alpha }^{\prime }}(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)=n(1-R-h\left(\delta \right)+o\left(1\right)).\end{array}$$

From (20) and from Remark 1,

$$\begin{array}{c}\hfill n(1-R_n-h\left(\delta \right))\le D_{\alpha }(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)\le D_{{\alpha }^{\prime }}(T_{b_{\delta n}}{f}_{{\mathcal{C}}_n}\parallel U_n)=n(1-R-h\left(\delta \right)+o\left(1\right)).\end{array}$$

Hence, the asymptotic equality in (20) is achievable for all $\alpha \in [0,\infty )$. □

The above theorem characterizes the $D_{\alpha }$-smoothing capacities with respect to ball noise.

Corollary 2. 

Let $\delta \in [0,1/2]$. Let $b\left(\delta \right)={\left(b_{\delta n}\right)}_n$ be a sequence of ball noise operators, where $\delta n$ is the radius corresponding to the n-th kernel. Then,

$$\begin{array}{cc}\hfill S_{\alpha }^{b\left(\delta \right)}& =1-h\left(\delta \right)\mathrm{for}\alpha \in [0,\infty ].\hfill \end{array}$$

The norms of $T_{b_t}{f}_{\mathcal{C}}$ can be used to bound the decoding error probability on a BSC. While estimating these norms for a given code is generally complicated, the second norm affords a compact expression based on the distance distribution of the code. In the next section, we bound the decoding error probability using the second norm of $T_{b_t}{f}_{\mathcal{C}}$. The following proposition provides closed-form expressions for $\parallel 2^n{T}_{b_t}{f}_{\mathcal{C}}{\parallel }_2^2$.

Proposition 4. 

$$\begin{array}{c}\hfill \parallel 2^n{T}_{b_t}{f}_{\mathcal{C}}{\parallel }_2^2=\frac{2^n}{\left|\mathcal{C}\right|V_t^2}\sum _{i=0}^n{\mu }_t\left(i\right)A_i=\frac{1}{V_t^2}\sum _{k=0}^n{L}_t{\left(k\right)}^2{A}_k^{\perp }.\end{array}$$

where ${\mu }_t\left(i\right)$ is defined in (1) and $L_t$ is the Lloyd polynomial of degree t (A2).

The proof is immediate from Proposition A1 in combination with (A2) and (A4).

6.2. Probability of Decoding Error on a BSC$\left(\delta \right)$

The idea that the smoothing of codes under some conditions implies good decoding performance has appeared in a number of papers using different language. The smoothing of capacity-achieving codes was considered in [18,46]. Hązła et al. [14] showed that if a code (sequence) is perfectly smoothable with respect to the Bernoulli noise, then the dual code is good for decoding (see Theorem A4, Corollary A1). Going from smoothing to decodability involves representing the $D_2$-smoothness of codes with respect to the Bernoulli noise as a potential energy form and comparing it to the Bhattacharyya bound for the dual codes. One limitation of this approach is that it cannot infer decodability for rates $R>1-log(1+2\sqrt{\delta (1-\delta )})$ (this is the region above the blue solid curve in Figure 2). Rao and Sprumont [15] and Hązła [34] proved that sufficient smoothing of codes implies the decodability of the codes themselves rather than their duals. However, these results are concerned with list decoding for rates above the Shannon capacity, resulting in an exponential list size, which is arguably less relevant from the perspective of communication.

Except for [15], the cited papers utilize perfect or near-perfect smoothing to infer decodability. For codes whose rates are below the capacity, perfect smoothing is impossible. At the same time, codes that possess sufficiently good smoothing properties are good for decoding. This property is at the root of the results for list decoding in [15]; however, their bounds were insufficient to make conclusions about list decoding below capacity.

Consider a channel where, for the input $X\sim f_{\mathcal{C}}$, the output Y is given by $Y=X+W$ with $W\sim b_t$. Define $F_t\left(y\right)=|\mathcal{C}\cap B(y,t)|$ as the number of codewords in the ball $B(y,t)$. Hence, for a received vector y, the possible number of codewords that can yield y is given by $F_t\left(y\right)$. Intuitively, the decoding error is small if $F_t\left(y\right)\approx 1$ for typical errors. Therefore, $F_t$ is of paramount interest in decoding problems. Since the typical errors for both ball noise and the Bernoulli noise are almost the same, this allows us to obtain a bound for decodability in the BSC channel. Using this approach, we show that the error probability of decoding on a $\mathrm{BSC}\left(\delta \right)$ can be expressed via the second moment of the number of codewords in the ball of radius $t\gtrsim \delta n$.

Assume, without loss of generality, that $\mathcal{C}$ is a linear code and $0^n$ is used for transmission. Let Y be the random Bernoulli vector of errors, and note that $Y\sim {\beta }_{\delta }$. The calculation below does not depend on whether we rely on unique or list decoding within a ball of radius t, so let us assume that the decoder outputs $L\ge 1$ candidate codewords conditioned on the received vector y, which is a realization of $Y.$

In this case, the list decoding error can be written as

$$\begin{array}{c}\hfill P_{L,t}(\mathcal{C},\mathrm{BSC}\left(\delta \right))=Pr\{F_t\left(Y\right)\ge L+1\cup \left|Y\right|>t\}.\end{array}$$

(23)

Theorem 11. 

Let t and $t^{\prime }$ be integers such that $0<t^{\prime }<t<n$. Then, for any $L\ge 1,$

$$\begin{array}{c}\hfill P_{L,t}(\mathcal{C},\mathrm{BSC}\left(\delta \right))\le \frac{{\beta }_{\delta }\left(t^{\prime }\right)}{L}\sum _{w=1}^n{\mu }_t\left(w\right)A_w+Pr\left(\right|Y|\le t^{\prime }\cup \left|Y\right|\ge t).\end{array}$$

(24)

Proof. 

Define $S_{t^{\prime },t}=B(0,t)\setminus B(0,t^{\prime })$. Clearly,

$$\begin{array}{cc}\hfill P_{L,t}(\mathcal{C},\mathrm{BSC}\left(\delta \right))& =Pr\{F_t\left(Y\right)\ge L+1\cup |Y|>t\}\hfill \\ \hfill & \le Pr\{(F_t\left(Y\right)\ge L+1)\cap (Y\in S_{t^{\prime },t})\}+Pr(Y\notin S_{t^{\prime },t}).\hfill \end{array}$$

Let us estimate the first of these probabilities.

$$\begin{array}{cc}\hfill & Pr\{(F_t\left(Y\right)\ge L+1)\cap (Y\in S_{t^{\prime },t})\}\hfill \\ \hfill & =\sum _{y\in S_{t^{\prime },t}}{\mathbb{1}}_{F_t\left(y\right)\ge L+1}{\beta }_{\delta }\left(y\right)\hfill \\ \hfill & \le \sum _{y\in S_{t^{\prime },t}}\frac{F_t\left(y\right)-1}{L}{\beta }_{\delta }\left(y\right)\hfill \\ \hfill & \le \frac{{\beta }_{\delta }\left(t^{\prime }\right)}{L}\sum _{y\in S_{t^{\prime },t}}(F_t\left(y\right)-1)\hfill \\ \hfill & \le \frac{{\beta }_{\delta }\left(t^{\prime }\right)}{L}\sum _{y\in B(0,t)}(F_t\left(y\right)-1)(\mathrm{because}\mathrm{for}\mathrm{all}y\in B(0,t),F_t\left(y\right)\ge 1)\hfill \\ \hfill & =\frac{{\beta }_{\delta }\left(t^{\prime }\right)}{L}\left(\sum _{y\in {\mathcal{H}}_n}({\mathbb{1}}_{\mathcal{C}}\ast {\mathbb{1}}_{B(0,t)})\left(y\right){\mathbb{1}}_{B(0,t)}\left(y\right)-V_t\right)\hfill \\ \hfill & =\frac{{\beta }_{\delta }\left(t^{\prime }\right)}{L}\left(\sum _{c\in \mathcal{C}}({\mathbb{1}}_{B(0,t)}\ast {\mathbb{1}}_{B(0,t)})\left(c\right)-V_t\right)\hfill \\ \hfill & =\frac{{\beta }_{\delta }\left(t^{\prime }\right)}{L}\sum _{i=1}^n{\mu }_t\left(i\right)A_i.\hspace{1em}\hspace{1em}\square \hfill \end{array}$$

Remark 7. 

In the case of $L=1$, the bound in (24) can be considered a slightly weaker version of Poltyrev’s bound [58], Lemma 1. By allowing this weakening, we obtain a bound in a somewhat more closed form, also connecting the decodability with smoothing. We also prove a simple bound for the error probability of list decoding expressed in terms of the code’s distance distribution (and, from (A4), also in terms of the dual distance distribution). The latter result seems not to have appeared in earlier literature.

The following version of this lemma provides an error bound, which is useful in the asymptotic setting.

Proposition 5. 

Let $t=\delta n+n^{\theta },$ where $\theta \in (1/2,1).$ Then,

$$\begin{array}{c}\hfill P_{L,t}(\mathcal{C},\mathrm{BSC}\left(\delta \right))\le \frac{\sqrt{2n}}{L{V}_t}{\left(\frac{1-\delta }{\delta }\right)}^{2n^{\theta }}\sum _{w=1}^n{\mu }_t\left(w\right)A_w+2e^{-n^{2\theta -1}}.\end{array}$$

In particular,

$$\begin{array}{c}\hfill P_{L,t}(\mathcal{C},\mathrm{BSC}\left(\delta \right))\le \frac{\sqrt{2n}}{V_t}{\left(\frac{1-\delta }{\delta }\right)}^{2n^{\theta }}\sum _{w=1}^n{\mu }_t\left(w\right)A_w+2e^{-n^{2\theta -1}}.\end{array}$$

Proof. 

Set $t^{\prime }=\delta n-n^{\theta }$. A direct calculation shows that

$${\beta }_{\delta }\left(t^{\prime }\right)V_t<\sqrt{2n}{\left(\frac{1-\delta }{\delta }\right)}^{2n^{\theta }}.$$

By the Hoeffding bound,

$$\begin{array}{c}\hfill Pr\left(\right|Y|\le t^{\prime }\cup \left|Y\right|\ge t)\le 2e^{-n^{2\theta -1}}.\end{array}$$

Together with Lemma 11, this implies our statements. □

A question of prime importance is whether the right-hand side quantities in Proposition 5 converge to 0. For $R<1-h\left(\delta \right)$, one can easily see that for random codes, ${\sum }_{w=1}^n\frac{{\mu }_t\left(w\right)}{V_t}{A}_w=2^{-\Theta \left(n\right)}$, where $t=\delta n+n^{\theta }$, showing that this is, in fact, the case.

From Proposition 4, it is clear that the potential energy ${\sum }_{w=1}^n{\mu }_t\left(w\right)A_w$ is a measure of the smoothness of $T_{b_t}{f}_{\mathcal{C}}$. This implies that codes that are sufficiently smoothable with respect to $b_t$ are decodable in the BSC with vanishing error probability. In other words, Proposition 5 establishes a connection between the smoothing and the decoding error probability.

7. Perfect Smoothing—The Finite Case

In this section, we briefly overview another form of perfect smoothing, which is historically the earliest application of these ideas in coding theory. It is not immediately related to the information-theoretic problems considered in the other parts.

We are interested in radial kernels that yield perfect smoothing for a given code. We often write $r\left(i\right)$ instead of $r\left(x\right)$ if $\left|x\right|=i,$ and call $\rho \left(r\right):=max(i:r(i)\ne 0)$ the radius of r. Note that the logarithm of the support size of r (as a function on the space ${\mathcal{H}}_n$) is exactly the 0-Rényi entropy of r. Therefore, kernels with smaller radii can be perceived as less random, supporting the view of the radius $\rho \left(r\right)$ as a general measure of randomness.

Definition 4. 

We say a code $\mathcal{C}$ is perfectly smoothable with respect to r if $T_r{f}_{\mathcal{C}}\left(x\right)=\frac{1}{2^n}$ for all $x\in {\mathcal{H}}_n$, and, in this case, we say that r is a perfectly smoothing kernel for $\mathcal{C}$.

Intuitively, such a kernel should have a sufficiently large radius. In particular, it should be as large as the covering radius of the code $\rho \left(\mathcal{C}\right)$ or otherwise smoothing does not affect the vectors that are $\rho$ away from the code. To obtain a stronger condition, recall that the external distance of code $\mathcal{C}$ is $\overline{d}\left(\mathcal{C}\right)=\left|\{i\ge 1:A_i^{\perp }\ne 0\}\right|.$

Proposition 6. 

Let r be a perfectly smoothing kernel of code $\mathcal{C}$. Then, $\rho \left(r\right)\ge \overline{d}\left(\mathcal{C}\right).$

Proof. 

Note that perfect smoothing of $\mathcal{C}$ with respect to r is equivalent to

$$\begin{array}{c}\hfill \parallel 2^n{T}_r{f}_{\mathcal{C}}{\parallel }_2^2=1,\end{array}$$

which by Proposition A1 is equivalent to the following condition:

$$\begin{array}{c}\hfill \sum _{i=1}^n\widehat{r}{\left(i\right)}^2{A}_i^{\perp }=0.\end{array}$$

Therefore,

$$\begin{array}{c}\hfill \overline{d}\left(\mathcal{C}\right)=|\{i\ge 1:A_i^{\perp }\ne 0\}|\le n-|\{i\ge 1:\widehat{r}\left(i\right)\ne 0\}|.\end{array}$$

By definition,

$$\begin{array}{c}\hfill \widehat{r}=\frac{1}{2^n}{K}^{⊺}r,\end{array}$$

where $K={\left(K_i\left(j\right)\right)}_{i,j=0}^n$ is the Krawtchouk matrix. Define $I_1=\{j\in \{1,2,\dots ,n\}:\widehat{r}\left(j\right)=0\}$ and $I_2=\{i\in \{1,2,\dots ,n\}:r\left(i\right)\ne 0\}$ then,

$$\begin{array}{c}\hfill 0=\widehat{r}{|}_{I_1}=\frac{1}{2^n}{K}^{⊺}{|}_{(I_1,:)}r=\frac{1}{2^n}{K}^{⊺}{{|}_{(I_1,I_2)}r|}_{I_2}.\end{array}$$

This relation implies that there exists a linear combination of Krawtchouk polynomials of degree at most $\rho \left(r\right)$ with $|I_1|$ roots. Therefore, $\overline{d}\left(\mathcal{C}\right)\le n-|supp\left({\left\{\widehat{r}\left(i\right)\right\}}_{i=1}^n\right)|=|I_1|\le \rho \left(r\right).$ □

Since $\rho \left(\mathcal{C}\right)\le \overline{d}\left(\mathcal{C}\right)$, this inequality strengthens the obvious condition $\rho \left(r\right)\ge \rho \left(\mathcal{C}\right).$ At the same time, there are codes that are perfectly smoothable by a radial kernel r such that $\rho \left(r\right)=\rho \left(\mathcal{C}\right)$.

Definition 5 

([59]). A code $\mathcal{C}$ is uniformly packed in the wide sense if there exists rational numbers ${\left\{{\alpha }_i\right\}}_{i=0}^{\rho }$ such that

$$\begin{array}{c}\hfill \sum _{i=0}^{\rho \left(\mathcal{C}\right)}{\alpha }_i{A}_i\left(x\right)=1\mathit{for}\mathit{all}x\in {\mathcal{H}}_n,\end{array}$$

where $A_i\left(x\right)$ is the weight distribution of the code $\mathcal{C}-x$.

Our main observation here is that some uniformly packed codes are perfectly smoothable with respect to noise kernels that are minimal in a sense. The following proposition states this more precisely.

Proposition 7. 

Let $\mathcal{C}$ be a code that is perfectly smoothable by a radial kernel of radius $\rho \left(r\right)=\rho \left(\mathcal{C}\right)$. Then, $\mathcal{C}$ is uniformly packed in the wide sense with ${\alpha }_i\ge 0$ for all i.

Proof. 

By definition, if $\mathcal{C}$ is perfectly smoothable with respect to r, then $2^n{T}_r{f}_{\mathcal{C}}=1$, which is tantamount to ${\sum }_{y\in {\mathcal{H}}_n}\frac{2^n}{\left|\mathcal{C}\right|}r\left(y\right){\mathbb{1}}_{\mathcal{C}}(x-y)=1$ for all $x\in {\mathcal{H}}_n$. This condition can be written as ${\sum }_{i=0}^{\rho }\left(\frac{2^n}{\left|\mathcal{C}\right|}r\left(i\right)\right)A_i\left(x\right)=1$ for all $x\in {\mathcal{H}}_n$, completing the proof. □

To illustrate this claim, we list several families of uniformly packed codes ([59,60,61]) that are perfectly smoothable by a kernel of radius equal to the covering radius of the code.

(i)

Perfect codes: $r=b_{\rho }$, where $\rho =\rho \left(\mathcal{C}\right)$ is the covering radius.

(ii)

2-error-correcting BCH codes of length $2^{2m+1},m\ge 2$. The smoothing kernel r is given by

$$r\left(0\right)=r\left(1\right)=L,r\left(2\right)=r\left(3\right)=\frac{3L}{n},r\left(i\right)=0,i\ge 4.$$

(iii)

Preparata codes. The smoothing kernel r is given by

$$r\left(0\right)=r\left(1\right)=L,r\left(2\right)=r\left(3\right)=\frac{6L}{n-1},r\left(i\right)=0,i\ge 4.$$

(iv)

Binary $(2^m-1,2^{2^m-3m+2},7)$ Goethals-like codes [60]. The smoothing kernel r is given by

$$r\left(0\right)=r\left(1\right)=L,r\left(2\right)=r\left(3\right)=\frac{65L}{2n},r\left(4\right)=r\left(5\right)=\frac{30L}{n(n-3)},r\left(i\right)=0,i\ge 4.$$

Here, L is a generic notation for the normalizing factor. More examples are found in a related class of completely regular codes [62].

Definition 5 does not include the condition that ${\alpha }_i\ge 0$, and, in fact, there are codes that are uniformly packed in the wide sense, but some of the ${\alpha }_i$’s are negative, and, thus, they are not smoothable by a noise kernel of radius $\rho \left(\mathcal{C}\right)$. One such family is the 3-error-correcting binary BCH codes of length $2^{2m+1},m\ge 2$ [60].