# Smoothing of Binary Codes, Uniform Distributions, and Applications

^{*}

## Abstract

**:**

## 1. Introduction

- Characterizing the ${D}_{\alpha}$-smoothing capacities of noise operators on the Hamming space for $\alpha \in (1,\infty ]$.
- Identifying some explicit code families that attain a smoothing capacity of the Bernoulli noise for $\alpha \in \{2,3,\dots ,\infty \}$;
- Obtaining rate estimates for the RM codes used on the BSC wiretap channel under the strong secrecy condition;
- Showing that codes possessing sufficiently good smoothing properties are suitable for error correction.

## 2. Preliminaries

#### 2.1. Notation

#### 2.2. ${D}_{\alpha}$- and ${L}_{\alpha}$-Smoothness

**Remark 1.**

#### 2.3. Resolvability

**Theorem 1**

**Theorem 2**

**Corollary 1**

## 3. Perfect Smoothing—The Asymptotic Case

**Definition 1.**

**Proposition 1.**

**Proof.**

**Definition 2.**

**Definition 3.**

**Lemma 1.**

**Proof.**

**Theorem 3.**

**Remark 2.**

**Remark 3.**

## 4. Bernoulli Noise

**Theorem 4.**

**Proof.**

**Proposition 2.**

**Proof.**

**Theorem 5**

**Lemma 2.**

**Theorem 6.**

**Proof.**

**Remark 4.**

**Remark 5.**

**Proposition 3.**

**Proof.**

## 5. Binary Symmetric Wiretap Channels

**Lemma 3.**

**Proof.**

**Theorem 7.**

- 1
- $d\left({\left({\mathcal{C}}_{e}^{n}\right)}^{\perp}\right)=\omega (logn),R\left({\mathcal{C}}_{e}^{n}\right)\to {R}_{e}$;
- 2
- $d\left({\mathcal{C}}_{b}^{n}\right)=\omega (logn),R\left({\mathcal{C}}_{b}^{n}\right)\to {R}_{b}$.

**Proof.**

**Theorem 8.**

**Proof.**

**Remark 6.**

#### Secrecy from $\alpha $-Divergence

**Theorem 9.**

## 6. Ball Noise and Error Probability of Decoding

#### 6.1. Ball Noise

**Theorem 10.**

**Proof.**

**Corollary 2.**

**Proposition 4.**

#### 6.2. Probability of Decoding Error on a BSC$\left(\delta \right)$

**Theorem 11.**

**Proof.**

**Remark 7.**

**Proposition 5.**

**Proof.**

## 7. Perfect Smoothing—The Finite Case

**Definition 4.**

**Proposition 6.**

**Proof.**

**Definition 5**

**Proposition 7.**

**Proof.**

- (i)
- Perfect codes: $r={b}_{\rho}$, where $\rho =\rho \left(\mathcal{C}\right)$ is the covering radius.
- (ii)
- 2-error-correcting BCH codes of length ${2}^{2m+1},m\ge 2$. The smoothing kernel r is given by$$r\left(0\right)=r\left(1\right)=L,r\left(2\right)=r\left(3\right)=\frac{3L}{n},r\left(i\right)=0,i\ge 4.$$
- (iii)
- Preparata codes. The smoothing kernel r is given by$$r\left(0\right)=r\left(1\right)=L,r\left(2\right)=r\left(3\right)=\frac{6L}{n-1},r\left(i\right)=0,i\ge 4.$$
- (iv)
- Binary $({2}^{m}-1,{2}^{{2}^{m}-3m+2},7)$ Goethals-like codes [60]. The smoothing kernel r is given by$$r\left(0\right)=r\left(1\right)=L,r\left(2\right)=r\left(3\right)=\frac{65L}{2n},r\left(4\right)=r\left(5\right)=\frac{30L}{n(n-3)},r\left(i\right)=0,i\ge 4.$$

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. L_{2} Smoothing

**Proposition A1.**

**Proof.**

## Appendix B. Proof of Theorem 3

**Lemma A1.**

**Proof.**

**Lemma A2.**

**Proof.**

**Theorem A1.**

**Proposition A2.**

**Proof.**

**Lemma A3.**

**Proof.**

**Lemma A4.**

**Proof.**

## Appendix C. Samorodnitsky’s Inequalities and Their Implications

**Theorem A2**

**Theorem A3**

**Theorem A4**

**Corollary A1**

## Appendix D. Proof of Lemma 2

**Lemma A5.**

**Proof.**

**Lemma A6.**

**Proof.**

**Lemma A7.**

**Proof.**

**Lemma A8.**

**Proof.**

## References

- Han, T.S.; Verdú, S. Approximation theory of output statistics. IEEE Trans. Inf. Theory
**1993**, 39, 752–772. [Google Scholar] [CrossRef] - Hayashi, M. General nonasymptotic and asymptotic formulas in channel resolvability and identification capacity and their application to the wiretap channel. IEEE Trans. Inf. Theory
**2006**, 52, 1562–1575. [Google Scholar] [CrossRef] - Yu, L.; Tan, V.Y. Rényi resolvability and its applications to the wiretap channel. IEEE Trans. Inf. Theory
**2019**, 65, 1862–1897. [Google Scholar] [CrossRef] - Debris-Alazard, T.; Ducas, L.; Resch, N.; Tillich, J.-P. Smoothing codes and lattices: Systematic study and new bounds. IEEE Trans. Inf. Theory
**2023**, 69, 6006–6027. [Google Scholar] [CrossRef] - Micciancio, D.; Regev, O. Worst-case to average-case reductions based on Gaussian measures. SIam J. Comput.
**2007**, 37, 267–302. [Google Scholar] [CrossRef] - Chen, W.W.L.; Skriganov, M.M. Explicit constructions in the classical mean squares problem in irregularities of point distribution. J. Fur Die Reine Und Angew. Math.
**2002**, 545, 67–95. [Google Scholar] [CrossRef] - Skriganov, M.M. Coding theory and uniform distributions. Algebra Anal.
**2001**, 13, 191–239, Translation in St. Petersburg Math. J.**2002**, 13, 301–337. [Google Scholar] - Samorodnitsky, A. On the entropy of a noisy function. IEEE Trans. Inf. Theory
**2016**, 62, 5446–5464. [Google Scholar] [CrossRef] - Samorodnitsky, A. An upper bound on ℓ
_{q}norms of noisy functions. IEEE Trans. Inf. Theory**2019**, 66, 742–748. [Google Scholar] [CrossRef] - Samorodnitsky, A. An improved bound on ℓ
_{q}norms of noisy functions. arXiv**2020**, arXiv:2010.02721. [Google Scholar] [CrossRef] - Bloch, M.R.; Laneman, J.N. Strong secrecy from channel resolvability. IEEE Trans. Inf. Theory
**2013**, 59, 8077–8098. [Google Scholar] [CrossRef] - Belfiore, J.-C.; Oggier, F. Secrecy gain: A wiretap lattice code design. In Proceedings of the 2010 International Symposium on Information Theory & Its Applications, Taichung, Taiwan, 17–20 October 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 174–178. [Google Scholar] [CrossRef]
- Luzzi, L.; Ling, C.; Bloch, M.R. Optimal rate-limited secret key generation from Gaussian sources using lattices. IEEE Trans. Inf. Theory
**2023**, 69, 4944–4960. [Google Scholar] [CrossRef] - Hązła, J.H.; Samorodnitsky, A.; Sberlo, O. On codes decoding a constant fraction of errors on the BSC. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, Virtual, 21–25 June 2021; pp. 1479–1488. [Google Scholar] [CrossRef]
- Rao, A.; Sprumont, O. A criterion for decoding on the BSC. arXiv
**2022**, arXiv:2202.00240. [Google Scholar] [CrossRef] - Arimoto, S. On the converse to the coding theorem for discrete memoryless channels (corresp.). IEEE Trans. Inf. Theory
**1973**, 19, 357–359. [Google Scholar] [CrossRef] - Polyanskiy, Y.; Verdú, S. Arimoto channel coding converse and Rényi divergence. In Proceedings of the 2010 48th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, 29 September–1 October 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 1327–1333. [Google Scholar] [CrossRef]
- Polyanskiy, Y.; Verdú, S. Empirical distribution of good channel codes with nonvanishing error probability. IEEE Trans. Inf. Theory
**2013**, 60, 5–21. [Google Scholar] [CrossRef] - Chou, R.A.; Bloch, M.R.; Kliewer, J. Empirical and strong coordination via soft covering with polar codes. IEEE Trans. Inf. Theory
**2018**, 64, 5087–5100. [Google Scholar] [CrossRef] - Cover, T.M.; Permuter, H.H. Capacity of coordinated actions. In Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France, 24–29 June 2007; IEEE: Piscataway, NJ, USA, 2007; pp. 2701–2705. [Google Scholar] [CrossRef]
- Cuff, P. Distributed channel synthesis. IEEE Trans. Inf. Theory
**2013**, 59, 7071–7096. [Google Scholar] [CrossRef] - Cuff, P.W.; Permuter, H.H.; Cover, T.M. Coordination capacity. IEEE Trans. Inf. Theory
**2010**, 56, 4181–4206. [Google Scholar] [CrossRef] - Chou, R.A.; Bloch, M.R.; Abbe, E. Polar coding for secret-key generation. IEEE Trans. Inf. Theory
**2015**, 61, 6213–6237. [Google Scholar] [CrossRef] - Brakerski, Z.; Lyubashevsky, V.; Vaikuntanathan, V.; Wichs, D. Worst-case hardness for LPN and cryptographic hashing via code smoothing. In Proceedings of the Annual International Conference on the Theory and Applications of Cryptographic Techniques, Paris, France, 30 April– 4 May 2017; Springer: Berlin/Heidelberg, Germany, 2019; pp. 619–635. [Google Scholar]
- Goldfeld, Z.; Kato, K.; Nietert, S.; Rioux, G. Limit distribution theory for smooth p-Wasserstein distances. arXiv
**2022**, arXiv:2203.00159. [Google Scholar] [CrossRef] - Goldfeld, Z.; Kato, K.; Rioux, G.; Sadhu, R. Statistical inference with regularized optimal transport. arXiv
**2022**, arXiv:2205.04283. [Google Scholar] [CrossRef] - Nietert, S.; Goldfeld, Z.; Kato, K. Smooth p-Wasserstein distance: Structure, empirical approximation, and statistical applications. In Proceedings of the International Conference on Machine Learning, Virtual, 8–24 July 2021; pp. 8172–8183. [Google Scholar]
- Liu, J.; Cuff, P.; Verdú, S. E
_{γ}-resolvability. IEEE Trans. Inf. Theory**2016**, 63, 2629–2658. [Google Scholar] [CrossRef] - Steinberg, Y.; Verdú, S. Simulation of random processes and rate-distortion theory. IEEE Trans. Inf. Theory
**1996**, 42, 63–86. [Google Scholar] [CrossRef] - Ordentlich, O.; Polyanskiy, Y. Entropy under additive Bernoulli and spherical noises. In Proceedings of the 2018 IEEE International Symposium on Information Theory (ISIT), Vail, CO, USA, 17–22 June 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 521–525. [Google Scholar] [CrossRef]
- Polyanskiy, Y. Hypercontractivity of spherical averages in Hamming space. Siam J. Discret. Math.
**2019**, 33, 731–754. [Google Scholar] [CrossRef] - Yu, L. Edge-isoperimetric inequalities and ball-noise stability: Linear programming and probabilistic approaches. J. Comb. Theory Ser. A
**2022**, 188, 105583. [Google Scholar] [CrossRef] - Wyner, A.; Ziv, J. A theorem on the entropy of certain binary sequences and applications–I. IEEE Trans. Inf. Theory
**1973**, 19, 769–772. [Google Scholar] [CrossRef] - Hązła, J.H. Optimal list decoding from noisy entropy inequality. arXiv
**2022**, arXiv:2212.01443. [Google Scholar] [CrossRef] - Wyner, A.D. The wire-tap channel. Bell Syst. Tech. J.
**1975**, 54, 1355–1387. [Google Scholar] [CrossRef] - Csiszár, I. Almost independence and secrecy capacity. Probl. Peredachi Informatsii
**1996**, 32, 48–57, English translation in Probl. Inform. Transm.**1996**, 32, 40–47. [Google Scholar] - Maurer, U.M. Secret key agreement by public discussion from common information. IEEE Trans. Inf. Theory
**1993**, 39, 733–742. [Google Scholar] [CrossRef] - Thangaraj, A.; Dihidar, S.; Calderbank, A.R.; McLaughlin, S.W.; Merolla, J.-M. Applications of LDPC codes to the wiretap channel. IEEE Trans. Inf. Theory
**2007**, 53, 2933–2945. [Google Scholar] [CrossRef] - Mahdavifar, H.; Vardy, A. Achieving the secrecy capacity of wiretap channels using polar codes. IEEE Trans. Inf. Theory
**2011**, 57, 6428–6443. [Google Scholar] [CrossRef] - Subramanian, A.; Suresh, A.T.; Raj, S.; Thangaraj, A.; Bloch, M.; McLaughlin, S. Strong and weak secrecy in wiretap channels. In Proceedings of the 2010 6th International Symposium on Turbo Codes & Iterative Information Processing, Brest, France, 6–10 September 2010; pp. 30–34. [Google Scholar] [CrossRef]
- Gulcu, T.C.; Barg, A. Achieving secrecy capacity of the wiretap channel and broadcast channel with a confidential component. IEEE Trans. Inf. Theory
**2016**, 63, 1311–1324. [Google Scholar] [CrossRef] - Barg, A. Stolarsky’s invariance principle for finite metric spaces. Mathematika
**2021**, 67, 158–186. [Google Scholar] [CrossRef] - Bilyk, D.; Dai, F.; Matzke, R. The Stolarsky principle and energy optimization on the sphere. Constr. Approx.
**2018**, 48, 31–60. [Google Scholar] [CrossRef] - Skriganov, M.M. Point distributions in two-point homogeneous spaces. Mathematika
**2019**, 65, 557–587. [Google Scholar] [CrossRef] - Simon, B. Real Analysis: A Comprehensive Course in Analysis, Part 1; American Mathematical Society: Providence, RI, USA, 2015. [Google Scholar] [CrossRef]
- Shamai, S.; Verdú, S. The empirical distribution of good codes. IEEE Trans. Inf. Theory
**1997**, 43, 836–846. [Google Scholar] [CrossRef] - Polyanskiy, Y.; Poor, H.V.; Verdú, S. Channel coding rate in the finite blocklength regime. IEEE Trans. Inf. Theory
**2010**, 56, 2307–2359. [Google Scholar] [CrossRef] - Bloch, M.R.; Luzzi, L.; Kliewer, J. Strong coordination with polar codes. In Proceedings of the 2012 50th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, 1–5 October 2012; IEEE: Piscataway, NJ, USA, 2012; pp. 565–571. [Google Scholar] [CrossRef]
- Tillich, J.-P.; Zémor, G. Discrete isoperimetric inequalities and the probability of a decoding error. Comb. Probab. Comput.
**2000**, 9, 465–479. [Google Scholar] [CrossRef] - Ashikhmin, A.; Barg, A. Bounds on the covering radius of linear codes. Des. Codes Cryptogr.
**2002**, 27, 261–269. [Google Scholar] [CrossRef] - Watanabe, S.; Hayashi, M. Strong converse and second-order asymptotics of channel resolvability. In Proceedings of the 2014 IEEE International Symposium on Information Theory, Honolulu, HI, USA, 29 June–4 July 2014; pp. 1882–1886. [Google Scholar] [CrossRef]
- Kudekar, S.; Kumar, S.; Mondelli, M.; Pfister, H.D.; Şaşoğlu, E.; Urbanke, R. Reed-Muller codes achieve capacity on erasure channels. In Proceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing, Cambridge, MA, USA, 19–21 June 2016; pp. 658–669. [Google Scholar] [CrossRef]
- Abbe, E.; Sandon, C. A proof that Reed-Muller codes achieve Shannon capacity on symmetric channels. arXiv
**2023**, arXiv:2304.02509. [Google Scholar] [CrossRef] - Reeves, G.; Pfister, H.D. Reed-Muller codes achieve capacity on BMS channels. arXiv
**2021**, arXiv:2110.14631. [Google Scholar] [CrossRef] - Renes, J.M. Duality of channels and codes. IEEE Trans. Inf. Theory
**2018**, 64, 577–592. [Google Scholar] [CrossRef] - Rengaswamy, N.; Pfister, H.D. On the duality between the BSC and quantum PSC. In Proceedings of the 2021 IEEE International Symposium on Information Theory (ISIT), Melbourne, Australia, 12–20 July 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 2232–2237. [Google Scholar] [CrossRef]
- Pfister, H.; Department of Electrical and Computer Engineering, Duke University, Durham, NC, USA; Rengaswamy, N.; Department of Electrical and Computer Engineering, University of Arizona, Tucson, AZ, USA. Personal Communication, 2023.
- Poltyrev, G. Bounds on the decoding error probability of binary linear codes via their spectra. IEEE Trans. Inf. Theory
**1994**, 40, 1284–1292. [Google Scholar] [CrossRef] - Semakov, N.; Zinov’ev, V.A.; Zaitsev, G. Uniformly packed codes. Probl. Peredachi Informatsii
**1971**, 7, 38–50. [Google Scholar] - Goethals, J.-M.; van Tilborg, H.C.A. Uniformly packed codes. Philips Res. Rep.
**1975**, 30, 9–36. [Google Scholar] - Tokareva, N. An upper bound for the number of uniformly packed codes. In Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France, 24–29 June 2007; IEEE: Piscataway, NJ, USA, 2007; pp. 346–349. [Google Scholar] [CrossRef]
- Borges, J.; Rifà, J.; Zinoviev, V.A. On completely regular codes. Probl. Inf. Transm.
**2019**, 55, 1–45. [Google Scholar] [CrossRef] - Delsarte, P. An algebraic approach to the association schemes of coding theory. Philips Res. Repts. Suppl.
**1973**, 10, 1973. [Google Scholar] - Oxley, J. Matroid Theory; Oxford University Press: Oxford, UK, 1992. [Google Scholar]

**Figure 1.**Capacities and achievable rates for perfect smoothing. The lowermost curve gives the Shannon capacity of the BSC$\left(\delta \right)$, the second curve from the bottom is the smoothing threshold for the duals of the BEC capacity-achieving codes, the third one is ${S}_{2}^{{\beta}_{\delta}}$ and the top one is ${S}_{\infty}^{{\beta}_{\delta}}$.

**Figure 2.**Achievable rates in the BSC wiretap channel with BEC capacity-achieving codes. The bottom curve is the lower bound on the code rate that guarantees decodability on a BSC$\left(\delta \right)$. The middle curve shows Shannon’s capacity and the top one is the ${D}_{1}$-smoothing threshold for the Bernoulli noise ${T}_{\delta}$.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pathegama, M.; Barg, A.
Smoothing of Binary Codes, Uniform Distributions, and Applications. *Entropy* **2023**, *25*, 1515.
https://doi.org/10.3390/e25111515

**AMA Style**

Pathegama M, Barg A.
Smoothing of Binary Codes, Uniform Distributions, and Applications. *Entropy*. 2023; 25(11):1515.
https://doi.org/10.3390/e25111515

**Chicago/Turabian Style**

Pathegama, Madhura, and Alexander Barg.
2023. "Smoothing of Binary Codes, Uniform Distributions, and Applications" *Entropy* 25, no. 11: 1515.
https://doi.org/10.3390/e25111515