The Fractality of Polar and Reed–Muller Codes ^†

Abstract

The generator matrices of polar codes and Reed–Muller codes are submatrices of the Kronecker product of a lower-triangular binary square matrix. For polar codes, the submatrix is generated by selecting rows according to their Bhattacharyya parameter, which is related to the error probability of sequential decoding. For Reed–Muller codes, the submatrix is generated by selecting rows according to their Hamming weight. In this work, we investigate the properties of the index sets selecting those rows, in the limit as the blocklength tends to infinity. We compute the Lebesgue measure and the Hausdorff dimension of these sets. We furthermore show that these sets are finely structured and self-similar in a well-defined sense, i.e., they have properties that are common to fractals.

1. Introduction

In his book on fractal geometry, Falconer characterizes a set $\mathcal{F}$ as a fractal if it has some of the following properties [1] (p. xxviii):

• $\mathcal{F}$ has a fine structure, i.e., there is detail on arbitrarily small scales
• $\mathcal{F}$ does not admit a description in traditional geometrical language, neither locally nor globally; it is irregular in some sense
• $\mathcal{F}$ has some form of self-similarity, at least approximate or statistical
• The fractal dimension of $\mathcal{F}$ exceeds its topological dimension
• $\mathcal{F}$ is defined in a simple, often recursive way

In this work, we investigate whether polar codes and Reed–Muller are fractal in above sense. For a blocklength of $2^n$, these codes are based on the n-fold Kronecker product $G\left(n\right):=F^{\otimes n}$, where

$$F:=\left[\begin{array}{cc}1& 0\\ 1& 1\end{array}\right]$$

(1)

i.e., on a simple, recursive operation. Based on this, it has long been suspected that Kronecker product-based codes possess a fractal nature. For example, the authors of [2] observed that $G\left(n\right)$, when converted to a picture, resembles the Sierpinski triangle. In a personal communication [3], Abbe expressed his suspicion that the set of “good” polarized channels is fractal. Nevertheless, to the best of the author’s knowledge, a definite statement regarding this fractal nature has not been presented yet.

A rate-$K/2^n$ Kronecker product-based code is uniquely defined by a set $\mathcal{F}$ of K indices: Its generator matrix is the submatrix of $G\left(n\right)$ consisting of the rows indexed by $\mathcal{F}$. Letting $\mathcal{F}$ index the K rows of $G\left(n\right)$ with the largest Hamming weight defines a Reed–Muller code. Alternatively, one can fix the order r of a Reed–Muller code, which defines $\mathcal{F}$ as the index set of all rows with a Hamming weight at least as large as r (see Section 4). For polar codes, the rows of $G\left(n\right)$ can be interpreted as a communication channels. Then, a rate-$K/2^n$ polar code is defined by the set $\mathcal{F}$ indexing the K channels with the lowest Bhattacharyya parameters [4] (the “good” channels, see Section 2).

Although the sets $\mathcal{F}$ are important for the construction of polar and Reed–Muller codes, surprisingly little is known about their fractal properties. Recently, Renes, Sutter, and Hassani stated conditions under which the good (bad) channels derived from one binary-input memoryless channel are also good (bad) for another channel [5]. Moreover, the authors of [6,7] observed the self-similar structure of $\mathcal{F}$ by showing that polar and Reed–Muller codes are decreasing monomial codes.

In this paper, we analyze the fractal properties of $\mathcal{F}$ for polar codes (Section 3) and Reed–Muller codes (Section 5). In contrast to [6,7], we study the properties of $\mathcal{F}$ for infinite blocklengths, i.e., for $n\to \infty$. Specifically, we compute the Hausdorff dimension of $\mathcal{F}$, show that it is self-similar, and that it has detail on arbitrarily small scales (e.g., $\mathcal{F}$ is symmetric and dense in some well-defined containing set). Each of these results is relatively easy to obtain once appropriate definitions have been put in place. Taken as a whole, however, they paint an interesting picture and make a convincing point for the claim that polar and Reed–Muller codes are fractal.

The presented results will improve our understanding of polar and Reed–Muller codes, even though we have to admit that their practical implication (e.g., in code construction) still eludes us. Nevertheless, our results may apply in areas beyond channel coding: Arıkan’s polarization technique was used to polarize Rényi information dimension [8] and to construct high-girth matrices [9]. Moreover, Nasser showed that a sufficient and necessary condition for a binary operation to be polarizing is that it is uniformity preserving and that its inverse is strongly ergodic [10,11]. We are convinced that fractality carries over to these applications as well and that an analysis similar to ours can deepen understanding.

Since we consider the case $n\to \infty$, the set $\mathcal{F}$ indexes a subset of $\Omega :={\{0,1\}}^{\infty }$, the set of infinite binary sequences. We let $b:=(b_1{b}_2\cdots )\in \Omega$ and abbreviate $b^n:=(b_1{b}_2\cdots b_n)$. Let $\overline{b}:=({\overline{b}}_1{\overline{b}}_2\cdots )$ where ${\overline{b}}_i:=1-b_i$. Let furthermore $(\Omega ,\mathfrak{A},\mathbb{P})$ be a probability space with $\mathfrak{A}$ the Borel field generated by the cylinder sets $S\left(b^n\right):=\{w\in \Omega :w_1=b_1,\cdots ,w_n=b_n\}$ and $\mathbb{P}$ a probability measure satisfying $\mathbb{P}\left(S\left(b^n\right)\right)=1/2^n$. In the following, we represent every infinite binary sequence $b\in \Omega$ by a point in the unit interval $[0,1]$. The mapping between $\Omega$ and $[0,1]$ is given by

$$f\left(b\right):=\sum _{n=1}^{\infty }\frac{b_n}{2^n}.$$

(2)

Lemma 1

([12] (Exercises 7–10, p. 80)). Let ${\mathfrak{B}}_{[0,1]}$ be the Borel σ-algebra on $[0,1]$ and let λ be the Lebesgue measure. Let furthermore $\mathbb{D}:=[0,1]\cap \{p/2^n:p\in \mathbb{Z},n\in \mathbb{N}\}$ denote the set of dyadic rationals in the unit interval. Then, the function $f:(\Omega ,\mathfrak{A})\to ([0,1],{\mathfrak{B}}_{[0,1]})$ in (2) satisfies the following properties:

1. 

f is measurable

2. 

f is bijective on $\Omega \backslash f^{-1}\left(\mathbb{D}\right)$

3. 

for all $I\in {\mathfrak{B}}_{[0,1]}$, $\mathbb{P}\left(f^{-1}\left(I\right)\right)=\lambda \left(I\right)$

Example 1.

Lemma 1 states that f is not injective in general. The reason is that dyadic rationals have a non-unique binary expansion. For example, f maps both $b=(01111111\cdots )$ and $b=(10000000\cdots )$ to $0.5$, where we call the latter binary expansion terminating.

2. Preliminaries for Polar Codes

Let $W:\{0,1\}\to \mathcal{Y}$ be a binary-input memoryless channel with finite output alphabet $\mathcal{Y}$, (symmetric) capacity $0\le I\left(W\right)\le 1$, and with Bhattacharyya parameter

$$Z\left(W\right):=\sum _{y\in \mathcal{Y}}\sqrt{W\left(y\right|0\left)W\right(y\left|1\right)}.$$

(3)

It can be shown using [13] (Proposition 1) that $Z\left(W\right)=0\iff I\left(W\right)=1$ and $Z\left(W\right)=1\iff I\left(W\right)=0$. We say that the channel W is symmetric if there exists a permutation $\pi :\mathcal{Y}\to \mathcal{Y}$ such that ${\pi }^{-1}=\pi$ and, for every $y\in \mathcal{Y}$, $W\left(y\right|0)=W(\pi \left(y\right)\left|1\right)$.

Arıkan’s polarization technique [13] combines and splits two channel uses of W into one use of a “worse” channel

$$W_2^0\left(y_1^2\right|u_1):=\frac{1}{2}\sum _{u_2}W\left(y_1\right|u_1\oplus u_2)W\left(y_2\right|u_2)$$

(4)

and one use of a “better” channel

$$W_2^1(y_1^2,u_1|u_2):=\frac{1}{2}W\left(y_1\right|u_1\oplus u_2)W\left(y_2\right|u_2)$$

(5)

where $u_1,u_2\in \{0,1\}$ and $y_1,y_2\in \mathcal{Y}$. The combining operation encodes two input bits by F in (1); transmitting them via two channel uses of W creates a vector channel. This vector channel can then be split into the two virtual binary-input memoryless channels indicated in (4) and (5). The better (worse) channel obtained by polarization has a larger (smaller) capacity than the original channel W, i.e., $I\left(W_2^0\right)\le I\left(W\right)\le I\left(W_2^1\right)$—the inequalities are strict if $0<I\left(W\right)<1$. The sum capacity equals two times the capacity of the original channel, i.e., $I\left(W_2^0\right)+I\left(W_2^1\right)=2I\left(W\right)$ [13] (Proposition 4). Similarly, polarization has an effect on the Bhattacharyya parameter:

Lemma 2 (Bounds on the Bhattacharyya Parameter).

$$\begin{array}{cc}\hfill Z\left(W_2^1\right)& =g_1\left(Z\left(W\right)\right):=Z^2\left(W\right)\le Z\left(W\right)\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill Z\left(W\right)\le Z\left(W\right)\sqrt{2-Z^2\left(W\right)}=:h_0\left(Z\left(W\right)\right)\le Z\left(W_2^0\right)& \stackrel{\left(a\right)}{\le }{g}_0\left(Z\left(W\right)\right):=2Z\left(W\right)-Z^2\left(W\right)\hfill \end{array}$$

(7)

with equality in $\left(a\right)$ if W is a binary erasure channel (BEC).

Proof. 

The equality and inequality in (6) follow from [13] (Proposition 7) and the fact that $Z\left(W\right)\le 1$, respectively. The inequalities in (7) follow from the fact that $Z\left(W\right)\le 1$, from [14] (Lemma 20), and from [13] (Proposition 7). The last inequality becomes an equality if W is a BEC [13] (Proposition 7). ☐

For larger blocklengths $2^n$, $n>1$, we apply the polarization procedure recursively and obtain, for $b^n\in {\{0,1\}}^n$,

$$\left(W_{2^n}^{b^n},W_{2^n}^{b^n}\right)\to \left(W_{2^{n+1}}^{b^{n}0},W_{2^{n+1}}^{b^{n}1}\right)$$

(8)

where $b^{n}0$ and $b^{n}1$ denote the sequences of zeros and ones obtained by appending 0 and 1 to $b^n$, respectively. Note that the functions $g_1$, $g_0$, and $h_0$ from Lemma 2 are non-negative and non-decreasing and map the unit interval onto itself. Hence, the inequalities in (7) are preserved under composition:

$$\begin{array}{cc}\hfill Z\left(W_{2^n}^{b^n}\right)& \le p_{b^n}\left(Z\left(W\right)\right):=g_{b_n}\left(g_{b_{n-1}}(\cdots g_{b_1}\left(Z\left(W\right)\right)\cdots )\right)\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill Z\left(W_{2^n}^{b^n}\right)& \ge q_{b^n}\left(Z\left(W\right)\right):=h_{b_n}\left(h_{b_{n-1}}(\cdots h_{b_1}\left(Z\left(W\right)\right)\cdots )\right)\hfill \end{array}$$

(10)

where $h_1\equiv g_1$.

Applying this recursive polarization infinitely often leads to a situation in which almost all channels are either perfect or useless, i.e., either $I\left(W_{\infty }^b\right)=1$ or $I\left(W_{\infty }^b\right)=0$ for $b\in {\{0,1\}}^{\infty }$. This is the assertion of Arıkan’s polarization theorem:

Proposition 1

([13] (Proposition 10)). With probability one, the limit RV $I_{\infty }\left(b\right):=I\left(W_{\infty }^b\right)$ takes values in the set $\{0,1\}$: $\mathbb{P}(I_{\infty }=1)=I\left(W\right)$ and $\mathbb{P}(I_{\infty }=0)=1-I\left(W\right)$.

If we stop the polarization procedure at a finite blocklength $2^n$ for n large enough, then still most of the resulting $2^n$ channels are either almost perfect or almost useless (i.e., the channel capacities are close to one or to zero). The idea of polar coding is to transmit data only on those channels that are almost perfect. The generator matrix of a blocklength-$2^n$ polar code is thus the submatrix of $G\left(n\right)$ consisting of rows indexed by $\mathcal{F}$, where $\mathcal{F}$ contains the indices corresponding to the K virtual channels with the largest capacities. Determining this set $\mathcal{F}$ is inherently difficult, since (whenever W is not a BEC) the cardinality of the output alphabet increases exponentially in $2^n$ [15] (Chapter 3.3), [16] (p. 36). Tal and Vardy proposed an approximate construction method based on reducing the output alphabet and showing that the resulting channels are either upgraded or degraded w.r.t. the channel of interest [17] (see also Korada’s PhD thesis [16] (Definition 1.7 & Lemma 1.8)). These upgrading/degrading properties are important tools in our proofs.

Definition 1 (Channel Upgrading and Degrading).

A channel $W^{-}:\{0,1\}\to \mathcal{Z}$ is degraded w.r.t. the channel W (short: $W^{-}\preccurlyeq W$) if there exists a channel $Q:\mathcal{Y}\to \mathcal{Z}$ such that

$$W^{-}\left(z\right|u)=\sum _{y\in \mathcal{Y}}W\left(y\right|u)Q\left(z\right|y).$$

(11)

A channel $W^{+}:\{0,1\}\to \mathcal{Z}$ is upgraded w.r.t. the channel W (short: $W^{+}\succcurlyeq W$) if there exists a channel $P:\mathcal{Z}\to \mathcal{Y}$ such that

$$W\left(y\right|u)=\sum _{z\in \mathcal{Z}}{W}^{+}\left(z\right|u)P\left(y\right|z).$$

(12)

Moreover, $W^{+}\succcurlyeq W$ if and only if $W\preccurlyeq W^{+}$.

Upgraded (degraded) channels remains upgraded (degraded) during polarization:

Lemma 3

([16] (Lemma 4.7) & [17] (Lemmas 3 & 5)). Suppose that $W^{-}\preccurlyeq W\preccurlyeq W^{+}$. Then,

$$\begin{array}{ccccc}I\left(W^{-}\right)& \le & I\left(W\right)& \le & I\left(W^{+}\right)\\ Z\left(W^{-}\right)& \ge & Z\left(W\right)& \ge & Z\left(W^{+}\right)\\ {\left(W^{-}\right)}_2^1& \preccurlyeq & W_2^1& \preccurlyeq & {\left(W^{+}\right)}_2^1\\ {\left(W^{-}\right)}_2^0& \preccurlyeq & W_2^0& \preccurlyeq & {\left(W^{+}\right)}_2^0.\end{array}$$

(13)

Lemma 4

([15] (p. 9) & [6] (Lemma 3)). $W\preccurlyeq W_2^1$. If W is symmetric, then $W_2^0\preccurlyeq W\preccurlyeq W_2^1$.

Proof. 

By choosing

$$P\left(y\right|y_1^2,u_1)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}y=y_2\hfill \\ 0,\hfill & \mathrm{else}.\hfill \end{array}\right.$$

(14)

one can show that $W\preccurlyeq W_2^1$. To show that also $W_2^0\preccurlyeq W$ for symmetric channels, take [6] (Lemma 3)

$$Q\left(y_1^2\right|y)=\left\{\begin{array}{cc}\frac{1}{2}W\left(y_2\right|0)\hfill & \mathrm{if}{y}_1=y\hfill \\ \frac{1}{2}W\left(y_2\right|1)\hfill & \mathrm{if}{y}_1=\pi \left(y\right)\hfill \\ 0\hfill & \mathrm{else}.\hfill \end{array}\right.$$

(15)

 ☐

3. Fractal Properties of the Sets of Good and Bad Channels

We next investigate the behavior of the set $\mathcal{F}$ as we let the blocklength tend to infinity, i.e., as $n\to \infty$. This set indexes all sequences b for which we obtain $I\left(W_{\infty }^b\right)=1$. With the help of (2), we map these sequences to a subset of the unit interval, which we will call the set of good channels.

Definition 2 (The Good and the Bad Channels).

Let $\mathcal{G}$ denote the set of good channels, i.e.,

$$x\in \mathcal{G}\iff \exists b\in f^{-1}\left(x\right):I\left(W_{\infty }^b\right)=1.$$

(16)

Let $\mathcal{B}$ denote the set of bad channels, i.e.,

$$x\in \mathcal{B}\iff \exists b\in f^{-1}\left(x\right):I\left(W_{\infty }^b\right)=0.$$

(17)

If $I\left(W\right)=0$, then all polarized channels are useless and we have $\mathcal{B}=[0,1]$. Similarly, if $I\left(W\right)=1$, then all polarized channels are perfect and we have $\mathcal{G}=[0,1]$. We hence assume throughout this section that the channel W is nontrivial, i.e., that $0<I\left(W\right)<1$.

Proposition 2 (Denseness).

$\mathcal{G}\cap \mathcal{B}=\mathbb{D}$, i.e., the good and bad channels are dense in the unit interval. Moreover, $\mathcal{G}\backslash \mathbb{D}$ and $\mathcal{B}\backslash \mathbb{D}$ are dense in $[0,1]$.

Proof. 

See Appendix A. ☐

It is not really surprising that $\mathcal{G}$ and $\mathcal{B}$ are not disjoint; this is a direct consequence of the fact that f is not injective. It is not obvious, however, that the intersection exhausts the set on which f is non-injective. A consequence of this proposition is that there is no interval that contains only good channels. This has implications for code construction techniques. Indeed, the authors of [18,19] suggest that, for a polar code of a given blocklength, one may stop polarizing channels at shorter blocklengths and use copies of these channels rather than their polarization. For example, they suggest to use the channels $(W_{2^n}^{b^n},W_{2^n}^{b^n})$ rather than the channels $(W_{2^{n+1}}^{b^{n}0},W_{2^{n+1}}^{b^{n}1})$ if $I\left(W_{2^n}^{b^n}\right)$ is sufficiently large. Such a procedure can be justified if further polarizing $W_{2^n}^{b^n}$ to the desired blocklength will lead to including all channels polarized from $W_{2^n}^{b^n}$ in the code. Such a justification can never appear for polar codes with unbounded blocklength: Stopping polarizing at a given blocklength $2^n$ for a given polarization sequence and using copies of the resulting channel $W_{2^n}^{b^n}$ is equivalent to including a dyadic interval in the index set. This dyadic interval contains, by Proposition 2, bad channels, which shows that this choice is suboptimal.

Proposition 3 (Symmetry).

There exists a function ϑ, defined for almost all values in $[0,1]$, that is independent of W and satisfies $0\le \vartheta \left(x\right)\le 1$ and $\vartheta (1-x)=1-\vartheta \left(x\right)$. Let $x\in [0,1]$ be such that $\vartheta \left(x\right)$ is defined. Then, $\vartheta \left(x\right)>Z\left(W\right)$ implies $x\in \mathcal{G}$. If W is a BEC, then $\vartheta \left(x\right)<Z\left(W\right)$ implies $x\in \mathcal{B}$.

Proof. 

See Appendix B. ☐

Proposition 3 has two implications. The first implication concerns the alignment of the sets $\mathcal{G}$ and ${\mathcal{G}}^{\prime }$ for two different channels W and $W^{\prime }$. Specifically, it is connected to the question whether $Z\left(W\right)\ge Z\left(W^{\prime }\right)$ implies $\mathcal{G}\subseteq {\mathcal{G}}^{\prime }$. In general, the answer is negative [5]. Indeed, it may happen that for some $b\in \Omega$, we have $I\left(W_{\infty }^b\right)=1$ despite $Z\left(W\right)>\vartheta \left(f\right(b\left)\right)$, i.e., that the polarized channel turns out to be good even though the sufficient condition from Proposition 3 is not fulfilled. Such a situation cannot occur for BECs, as Proposition 3 shows. Hence, the set of good channels for a BEC is also good for any binary-input memoryless channel with a smaller Bhattacharyya parameter [20].

The second implication is that, at least for BECs, the sets $\mathcal{G}$ and $\mathcal{B}$ are symmetric. Indeed, if $\vartheta \left(x\right)\ne Z\left(W\right)$, then $x\in \mathcal{G}$ implies $1-x\in \mathcal{B}$. This symmetry is visible in the polar fractal that we display in Figure 1.

It is possible to define $\vartheta$ for $x\in \mathbb{D}$. We know from Proposition 2 that dyadic rationals are both good and bad, hence setting $\vartheta \left(x\right)=1$ for every $x\in \mathbb{D}$ leads to $\mathbb{D}\subseteq \mathcal{G}$. (The fact that also $\mathbb{D}\subseteq \mathcal{B}$ is not captured by nor in conflict with this setting.) The question whether the function $\vartheta$ can be defined for $x\in \mathbb{Q}\backslash \mathbb{D}$ is more interesting. In this case, the binary expansion is unique and recurring, i.e., there is a length-k sequence $a^k\in {\{0,1\}}^k$ such that $f(b^n{a}^k{a}^k{a}^k\cdots )=x$ for some $b^n\in {\{0,1\}}^n$. It is straightforward to show that for every non-trivial sequence $a_k$ (i.e., $a_k$ contains zeros and ones), $p_{a^k}$ is from $[0,1]$ to $[0,1]$, non-negative, and non-decreasing with vanishing derivatives at 0 and 1. Since this ensures that $p_{a^k}\left(z\right)<z$ for z close to zero and $p_{a^k}\left(z\right)>z$ for z close to one, the operation $z_{i+1}=p_{a^k}\left(z_i\right)$ constitutes an iterated function system with attracting fixed points at $z=0$ and $z=1$. Note further that, since $p_{a^k}$ corresponds to the recurring part of the binary expansion of x, $Z(W_{\infty }^{b^n{a}^k{a}^k\cdots })$ will be bounded from above by the value to which this iterated function system converges after being initialized with $Z(W_{2^n}^{b^n})$. To show that Proposition 3 holds for $x\in \mathbb{Q}\backslash \mathbb{D}$ requires showing that $p_{a^k}$ intersects the identity function only once on $(0,1)$, i.e., that there is no attracting fixed point on this open interval. We leave this problem for future investigation.

Example 2.

Let $x=2/3$, hence $f^{-1}\left(x\right)=101010101\cdots$. We determine the fixed points of the iterated function system corresponding to one period of the recurring sequence, i.e, the fixed points of $p_{10}\left(z\right)=2z^2-z^4$. These are given by the roots of $p_{10}\left(z\right)-z$, which are $z=0$, $z=1$, and $z=(\pm \sqrt{5}-1)/2$. One of these latter nontrivial roots lies outside $[0,1]$ and is hence irrelevant. The remaining root determines the threshold, $\vartheta (2/3)=(\sqrt{5}-1)/2$.

Now suppose that W is a BEC with Bhattacharyya parameter $Z\left(W\right)=\vartheta (2/3)$. Since $\vartheta (2/3)$ is a fixed point, we get $Z(W_{\infty }^{f^{-1}(2/3)})=Z\left(W\right)\notin \{0,1\}$. This illustrates that Proposition 1 holds only almost surely.

Proposition 4 (Lebesgue Measure & Hausdorff Dimension).

$\mathcal{G}$ is a Borel set and has Lebesgue measure $\lambda \left(\mathcal{G}\right)=I\left(W\right)$. $\mathcal{B}$ is a Borel set and has Lebesgue measure $\lambda \left(\mathcal{B}\right)=1-I\left(W\right)$. Therefore, the Hausdorff dimensions of $\mathcal{G}$ and $\mathcal{B}$ satisfy $d\left(\mathcal{G}\right)=d\left(\mathcal{B}\right)=1$.

Proof. 

See Appendix C. ☐

Loosely speaking, the Lebesgue measure of $\mathcal{G}$ is the asymptotic equivalent of the rate of the “infinite-blocklength” polar code for the channel W. The fact that $\lambda \left(\mathcal{G}\right)=I\left(W\right)$ states that the rate approaches the symmetric capacity of W. A positive Lebesgue measure and a Hausdorff dimension equal to one are not indicators of fractality.

The last fractal property we consider is self-similarity. As Falconer notes [1] (p. xxviii), self-similarity often occurs only approximately. What we show in the following proposition is that the set $\mathcal{G}$ is quasi self-similar. Along the same lines, the quasi self-similarity of $\mathcal{B}$ can be shown.

Proposition 5 (Self-Similarity).

Let ${\mathcal{G}}_n\left(k\right):=\mathcal{G}\cap [(k-1)2^{-n},k{2}^{-n}]$ for $k=1,\dots ,2^n$. $\mathcal{G}={\mathcal{G}}_0\left(1\right)$ is quasi self-similar in the sense that, for all n and all k, ${\mathcal{G}}_n\left(k\right)={\mathcal{G}}_{n+1}(2k-1)\cup {\mathcal{G}}_{n+1}\left(2k\right)$ is quasi self-similar to its right half:

$${\mathcal{G}}_n\left(k\right)\subseteq 2{\mathcal{G}}_{n+1}\left(2k\right)-k{2}^{-n}$$

(18)

If W is symmetric, ${\mathcal{G}}_n\left(k\right)$ is quasi self-similar:

$$2{\mathcal{G}}_{n+1}(2k-1)-(k-1)2^{-n}\subseteq {\mathcal{G}}_n\left(k\right)\subseteq 2{\mathcal{G}}_{n+1}\left(2k\right)-k{2}^{-n}$$

(19)

Proof. 

See Appendix D. ☐

In other words, at least for a symmetric channel, $\mathcal{G}$ is composed of two similar copies of itself (see Figure 1). The self-similarity is closely related to the fact that polar codes are decreasing monomial codes [6] (Theorem 1).

Example 3.

We want to determine whether $1/3\in \mathcal{G}$ for a given BEC W. This question translates the questions whether $1/6\in {\mathcal{G}}_1\left(1\right)$ and whether $2/3\in {\mathcal{G}}_1\left(2\right)$. Along the lines of Example 2, we obtain $\vartheta (1/6)\approx 0.214$, $\vartheta (1/3)\approx 0.382$, and $\vartheta (2/3)\approx 0.618$, i.e., $\vartheta (1/6)<\vartheta (1/3)<\vartheta (2/3)$. Since W is a BEC, we can connect this with Proposition 3 and thus obtain the inclusion indicated in Proposition 5.

4. Preliminaries for Reed–Muller Codes

An order-r, length-$2^n$ Reed–Muller code is defined by having a generator matrix $G_{RM}(r,n)$ composed of all length-$2^n$ sequences with a Hamming weight larger than $2^{n-r}$. For example, we have $G_{RM}(n,n)=G\left(n\right)$, while $G_{RM}(0,n)$ is a single row vector containing only ones (length-$2^n$ repetition code). To make this more precise, let $w\left(b^n\right)={\sum }_{i=1}^n{b}_i$ be the Hamming weight of $b^n\in {\{0,1\}}^n$ and let $s_i\left(n\right)$ be the i-th row of $G\left(n\right)$. Then, the generator matrix $G_{RM}(r,n)$ of an order-r, length-$2^n$ Reed–Muller code consists of the rows of $G\left(n\right)$ indexed by [4]

$$\mathcal{F}=\{i\in \{1,\dots ,2^n\}:w\left(s_i\left(n\right)\right)\ge 2^{n-r}\}.$$

(20)

To analyze the effect of doubling the block length, note that

$$G(n+1):=\left[\begin{array}{cc}G\left(n\right)& 0\\ G\left(n\right)& G\left(n\right)\end{array}\right].$$

(21)

Assume that we indicate the rows of $G\left(n\right)$ by a sequence of binary numbers, i.e., let the i-th row be indexed by $h_n\left(b^n\right):=2^n{\sum }_{l=1}^n{b}_l{2}^{-l}$. Furthermore, let $0b^n$ and $1b^n$ denote the sequences of zeros and ones obtained by prepending 0 and 1 to $b^n$, respectively. Clearly, $h_{n+1}\left(0b^n\right)=h_n\left(b^n\right)$ and $h_{n+1}\left(1b^n\right)=h_n\left(b^n\right)+2^n$. Combining this with (21) yields

$$\begin{array}{cc}\hfill w\left(s_{h_{n+1}\left(0b^n\right)}(n+1)\right)& =w\left(s_{h_n\left(b^n\right)}\left(n\right)\right)\hfill \\ \hfill w\left(s_{h_{n+1}\left(1b^n\right)}(n+1)\right)& =2w\left(s_{h_n\left(b^n\right)}\left(n\right)\right).\hfill \end{array}$$

Defining $G\left(0\right):=1$, we thus get

$$w\left(s_{h_n\left(b^n\right)}\left(n\right)\right)=2^{w\left(b^n\right)}$$

(22)

and

$$\mathcal{F}=h_n\left(\{b^n\in {\{0,1\}}^n:2^{w\left(b^n\right)}\ge 2^{n-r}\}\right).$$

(23)

In Section 5, we will analyze the properties of $\mathcal{F}$ in the limit as n tends to infinity. An important ingredient in our proofs is the concept of normal numbers.

Definition 3 (Normal Numbers).

A number $x\in [0,1]$ is called simply normal to base 2($x\in \mathcal{N}$) if and only if

$$\exists b\in f^{-1}\left(x\right):\underset{n\to \infty }{\lim }\frac{w\left(b^n\right)}{n}=\frac{1}{2}.$$

(24)

In general, a number is simply normal to base M if the fraction of its digits used in its M-ary expansion is $1/M$. A number is called normal if this property not only holds for digits, but also for subsequences: a number is normal in base M if, for each $k\ge 1$, the fraction of each length-k sequences used in its M-ary expansion is $1/M^k$. It immediately follows that a normal number is simply normal. The converse is in general not true:

Example 4.

Let $x=1/3$, hence $b=010101\cdots$. x is simply normal to base 2, but not normal (since the sequences 00 and 11 never occur). Let $x=1/7$, hence $b=001001001\cdots$. x is neither normal nor simply normal. Let $x\in \mathbb{D}$, hence b is either terminating (${\lim }_{n\to \infty }w\left(b^n\right)/n=0$) or non-terminating (${\lim }_{n\to \infty }w\left(b^n\right)/n=1$). Dyadic rationals are not simply normal.

Lemma 5

(Borel’s Law of Large Numbers, cf. [21] (Corollary 8.1, p. 70)). Almost all numbers in $[0,1]$ are simply normal, i.e.,

$$\lambda \left(\mathcal{N}\right)=1.$$

(25)

Despite this result, there are uncountably many numbers in the unit interval which are not normal. Moreover, the set of numbers that are not normal is superfractal, i.e., it has a Hausdorff dimension equal to one although it has zero Lebesgue measure [22].

5. Fractal Properties of the Set of Heavy Codewords

If we let n tend to infinity, the definition of $\mathcal{F}$ in (23) becomes problematic. Rather than looking at order-r, length-$2^n$ Reed–Muller codes, we investigate order-$(1-\rho )n$, length-$2^n$ codes, where we assume that $\rho n$ is integer. In other words, we assume that the threshold for the Hamming weight increases linearly with the blocklength. This gives rise to the definition of heavy codewords:

Definition 4 (The Heavy Codewords).

Let $\mathcal{H}\left(\rho \right)$ denote the set of ρ-heavy codewords, i.e.,

$$x\in \mathcal{H}\left(\rho \right)\iff \exists b\in f^{-1}\left(x\right):\underset{n\to \infty }{\lim \inf }\frac{2^{w\left(b^n\right)}}{2^{n\rho }}\ge 1.$$

(26)

Loosely speaking, the set of heavy codewords corresponds to those rows of $G\left(n\right)$ that asymptotically have a fractional Hamming weight larger than a given threshold.

Example 5.

$\mathcal{H}\left(1\right)=\left\{1\right\}$. This follows from the fact that 1 is the only number in the unit interval with a binary expansion consisting only of ones. $\mathcal{H}\left(0\right)=[0,1]$. This follows from the fact that $w\left(b^n\right)\ge 0$.

Proposition 6 (Denseness).

For all $\rho \in [0,1)$, $\mathbb{D}\subset \mathcal{H}\left(\rho \right)$. Moreover, for $\rho \in (0,1)$, $\mathcal{H}\left(\rho \right)\backslash \mathbb{D}$ and its complement are dense in $[0,1]$.

Similarly as for polar codes, also Reed–Muller codes are such that no interval is contained in either $\mathcal{H}\left(\rho \right)$ or its complement (unless in the trivial cases $\mathcal{H}\left(0\right)$ and $\mathcal{H}\left(1\right)$). This is again in contrast with the intuition one obtains for Reed–Muller code with finite blocklength. Suppose we fix n to be even and set $r=n/2$, i.e., we require that at least one half of the bits in $b^n$ are one. The matrix $G\left(n\right)$ resembles a Sierpinski triangle, as depicted in [2] (Figure 2). In our notation, the set $\mathcal{F}$ indexes none of the first $2^{n/2}-1$ rows of $G\left(n\right)$, since they cannot have sufficient Hamming weight. Consequently, the transition as $n\to \infty$ creates complications that are not present for finite n, and one needs to depart from intuition based on these finite-blocklength considerations.

Proof. 

See Appendix E. ☐

Proposition 7 (Lebesgue Measure & Hausdorff Dimension).

$\mathcal{H}\left(\rho \right)$ is Lebesgue measurable and has Lebesgue measure

$$\lambda \left(\mathcal{H}\left(\rho \right)\right)=\left\{\begin{array}{cc}1,\hfill & \mathit{if}\rho <1/2\hfill \\ 0,\hfill & \mathit{if}\rho \ge 1/2.\hfill \end{array}\right.$$

(27)

The Hausdorff dimension satisfies

$$d\left(\mathcal{H}\left(\rho \right)\right)\left\{\begin{array}{cc}=1,\hfill & \mathit{if}\rho \le 1/2\hfill \\ \ge h_2\left(\rho \right),\hfill & \mathit{if}\rho >1/2\hfill \end{array}\right.$$

(28)

where $h_2\left(x\right):=-x{\log }_{2}x-(1-x){\log }_2(1-x)$.

Proof. 

See Appendix F. ☐

Loosely speaking, the Lebesgue measure of $\mathcal{H}\left(\rho \right)$ is the asymptotic equivalent of the rate of the fractional order-$\rho$ Reed–Muller code. As we showed in Proposition 4, the Lebesgue measure of $\mathcal{G}$ is equal to the symmetric capacity of W. In contrast, the set $\mathcal{H}\left(\rho \right)$ does not depend on W. Rather, Proposition 7 suggests that the order parameter $\rho$ induces a phase transition for the rate of Reed–Muller codes: If $\rho <1/2$, the “infinite-blocklength” Reed–Muller code consists of almost all (in the sense of Lebesgue measure) possible binary sequences. In contrast, if $\rho \ge 1/2$, the “infinite-blocklength” Reed–Muller code consists of almost no codewords (again, in the sense of Lebesgue measure).

Let us briefly consider the case $\rho =1/2$. For this case, Proposition 7 states that $\mathcal{H}\left(\rho \right)$ is a Lebesgue null set that has a Hausdorff dimension equal to 1. Thus, the set $\mathcal{H}(1/2)$ is a superfractal. Unfortunately, we were not able to give an exact expression for the Hausdorff dimension of $\mathcal{H}\left(\rho \right)$ for $\rho >1/2$. While the set of all non-normal numbers is superfractal, we are not sure if this holds also for the specific proper subset $\mathcal{H}\left(\rho \right)$.

The sets $\mathcal{G}$ and $\mathcal{B}$ exhibit self-similarity, i.e., detailed structure on every scale (cf. Figure 1). We next show that also $\mathcal{H}\left(\rho \right)$ is self-similar. At least for $\mathcal{H}\left(0\right)$ and $\mathcal{H}\left(1\right)$ (cf. Example 5) this is as trivial as the self-similarity of a point or a line. For $\rho \in (0,1)$ this self-similarity is more interesting, and related to the fact that Reed–Muller codes are decreasing monomial codes [6] (Proposition 2).

Proposition 8 (Self-Similarity).

Let ${\mathcal{H}}_n(\rho ,k):=\mathcal{H}\left(\rho \right)\cap [(k-1)2^{-n},k{2}^{-n}]$ for $k=1,\dots ,2^n$. $\mathcal{H}\left(\rho \right)={\mathcal{H}}_0(\rho ,1)$ is quasi self-similar in the sense that, for all n and all k, ${\mathcal{H}}_n(\rho ,k)={\mathcal{H}}_{n+1}(\rho ,2k-1)\cup {\mathcal{H}}_{n+1}(\rho ,2k)$ is quasi self-similar:

$$2{\mathcal{H}}_{n+1}(\rho ,2k-1)-(k-1)2^{-n}\subseteq {\mathcal{H}}_n(\rho ,k)\subseteq 2{\mathcal{H}}_{n+1}(\rho ,2k)-k{2}^{-n}.$$

(29)

Proof. 

See Appendix G. ☐

6. Discussion and Outlook

That Kronecker product-based codes possess fractal properties has long been suspected. The present manuscript contains several results that back this suspicion with solid mathematical analyses. Specifically, we assumed that the blocklength tends to infinity and investigated the properties of the set $\mathcal{G}$ of virtual channels that are perfect and the set $\mathcal{H}\left(\rho \right)$ of codewords that have a fractional Hamming weight no less than $\rho$. Since both polar codes and Reed–Muller codes are obtained by a simple, recursive procedure, it remains to investigate whether the sets $\mathcal{G}$ and $\mathcal{H}\left(\rho \right)$ satisfy any of the following properties [1] (p. xxviii):

• The set has a fine structure, i.e., there is detail on arbitrarily small scales;
• It does not admit a description in traditional geometrical language, neither locally nor globally; it is irregular in some sense;
• It has some form of self-similarity, at least approximate or statistical;
• The fractal dimension of the set exceeds its topological dimension.

Indeed, the sets $\mathcal{G}$ and $\mathcal{H}\left(\rho \right)$ possess a fine structure in the sense that they are dense in the unit interval, but that also their complements are dense in the unit interval (cf. Propositions 2 and 6). Therefore, at an arbitrarily small scale, the sets $\mathcal{G}$ and $\mathcal{H}\left(\rho \right)$ admit no simple description in geometrical language. Both of these sets are self-similar in a specific sense, as we outlined in Propositions 5 and 8. Finally, while $\mathcal{G}$ has a fractal dimension of one (cf. Proposition 4), the set $\mathcal{H}\left(\rho \right)$ has, for a certain range of $\rho$, a positive (fractional?) Hausdorff dimension despite being a Lebesgue null set. This result, which we proved in Proposition 7, is one of the defining properties of a fractal set.

One reviewer pointed out that our definition of $\mathcal{H}\left(\rho \right)$ can be complemented by a different one. Specifically, while $\mathcal{H}\left(\rho \right)$ indexes the codewords with a fractional Hamming weight not smaller than $\rho$, one could define a set ${\mathcal{H}}^{\prime }\left(R\right)$ indexing the codewords of a Reed–Muller code with rate R. In other words, while $\mathcal{H}\left(\rho \right)$ is parameterized via the fractional order of the code, ${\mathcal{H}}^{\prime }\left(R\right)$ is parameterized via its rate. We expect that the Lebesgue measure of the (adequately defined) set ${\mathcal{H}}^{\prime }\left(R\right)$ should be R and that, thus, its Hausdorff dimension equals one. An appropriate definition of ${\mathcal{H}}^{\prime }\left(R\right)$ is tied to the set $\mathcal{F}$ of a rate-R, length-$2^n$ Reed–Muller code (such as is our Definition 4). Since finding such a definition has so far eluded us, we postpone this investigation to future work.

Another obvious extension of our work are non-binary polar and Reed–Muller codes. For example, consider an $\ell \times \ell$ matrix with entries from $\{0,\dots ,q-1\}$, where q is prime. One can show that this matrix is polarizing as long as it is not upper-triangular [15] (Theorem 5.2). We believe that our analysis can be replicated by considering the ℓ-ary expansion of real numbers in $[0,1]$. Along the same lines, it would be interesting to examine the properties of q-ary Reed–Muller codes, e.g., [23,24].