Error Exponents of LDPC Codes under Low-Complexity Decoding

Abstract

This paper deals with the specific construction of binary low-density parity-check (LDPC) codes. We derive lower bounds on the error exponents for these codes transmitted over the memoryless binary symmetric channel (BSC) for both the well-known maximum-likelihood (ML) and proposed low-complexity decoding algorithms. We prove the existence of such LDPC codes that the probability of erroneous decoding decreases exponentially with the growth of the code length while keeping coding rates below the corresponding channel capacity. We also show that an obtained error exponent lower bound under ML decoding almost coincide with the error exponents of good linear codes.

1. Introduction

Low-density parity-check (LDPC) codes [1] are known for their very efficient low-complexity decoding algorithms. This paper’s central question is: Are there LDPC codes that asymptotically achieve the capacity of binary-symmetric channel (BSC) under a low-complexity decoding algorithm? The following results help us construct LDPC code with specific construction and develop a decoding algorithm to answer yes to this question. So, Zyablov and Pinsker showed in [2] that the ensemble of LDPC codes, proposed by Gallager (G-LDPC codes), includes codes that can correct a number of the errors that grow linearly with the code length n while the decoding complexity remains $\mathcal{O}\left(nlogn\right)$. Later the lower bound on this fraction of errors was improved in [3,4,5]. Thus, the main idea of LDPC code construction and decoding algorithm, considered in this paper, is as follows. We need to introduce to the construction of G-LDPC code some “good” codes that reduce the number of errors from the channel in such a way that the low-complexity majority decoding can correct the rest errors. As “good” codes, we select the codes with the error exponent of good codes under ML decoding [6]. To introduce these “good” codes to the construction, we compose the parity-check matrix layer with the parity-check matrices of “good” codes. To meet the requirements on low-complexity ($\mathcal{O}\left(nlogn\right)$) of decoding algorithm we impose restrictions on the length of “good” codes (it must be small ($loglog\left(n\right)$) compared to the length of whole construction). To show that the proposed construction asymptotically achieves the capacity of BSC we consider the estimation on error-exponent under the proposed low-complexity decoding algorithm.

It worth mention that papers [7,8] introduce expander codes achieving the BSC capacity under an iterative decoding algorithm with low complexity. But in this paper, we are interested in LDPC-code construction and corresponding decoding algorithm.

To show that the proposed construction of LDPC code is good, we also estimate the error exponent under ML decoding and compare it with the error exponent under the proposed low-complexity decoding algorithm. Previously the authors of [9,10] have derived the upper and lower bounds on the G-LDPC codes error exponent under the ML decoding assumption. Moreover, one can conclude from [10] that the lower bound on the error exponent under ML decoding of G-LDPC codes almost coincides with the lower bound obtained for good linear codes (from [6]) under ML.

Some parts of this paper were previously presented (with almost all of the proofs omitted) in the conference paper [11]. The low-complexity decoding algorithm that we use for our analysis was proposed in [12,13]. Unlike in previous papers, Corollary 1 is significantly enhanced and proved in detail in this paper. Moreover, the results for the error-exponent bound under ML decoding and corresponding proofs are added. We compare the obtained lower bounds on the error exponents under the low-complexity decoding and the ML decoding. We evaluate the error exponents numerically for different code parameters.

2. LDPC Code Construction

Let us briefly consider the LDPC code construction from [11,12]. First, let us consider the G-LDPC code parity-check matrix ${\mathbf{H}}_2$ of size $\ell \times b_0{n}_0$ from [1]:

$${\mathbf{H}}_2=\left(\begin{array}{c}{\pi }_1\left({\mathbf{H}}_{b_0}\right)\\ {\pi }_2\left({\mathbf{H}}_{b_0}\right)\\ ⋮\\ {\pi }_{\ell }\left({\mathbf{H}}_{b_0}\right)\end{array}\right).$$

Here we denote ${\pi }_l\left({\mathbf{H}}_{b_0}\right)$, $l=1,\cdots ,\ell$, as a random column permutation of ${\mathbf{H}}_{b_0}$, which is given by

$${\mathbf{H}}_{b_0}=\left(\underset{b_0}{\underbrace{\begin{array}{cccc}{\mathbf{H}}_0& \mathbf{0}& \dots & \mathbf{0}\\ \mathbf{0}& {\mathbf{H}}_0& \dots & \mathbf{0}\\ ⋮& ⋮& \ddots & ⋮\\ \mathbf{0}& \mathbf{0}& \dots & {\mathbf{H}}_0\end{array}}}\right),$$

where ${\mathbf{H}}_0$ is the parity-check matrix of the constituent single parity check (SPC) code of length $n_0$.

The elements of the Gallager’s LDPC codes ensemble ${\mathcal{E}}_G\left(\ell ,n_0,b_0\right)$ are obtained by independently selecting the equiprobable permutations ${\pi }_l$, l = 1, 2, …, ℓ.

One can write a lower bound on the G-LDPC code rate ${\mathcal{E}}_G\left(\ell ,n_0,b_0\right)$ as

$$R_2\ge 1-\ell \left(1-R_0\right),$$

(1)

where $R_0=\frac{n_0-1}{n_0}$ is a SPC code rate.

The equality achieved if and only if the matrix ${\mathbf{H}}_2$ has full rank.

Consider a G-LDPC parity check matrix with an additional layer consisting of linear codes (LG-LDPC code). Let us denote this matrix as $\mathbf{H}$:

$$\mathbf{H}=\left(\begin{array}{c}{\pi }_1\left({\mathbf{H}}_{b_0}\right)\\ {\pi }_2\left({\mathbf{H}}_{b_0}\right)\\ ⋮\\ {\pi }_{\ell }\left({\mathbf{H}}_{b_0}\right)\\ {\pi }_{\ell +1}\left({\mathbf{H}}_{b_1}\right)\end{array}\right),$$

where ${\mathbf{H}}_{b_1}$ is given by

$${\mathbf{H}}_{b_1}=\left(\underset{b_1}{\underbrace{\begin{array}{cccc}{\mathbf{H}}_1& \mathbf{0}& \dots & \mathbf{0}\\ \mathbf{0}& {\mathbf{H}}_1& \dots & \mathbf{0}\\ ⋮& ⋮& \ddots & ⋮\\ \mathbf{0}& \mathbf{0}& \dots & {\mathbf{H}}_1\end{array}}}\right),$$

where $b_1$ is such that $b_1{n}_1=b_0{n}_0$. As soon as the first ℓ layers of matrix $\mathbf{H}$ is the G-LDPC parity-check matrix, we can write $\mathbf{H}$ as

$$\mathbf{H}=\left(\begin{array}{c}{\mathbf{H}}_2\\ {\pi }_{\ell +1}\left({\mathbf{H}}_{b_1}\right)\end{array}\right).$$

For a given SPC code with the code length $n_0$ and the parity-check matrix ${\mathbf{H}}_0$ and for a given linear code with the code length $n_1$ and the parity-check matrix ${\mathbf{H}}_1$, the elements of the LG-LDPC codes ensemble ${\mathcal{E}}_{LG}\left(\ell ,n_0,b_0,R_1,n_1,b_1\right)$ are obtained by independently selecting the equiprobable permutations ${\pi }_l$, l = 1, 2, …, $\ell +1$.

The length of the constructed LG-LDPC code is $n=b_0{n}_0=b_1{n}_1$, and the code rate R is lower bounded by

$$R\ge R_1-\ell \left(1-R_0\right),$$

According to (1),

$$R\ge R_1+R_2-1.$$

3. Decoding Algorithms

In this paper, we consider two decoding algorithms of the proposed construction. The first one is the well-known maximum likelihood decoding algorithm ${\mathcal{A}}_{ML}$. Under the second decoding algorithm ${\mathcal{A}}_C$, the LG-LDPC code is decoded as a concatenated code. In other words, in the first step, we decode the received sequence using linear codes with the parity-check matrix ${\mathbf{H}}_1$ from the $\ell +1$ layer of $\mathbf{H}$. Then, in the second step, we decode the sequence obtained in the previous step using the G-LDPC code with the parity-check matrix ${\mathbf{H}}_2$. Thus, this algorithm ${\mathcal{A}}_C$ consists of the following two steps:

• The received sequence is separately decoded with the well-known maximum likelihood algorithm by $b_1$ linear codes with the parity-check matrix ${\mathbf{H}}_1$ from the $\ell +1$ layer of $\mathbf{H}$.
• The tentative sequence is decoded with the well-known bit-flipping (majority) decoding algorithm ${\mathcal{A}}_M$ by the G-LDPC code with the parity-check matrix ${\mathbf{H}}_2$.

An important note here is that ${\mathcal{A}}_C$ is a two-step decoding algorithm, and each step is performed only once. It first decodes the received sequence once by the ML decoding algorithm using linear codes ${\mathbf{H}}_1$. Then it applies the iterative bit-flipping (majority) algorithm ${\mathcal{A}}_M$ using the G-LDPC code to the tentative sequence.

It is also worth noting that the complexity of the proposed decoding algorithm ${\mathcal{A}}_C$ is $\mathcal{O}\left(nlogn\right)$ with some restrictions on the length of the linear codes with the parity-check matrix ${\mathbf{H}}_1$ (see Lemma 3). At the same time the complexity of ML decoding is exponential.

4. Main Results

Consider a BSC with a bit error probability p. Let a decoding error probability P be the probability of the union of decoding denial and the erroneous decoding events. In this paper, we consider the decoding error probability P in the following form

$$P\le exp\left\{-nE\left(·\right)\right\},$$

with the $E\left(·\right)$ being the required error exponent.

Let us define two error exponents: $E_C\left(·\right)$ and $E_{ML}\left(·\right)$ corresponding to the ${\mathcal{A}}_C$ decoding algorithm (having $\mathcal{O}\left(nlogn\right)$ complexity) and the ${\mathcal{A}}_{ML}$ decoding algorithm (having an exponential complexity) respectively. Let us consider first the error exponent $E_C\left(·\right)$.

Theorem 1.

Let there exist in the ensemble ${\mathcal{E}}_G\left(\ell ,n_0,b_0\right)$ of the G-LDPC codes a code with the code rate $R_2$ that can correct any error pattern of weight up to $⌊{\omega }_{t}n⌋$ while decoding with the bit-flipping (majority) algorithm ${\mathcal{A}}_M$.

Let there exist a linear code with code length $n_1$, code rate $R_1$ and an error exponent under maximum likelihood decoding lower bounded by $E_0\left(R_1,p\right)$.

Then, in the ensemble ${\mathcal{E}}_{LG}\left(\ell ,n_0,b_0,R_1,n_1,b_1\right)$ of the LG-LDPC codes, there exists a code with code length n,

$$n=n_0{b}_0=n_1{b}_1,$$

code rate R,

$$R\ge R_1+R_2-1,$$

and an error exponent over the memoryless BSC with BER p under the decoding algorithm ${\mathcal{A}}_C$ with complexity $\mathcal{O}\left(nlogn\right)$ lower bounded by $E_C\left(·\right)$:

$$E_C\left(R_1,n_1,{\omega }_t,p\right)=\underset{{\omega }_t\le \beta \le {\beta }_0}{min}\left\{\beta E_0\left(R_1,p\right)+E_2\left(\beta ,{\omega }_t,p\right)-\frac{1}{n_1}H\left(\beta \right)\right\},$$

(2)

where ${\beta }_0=min\left(\frac{{\omega }_t}{2p},1\right)$, $H\left(\beta \right)=-\beta ln\beta -\left(1-\beta \right)ln\left(1-\beta \right)$ – an entropy function – and $E_2\left(\beta ,{\omega }_t,p\right)$ is given by

$$E_2\left(\beta ,{\omega }_t,p\right)=\frac{1}{2}\left({\omega }_{t}ln\frac{{\omega }_t}{p}+\left(2\beta -{\omega }_t\right)ln\frac{2\beta -{\omega }_t}{1-p}\right)-\beta ln\left(2\beta \right),$$

where $n_1$ satisfies the following condition:

$$\frac{-ln{\beta }_0}{E_0\left(R_1,p\right)}\le n_1\le \frac{1}{R_1}{log}_2{log}_2\left(n\right).$$

(3)

Corollary 1.

$E_C(·)>0$, if $R\to C$, where C is the capacity of a memoryless BSC with error probability p, such that $R_1\to C$ and $R_2<1$.

Thus, according to Corollary 1, we can state that there exists an LG-LDPC code such that the error probability of the low-complexity decoding algorithm ${\mathcal{A}}_C$ exponentially decreases with the code length for all code rates below the channel capacity C.

Remark 1.

We have obtained the lower bound on $E_C\left(R_1,n_1,{\omega }_t,p\right)$ assuming $n\to \infty$, where $n_0=const$, $n_1=const$, $b_0\to \infty$, and $b_1\to \infty$. As a result, the complexity of ${\mathcal{A}}_C$ algorithm equals to $\mathcal{O}\left(nlogn\right)$, and we have the right inequality of condition (3) for $n_1$.

Theorem 1 was obtained in [12]. The main idea of the proof is based on the following results. Our previous results [3,4] show that in the ensemble ${\mathcal{E}}_G\left(\ell ,n_0,b_0\right)$ of G-LDPC codes, there exists a code that can correct any error pattern of weight up to $⌊{\omega }_{t}n⌋$ under the algorithm ${\mathcal{A}}_M$ with complexity $\mathcal{O}\left(nlogn\right)$. In [6], it was shown that there exists a linear code for which the error exponent under ML decoding is lower bounded by $E_0\left(R,p\right)$, where $E_0\left(R,p\right)>0$ for $\forall R<C$.

Let us now consider the lower bound on the error exponent $E_{ML}\left(·\right)$.

Theorem 2.

In the ensemble ${\mathcal{E}}_{LG}\left(\ell ,n_0,b_0,R_1,n_1,b_1\right)$, there exists an LG-LDPC code such that the error exponent of this code over the memoryless BSC with BER p under the decoding algorithm ${\mathcal{A}}_{ML}$ is lower bounded by

$$E_{ML}\left(p\right)=\underset{{\omega }_0\le {\omega }_c\le 1}{max}\left\{min\left(E_{\delta }\left({\omega }_c,p\right),E_{{\omega }_c}\left({\omega }_c,p\right)\right)\right\},$$

where ${\omega }_0=max\left(\delta ,p\right)$, δ is the code distance of this code, and $E_{\delta }\left({\omega }_c,p\right)$ is given by

$$E_{\delta }\left({\omega }_c,p\right)=\underset{\delta \le \omega \le {\omega }_c}{max}\left\{\nu \left(\omega \right)+\omega \left(ln2+ln\sqrt{p\left(1-p\right)}\right)\right\},$$

where $\nu \left(\omega \right)$ is an asymptotic spectrum of the LG-LDPC code:

$$\nu \left(\omega \right)=\underset{n\to \infty }{lim}\frac{ln\overline{N}\left(\omega n\right)}{n},$$

where $\overline{N}\left(\omega n\right)$ is the average number of codewords and $E_{{\omega }_c}\left({\omega }_c,p\right)$ is given by

$$E_{{\omega }_c}\left({\omega }_c,p\right)=\left(1-{\omega }_c\right)ln\frac{1-{\omega }_c}{1-p}+{\omega }_{c}ln\frac{{\omega }_c}{p}.$$

We have obtained Theorem 2 using the methods developed in [14] to estimate the error exponent under the ML decoding of codes with the given spectrum. We have taken ideas of [1] for G-LDPC codes to construct the upper bound on the code spectrum and the lower bound on the code distance of the proposed LDPC construction (see Lemmas 1 and 2).

Lemma 1.

The value of $\nu \left(\omega \right)$ for the codes from the ensemble ${\mathcal{E}}_{LG}\left(\ell ,n_0,b_0,R_1,n_1,b_1\right)$ of LG-LDPC codes is upper bounded by

$$\nu \left(\omega \right)\le {\nu }_0\left(\omega \right)=-\left(\ell -1\right)H\left(\omega \right)+\underset{s>0}{min}\left\{\frac{\ell -1}{n_0}ln{g}_0\left(s,n_0\right)+\frac{1}{n_1}ln{g}_1\left(s,R_1,n_1\right)-\omega \ell lns\right\},$$

where $g_0\left(s,n_0\right)$ is a spectrum function of the constituent SPC code with length $n_0$,

$$g_0\left(s,n_0\right)=\sum _{i=0}^{n_0}\frac{{\left(1+s\right)}^{n_0}+{\left(1-s\right)}^{n_0}}{2},$$

and $g_1\left(s,R_1,n_1\right)$ is a spectrum function of the constituent linear code with a good spectrum, code rate $R_1$ and length $n_1$ obtained in [14]:

$$g_1\left(s,R_1,n_1\right)\le 1+n_1{2}^{-\left(1-R_1\right)n_1}\sum _{i=⌈{\delta }_{VG}{n}_1⌉}^{n_1}\left(\begin{array}{c}{n}_1\\ i\end{array}\right)s^i,$$

where ${\delta }_{VG}$ is given by the Varshamov-Gilbert bound.

Lemma 2.

Let the positive root ${\delta }_0$ of the following equation exist:

$${\nu }_0\left({\delta }_0\right)=0.$$

Then, a code with minimum code distance $\delta \ge {\delta }_0$ exists in ensemble ${\mathcal{E}}_{LG}\left(\ell ,n_0,b_0,R_1,n_1,b_1\right)$.

5. Numerical Results

One can see from Theorems 1 and 2 that the obtained lower-bounds $E_C(·)$ and $E_{ML}(·)$ depend on the set of parameters: error probability p of BSC, code rate $R_1$ and length $n_1$ of linear code from added layer, code rate $R_2$ and constituent code length $n_0$ of G-LDPC code (the value of ${\omega }_t$, used in $E_C(·)$ bound, depends on these parameters), wherein the code rate R of whole construction depends on $R_1$ and $R_2$.

Thus, to simplify the analysis let us first fix the parameters $R_1=0.85$, $n_1=2000$, $R=0.5$ and $p={10}^{-3}$ and find how $E_C\left(·\right)$ and $E_{ML}\left(·\right)$ depend on the SPC code length $n_0$ (see Figure 1). Then, let us consider the maximized $E_{ML}\left(·\right)$ and $E_C\left(·\right)$ over the values of $n_0$ (see Figure 2).

We can explain the different behaviors of $E_{ML}\left(·\right)$ and $E_C\left(·\right)$ shown in Figure 1 and Figure 2 by the following: the value of $E_{ML}\left(·\right)$ significantly depends on the value of the code distance $\delta$ of the LG-LDPC code and the value of $E_C\left(·\right)$ depends on the value of the error fraction ${\omega }_t$, which is guaranteed to be corrected by the G-LDPC code. And it is known that for the fixed code rate R code distance of LDPC code increases with the growth of constituent code length $n_0$ and guaranteed corrected error fraction ${\omega }_t$ has the maximum for the certain parameters $n_0$ and ℓ.

In Figure 3, we compare the dependencies on R for fixed $p={10}^{-3}$ of the obtained lower bound $E_{ML}\left(·\right)$, maximized over the values of $n_0$ and $R_1$, and of the lower bound $E_0\left(·\right)$.

Figure 4 shows the dependencies on R of the maximum values of $E_{ML}\left(·\right)$ and $E_C\left(·\right)$ for fixed $p={10}^{-3}$ (the maximization was performed over the values of $n_0$ and $R_1$).

As observed from Figure 4, $E_C\left(·\right)$ is approximately two orders of magnitude smaller than $E_{ML}\left(·\right)$, which almost reaches the lower bound on the error exponent $E_0\left(·\right)$ of the good linear code (see Figure 3). However, it is important to note that $E_{ML}\left(·\right)$ encounters only exponential decoding complexity and $E_C\left(·\right)$ encounters the decoding complexity of $\mathcal{O}\left(nlogn\right)$.

6. Conclusions

The main result of this paper is that we prove (see Corollary 1) the existence of such LDPC code with specific construction that the probability of erroneous decoding with low-complexity algorithm ($\mathcal{O}\left(nlogn\right)$) decreases exponentially with the growth of the code length for all code rates below BSC capacity. We also obtain the lower bound on error exponent under ML decoding for proposed construction (see Theorem 2) and show with numeric results that obtained lower-bound almost coincide with the error exponents of good linear codes for the certain parameters.

As a future work to improve the lower-bound for the low-complexity decoder, we plan to consider error-reducing codes instead of good linear codes and generalize our results for channels with reliability (e.g., channels with additive white Gaussian noise (AWGN) and “soft” reception).

7. Proofs of the Main Results

7.1. Error Exponent for Decoding Algorithm ${\mathcal{A}}_C$

Theorem 1 was proved in [12]. Here, we provide the proof for convenience of the reader in more detail, especially for the essential Corollary 1.

Let us first consider the complexity of the decoding algorithm ${\mathcal{A}}_C$ of an LG-LDPC code.

Lemma 3.

The complexity of the decoding algorithm ${\mathcal{A}}_C$ of an LG-LDPC code with length n is of order $O(nlogn)$ if the length of the linear code satisfies the inequality $n_1\le \frac{1}{R_1}{log}_2{log}_2\left(n\right)$.

Proof. 

Since the length of the linear code is equal to $n_1$ and the code rate is $R_1$, the complexity of the maximum likelihood decoding algorithm for the single code is of order $O\left(2^{R_1{n}_1}\right)$. The total number of codes is $b_1$, which is proportional to n, and then, the complexity of decoding all of the codes is of order $O\left(n{2}^{R_1{n}_1}\right)$.

In [4], it was shown that the complexity of the bit-flipping decoding algorithm of LDPC codes is $O(nlogn)$.

Therefore, the complexity of decoding algorithm ${\mathcal{A}}_C$ is of order $O\left(n{log}_{2}n\right)$ if the following condition is satisfied:

$$n{2}^{R_1{n}_1}\le n{log}_2\left(n\right).$$

Here we find the condition on $n_1$:

$$n_1\le \frac{1}{R_1}{log}_2{log}_2\left(n\right).$$

(4)

 □

Let us now consider the proof of Theorem 1.

Proof. 

Assume that in the first step of the decoding algorithm ${\mathcal{A}}_C$ of LG-LDPC code, the decoding error occurred exactly in i linear codes. Since each code contains no more than $n_1$ errors, the total number of errors W after the first step of decoding is no greater than $i{n}_1$. Let $i=\beta b_1$, where $\beta$ is the fraction of linear codes in which the decoding failure occurred; then,

$$W\le \beta b_1{n}_1=\beta n.$$

According to [4], LDPC code is capable of correcting any error pattern with weight less than W, that is,

$$W<W_0=\lfloor {\omega }_{t}n\rfloor ,$$

where ${\omega }_t$ is the fraction of errors guaranteed corrected by the G-LDPC code [4] (Theorem 1). Consequently, for the case where $\beta <{\omega }_t$, the decoding error probability P for LG-LDPC code under decoding algorithm ${\mathcal{A}}_C$ is equal to 0:

$$P=0,\beta <{\omega }_t.$$

At $\beta >{\omega }_t$, the error decoding probability is defined as

$$P=\sum _{i=⌊{\omega }_t{b}_1⌋}^{b_1}\left(\begin{array}{c}{b}_1\\ i\end{array}\right)P_2\left(W\ge W_0|i\right)P_1^i{\left(1-P_1\right)}^{b_1-i},$$

(5)

where $P_1$ is the error decoding probability of linear code,

$$P_1\le exp\{-n_1{E}_0(R_1,p)\},$$

and $P_2\left(W\ge W_0|i\right)$ is the probability that the number of errors after the first step of the decoding algorithm ${\mathcal{A}}_C$ is not less than $W_0$ under the condition that the decoding error occurred exactly in i linear codes.

Since the number of errors no more than doubles in a block in the case of error decoding with the maximum likelihood decoding algorithm, it must be greater than $\frac{W_0}{2}$ errors before the first step in i erroneous blocks to have more than $W_0$ errors after the first step of decoding algorithm ${\mathcal{A}}_C$. Then, we can write $P_2(W\ge W_0|i)$ as

$$P_2\left(W\ge W_0|i\right)=\sum _{j=⌊\frac{{\omega }_{t}n}{2}⌋}^{i{n}_1}\left(\begin{array}{c}i{n}_1\\ j\end{array}\right)p^j{\left(1-p\right)}^{i{n}_1-j}.$$

Using the Chernoff bound, we obtain

$$P_2(W\ge W_0|i)\le exp\{-n{E}_2(\beta ,{\omega }_t,p)\},$$

where

$$E_2\left(\beta ,{\omega }_t,p\right)=\left\{\begin{array}{c}\frac{1}{2}\left({\omega }_{t}ln\frac{{\omega }_t}{p}+\left(2\beta -{\omega }_t\right)ln\frac{2\beta -{\omega }_t}{1-p}\right)-\beta ln2\beta ,\beta <{\beta }_0,\hfill \\ 0,\beta \ge {\beta }_0.\hfill \end{array}\right.$$

(6)

Here $\beta =\frac{i}{b_1}>{\omega }_t$, and

$${\beta }_0=min\left(\frac{{\omega }_t}{2p},1\right)$$

because $\beta >1$ has no sense.

In accordance with (6), the probability $P_2(W\ge W_0|i)$ can be replaced with the trivial estimation $P_2(W\ge W_0|i)\le 1$ for $i\ge \lceil {\beta }_0{b}_1\rceil$, and then, sum (5) is upper bounded as follows:

$$P⩽\sum _{i=⌊{\omega }_t{b}_1⌋}^{⌊{\beta }_0{b}_1⌋}\left(\begin{array}{c}{b}_1\\ i\end{array}\right)P_2\left(W\ge W_0|i\right)P_1^i{\left(1-P_1\right)}^{b_1-i}+\sum _{i=⌈{\beta }_0{b}_1⌉}^{b_1}\left(\begin{array}{c}{b}_1\\ i\end{array}\right)P_1^i{\left(1-P_1\right)}^{b_1-i}.$$

Let $P_{\mathrm{II}}$ denote the first sum in the right part of this inequality and $P_{\mathrm{I}}$ denote the second sum. Let us consider each sum separately.

The sum $P_{\mathrm{I}}$ can be easily estimated as a tail of the binomial distribution with probability $P_1$ using the Chernoff bound:

$$P_{\mathrm{I}}\le exp\{-n{E}_{\mathrm{I}}(R_1,n_1,{\omega }_t,p)\},$$

where

$$E_{\mathrm{I}}(R_1,n_1,p)={\beta }_0{E}_0(R_1,p)-\frac{1}{n_1}H\left({\beta }_0\right),$$

and $P_1$ satisfies the condition

$$P_1(n_1,R_1,p)\le {\beta }_0.$$

Thus,

$$n_1\ge \frac{-ln{\beta }_0}{E_0(R_1,p)}.$$

(7)

Let us now consider the sum $P_{\mathrm{II}}$:

$$P_{II}\le ⌈\left({\beta }_0-{\omega }_t\right)b_1⌉\times \underset{{\omega }_t\le \beta \le {\beta }_0}{max}\left\{\left(\begin{array}{c}{b}_1\\ \beta b_1\end{array}\right)P_2\left(W\ge W_0|\beta b_1\right)P_1^{\beta b_1}{\left(1-P_1\right)}^{\left(1-\beta \right)b_1}\right\}.$$

Hence, at $n\to \infty$ ($b_1\to \infty$ and $b_0\to \infty$), we obtain

$$E_{\mathrm{II}}(R_1,n_1,{\omega }_t,p)=\underset{{\omega }_t\le \beta \le {\beta }_0}{min}\left\{E_2(\beta ,{\omega }_t,p)+\beta E_0(R_1,p)-\frac{1}{n_1}H\left(\beta \right)\right\}.$$

(8)

Let us note that if a minimum is achieved at ${\beta }_0$ in the right part of equality (8), then according to (6), we obtain $E_{\mathrm{II}}=E_{\mathrm{I}}$. Consequently, $E_{\mathrm{II}}\le E_{\mathrm{I}}$.

It is easy to see that at $n\to \infty$, the following inequality is satisfied:

$$P\le exp\{-nE(R_1,n_1,{\omega }_t,p)\},$$

where $E(R_1,n_1,{\omega }_t,p)$ = $min\{E_{\mathrm{II}},E_{\mathrm{I}}\}$ = $E_{\mathrm{II}}$.

According to the proved lemma, the complexity of the decoding algorithm ${\mathcal{A}}_C$ is of order $O(nlogn)$ if the condition (4) is satisfied, but for the obtained estimation, the condition (7) must also be satisfied. Thus,

$$\frac{-ln{\beta }_0}{E_0(R_1,p)}\le n_1\le \frac{1}{R_1}{log}_2{log}_{2}n.$$

This completes the proof.  □

Before proving Corollary 1, we need to consider the lower-bound behavior of the error fraction ${\omega }_t$ guaranteed corrected by G-LDPC code. In [4], the new estimation of the error fraction ${\omega }_t$ guaranteed corrected by generalized LDPC code with a given constituent code was obtained. Let us formulate this result for G-LDPC code:

Theorem 3.

Let the root ${\omega }_0$ exist for the following equation:

$$h\left({\omega }_0\right)-\ell F_e\left({\omega }_0,n_0\right)=0,$$

(9)

where $F_e\left({\omega }_0,n_0\right)$ is given by

$$F_e\left({\omega }_0,n_0\right)=h\left({\omega }_t\right)+\underset{s>0,0<v<1}{max}\left\{{\omega }_0{log}_{2}sv-\frac{1}{n_0}{log}_2\left(g_e\left(s,v,n_0\right)+g_0\left(s,n_0\right)\right)\right\},$$

where $g_0\left(s,n_0\right)$ and $g_e\left(s,v,n_0\right)$ have the following forms:

$$g_0\left(s,n_0\right)=\frac{{\left(1+s\right)}^{n_0}+{\left(1-s\right)}^{n_0}}{2},g_e\left(s,v,n_0\right)=g_d\left(s{v}^2,n_0\right),$$

where

$$g_d\left(s,n_0\right)={\left(1+s\right)}^{n_0}-g_0\left(s,n_0\right).$$

Let for the found value ${\omega }_0$, the root ${\alpha }_0$ exist for the following equation:

$$h\left({\omega }_0\right)-\ell F_{\mathbf{s}}\left(\alpha ,{\omega }_0,n_0,\ell \right)=0,$$

(10)

where $F_{\mathbf{s}}\left(\alpha ,{\omega }_0,n_0,\ell \right)$ is given by

$$F_{\mathbf{s}}\left(\alpha ,{\omega }_0,n_0,\ell \right)=h\left({\omega }_0\right)+\underset{s>0,0<v<1}{max}\left\{{\omega }_0\left({log}_{2}s+\frac{\ell -\frac{1-\alpha }{\alpha }}{\ell }{log}_{2}v\right)-\frac{1}{n_0}{log}_2\left(g_d\left(s,n_0\right)v+g_0\left(s,n_0\right)\right)\right\}.$$

Then, there exists a code (with $p_n:\underset{n\to \infty }{lim}{p}_n=1$) in the ensemble ${\mathcal{E}}_G\left(\ell ,n_0,b_0\right)$ of G-LDPC codes that can correct any error pattern with weight less than $⌊{\omega }_{t}n⌋$, where ${\omega }_t={\alpha }_0{\omega }_0$, with decoding complexity $\mathcal{O}\left(nlogn\right)$.

For Theorem 3, we obtain the following:

Corollary 2.

For the given code rate $R<1$, there exists a G-LDPC code in the ensemble ${\mathcal{E}}_G\left(\ell ,n_0,b_0\right)$ with $\ell >2$ such that equation (9) has a positive root ${\omega }_t>0$.

The proof of Theorem 3 was given in a more generalized form in [4]. Here, we consider only the proof of Corollary 2. For this purpose, let us formulate some useful facts proved in [4].

First, let us formulate the condition of the existence of a symbol that upon inversion, reduces the number of unsatisfied checks:

Lemma 4.

At least one such symbol exists that will be inverted during one iteration of decoding algorithm ${\mathcal{A}}_M$ for G-LDPC code if the following condition is satisfied:

$$E_{\sum }^{\left(W\right)}=2\sum _{j=1}^W{e}_{A_{1\to 0}}^{\left(i_j\right)}+\sum _{j=1}^W{e}_{A_{1\to 1}}^{\left(i_j\right)}>W\ell ,$$

(11)

where W is the number of errors in the received sequence, $i_1,i_2,\dots ,i_W$ are indices of erroneous symbols, $e_{A_{1\to 0}}^{\left(i\right)}$ is the number of edges emanating from the ith variable-node to the set of check-nodes for which the checks become satisfied after the inversion of this symbol, and $e_{A_{1\to 1}}^{\left(i\right)}$ is the number of the edges emanating from the ith variable-node to the set of check-nodes for which the checks remain unsatisfied after the inversion of this symbol.

Now, let us consider the estimation of the probability that the above condition is not satisfied:

Lemma 5.

The probability $P_W\left(E_{\sum }^{\left(W\right)}⩽W\ell \right)$ for the fixed pattern of errors of weight W that condition (11) is not satisfied, e.g., $E_{\sum }^{\left(W\right)}⩽W\ell$, is upper bounded as follows:

$$P_W\left(E_{\sum }^{\left(W\right)}⩽W\ell \right)⩽2^{-n\ell F_e\left(\omega ,n_0\right)+o\left(n\right)},\omega =\frac{W}{n}.$$

Now, let us consider the proof of Corollary 2.

Proof. 

Let us select an arbitrary small value ${\epsilon }^{\prime }$ and write the following condition:

$$\underset{n\to \infty }{lim}\sum _{W=1}^{⌊{\epsilon }^{\prime }n⌋}{2}^{-n\left(\ell F_e\left(\frac{W}{n},n_0\right)-h\left(\frac{W}{n}\right)\right)}<1.$$

In the left part of the inequality is the upper bound on the probability of the code that condition (11) is not satisfied for some sequences.

Let us introduce the following function $G\left(\omega \right)$:

$$G\left(\omega \right)=\ell F_e\left(\omega ,n_0\right)-h\left(\omega \right)=\left(\ell -1\right)h\left(\omega \right)+\ell \underset{s>0,0<v<1}{max}\left\{\omega {log}_{2}sv-\frac{1}{n_0}{log}_2\left(g_e\left(s,v,n_0\right)+g_0\left(s,n_0\right)\right)\right\}.$$

Since the variables s and v are dummies, they can be equal to an arbitrary value if the conditions $s>0$ and $0<v<1$ are satisfied. Then, let us set $s=v=\sqrt[4]{\omega }$ (this choice is justified by the fact that due to the structure of the parity-check matrix, the conditions $\ell >2$ should be satisfied):

$$G^{*}\left(\omega \right)=\left(\ell -1\right)h\left(\omega \right)+\ell \left(\frac{\omega }{2}{log}_2\omega -\frac{1}{n_0}{log}_2\left(g_e\left(\sqrt[4]{\omega },\sqrt[4]{\omega },n_0\right)+g_0\left(\sqrt[4]{\omega },n_0\right)\right)\right).$$

Let us transform $G^{*}\left(\omega \right)$ as follows:

$$G^{*}\left(\omega \right)=-\left(\frac{\ell }{2}-1\right)\omega {log}_2\omega -\left(\ell -1\right)\left(1-\omega \right){log}_2\left(1-\omega \right)-\frac{\ell }{n_0}{log}_2\left(g_e\left(\sqrt[4]{\omega },\sqrt[4]{\omega },n_0\right)+g_0\left(\sqrt[4]{\omega },n_0\right)\right).$$

It is easy to show that $g_e\left(s,v,n_0\right)+g_0\left(s,n_0\right)⩽{\left(1+s\right)}^{n_0}$ for $0<s<1$ and $0<v<1$. Then, we obtain

$$G^{*}\left(\omega \right)=-\left(\frac{\ell }{2}-1\right)\omega {log}_2\omega +\mathcal{O}\left(\omega \right).$$

It is easily noted that $G\left(\omega \right)⩾G^{*}\left(\omega \right)$ implies

$$\underset{n\to \infty }{lim}\sum _{W=1}^{⌊{\epsilon }^{\prime }n⌋}{2}^{-nG\left(\frac{W}{n}\right)}⩽\underset{n\to \infty }{lim}\sum _{W=1}^{⌊{\epsilon }^{\prime }n⌋}{2}^{-n{G}^{*}\left(\frac{W}{n}\right)}.$$

Since LDPC code construction requires $\ell >2$, $\left(\frac{\ell }{2}-1\right)>0$, and consequently,

$$G\left(\omega \right)⩾-c_1\omega {log}_2\omega +c_2\omega +o\left(\omega \right),c_1>0.$$

$$\underset{n\to \infty }{lim}\sum _{W=1}^{⌊{\epsilon }^{\prime }n⌋}{2}^{-nG\left(\frac{W}{n}\right)}⩽\underset{n\to \infty }{lim}\sum _{W=1}^{⌊{\epsilon }^{\prime }n⌋}{2}^{n·c_1·\frac{W}{n}·{log}_2\frac{W}{n}-n·c_2·\frac{W}{n}}=\underset{n\to \infty }{lim}\sum _{W=1}^{⌊{\epsilon }^{\prime }n⌋}{\left(\frac{W}{n}\right)}^{c_{1}W}{2}^{-c_{2}W}⩽$$

$$⩽\underset{n\to \infty }{lim}\sum _{W=1}^{⌊{\epsilon }^{\prime }n⌋}{\left({\left({\epsilon }^{\prime }\right)}^{c_1}{2}^{-c_2}\right)}^W=\frac{{\left({\epsilon }^{\prime }\right)}^{c_1}{2}^{-c_2}}{1-{\left({\epsilon }^{\prime }\right)}^{c_1}{2}^{-c_2}}={\epsilon }^{″}.$$

It should be noted that the sign of $c_2$ is not important because ${\epsilon }^{″}$ can be made arbitrarily small by a correct choice of ${\epsilon }^{\prime }$.

Thus,

$$\underset{n\to \infty }{lim}\sum _{W=1}^{⌊{\epsilon }^{\prime }n⌋}{2}^{-n\left(\ell F_e\left(\frac{W}{n},n_0\right)-h\left(\frac{W}{n}\right)\right)}⩽{\epsilon }^{″}<1.$$

Consequently, the code for which the condition (11) is satisfied for all values of ${\omega }_t<{\epsilon }^{\prime }$ exists with non-zero probability in the ensemble of G-LDPC codes.  □

Finally, let us consider the proof of Corollary 1.

Proof. 

The correctness of the corollary is easy to see if we note that $E_0(·)>0$ for $R_1<C$ [6] and $E_2(·)\ge 0$, which follows from (6), and we can always select $n_1$ such that $\frac{1}{n_1}H\left(\beta \right)<\beta E_0(·)+E_2(·)0pt12.5pt$ because $n_1$ can be arbitrarily large according to condition (3). Therefore, according to Corollary 2, the construction of G-LDPC code with ${\omega }_t>0$ for any code rate $R_2<1$ exists, helping us omit this condition in the corollary formulation (unlike the formulation of a similar corollary in [12]).  □

7.2. Error Exponent for Decoding Algorithm ${\mathcal{A}}_{ML}$

Let us consider the proof of Theorem 2.

Proof. 

To simplify the proof without loss of generality, let us consider the transmission of a zero codeword over the BSC with BER p. Let the probability of the transition of the zero codeword to each of $N\left(w\right)$ codewords with weight w during decoding with algorithm ${\mathcal{A}}_{ML}$ be equal to $P_{\delta }\left(w\right)$. Moreover, let there exist a critical value $w_c$ of the number of errors that leads to erroneous decoding with algorithm ${\mathcal{A}}_{ML}$. Then, we can write

$$P_{ML}=\sum _{w=d}^{w_c}N\left(w\right)P_{\delta }\left(w\right)+P\left(w\ge w_c\right),$$

where d is the code distance of the LG-LDPC code.

To obtain the upper bound, it is sufficient to consider the case when the zero codeword becomes the word with weight w if there are more than $w/2$ errors:

$$P_{\delta }\left(w\right)=\sum _{i=\frac{w}{2}}^w\left(\begin{array}{c}w\\ i\end{array}\right)p^i{\left(1-p\right)}^{w-i}\le 2^{w-1}{p}^{\frac{w}{2}}{\left(1-p\right)}^{\frac{w}{2}},$$

where $p\le \frac{1}{2}$.

From this inequality, we easily obtain

$$E_{\delta }\left({\omega }_c,p\right)=\underset{\delta \le \omega \le {\omega }_c}{max}\left\{\nu \left(\omega \right)+\omega \left(ln2+ln\sqrt{p\left(1-p\right)}\right)\right\},$$

where $\nu \left(\omega \right)$ is an asymptotic spectrum of the LG-LDPC code given by Lemma 1 and $\delta$ is the relative code distance of the LG-LDPC code given by Lemma 2.

With the help of the Chernoff bound, we obtain the exponent of the probability that more than $w_c$ errors have occurred:

$$P\left(w\ge w_c\right)\le exp\left\{-n{E}_{{\omega }_c}\left({\omega }_c,p\right)\right\}.$$

$$E_{{\omega }_c}\left({\omega }_c,p\right)=\left(1-{\omega }_c\right)ln\frac{1-{\omega }_c}{1-p}+{\omega }_{c}ln\frac{{\omega }_c}{p},{\omega }_c\ge p.$$

Consequently,

$$E_{ML}\left(p\right)=\underset{{\omega }_0\le {\omega }_c\le 1}{max}\left\{min\left(E_{\delta }\left({\omega }_c,p\right),E_{{\omega }_c}\left({\omega }_c,p\right)\right)\right\},$$

$${\omega }_0=max\left(\delta ,p\right).$$

 □

The estimations given in Lemmas 1 and 2 were obtained by the slightly modified classical Gallager’s method [1]. Thus, in this paper, we give only a sketch of the proof.

Let us first consider the proof of Lemma 1.

Proof. 

Let us consider the fixed word of weight W and find the probability of there being a code in the LG-LDPC code ensemble such that this word is a codeword for this code. For this purpose, let us consider the first layer of the parity-check matrix of some LG-LDPC code from the ensemble composed of the parity-check matrices of the single parity check code. We can write the probability that the considered word is a codeword for a given layer as follows:

$$P_W^{\left(1\right)}=\frac{N_1\left(W\right)}{\left(\genfrac{}{}{0pt}{}{n}{W}\right)},$$

where $N_1\left(W\right)$ is the number of layers, and the word of weight W is a codeword.

We estimate $N_1\left(W\right)$ as

$$N_1\left(W\right)⩽\underset{s>0}{min}\left\{\frac{g_0^{b_0}\left(s,n_0\right)}{s^W}\right\},$$

where $g_0\left(s,n_0\right)$ is a spectrum function of the SPC code.

Thus,

$$P_W^{\left(1\right)}⩽{\left(\genfrac{}{}{0pt}{}{n}{w}\right)}^{-1}\underset{s>0}{min}\left\{g_0^{b_0}\left(s,n_0\right)s^{-W}\right\}.$$

It is clear that the obtained estimation is the same for all $\ell -1$ layers:

$$P_W^{\left(i\right)}⩽{\left(\genfrac{}{}{0pt}{}{n}{w}\right)}^{-1}\underset{s>0}{min}\left\{g_0^{b_0}\left(s,n_0\right)s^{-W}\right\},i=1\dots \ell -1.$$

Similarly, we can write the probability that the considered word of weight W is a codeword for the ℓth layer of the parity-check matrix composed of “optimal” linear codes:

$$P_W^{\left(\ell \right)}⩽{\left(\genfrac{}{}{0pt}{}{n}{w}\right)}^{-1}\underset{s>0}{min}\left\{g_1^{b_1}\left(s,R_1,n_1\right)s^{-W}\right\},$$

where $g_1\left(s,R_1,n_1\right)$ is a spectrum of the code with a good spectrum.

Since the layer permutations are independent, we can write the probability that the given word of weight W is a codeword for the whole code construction as

$$P_W=\prod _{i=1}^{\ell }{P}_W^{\left(i\right)}⩽{\left(\genfrac{}{}{0pt}{}{n}{W}\right)}^{-\ell }\underset{s>0}{min}\left\{g_0^{b_0\left(\ell -1\right)}\left(s,n_0\right)g_1^{b_1}\left(s,R_1,n_1\right)s^{-W\ell }\right\}.$$

Consequently, the average number of weight W codewords is given by

$$\overline{N}\left(W\right)=\left(\genfrac{}{}{0pt}{}{n}{W}\right)P_W⩽{\left(\genfrac{}{}{0pt}{}{n}{W}\right)}^{-\left(\ell -1\right)}\underset{s>0}{min}\left\{g_0^{b_0\left(\ell -1\right)}\left(s,n_0\right)g_1^{b_1}\left(s,R_1,n_1\right)s^{-W\ell }\right\}.$$

For $W=\omega n$, we obtain

$$\nu \left(\omega \right)=\underset{n\to \infty }{lim}\frac{lnN\left(\omega n\right)}{n}⩽{\nu }_0\left(\omega \right).$$

 □

Now, let us consider the proof of Lemma 2.

Proof. 

If the average number of codewords $\overline{N}\left(W\right)$ in the ensemble of LG-LDPC codes satisfies the condition

$$\sum _{W=1}^{d_0}\overline{N}\left(W\right)⩽1,$$

then the code with code distance $d⩾d_0$ exists in this ensemble.

It is easy to show that the sum of the right part of the inequality can be estimated with the last member of this sum. Therefore, using the estimation obtained in the previous lemma, we can write

$${\nu }_0\left(\delta \right)⩽0,$$

where $\delta =d/n$ is the relative code distance.

Thus, we can obtain the maximum value of ${\delta }_0$ such that the above-considered condition is satisfied for all smaller values $\delta ⩽{\delta }_0$0 as the smallest positive root of the following equation:

$${\nu }_0\left({\delta }_0\right)=0.$$

 □