Adaptive Learned Belief Propagation for Decoding Error-Correcting Codes

Abstract

Weighted belief propagation (WBP) for the decoding of linear block codes is considered. In WBP, the Tanner graph of the code is unrolled with respect to the iterations of the belief propagation decoder. Then, weights are assigned to the edges of the resulting recurrent network and optimized offline using a training dataset. The main contribution of this paper is an adaptive WBP where the weights of the decoder are determined for each received word. Two variants of this decoder are investigated. In the parallel WBP decoders, the weights take values in a discrete set. A number of WBP decoders are run in parallel to search for the best sequence- of weights in real time. In the two-stage decoder, a small neural network is used to dynamically determine the weights of the WBP decoder for each received word. The proposed adaptive decoders demonstrate significant improvements over the static counterparts in two applications. In the first application, Bose–Chaudhuri–Hocquenghem, polar and quasi-cyclic low-density parity-check (QC-LDPC) codes are used over an additive white Gaussian noise channel. The results indicate that the adaptive WBP achieves bit error rates (BERs) up to an order of magnitude less than the BERs of the static WBP at about the same decoding complexity, depending on the code, its rate, and the signal-to-noise ratio. The second application is a concatenated code designed for a long-haul nonlinear optical fiber channel where the inner code is a QC-LDPC code and the outer code is a spatially coupled LDPC code. In this case, the inner code is decoded using an adaptive WBP, while the outer code is decoded using the sliding window decoder and static belief propagation. The results show that the adaptive WBP provides a coding gain of 0.8 dB compared to the neural normalized min-sum decoder, with about the same computational complexity and decoding latency.

1. Introduction

Neural networks (NNs) have been widely studied to improve communication systems. The ability of NNs to learn from data and model complex relationships makes them indispensable tools for tasks such as equalization, monitoring, modulation classification, and beamforming [1]. While NNs have also been considered for decoding error-correcting codes for quite some time [2,3,4,5,6,7,8,9,10], interest in this area has surged significantly in recent years due to advances in NNs and their widespread commercialization [11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46].

Two categories of neural decoders may be considered. In model-agnostic decoders, the NN has a general architecture independent of the conventional decoders in coding theory [12,34,42]. Many of the common architectures have been studied for decoding, including multi-layer perceptrons [5,12,47], convolutional NNs (CNNs) [48], recurrent neural networks (RNNs) [13], autoencoders [19,35,40], convolutional decoders [6], graph NNs [33,43], and transformers [34,41]. These models have been used to decode linear block codes [4,12], Reed–Solomon codes [49], convolutional codes [4,48,50], Bose–Chaudhuri–Hocquenghem (BCH) codes [11,16,18,31], Reed–Muller codes [5,25], turbo codes [13], low-density parity-check (LDPC) codes [37,51], and polar codes [52].

Training neural decoders is challenging because the number of codewords to classify depends exponentially on the number of information bits. Furthermore, the sample complexity of the NN is high for the small bit error rates (BERs) and long block lengths required in some applications. As a consequence, model-agnostic decoders often require a large number of parameters and may overfit, which makes them impractical unless the block length is short.

In model-based neural decoders, the architecture of the NN is based on the structure of a conventional decoder [11,14,25,28,31,53]. An example is weighted belief propagation (WBP), where the messages exchanged across the edges of the Tanner graph of the code are weighted and optimized [11,31,54]. This gives rise to a decoder in the form of a recurrent network obtained by unfolding the update equations of the belief propagation (BP) over the iterations. Since the WBP is a biased model, it has fewer parameters than the model-agnostic NNs at the same accuracy.

Prior work has demonstrated that the WBP outperforms BP for block lengths up to around 1000, particularly with structured codes, low-to-moderate code rates, and high signal-to-noise ratios (SNRs) [17,28,31,37,38,39,54,55,56,56]. It is believed that the improvement is achieved by altering the log-likelihood ratios (LLRs) that are passed along short cycles. For example, for BCH and LDPC codes with block lengths under 200, WBP provides frame error rate (FER) improvements of up to 0.4 dB in the waterfall region and up to 1.5 dB in the error-floor region [23,57,58]. Protograph-based (PB) QC-LDPC codes have been similarly decoded using the learned weighted min-sum (WMS) decoder [28].

The WBP does not generalize well at low bit error rates (BERs) due to the requirement of long block lengths and the resulting high sample and training complexity [44]. For example, in optical fiber communication, the block length can be up to tens of thousands to achieve a BER of ${10}^{-15}$. In this case, the sample complexity of WBP is high, and the model does not generalize well when trained with a practically manageable number of examples.

The training complexity and storage requirements of the WBP can be reduced through parameter sharing. Lian et al. introduced a WBP decoder wherein the parameters are shared across or within the layers of the NN [18,39]. A number of parameter-sharing schemes in WBP are studied in [28,39]. Despite intensive research in recent years, WBP remains impractical in most real-world applications.

In this work, we improve the generalization of WBP to enhance its practical applicability. The WBP is a static NN, trained offline based on a dataset. The main contribution of this paper is the proposal of adaptive learned message-passing algorithms, where the weights assigned to messages are determined for each received word. In this case, the decoder is dynamic, changing its parameters for each transmission in real time.

Two variants of this decoder are proposed. In the parallel decoder architecture, the weights take values in a discrete set. A number of WMS decoders are run in parallel to find the best sequence of weights based on the Hamming weight of the syndrome of the received word. In the two-stage decoder, a secondary NN is trained to compute the weights to be used in the primary NN decoder. The secondary NN is a CNN that takes the LLRs of the received word and is optimized offline.

The performance and computational complexity of the static and adaptive decoders are compared in two applications. In the first application, a number of regular and irregular quasi-cyclic low-density parity-check (QC-LDPC) codes, along with a BCH and a polar code, are evaluated over an additive white Gaussian noise (AWGN) channel in both low- and high-rate regimes. The results indicate that the adaptive WMS decoders achieve decoding BERs up to an order of magnitude less than the BERs of the static WMS decoders, at about the same decoding complexity, depending on the code, its rate, and the SNR. The coding gain is 0.32 dB at a bit error rate of ${10}^{-4}$ in one example.

The second application is coding over a nonlinear optical fiber link with wavelength division multiplexing (WDM). The data rates in today’s optical fiber communication system approach terabits/s per wavelength. Here, the complexity, power consumption, and latency of the decoder are important considerations. We apply concatenated coding by combining a low-complexity short-block-length soft-decision inner code with a long-block-length hard-decision outer code. This approach allows the component codes to have much shorter block lengths and higher BERs than the combined code. As a result, it becomes feasible to train the WBP for decoding the inner code, addressing the curse of dimensionality and sample complexity issues. For PB QC-LDPC inner codes and a spatially coupled (SC) QC-LDPC outer code, the results indicate that the adaptive WBP outperforms the static WBP by 0.8 dB at about the same complexity and decoding latency in a 16-QAM 8 × 80 km 32 GBaud WDM system with five channels.

The remainder of this paper is organized as follows. Section 2 introduces the notation, followed by the channel models in Section 3. In Section 4, we introduce the WBP, and in Section 5, two adaptive learned message-passing algorithms. In Section 6, we compare the performance and complexity of the static and adaptive decoders, and in Section 7, we conclude the paper. Appendix A and Appendix B provide supplementary information, and Appendix C presents the parameters of the codes.

2. Notation

Natural, real, complex and non-negative numbers are denoted by $\mathbb{N}$, $\mathbb{R}$, $\mathbb{C}$, and ${\mathbb{R}}_{+}$, respectively. The set of integers from m to n is shown as $[m:n]=\left\{m,m+1,\cdots ,n\right\}$. The special case $m=1$ is shortened to $\left[n\right]\stackrel{\Delta }{=}[1:n]$. $\lfloor x\rfloor$ and $\lceil x\rceil$ denote, respectively, the floor and ceiling of $x\in \mathbb{R}$. The Galois field GF(q) with $q\in \mathbb{N}$, $q\ge 2$, is ${\mathbb{F}}_q$. The set of matrices with m rows, n columns, and elements in $[0:q-1]$ is ${\mathbb{F}}_q^{m\times n}$.

A sequence of length n is denoted as $x^n=(x_1,x_2,\cdots ,x_n)$. Deterministic vectors are denoted by boldface font, e.g., $\mathbf{x}\in {\mathbb{R}}^n$. The $i^{\mathrm{th}}$ entry of $\mathbf{x}$ is ${\left[\mathbf{x}\right]}_i$. Deterministic matrices are shown by upper-case letters with mathrm font, e.g., $\mathrm{A}$.

The probability density function (PDF) of a random variable X is denoted by ${Pr}_X\left(x\right)$, shortened to $Pr\left(x\right)$ if there is no ambiguity. The conditional PDF of Y given X is ${Pr}_{Y|X}\left(y\right|x)$. The expected value of a random variable X is denoted by $\mathbb{E}\left(X\right)$. The real Gaussian PDF with mean $\theta$ and standard deviation $\sigma$ is denoted by $N(\theta ,{\sigma }^2)$. The Q function is $Q\left(x\right)={\displaystyle \frac{1}{2}}erfc\left({\displaystyle \frac{x}{\sqrt{2}}}\right)$, where $erfc\left(x\right)$ is the complementary error function. The binary entropy function is $H_b\left(x\right)=-(x{log}_2\left(x\right)+(1-x){log}_2(1-x))$, $x\in (0,1)$.

3. Channel Models

3.1. AWGN Channel

Encoder: We consider an $(n,k)$ binary linear code $\mathcal{C}$ with the parity-check matrix (PCM) $\mathrm{H}\in {\mathbb{F}}_2^{m\times n}$, where n is the code length, k is the code dimension, and $m\ge n-k$, $m\in \mathbb{N}$. The rate of the code is $r=k/n\ge 1-{\displaystyle \frac{m}{n}}$. A PB QC-LDPC code is characterized by a lifting factor $M\in \mathbb{N}$, a base matrix $\mathrm{B}\in {\{-1,0,1\}}^{\lambda \times \omega }$, and an exponent matrix $\mathrm{P}\in {\{0,1,\cdots ,M-1\}}^{\lambda \times \omega }$ where, $\lambda ,\omega \in \mathbb{N}$, $\lambda <\omega$. Given $(\lambda ,\omega ,M,\mathrm{P})$, the PCM is obtained according to the procedure in Appendix A.

We evaluate a BCH code, seven regular and irregular QC-LDPC codes, and a polar code in the low- and high-rate regimes. These codes are summarized in Table 1 and described in Section 6.1. The parameters of the QC-LDPC codes are given in Appendix C.

Table 1. Codes in this paper.

AWGN Channel
Low rate	High rate
BCH $C_1(63,36)$, $r=0.57$	QC-LDPC ${\mathcal{C}}_6(1050,875)$, $0.83$
QC-LDPC $C_2(3224,1612)$, $0.5$	QC-LDPC ${\mathcal{C}}_7(1050,850)$, $0.81$
QC-LDPC $C_3(4016,2761)$, $0.69$	QC-LDPC ${\mathcal{C}}_8(4260,3834)$, $0.9$
Irregular LDPC $C_4(420,180)$, $0.43$	Polar ${\mathcal{C}}_9(1024,854)$, $0.83$
Irregular LDPC $C_5(128,64)$, $0.5$
Optical Fiber Channel
Inner code	Outer code
Single-edge QC-LDPC ${\mathcal{C}}_{10}(4000,3680)$, $0.92$	Multi-edge QC-LDPC ${\mathcal{C}}_{11}(3680,3520)$, $0.96$
Non-binary multi-edge ${\mathcal{C}}_{12}(800,32)$

The encoder maps a sequence of information bits $\mathbf{b}=(b_1,b_2,\dots ,b_k)$, $b_i\in \{0,1\}$, $i\in \left[k\right]$, to a codeword $\mathbf{c}=(c_1,c_2,\dots ,c_n)$ as $\mathbf{c}=\mathbf{b}\mathrm{G}$, where $\mathrm{G}\in {\mathbb{F}}_2^{k\times n}$ is the generator matrix of the code.

Channel model: The codeword $\mathbf{c}$ is modulated with a binary phase shift keying with symbols $\pm A\in \mathbb{R}$, and transmitted over an AWGN channel. The vector of received symbols is $\mathbf{y}=(y_1,\cdots ,y_n)$, where

$$\begin{array}{c}\hfill y_i={(-1)}^{c_i}A+z_i,i=1,2,3,\cdots ,\end{array}$$

(1)

where $z_j\sim i.i.d.N(0,{\sigma }^2)$. If $\rho$ is the SNR, the channel can be normalized so that $A=1$, and ${\sigma }^2={\left(r\rho \right)}^{-1}$.

The LLR function $L:{\mathbb{R}}^n\mapsto {\mathbb{R}}^n$ of $\mathbf{c}$ conditioned on $\mathbf{y}$ is

$$\begin{array}{c}\hfill {\left.[\mathrm{L}\left(\mathbf{y}\right)\right]}_i\stackrel{\Delta }{=}log\left({\displaystyle \frac{Pr\left(c_i=0\mid \mathbf{y}\right)}{Pr\left(c_i=1\mid \mathbf{y}\right)}}\right)\end{array}$$

$$=log\left({\displaystyle \frac{Pr\left(y_i\mid c_i=0\right)}{Pr\left(y_i\mid c_i=1\right)}}\right)$$

(2)

$$=4r\rho y_i$$

(3)

for each $i\in \left[n\right]$. Equation (2) holds under the assumption that $c_i$ are independent and uniformly distributed, and (3) is obtained from Gaussian $Pr\left(y_i\right|c_i)$ from (1).

Decoder: We compare the performance and complexity of the static and adaptive belief propagation. The static decoders are tanh-based BP, the auto-regressive BP and WBP with different levels of parameter sharing, including BP with simple scaling and parameter-adapter networks (SS-PAN) [18]. Additionally, to assess the achievable performance with a large number of parameters in the decoder, we include a comparison with two model-agnostic neural decoders based on transformers [41] and graph NNs [33,43].

3.2. Optical Fiber Channel

In this application, we consider a multi-user fiber-optic transmission system using WDM with $N_c$ users, each of bandwidth $B_0$ Hz, as shown in Figure 1.

Figure 1. Block diagram of an optical fiber transmission system.

Transmitter (TX): A binary source generates a pseudo-random bit sequence of $n_b$ information bits ${\mathbf{b}}_u=(b_{u,1},b_{u,2},\dots ,b_{u,n_b})$, $b_{u,j}\in \{0,1\}$, for the WDM channel $u\in [u_1:u_2]$, $u_1=-\lfloor N_c/2\rfloor$, $u_2=\lceil N_c/2\rceil -1$, $j\in [1:n_b]$. Bit-interleaved coded modulation (BICM) with concatenated coding is applied in WDM channels independently. The BICM comprises an outer encoder for the code ${\mathcal{C}}_{\mathrm{o}}(n_{\mathrm{o}},k_{\mathrm{o}})$ with rate $r_{\mathrm{o}}$, an inner encoder for ${\mathcal{C}}_i(n_{\mathrm{i}},k_{\mathrm{i}})$ with rate $r_{\mathrm{i}}$, a permuter $\pi$, and a mapper $\mu$, where $n_{\mathrm{o}}/k_{\mathrm{i}}$ is assumed to be an integer. The concatenated code $\mathcal{C}={\mathcal{C}}_i\circ {\mathcal{C}}_o$ has parameters $k=k_{\mathrm{o}}$ and $n=n_{\mathrm{o}}/r_{\mathrm{i}}$ and $r_{\mathrm{total}}=r_{\mathrm{i}}{r}_{\mathrm{o}}$. Each consecutive subsequence of ${\mathbf{b}}_u$ of length $k_o$ is mapped to ${\tilde{\mathbf{c}}}_u\in {\mathcal{C}}_o\subset {\{0,1\}}^{n_{\mathrm{o}}}$ by the outer encoder and subsequently to ${\mathbf{c}}_u\in \mathcal{C}\subset {\{0,1\}}^n$ by the inner encoder. Next, ${\mathbf{c}}_u$ is mapped to ${\overline{\mathbf{c}}}_u=\pi \left({\mathbf{c}}_u\right)$ by a random uniform permuter $\pi :{\mathbb{F}}_2^n\mapsto {\mathbb{F}}_2^n$. The mapper $\mu :{\mathbb{F}}_2^m\mapsto \mathcal{A}$ maps consecutive sub-sequences of ${\overline{\mathbf{c}}}_u$ of length m to a symbol in a constellation $\mathcal{A}$ of size $\mathfrak{M}=2^m$. Thus, the BICM maps ${\mathbf{b}}_u$ to a sequence of symbols ${\mathbf{s}}_u=(s_{u,{\mathcal{l}}_1},s_{u,{\mathcal{l}}_1+1},\dots ,s_{u,{\mathcal{l}}_2})$, where $s_{u,\mathcal{l}}\in \mathcal{A}$, $\mathcal{l}\in [{\mathcal{l}}_1:{\mathcal{l}}_2]$, ${\mathcal{l}}_1=-\lfloor n_s/2\rfloor$, ${\mathcal{l}}_2=\lceil n_s/2\rceil -1$, $n_s=n_b/m$.

The symbols $s_{u,\mathcal{l}}$ are modulated with a root raised cosine (RRC) pulse shape $p\left(t\right)$ at symbol rate $R_s$, where t is the time. The resulting electrical signal of each channel $x_u\left(t\right)$ is converted to an optical signal and subsequently multiplexed by a WDM multiplexer. The baseband representation of the transmitted signal is

$$x\left(t\right)=\sum _{u=u_1}^{u_2}\sum _{\mathcal{l}={\mathcal{l}}_1}^{{\mathcal{l}}_2}{s}_{u,\mathcal{l}}p(t-\mathcal{l}{T}_s)e^{j2\pi u{B}_{0}t},$$

(4)

where $T_s=1/R_s$ and $s_{u,\mathcal{l}}$ are i.i.d. random variables. The average power of the transmitted signal is $\mathcal{P}$; thus, $\mathbb{E}|s_{u,\mathcal{l}}{|}^2=\mathcal{P}$, $\forall u,\mathcal{l}$.

Encoder: SC LDPC codes are attractive options for optical communications [59]. These codes approach the capacity of the canonical communication channels [60,61] and have a flexible performance–complexity trade-off. They are decoded with the BP and the sliding window decoder (SWD). Another class of codes in optical communication is the SC product-like codes, braided block codes [62], SC turbo product codes [63], staircase codes [64,65,66] and their generalizations [67,68]. These codes are decoded with iterative, algebraic hard decision algorithms and prioritize low-complexity, hardware-friendly decoding over coding gain.

In this paper, the encoding in BICM combines an inner (binary or non-binary) QC-LDPC code ${\mathcal{C}}_i$ with an outer SC QC-LDPC code ${\mathcal{C}}_o$ whose component code is a multi-edge QC-LDPC code, as outlined in Table 1. The construction and parameters of the codes are given in Appendix B.1 and Appendix C, respectively.

The choice of the inner code is due to the decoder complexity. Other options have been considered in the literature, for instance, algebraic codes, e.g., the BCH (Section 3.3 [69]) or Reed-Solomon codes (Section 3.4 [69]), or polar codes [70]. However, the QC-LDPC codes are simpler to decode, especially at high rates. The outer code can be an LDPC code [71,72], a staircase code [71,73], or a SC-LDPC code [72].

Fiber-optic link: The channel is an optical fiber link with $N_{sp}$ spans of length $L_{sp}$ of the standard single-mode fiber, with parameters in Table 2.

Table 2. The parameters of the fiber-optic link.

Parameter Name	Value
Transmitter parameters
WDM channels	5
Symbol rate $R_s$	32 Gbaud
RRC roll-off	0.01
Channel frequency spacing	33 GHz
Fiber channel parameters
Attenuation $\left(\alpha \right)$	0.2 dB/km
Dispersion parameter (D)	17 ps/nm/km
Nonlinearity parameter $\left(\gamma \right)$	1.2 l/(W·km)
Span configuration	8 × 80 km
EDFA gain	16 dB
EDFA noise figure	5 dB

Let $q(t,z):\mathbb{R}\times {\mathbb{R}}^{+}\mapsto \mathbb{C}$ be the complex envelope of the signal as a function of time t and distance z along the fiber. The propagation of the signal in one polarization over one span of optical fiber is modeled by the nonlinear Schrödinger equation [74]

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial q(t,z)}{\partial z}}=-{\displaystyle \frac{\alpha }{2}}q(t,z)-{\displaystyle \frac{j{\beta }_2}{2}}{\displaystyle \frac{{\partial }^{2}q(t,z)}{\partial t^2}}+j\gamma {\left|q(t,z)\right|}^{2}q(t,z),\end{array}$$

(5)

where $\alpha$ is the loss constant, ${\beta }_2$ is the chromatic dispersion coefficient, $\gamma$ is the Kerr nonlinearity parameter, and $j=\sqrt{-1}$. The transmitter is located at $z=0$ and the receiver at $z=\mathcal{L}$. The continuous-time model (5) can be discretized to a discrete-time discrete-space model using the split-step Fourier method (Section III.B [75]). The optical fiber channel described by the partial differential Equation (5) differs significantly from the AWGN channel due to the presence of nonlinearity.

An erbium doped fiber amplifier (EDFA) is placed at the end of each span, which compensates for the fiber loss, and introduces amplified spontaneous emission noise. The input $x_i\left(t\right)$–output $x_o\left(t\right)$ relation of the EDFA is given by $x_o\left(t\right)=G{x}_i\left(t\right)+n\left(t\right)$, where $G=e^{\alpha {\mathcal{L}}_{sp}}$ is the amplifier’s gain, and $n\left(t\right)$ is zero-mean circularly symmetric complex Gaussian noise process with the power spectral density

$$\begin{array}{c}\hfill {\sigma }^2={\displaystyle \frac{1}{2}}(G-1)h{f}_{0}NF,\end{array}$$

where NF is the noise figure, h is a Planck constant, and $f_0$ is the carrier frequency at 1550 nm.

Receiver: The advent of the coherent detection paved the way for the compensation of transmission effects in optical fiber using digital signal processing (DSP). As a result, the linear effects in the channel, such as the chromatic dispersion and polarization-induced impairments, and some of the nonlinear effects, can be compensated with DSP.

At the receiver, a demultiplexer filters the signal of each WDM channel. The optical signal for each channel is converted to an electrical signal by a coherent receiver. Next, DSP followed by bit-interleaved coded demodulation (BICD) is applied. The continuous-time electrical signal is converted to the discrete-time signals by analogue-to-digital converters, down-sampled, and passed to a digital signal processing unit for the mitigation of the channel impairments. For equalization, digital back-propagation (DBP) based on the symmetric split-step Fourier method is applied to compensate for most of the linear and nonlinear fiber impairments [76].

After DSP, the symbols are still subject to signal-dependent noise, which is mitigated by the bit-interleaved coded demodulator (BICD). Let $\mathbf{y}\in {\mathbb{C}}^{n_s}$ denote the equalized signal samples for the transmitted symbols $\mathbf{s}\in {\mathcal{A}}^{n_s}$ in the WDM channel of interest. Given that the deterministic effects were equalized, we assume that the channel $\mathbf{s}\mapsto \mathbf{y}$ is memoryless so that $Pr\left(\mathbf{y}\right|\mathbf{s})={\prod }_{\mathcal{l}=1}^{n_s}Pr\left(y_{\mathcal{l}}\right|s_{\mathcal{l}})$. For $s\in \mathcal{A}$, let ${\mu }^{-1}\left(s\right)=(b_1\left(s\right),\cdots ,b_m\left(s\right))$. From the symbol-to-symbol channel $Pr\left(y\right|s)$, $s\in \mathcal{A}$, $y\in \mathbb{C}$, we obtain m bit-to-symbol channels

$$\begin{array}{c}\hfill {Pr}_j\left(y\right|b)={\displaystyle \sum _{s\in \mathcal{A},b_j\left(s\right)=b}}Pr\left(y\right|s),\end{array}$$

(6)

where $b\in {\mathbb{F}}_2$, and $j\in \left[m\right]$.

Let $\overline{\mathbf{c}}=(b_1\left(s_1\right),\cdots ,b_m\left(s_1\right),\cdots ,b_1\left(s_{n_s}\right),\cdots ,b_m\left(s_{n_s}\right))$, $n=m{n}_s$. The LLR function $L:{\mathbb{C}}^{n_s}\mapsto {\mathbb{R}}^n$ of $\mathbf{c}$ conditioned on $\mathbf{y}$ is, for each $i\in \left[n\right]$,

$$\begin{array}{cc}\hfill {\left.[\mathrm{L}\left(\mathbf{y}\right)\right]}_i& \stackrel{\Delta }{=}log\left({\displaystyle \frac{Pr\left(c_i=0\mid \mathbf{y}\right)}{Pr\left(c_i=1\mid \mathbf{y}\right)}}\right)\hfill \\ \hfill & =log\left({\displaystyle \frac{Pr\left({\overline{c}}_{i^{\prime }}=0\mid \mathbf{y}\right)}{Pr\left({\overline{c}}_{i^{\prime }}=1\mid \mathbf{y}\right)}}\right)\hfill \\ \hfill & =log\left({\displaystyle \frac{{Pr}_j\left(y\mid b=0\right)}{{Pr}_j\left(y\mid b=1\right)}}\right),\hfill \end{array}$$

(7)

where $i^{\prime }$ is obtained from i according to $\pi$, $j=i^{\prime }modm$, and ${Pr}_j\left(y\right|b)$ is defined in (6).

Decoder: The decoding of ${\mathcal{C}}_i\circ {\mathcal{C}}_o$ consists of two steps. First, ${\mathcal{C}}_i$ is decoded using an adaptive WBP in Appendix B, which takes the soft information $\mathrm{L}\left(\mathbf{y}\right)\in {\mathbb{R}}^n$ and corrects some errors. Second, ${\mathcal{C}}_o$ is decoded using the min-sum (MS) decoder with SWD in Appendix B, which further lowers the BER, and outputs the decoded information bits $\widehat{\mathbf{b}}$. The LLRs in the inner decoder are represented with 32 bits, and in the outer decoder are quantized at 4 bits with per-window configuration.

In optical communication, the forward error correction (FEC) overhead 6–25% is common [77]. Thus, the inner code typically has a high rate of ≥0.9 and a block length of several thousands, achieving a BER of ${10}^{-6}$–${10}^{-2}$. The outer code has a length of up to tens of thousands, lowering the BER to an error floor to ∼${10}^{-15}$.

3.3. Performance Metrics

Q-factor: The SNR per bit in the optical fiber channel is $E_b/N_o$, where $E_b=\mathcal{P}/m$ is the bit energy, and $N_o={\sigma }^{2}B{N}_{sp}$ is the total noise power in the link, where $B=B_0{N}_c$. The performance of the uncoded communication system is often measured by the BER. The Q-factor for a given BER is the corresponding SNR in an additive white Gaussian noise channel with binary phase-shift keying modulation:

$$\begin{array}{c}\hfill \mathrm{QF}=20{log}_{10}\left(\sqrt{2}{erfc}^{-1}\left(2\mathrm{BER}\right)\right),\mathrm{dB}.\end{array}$$

Coding gain: Let ${\mathrm{BER}}_i$ and ${\mathrm{QF}}_i$ (respectively, ${\mathrm{BER}}_o$ and ${\mathrm{QF}}_o$) denote the BER and Q-factor at the input (respectively, output) of the decoder. The coding gain (CG) in dB is the reduction in the Q-factor

$$\begin{array}{ccccc}\hfill \mathrm{CG}& =\hfill & {\mathrm{QF}}_o-{\mathrm{QF}}_i& =& 20{log}_{10}{\mathrm{erfc}}^{-1}\left(2{\mathrm{BER}}_o\right)-20{log}_{10}{\mathrm{erfc}}^{-1}\left(2{\mathrm{BER}}_i\right).\end{array}$$

The corresponding net CG (NCG) is

$$\begin{array}{c}\hfill \mathrm{NCG}=\mathrm{CG}+10{log}_{10}{r}_{\mathrm{total}}.\end{array}$$

(8)

Finite block-length NCG: If n is finite, the rate $r_{\mathrm{total}}$ in (8) may be replaced with the information rate in the finite block-length regime [78]

$$C_f\approx C-{log}_2\left(e\right)\sqrt{\frac{{\mathrm{BER}}_i(1-{\mathrm{BER}}_i)}{n}}{Q}^{-1}({\mathrm{BER}}_o),$$

where $Q\left(x\right)={\displaystyle \frac{1}{2}}\mathrm{erfc}\left({\displaystyle \frac{x}{\sqrt{2}}}\right).$

4. Weighted Belief Propagation

Given a code $\mathcal{C}$, one can construct a bipartite Tanner graph $T_{\mathcal{C}}=(C,V,E)$, where $C=\left[m\right]$, $\left[m\right]\stackrel{\Delta }{=}1,2,\dots ,m$, $V=\left[n\right]$, and $E=\{(c,v)\in C\times V\mid {\mathrm{H}}_{c,v}\ne 0\}$ are, respectively, the set of check nodes, variable nodes and the edges connecting them. Let $V_c=\{v\in V\mid (c,v)\in E\}$, $C_v=\{c\in C\mid (c,v)\in E\}$, and $d_c$ and $d_v$ be the degree of c and v in $T_{\mathcal{C}}$, respectively.

The WBP is an iterative decoder based on the exchange of the weighted LLRs between the variable nodes and the check nodes in $T_{\mathcal{C}}$ [11,79]. Let ${\mathrm{L}}_{c2v}^{\left(t\right)}$ denote the extrinsic LLR from the check node c to the variable node v at iteration t. Define similarly ${\mathrm{L}}_{v2c}^{\left(t\right)}$.

The decoder is initialized at $t=1$ with ${\mathrm{L}}_{v2c}^{\left(0\right)}=\mathrm{L}\left(y_j\right)$, where v is the j-th variable node, and $\mathrm{L}\left(y_j\right)$ is obtained from (3) or (7). For iteration $t\in \left[T\right]$, the LLRs are updated in two steps.

The check node update:

$$\begin{array}{c}{L}_{c2v}^{\left(t\right)}\stackrel{\left(a\right)}{=}2{tanh}^{-1}\left\{{\displaystyle \prod _{v^{\prime }\in {\mathrm{V}}_c\setminus \left\{v\right\}}}tanh{\displaystyle \frac{{\gamma }_{v^{\prime },c}^{\left(t\right)}}{2}}{\mathrm{L}}_{v^{\prime }2c}^{(t-1)}\right\}\\ \stackrel{\left(b\right)}{\approx }{\underset{¯}{\mathrm{L}}}_{c,v}{\displaystyle \prod _{v^{\prime }\in {\mathrm{V}}_c\setminus \left\{v\right\}}}{\gamma }_{v^{\prime },c}^{\left(t\right)}sign\left({\mathrm{L}}_{v^{\prime }2c}^{(t-1)}\right),\end{array}$$

(9)

where

$${\underset{¯}{\mathrm{L}}}_{c,v}=\underset{v^{\prime }\in {\mathrm{V}}_c\setminus \left\{v\right\}}{min}\left\{\left|{\mathrm{L}}_{v^{\prime }2c}^{(t-1)}\right|\right\}.$$

The equation in $\left(a\right)$ represents the update relation in the BP [69], where the LLR messages are scaled by non-negative weights $\{{\gamma }_{v,c}^{\left(t\right)}:v\in V,c\in C_v,t\in \left[T\right]\}$. Further, $\left(b\right)$ is obtained from $\left(a\right)$ though an approximation to lower the computational cost. The WBP and WMS decoders use $\left(a\right)$ and $\left(b\right)$, respectively.

The variable-node update:

$$\begin{array}{c}\hfill L_{v2c}^{\left(t\right)}={\alpha }_v^{\left(t\right)}\mathrm{L}\left(\mathbf{y}\right)+{\displaystyle \sum _{c^{\prime }\in {\mathrm{C}}_v\setminus \left\{c\right\}}}{\beta }_{c^{\prime },v}^{\left(t\right)}{\mathrm{L}}_{c^{\prime }2v}^{(t-1)}.\end{array}$$

(10)

This is the update relation in the BP, to which the sets of non-negative weights $\{{\alpha }_v^{\left(t\right)}:v\in V,t\in \left[T\right]\}$ and $\{{\beta }_{c,v}^{\left(t\right)}:c\in C,v\in V_c,t\in \left[T\right]\}$ are introduced.

At the end of each iteration t, a hard decision is made

$$\begin{array}{c}\hfill {\overline{y}}_j=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{\mathrm{L}}_v^{\left(t\right)}<0,\hfill \\ 0,\hfill & \mathrm{if}{\mathrm{L}}_v^{\left(t\right)}\ge 0,\hfill \end{array}\right.\end{array}$$

(11)

where

$$\begin{array}{c}\hfill L_v^{\left(t\right)}=\mathrm{L}\left(y_j\right)+{\displaystyle \sum _{c\in C_v}}{\mathrm{L}}_{c2v}^{\left(t\right)}.\end{array}$$

(12)

Let $\overline{\mathbf{y}}=\left({\overline{y}}_1,{\overline{y}}_2,\cdots ,{\overline{y}}_n\right)\in {\mathbb{F}}_2^n$, and let $\mathbf{s}=\overline{\mathbf{y}}{\mathrm{H}}^T\in {\mathbb{F}}_2^m$ be the syndrome. The algorithm stops if $\mathbf{s}=\mathbf{0}$ or $t=T$.

The computation in (9) and (10) can be expressed with an NN. The Tanner graph $T_{\mathcal{C}}$ is unrolled over the iterations to obtain a recurrent network with $2T$ layers (see Figure 2), in which the weights ${\gamma }_{v,c}^{\left(t\right)}$ and ${\beta }_{c,v}^{\left(t\right)}$ are assigned to the edges of $T_{\mathcal{C}}$, and the weights ${\alpha }_v^{\left(t\right)}$ to the outputs [16]. The weights are obtained by minimizing a loss function evaluated over a training dataset using the standard optimizers for NNs.

Figure 2. Tanner graph unrolled to an RNN.

4.1. Parameter Sharing Schemes

The training complexity of WBP can be reduced through parameter sharing at the cost of performance loss. We consider dimensions $(t,v,c)$ for the ragged arrays ${\gamma }_{v,c}^{\left(t\right)}$ and ${\beta }_{c,v}^{\left(t\right)}$. In Type T parameter sharing over ${\gamma }_{v,c}^{\left(t\right)}$, parameters are shared with respect to iterations t. In Type $T_a$ scheme, ${\gamma }_{v,c}^{\left(t\right)}={\beta }_{c,v}^{\left(t\right)}={\gamma }_{v,c}$, $\forall (c,v)\in E$, $\forall t\in \left[T\right]$. In this case, there is a single ragged array with $\left|E\right|$ trainable parameters ${\left\{{\gamma }_{v,c}\right\}}_{c\in C_v,v\in V_c}$. For the regular LDPC code, $\left|E\right|=n{d}_v=m{d}_c$. It has been observed that for typical block lengths, indeed, the weights do not change significantly with iterations [28]. In Type $T_b$, there are T arrays ${\gamma }_{v,c}^{\left(t\right)}={\beta }_{c,v}^{\left(t\right)}$, while in Type $T_c$, there are two arrays ${\gamma }_{v,c}^{\left(t\right)}={\gamma }_{v,c}$ and ${\beta }_{c,v}^{\left(t\right)}={\beta }_{c,v}$. Type $T_a$ and $T_c$ decoders can be referred to as BP-RNN decoders and Type $T_b$ as feedforward BP. In Type V sharing, ${\gamma }_{v,c}^{\left(t\right)}={\gamma }_c^{\left(t\right)}$ is independent of v. This corresponds to one weight per check node. Likewise, in Type C sharing, there is one weight per check node update, and ${\gamma }_{v,c}^{\left(t\right)}={\gamma }_v^{\left(t\right)}$.

These schemes can be combined. For instance, in Type $T_{a}VC$ parameter sharing, ${\beta }_{c,v}^{\left(t\right)}={\gamma }_{v,c}^{\left(t\right)}=\gamma$. Thus, a single parameter $\gamma$ is introduced in all layers of the NN. This decoder is referred to as the neural normalized BP, e.g., neural normalized min-sum (NNMS) decoder when BP is based on the MS algorithm. The latter is similar to the normalized MS decoder, except that the parameter $\gamma$ is empirically determined there. In the Type $T_{b}VC$ scheme, ${\beta }_{c,v}^{\left(t\right)}={\gamma }_{v,c}^{\left(t\right)}={\gamma }^{\left(t\right)}$. Here, there is one weight per iteration. In this paper, ${\alpha }_v=1$$\forall t\in T$.

4.2. WBP over ${\mathbb{F}}_{\mathit{q}}$

The construction and decoding of the PB QC-LDPC binary codes can be extended to codes over a finite field ${\mathbb{F}}_q$ [80,81]. Here, there are $q-1$ LLR messages sent from each node, defined in Equation (1) [81]. The update equations of the BP are similar to (9)–(10), and presented in [81] for the extended min-sum (EMS) and in [82] for the weighted EMS (WEMS) decoder.

The parameter sharing for the four-dimensional ragged array ${\left\{{\gamma }_{v,c,q^{\prime }}^{\left(t\right)}\right\}}_{t,v\in C_v,c\in V_c,q^{\prime }\in {\mathbb{F}}_q}$ is defined in Section 4.1. In the check-node update of the WEMS algorithm, it is possible to assign a distinct weight to each coefficient $q^{\prime }\in {\mathbb{F}}_q$ for every variable node. For instance, in the Type $T_{c}CQ$ scheme, ${\gamma }_{v,c,q^{\prime }}^{\left(t\right)}={\gamma }_v$ and ${\beta }_{c,v,q^{\prime }}^{\left(t\right)}={\beta }_v$, so there is one weight per variable and one per check node $\forall t\in T$. In the case of Type $T_{b}VCQ$, there is only one weight per variable, iteration, and coefficient. In this case, if BP is based on the non-binary EMS algorithm, the decoder is called the neural normalized EMS (NNEMS).

Remark 1. 

The EMS decoder has a truncation factor in $\{1,2,\cdots ,q\}$ that provides a trade-off between complexity and accuracy. In this paper, it is set to q to investigate the maximum performance.

5. Adaptive Learned Message Passing Algorithms

The weights of the static WBP are obtained by training the network offline using a dataset. A WBP where the weights are determined for each received word $\mathbf{y}$ is an adaptive WBP. The weights must therefore be found by online optimization. To manage the complexity, we consider a WMS decoder with Type $T_a$ parameter sharing. Thus, the decoder has one weight per T iterations, which must be determined for a received $\mathbf{y}$.

Let $\mathbf{c}\in {\mathbb{F}}_2^n$ be a codeword and $\mathbf{y}\in U$ be the corresponding received word, where $U={\mathbb{R}}^n$ for the AWGN channel and $U={\mathbb{C}}^{n_s}$ for the optical fiber channel. Let $\overline{\mathbf{y}}={\mathcal{D}}_{\mathit{\gamma }}\left(\mathbf{y}\right)$ be the word decoded by a Type $T_a$ WBP decoder with weight ${\gamma }^{\left(t\right)}\in {\mathbb{R}}^{+}$ in iteration $t\in \left[T\right]$, where $\mathit{\gamma }=({\gamma }^{\left(1\right)},{\gamma }^{\left(2\right)},\cdots ,{\gamma }^{\left(T\right)})$. In the adaptive decoder, we wish to find a function $g:U\mapsto \mathcal{X}$, $\mathcal{X}\subseteq {\mathbb{R}}_{+}^T$, $\mathit{\gamma }=g\left(\mathbf{y}\right)$ that minimizes the probability that ${\mathcal{D}}_{g\left(\mathbf{y}\right)}\left(\mathbf{y}\right)$ makes an error

$$\begin{array}{c}\hfill {\displaystyle \underset{g\in \mathcal{H}}{min}}\mathrm{Pr}\left(\overline{\mathbf{y}}\ne \mathbf{c}\right),\end{array}$$

(13)

where $\mathcal{H}$ is a functional class. The static decoder is a special case where $g(.)$ is a constant function. Two variants of this decoder are proposed, illustrated in Figure 3.

Figure 3. Adaptive decoders. (a) Parallel decoders; (b) the two-stage decoder. $\mathrm{WMS}(\mathit{\gamma },L(\mathbf{y}\left)\right)$ refers to ${\mathcal{D}}_{\mathit{\gamma }}\left(\mathbf{y}\right)$.

5.1. Parallel Decoders

Architecture: In parallel decoders, $g(.)$ is found through searching. Here, ${\gamma }^{\left(t\right)}$ takes value in a discrete set ${\mathcal{X}}_t=\left\{x_1^{\left(t\right)},x_2^{\left(t\right)},\cdots ,x_{K_t}^{\left(t\right)}\right\}$, $K_t\in \mathbb{N}$, and thus $\mathit{\gamma }\in \mathcal{X}:={\prod }_{t=1}^T{\mathcal{X}}_t=\left\{{\mathit{\gamma }}_1,\cdots ,{\mathit{\gamma }}_{\nu }\right\}$. The parallel decoders consist of $\nu$ independent decoders ${\overline{\mathbf{y}}}_i={\mathcal{D}}_{{\mathit{\gamma }}_i}\left(\mathbf{y}\right)$, $i\in \left[\nu \right]$, running concurrently. Since $\mathrm{Pr}\left({\overline{\mathbf{y}}}_i\ne \mathbf{c}\right)$ in (13) is generally intractable, a sub-optimal $g(.)$ is selected as follows. At the end of decoding by ${\mathcal{D}}_{{\mathit{\gamma }}_i}$, the syndrome ${\mathbf{s}}_i={\overline{\mathbf{y}}}_i{H}^{\mathrm{T}}$ is computed. Let

$$\begin{array}{c}\hfill i^{*}={argmin}_{i\in \left[\nu \right]}\left\{\left|\right|{\mathbf{s}}_i{\left|\right|}_H:{\mathbf{s}}_i={\overline{\mathbf{y}}}_i{H}^{\mathrm{T}}\right\},\end{array}$$

(14)

be the index of the decoder whose syndrome has the smallest Hamming weight. Then, $g\left(\mathbf{y}\right)={\mathit{\gamma }}_{i^{*}}$, and $\overline{\mathbf{y}}={\mathcal{D}}_{{\mathit{\gamma }}_{i^{*}}}\left(\mathbf{y}\right)$.

In practice, the search can be performed up to depth $T_1=5$ iterations. However, the BP decoder often has to run for more iterations. Thus, a WBP decoder with weights ${\mathit{\gamma }}_{i^{*}}$ can continue the output with $T_2$ iterations.

Remark 2. 

The decoder obtained via (14) is generally sub-optimal. Minimizing ${\parallel {\mathbf{s}}_i\parallel }_H$ does not necessarily minimize the number of errors. However, for random codes, the decoder obtained from (14) outperforms the static decoder.

Remark 3. 

If $D_{{\gamma }_1}$ and $D_{{\gamma }_2}$ yield the same number of errors, the decoder with the smaller weight vector is selected, which tends to output smaller LLRs.

Obtaining $x_k^{\left(t\right)}$ from the distribution of weights.: The values of $x_k^{\left(t\right)}$ can be determined by dividing a sub-interval in $[0,1]$ uniformly. The resulting parallel WMS decoder outperforms WMS; however, the performance can be improved by choosing $x_k^{\left(t\right)}$ based on the probability distribution of the weights.

The probability distribution of the channel noise induces a distribution on $\mathbf{y}$ and consequently on $\mathit{\gamma }=g\left(\mathbf{y}\right)$. Let ${\mathsf{\Gamma }}^{\left(t\right)}$ be a random variable representing ${\gamma }^{\left(t\right)}$. Denote the corresponding mean by ${\theta }^{\left(t\right)}$, standard deviation by ${\sigma }^{\left(t\right)}$, and the cumulative distribution function by $C_t(.)\stackrel{\Delta }{=}{C}_{{\mathsf{\Gamma }}^{\left(t\right)}}(.)$. For $ϵ>0$, set

$$\begin{array}{c}\hfill x_k^{\left(t\right)}=inf\left\{x:C_t\left(x\right)>{\displaystyle \frac{ϵ}{2}}+(k-1){\displaystyle \frac{1-ϵ}{K_t}}\right\}.\end{array}$$

(15)

The numbers $x_1^{\left(t\right)}<x_2^{\left(t\right)}<\cdots <x_{K_t}^{\left(t\right)}$ partition the real line into intervals such that $Pr\left({\mathsf{\Gamma }}^{\left(t\right)}\in [x_1^{\left(t\right)},x_{K_t}^{\left(t\right)}]\right)$$=1-ϵ$ and $Pr({\mathsf{\Gamma }}^{\left(t\right)}\in [x_k^{\left(t\right)},x_{k+1}^{\left(t\right)}]\left|{\mathsf{\Gamma }}^{\left(t\right)}\in [x_1^{\left(t\right)},x_{K_t}^{\left(t\right)}]\right)$$={\displaystyle \frac{1}{K_t}}$. In practice, ${\mathsf{\Gamma }}^{\left(t\right)}$ has a distribution close to Gaussian, in which case $x_k^{\left(t\right)}$ values are given by the explicit formulas in Lemma 1.

Lemma 1. 

Let ${\mathsf{\Gamma }}^{\left(t\right)}$ have a cumulative distribution function $C_t(.)$ that is continuous and strictly monotonic. For $k\in \left[K_t\right]$,

$$\begin{array}{c}\hfill x_k^{\left(t\right)}=C_t^{-1}\left({\displaystyle \frac{ϵ}{2}}+(k-1){\displaystyle \frac{1-ϵ}{K_t}}\right).\end{array}$$

If ${\mathsf{\Gamma }}^{\left(t\right)}$ has a Gaussian distribution with a mean ${\theta }^{\left(t\right)}$ and standard deviation ${\sigma }^{\left(t\right)}$, then

$$\begin{array}{c}\hfill x_k^{\left(t\right)}={\theta }^{\left(t\right)}+\sqrt{2}{\sigma }^{\left(t\right)}{erfc}^{-1}\left(2-ϵ-{\displaystyle \frac{2(k-1)(1-ϵ)}{K_t}}\right),\end{array}$$

(16)

where erfc is the complementary error function.

Proof. 

The proof is based on elementary calculus. □

Obtaining the distribution of weights: To apply (15) or (16), $C_t(.)$ is required. To this end, a static WBP (with no parameter sharing) is trained offline given a dataset ${\left\{({\mathbf{y}}^{\left(i\right)},{\mathbf{c}}^{\left(i\right)})\right\}}_i$. The empirical cumulative distribution of the weights in each iteration is computed as an approximation to $C_t(.)$. However, if the BER is low, it can be difficult to obtain a dataset that contains a sufficient number of examples corresponding to incorrectly decoded words required to obtain good generalization.

To address this issue, we apply active learning [23,83]. This approach is based on the fact that the training examples near the decision boundary of the optimal classifier determine the classifier the most. Hence, input examples are sampled from a probability distribution with a support near the decision boundary.

The following approach to active learning is considered. At epoch e in the training of the WBP, random codewords $\mathbf{c}$ and the corresponding outputs $\mathbf{y}$ are computed. The decoder from the epoch $e-1$ is applied to decode $\mathbf{y}$ to $\widehat{\mathbf{c}}=\mathrm{WBP}(\mathit{\gamma },L\left(\mathbf{y}\right))$. An acquisition function $A_f(\mathbf{c},\widehat{\mathbf{c}}):\mathcal{C}\times {\mathbb{F}}_2^n\mapsto {\mathbb{R}}^{+}$ evaluates whether the example pair $(\mathbf{y},\mathbf{c})$ should be retained. A candidate example is retained if $A_f(\mathbf{c},\widehat{\mathbf{c}})$ is in a given range.

The choice of the acquisition function depends on the specific problem being solved, the architecture of the NN, and the availability of the labeled data [83]. In the context of training the NN decoders for channel coding, the authors of [23] use distance parameters and reliability parameters. Inspired by [23], the authors of [84] define the acquisition function using importance sampling. In this paper, the acquisition function is the number of errors $A_f(\mathbf{c},\widehat{\mathbf{c}})={\parallel \mathbf{c}-\widehat{\mathbf{c}}\parallel }_H=d_H(\mathbf{c},\widehat{\mathbf{c}})$, where $d_H$ is the Hamming distance.

The dataset is incrementally generated and pruned as follows. At each epoch e, a subset $S_e={\left\{({\mathbf{y}}^{\left(i\right)},{\mathbf{c}}^{\left(i\right)})\right\}}_{i=1}^{{\mathfrak{b}}_1}$ of ${\mathfrak{b}}_1$ examples, filtered by the acquisition function, is selected. The entire dataset at epoch e is $S^e=\mathrm{Prune}\left({\cup }_{e^{\prime }=1}^e{S}_{e^{\prime }}\right)$ and has size ${\mathfrak{b}}_2>{\mathfrak{b}}_1$. The operator Prune removes the subsets $S_{e^{\prime }}$ introduced in old epochs $e^{\prime }\in [e-e_0]$ if $e>e_0\stackrel{\Delta }{=}{\mathfrak{b}}_2/{\mathfrak{b}}_1$ and otherwise leaves its input intact. At each epoch e, the loss function is averaged over a batch set of size ${\mathfrak{b}}_s$ obtained by randomly sampling from $S^e$.

Complexity of the parallel decoders: The computational complexity of the decoder is measured in real multiplications (RMs). For instance, the complexity of the WMS with T iterations, ${\alpha }_v=1$, without parameter sharing, or with Type $T_b$ parameter sharing, is $\mathrm{RM}=2T\left|E\right|$, where $\left|E\right|=n{d}_v=m{d}_c$ is the number of edges of the Tanner graph of the code. For the WMS decoder with Type $T_a$ or Type $T_{b}VC$ parameter sharing, $\mathrm{RM}=T\left(m+n\right)$. The latter arises from the fact that equal weights factor out of the ∑ and the min terms in BP and are applied once. Thus, the complexity of ν parallel WMS decoders with Type $T_a$ parameter sharing is $\mathrm{RM}=\nu T\left(m+n\right)$. If ${\alpha }_v\ne 1$, $nT$ is added to the above formulas. Finally, the complexity of Type $T_{a}VC$ decoder is RM $=n$ per single iteration. These expressions neglect the cost of the syndrome check.

5.2. Two-Stage Decoder

In a parallel decoder, the weights are restricted in a discrete set. The number of parallel decoders depends exponentially on the size of this set. The two-stage decoder predicts arbitrary non-negative weights, without the exponential complexity of the parallel decoders. Further, since the weights are arbitrary, the two-stage decoder can improve upon the performance of the parallel decoders, when the output LLRs are sensitive to the weights.

Architecture: Recall that we wish to find a function $g\left(\mathbf{y}\right)$ that minimizes the BER in (13). In a two-stage decoder, this function is expressed by an NN $\overline{\mathit{\gamma }}=g_{\mathit{\theta }}\left(\mathbf{y}\right)$ parameterized by vector θ. Thus, the two-stage decoder is a combination of an NN and a WBP. First, the NN takes as input either the LLRs at the channel output $\mathrm{L}\left(\mathbf{y}\right)$ or $\left(\mathrm{L}\right(\mathbf{y}),\mathbf{y})$ and outputs the vector of weights $\overline{\mathit{\gamma }}$. Then, the WBP decoder takes the channel LLRs $\mathrm{L}\left(\mathbf{y}\right)$ and weights $\overline{\mathit{\gamma }}$ and outputs the decoded word $\overline{\mathbf{y}}$.

The parameters θ are found using a dataset of examples ${\left\{({\mathbf{y}}^{\left(i\right)},{\mathit{\gamma }}^{\left(i\right)})\right\}}_i$, where ${\mathit{\gamma }}^{\left(i\right)}$ is the target weight. This dataset can be obtained through a simulation, i.e., transmitting a codeword ${\mathbf{c}}^{\left(i\right)}$, receiving ${\mathbf{y}}^{\left(i\right)}$, and using, e.g., an offline parallel decoder to determine the target weight ${\mathit{\gamma }}^{\left(i\right)}$. In this manner, $g_{\mathit{\theta }}\left(\mathbf{y}\right)$ is expressed in a functional form instead of being determined by real-time search, which may be more expensive.

In this paper, the NN is a CNN consisting of a cascade of two one-dimensional convolutional layers ${\mathbf{Conv}}_1$ and ${\mathbf{Conv}}_2$, followed by a dense layer $\mathbf{Dens}$. ${\mathbf{Conv}}_i$ applies $F_i$ filters of size $S_i$ and stride 1, and the rectified linear unit (ReLU) activation, $i=1,2$. The output of ${\mathbf{Conv}}_2$ is flattened and passed to $\mathbf{Dense}$, which produces the vector of weights $\overline{\mathit{\gamma }}$ of length T. This final layer is a linear transformation with ReLU activation to produce non-negative weights.

The model is trained by minimizing the quantile loss function

$$\begin{array}{c}{\mathcal{l}}_{\xi }(\mathit{\gamma },\overline{\mathit{\gamma }})=\mathrm{mean}\left(max\left(\xi (\mathit{\gamma }-\overline{\mathit{\gamma }}),(\xi -1)(\mathit{\gamma }-\overline{\mathit{\gamma }})\right)\right),\end{array}$$

where $\xi \in (0,1)$ is the quantile parameter and max is applied per entry and mean over vector entries. The choice of loss is obtained by cross-validating the validation error over a number of candidate functions. This is an asymmetric absolute-like loss, which, if $\xi \ge 1/2$ as in Section 6, encourages entries of $\overline{\mathit{\gamma }}$ to be close to entries of γ from above rather than below.

Complexity of the two-stage decoder: The computational complexity of the two-stage decoder in the inference mode is the sum of the complexity of the CNN and WMS decoder

$$\mathrm{RM}=T(m+n)+\mathrm{RM}\left(\mathrm{CNN}\right),$$

where the computational complexity of the CNN is

$$\begin{array}{cc}\hfill \mathrm{RM}\left(\mathrm{CNN}\right)& =\mathrm{RM}\left({\mathbf{Conv}}_1\right)+\mathrm{RM}\left({\mathbf{Conv}}_2\right)+\mathrm{RM}\left(\mathbf{Dense}\right)\hfill \\ \hfill & =F_1(n-S_1+1)S_1+F_1{F}_2(n-S_1-S_2+2)S_2+F_2(n-S_1-S_2+2).\hfill \end{array}$$

(17)

The complexity can be significantly reduced by pruning the weights, for example, by setting to zero the weights below a threshold ${\tau }_{\mathrm{prun}}$.

Remark 4. 

Neural decoders are sensitive to distribution shifts and often require retraining when the input distribution or channel conditions change. To lower the training complexity, [18] proposed a decoder that learns a mapping from the input SNR to the weights in WBP, enabling the decoder to operate across a range of SNRs. However, the WBP decoder in [18] is static, since the weights remain fixed throughout the transmission once chosen, despite being referred to as dynamic WBP. We do not address the problem of distribution shift in this paper.

6. Performance and Complexity Comparison

In this section, we study the performance and complexity trade-off of the static and adaptive decoders for the AWGN in Section 3.1 and optical fiber channel in Section 3.2.

6.1. AWGN Channel

Low-rate regime: To investigate the error correction performance of the decoders at low rates, we consider a BCH code ${\mathcal{C}}_1(63,36)$ of rate $0.57$ with the cycle-reduced parity check matrix ${\mathrm{H}}_{\mathrm{cr}}$ in [85] and two QC-LDPC codes ${\mathcal{C}}_2(3224,1612)$ and ${\mathcal{C}}_3(4016,2761)$, which are, respectively, $(4,8)$- and $(5,16)$-regular with rates of $0.5$ and $0.69$. The parity check matrix of each QC-LDPC code is constructed using an exponent matrix obtained from the random progressive edge growth (PEG) algorithm [86], with a girth-search depth of two, which is subsequently refined manually to remove the short cycles in their Tanner graphs. The parameters of the QC-LDPC codes, including the exponent matrices $P_2$ and $P_3$, are given in Appendix C. In addition, we consider the irregular LDPC codes ${\mathcal{C}}_4(420,180)$ specified in the 5G New Radio (NR) standard and ${\mathcal{C}}_5(128,64)$ in the Consultative Committee for Space Data Systems (CCSDS) standard [85]. The code parameters, such as exponent matrices, are also available in the public repository ([87] v0.1).

We compare our adaptive decoders with tanh-based BP, the auto-regressive BP and several static WMS decoder with different levels of parameter sharing, such as BP with SS-PAN [18]. The latter is a Type $T_{a}VC$ WBP with ${\alpha }_v=\alpha$, i.e., a BP with two parameters. Additionally, to assess the achievable performance with a large number of parameters in the decoder, we include a comparison with two model-agnostic neural decoders based on the transformer [41] and graph NNs [33,43].

The number of iterations in the WMS decoders of the parallel decoders is chosen so that the total computational complexities of the parallel decoders and the static WMS decoder are about the same. In Figure 4a, this value is $T=5$ for ${\mathcal{C}}_1$, where $T=T_1+T_2$ is the total number of iterations; in Figure 4b–d, $T=4$ for ${\mathcal{C}}_2$ and ${\mathcal{C}}_3$, $T=6$ for ${\mathcal{C}}_4$, and $T=10$ for ${\mathcal{C}}_5$; in Figure 4f, $T=10$ for ${\mathcal{C}}_1$. Furthermore, $K_t=4$ for all $t\in \left[T\right]$.

Figure 4. BER versus SNR ρ, for the AWGN channel in the low-rate regime. (a) BCH code ${\mathcal{C}}_1(63,36)$. Here, the curve for WMS, Type $T_a$ & $T_b$ is from ([16] Figure 8) and the curve for WMS SS-PAN is from ([18] Figure 5a). (b) QC-LDPC code ${\mathcal{C}}_2(3224,1612)$, (c) QC-LDPC code ${\mathcal{C}}_3(4016,2761)$, (d) 5G-NR LDPC code ${\mathcal{C}}_4(420,180)$. Here, the curve for Graph NN is from ([33] Figure 5). (e) CCSDS LDPC code ${\mathcal{C}}_5(128,64)$. In this and the next sub-figure, the Autoregressive BP and Transformers curves are from [30] and [34], respectively. (f) BCH code ${\mathcal{C}}_1(63,36)$. Figures (d–f) show that adaptive decoders achieve the performance of the static decoders with less complexity.

To compute the value of weights $x_k^{\left(t\right)}$, the probability distribution of ${\mathsf{\Gamma }}^{\left(t\right)}$ is required. For this purpose, a WMS decoder is trained offline. The training dataset is a collection of examples obtained using the AWGN channel with a range of SNRs $\rho \in \left\{5.8,6.0,6.2\right\}$ dB for ${\mathcal{C}}_1$ or $\rho \in \left\{3.8,3.9,4.0,4.1,4.2\right\}$ dB for ${\mathcal{C}}_2$ and ${\mathcal{C}}_3$. The datasets for ${\mathcal{C}}_4$ and ${\mathcal{C}}_5$ are obtained similarly, with different sets of SNRs. The acquisition function $A_f(.,.)$ in active learning is the Hamming distance. A candidate example for the training dataset is retained if $A_f\le 10$. The parameters of the active learning are $b_1=b_s=2000$, and $b_2=$ 40,000. The loss function is the binary cross-entropy. The models are trained using the Adam optimizer with a learning rate of $0.0005$. It is observed that the distribution of ${\mathsf{\Gamma }}^{\left(t\right)}$ is nearly Gaussian. Thus, we obtain $x_k^{\left(t\right)}$ from (16). Table 3 presents the mean and variance of this distribution, and $x_k^{\left(t\right)}$, for the three codes considered.

Table 3. The mean and variance $({\theta }^{\left(t\right)},{\sigma }^{\left(t\right)})$ of $(x_1^{\left(t\right)},\cdots ,x_4^{\left(t\right)})$ in WMS for the AWGN channel.

Code	${\mathcal{C}}_1$	${\mathcal{C}}_2$	${\mathcal{C}}_3$
$t=1$	$\left(0.99,0.019\right)$ $\left(0.96,0.98,0.99,1.02\right)$	$\left(0.90,0.026\right)$ $\left(0.86,0.89,0.90,0.94\right)$	$\left(0.91,0.023\right)$ $\left(0.87,0.90,0.92,0.95\right)$
$t=2$	$\left(0.97,0.036\right)$ $\left(0.91,0.96,0.98,1.02\right)$	$\left(0.84,0.029\right)$ $\left(0.79,0.83,0.85,0.89\right)$	$\left(0.86,0.030\right)$ $\left(0.81,0.85,0.87,0.90\right)$
$t=3$	$\left(0.91,0.049\right)$ $\left(0.83,0.89,0.92,0.99\right)$	$\left(0.73,0.032\right)$ $\left(0.68,0.72,0.74,0.78\right)$	$\left(0.75,0.031\right)$ $\left(0.69,0.74,0.76,0.80\right)$
$t=4$	$\left(0.70,0.086\right)$ $\left(0.56,0.67,0.73,0.84\right)$	$\left(0.63,0.036\right)$ $\left(0.57,0.62,0.64,0.69\right)$	$\left(0.63,0.034\right)$ $\left(0.57,0.61,0.64,0.68\right)$
$t=5$	$\left(0.40,0.175\right)$ $\left(0.12,0.34,0.46,0.68\right)$	–	–

For the two-stage decoder, we use a CNN with $F_1=5$, $S_1=3$, $F_2=8$, and $S_2=2$, determined by cross-validation. The CNN is trained with a dataset of size 80,000, batch set size 300, and the quantile loss function with $\xi =0.75$. The number of iterations of the WMS decoder for each code is the same as above.

Figure 4 illustrates the BER vs. SNR for different codes, and different decoders for the same code. In each of Figure 4a–d, one can compare different decoders at about the same complexity (except for the parallel decoder with the largest ν that shows the smallest achievable BER). For instance, it can be seen in Figure 4a that the two-stage decoder achieves half the BER of the WMS with SS-PAN decoder at SNR $6.4$ dB for the short length code ${\mathcal{C}}_1$, or approximately $0.32$ dB gain in SNR at a BER of ${10}^{-4}$. For this code, the parallel WMS decoders with 3 iterations and $\nu =9$ outperforms the tanh-based BP with nearly the same complexity. Figure 4b,c show that the two-stage decoder offers about an order-of-magnitude improvement in the BER compared to the Type $T_a$ WMS decoder at $4.2$ dB for moderate-length codes ${\mathcal{C}}_2$ and ${\mathcal{C}}_3$, or over $0.1$ dB gain at a BER of ${10}^{-6}$. The performance gains vary with the code, parameters, and SNR.

Figure 4d–f compare decoders with different complexities at about the same performance. The proposed adaptive model-based decoders achieve the same performance of the model-agnostic static decoders, with far fewer parameters.

The computational complexity of the decoders are presented in Table 4. For the CNN, from (17), $\mathrm{RM}=n(8T+95)-24T-310$, which is further reduced by a factor of 4 upon pruning at the threshold ${\tau }_{\mathrm{prun}}=0.001$, with minimal impact on BER. Thus, the two-stage decoder requires less than half of the RM of the WMS decoder with no or Type $T_a$ parameter sharing. Moreover, the two-stage decoder requires approximately one-fifth of the RM of the parallel decoders with $\nu =16$. Compared to the WMS decoder with SS-PAN [18], the two-stage decoder has nearly double the complexity, albeit with much lower BER, as seen in Figure 4a.

Table 4. Computational complexity of decoders, for the AWGN channel.

	${\mathit{\gamma }}^{*\left(\mathit{t}\right)}$	${\mathit{\alpha }}^{*\left(\mathit{t}\right)}$	Average RM per Iteration
			${\mathcal{C}}_{\mathbf{1}}$	${\mathcal{C}}_{\mathbf{2}}$	${\mathcal{C}}_{\mathbf{3}}$
No weight sharing
WMS [16]	${\gamma }_{v,c}^{\left(t\right)}$	1	768	25,792	40,160
Weight sharing
WMS, Type $T_a$	${\gamma }_{v,c}$	1	768	25,792	40,160
WMS, Type $T_{a}VC$	$\gamma$	1	63	3226	4016
Parallel WMS Type $T_{b}VC$, $\nu =16$	${\gamma }^{\left(t\right)}$	1	1440	77,376	84,336
Parallel WMS Type $T_{b}VC$, $\nu =64$	${\gamma }^{\left(t\right)}$	1	–	≃$3.09\times {10}^5$	≃$3.37\times {10}^5$
Parallel WMS Type $T_{b}VC$, $\nu =1024$	${\gamma }^{\left(t\right)}$	1	92,340	–	–
Two-stage decoder Type $T_{b}VC$, ${\tau }_{\mathrm{prun}}=0.001$	${\gamma }^{\left(t\right)}$	1	≃300	≃14,093	≃17,558
WMS SS−PAN, Type $T_{a}VC$ [18]	${\gamma }^{\left(t\right)}$	1	153	8060	9287

High-rate regime: To further investigate the error correction performance of the decoders at high rates, we consider three single-edge QC-LDPC codes, ${\mathcal{C}}_6(1050,875)$, ${\mathcal{C}}_7(1050,850)$, and ${\mathcal{C}}_8(4260,3834)$, associated, respectively, with the PCMs ${\mathrm{H}}_6(\lambda =7,\omega =42,M=25,{\mathrm{P}}_6)$, ${\mathrm{H}}_7(8,42,25,{\mathrm{P}}_7)$, and ${\mathrm{H}}_8(6,60,71,{\mathrm{P}}_8)$. These codes have rates $r=0.84$, $0.81$, and $0.9$, respectively, and are constructed using the PEG algorithm. The PEG algorithm requires the degree distributions of the Tanner graph, which are optimized using the stochastic extrinsic information transfer (EXIT) chart described in Appendix C. Additionally, we include the polar code ${\mathcal{C}}_9(1024,854)$ with $r=0.84$ from the 5G-NR standard as a state-of-the-art benchmark. The code parameters, including degree distribution polynomials and the exponent matrices, are given in Appendix C.

The acquisition function with active learning in the parallel decoders is based on Figure 5. The figure shows the scatter plot of the pre-FEC error ${\mathfrak{e}}_1=\parallel \mathbf{c}-\overline{\mathbf{y}}\parallel$ versus post-FEC error ${\mathfrak{e}}_2={\parallel \mathbf{c}-\widehat{\mathbf{c}}\parallel }_H$, for 340 examples $(\mathbf{c},\mathbf{y})$ for ${\mathcal{C}}_6$ at $E_b/N_0=4.25$ dB. Here, $\overline{\mathbf{y}}$ is the hard decision of the LLRs at the channel output defined in (11), and $\widehat{\mathbf{c}}$ is decoded with the best decoder at epoch e, i.e., the WMS with weights from epoch $e-1$. The acquisition function retains $(\mathbf{c},\mathbf{y})$ if ${\mathfrak{e}}_1=0$ (no error) or if $({\mathfrak{e}}_1,{\mathfrak{e}}_2)$ falls in the rectangle in Figure 5 (with error). The rectangle is defined such that $Pr(({\mathfrak{e}}_1,{\mathfrak{e}}_2)\in \mathcal{S})\ge 0.95$. It is ensured that 70% of examples satisfy ${\mathfrak{e}}_1=0$ and 30% with $({\mathfrak{e}}_1,{\mathfrak{e}}_2)$ in the rectangle $\mathcal{S}$. In this example, ${\mathfrak{e}}_1\in \{80,81,\cdots ,100\}$, and ${\mathfrak{e}}_2\in [\mu -2\sigma ,\mu +2\sigma ]$, $\mu =149.97$, $\sigma =12.2$. We use ${\mathfrak{b}}_1=2000$, ${\mathfrak{b}}_2=$ 20,000, ${\mathfrak{b}}_s=500$ and the learning rate $0.001$.

Figure 5. The scatter plot of $({\mathfrak{e}}_1,{\mathfrak{e}}_2)$ for ${\mathcal{C}}_9$ at $E_b/N_0=4.25$ dB, for the AWGN channel. The scaled Gaussian approximation curve is fitted per axis.

For the adaptive decoder, we consider five parallel decoders with $T_1=4$ iterations. The decoder for the binary codes ${\mathcal{C}}_6$, ${\mathcal{C}}_7$ and ${\mathcal{C}}_8$ is WMS with Type $T_{a}VC$ sharing. The output of the decoder with the smallest syndrome is continued with an MS decoder with $T_2=4$ iterations. The polar code, however, is decoded with either a cyclic redundancy check (CRC) and successive cancellation list (SCL) with list size L [88] or the optimized successive cancellation (OSC) [89].

Figure 6 shows the performance of the adaptive and static MS decoders for ${\mathcal{C}}_6$, ${\mathcal{C}}_7$ and ${\mathcal{C}}_9$. The polar code ${\mathcal{C}}_9$ with 24 CRC bits is simulated using AFF3CT software toolbox ([90] v3.0.2). It can be seen that at high SNRs, $E_b/N_0\ge 4.6$, ${\mathcal{C}}_6$ and ${\mathcal{C}}_7$ decoded with adaptive parallel decoders outperform ${\mathcal{C}}_9$. Given this, and the higher complexity of decoding the polar code with either SCL or OSC [88], the choice of QC-LDPC codes for the inner code for the optical fiber channel in Section 6.2 is justified.

Figure 6. Performance of the polar code ${\mathcal{C}}_9\left(1024,854\right)$ versus QC-LDPC codes ${\mathcal{C}}_6\left(1050,875\right)$ and ${\mathcal{C}}_7\left(1050,850\right)$, for the AWGN channel in the high-rate regime. The curve for OSC decoder is from [89].

Figure 7 shows the performance of ${\mathcal{C}}_8(4260,3834)$ with rate $0.9$. The adaptive WMS decoder with $T_1+T_2=8$ iterations outperforms the static MS decoder with $T=8$ iterations at $E_b/N_0=5$ by an order of magnitude in BER.

Figure 7. Performance of the static and adaptive MS decoder for ${\mathcal{C}}_8\left(4260,3834\right)$ at $E_b/N_0=4$ dB, for the AWGN channel in the high-rate regime.

The gains of WBP depend on parameters such as the block length or SNR [91] (Section IV. d [44]). In general, the gain is decreased when the block length is increased, with other parameters remaining fixed.

6.2. Optical Fiber Channel

We simulate a 16-QAM WDM optical communication system described in Section 3.2, with parameters described in Table 2. The continuous-time model (5) is simulated with the split-step Fourier method with a spatial step size of 100 m and a simulation bandwidth of 200 GHz. DBP with two samples/symbol is applied to compensate for the physical effects and to obtain the per-symbol channel law $Pr\left(y\right|s)$, $s\in \mathcal{A}$, $y\in \mathbb{C}$. For the inner code in the concatenated code, we consider two QC-LDPC codes of rate $r_i=0.92$: binary single-edge code ${\mathcal{C}}_{10}(4000,3680)$ and non-binary multi-edge code ${\mathcal{C}}_{11}(800,32)$ over ${\mathbb{F}}_{32}$, respectively, with PCMs ${\mathrm{H}}_{10}(4,50,80,{\mathrm{P}}_{10})$ and ${\mathrm{H}}_{11}(2,25,$$32,$${\mathrm{P}}_{11})$, given in Appendix C. For the component code used in the outer spatially coupled code, we consider multi-edge QC-LDPC code ${\mathcal{C}}_{12}(3680,3520)$ with the PCM ${\mathrm{H}}_{12}(1,23,160,{\mathrm{P}}_{12})$. For $m_s=2$ and $L=100$, the resulting SC-QC-LDPC code has the PCM ${\mathrm{H}}_{\mathrm{SC}}(1,23,160,{\mathrm{P}}_{12},2,100,{\overline{\mathrm{B}}}_{12})$, where ${\overline{\mathrm{B}}}_{12}$ is the spreading matrix. The outer SC-QC-LDPC code is encoded with the sequential encoder [92]. This requires that the top-left $\lambda M\times \omega M$ block ${\mathrm{H}}_0\left(0\right)$ of ${\mathrm{H}}_{\mathrm{SC}}$ in Equation (A2) is of full rank. Thus, ${\overline{\mathrm{B}}}_{12}$ is designed to fulfill this condition. In Equation (A2), we have ${\mathrm{H}}_t(\lambda =1,\omega =23,M=160)\stackrel{\Delta }{=}\mathrm{H}\in {\mathbb{F}}_2^{160\times 3680}$, $t=0,1,2$, ${\mathrm{H}}_0$ is of full rank, and ${\mathrm{H}}_{\mathrm{SC}}\in {\mathbb{F}}_2^{16320\times 368000}$. The rate of the component code is $r_{\mathrm{QC}}=1-1/23\approx 0.956$, and the rate of outer SC code is $r_{\mathrm{o}}$$=r_{\mathrm{QC}}-{\displaystyle \frac{m_s\lambda }{L\omega }}$$\approx 0.955$, so $r_{\mathrm{total}}=r_{\mathrm{i}}·r_{\mathrm{o}}\approx 0.88$. The ${\mathrm{P}}_{12}$ and ${\overline{\mathrm{B}}}_{12}$ matrices are constructed heuristically and are given in Appendix C.

The inner code is decoded with the parallel decoder, with five decoders with four iterations each. The decoder for the binary code ${\mathcal{C}}_9$ is WMS with Type $T_{a}VC$ parameter sharing, while for the non-binary code, ${\mathcal{C}}_{11}$ is WEMS with Type $T_{a}VCQ$ sharing and ${\beta }_{c,v,q^{\prime }}^{\left(t\right)}=1$. The static EMS algorithm [93] is parameterized as in Section 4, initialized with the LLRs computed from Equation (1) [81]. The outer code is decoded with the SWD, with the static MS decoding of a maximum of 26 iterations per window.

Table 5 and Table 6 contain a summary of the numerical results. ${\mathrm{BER}}_i$ is pre-FEC BER, and the reference BER for the coding gain is ${\mathrm{BER}}_o={10}^{-12}$. $\mathcal{P}=-10$ dBm, the total gap to NCG_f for the adaptive weighted min-sum AWSM (resp., WEMS) decoder is 2.51 (resp., 1.75), while this value is 3.31 (resp., 2.29) and 3.44 (resp., 2.69), respectively, for the NNMS (resp., NNEMS) and MS (resp., EMS) decoders. Thus, the adaptive WBP provides a coding gain of 0.8 dB compared to the static NNMS decoder with about the same computational complexity and decoding latency

Table 5. Concatenated inner binary QC-LDPC code ${\mathcal{C}}_{10}$ and outer SC-QC-LDPC code ${\mathcal{C}}_{12}$ with an $r_{\mathrm{total}}=0.88$ for the optical fiber channel. The sections for ${\mathrm{BER}}_{\mathrm{i}}$ 0.012 and 0.025 correspond to average powers −10 and −11 dBm, respectively. NCGs are in dB.

${\mathrm{BER}}_{\mathrm{i}}$	Inner-SD Decoder	${\mathbf{BER}}_{\mathbf{o}}$ Inner	${\mathbf{BER}}_{\mathbf{o}}$ Total	NCG Inner	NCG Total	${\mathbf{NCG}}_{\mathit{f}}$ Inner	${\mathbf{NCG}}_{\mathit{f}}$ Total	$\mathbf{Gap}\mathbf{to}{\mathbf{NCG}}_{\mathit{f}}$ Inner	$\mathbf{Gap}\mathbf{to}{\mathbf{NCG}}_{\mathit{f}}$ Total
$0.012$	AWMS	$3.29\times {10}^{-6}$	$4.52\times {10}^{-8}$	$5.64$	$6.93$	$9.38$	$9.44$	$3.74$	$2.51$
	NNMS $\theta =0.75$	$4.02\times {10}^{-6}$	$5.43\times {10}^{-7}$	$5.56$	$6.13$	$9.38$	$9.44$	$3.82$	$3.31$
	MS	$4.77\times {10}^{-6}$	$7.75\times {10}^{-7}$	$5.49$	$6.00$	$9.38$	$9.44$	$3.89$	$3.44$
$0.025$	AWMS	$0.019$	$0.017$	$0.13$	$0.12$	$10.20$	$10.28$	$10.07$	$10.16$
	NNMS $\theta =0.72$	$0.02$	$0.018$	$0.04$	$0.03$	$10.20$	$10.28$	$10.16$	$10.25$
	MS	$0.023$	$0.02$	$-0.20$	$-0.15$	$10.20$	$10.28$	$10.40$	$10.43$

Table 6. Concatenated inner non-binary QC-LDPC code ${\mathcal{C}}_{11}$ and outer SC-QC-LDPC code ${\mathcal{C}}_{12}$ with $r_{\mathrm{total}}=0.88$ for the optical fiber channel. The sections for ${\mathrm{BER}}_{\mathrm{i}}$ 0.012 and 0.025 correspond to average powers −10 and −11 dBm, respectively. NCGs are in dB.

${\mathrm{BER}}_i$	Inner-SD Decoder	${\mathrm{BER}}_{\mathrm{o}}$ Inner	${\mathbf{BER}}_{\mathrm{o}}$ Total	NCG Inner	NCG Total	${\mathbf{NCG}}_{\mathit{f}}$ Inner	${\mathbf{NCG}}_{\mathit{f}}$ Total	$\mathbf{Gap}\mathbf{to}{\mathbf{NCG}}_{\mathit{f}}$ Inner	$\mathbf{Gap}\mathbf{to}{\mathbf{NCG}}_{\mathit{f}}$ Total
$0.012$	AWEMS	$3.21\times {10}^{-8}$	$2.74\times {10}^{-9}$	$7.23$	$7.69$	$9.38$	$9.44$	$2.15$	$1.75$
	NNEMS $\theta =0.2$	$4.61\times {10}^{-8}$	$2.11\times {10}^{-8}$	$7.12$	$7.15$	$9.38$	$9.44$	$2.26$	$2.29$
	EMS	$2.44\times {10}^{-7}$	$8.20\times {10}^{-8}$	$6.60$	$6.75$	$9.38$	2$9.44$	$2.78$	$2.69$
$0.025$	AWEMS	$0.0063$	$0.0051$	$1.73$	$1.80$	$10.20$	$10.28$	$8.47$	$8.48$
	NNEMS $\theta =0.25$	$0.0087$	$0.0075$	$1.32$	$1.32$	$10.20$	$10.28$	$8.88$	$8.96$
	EMS	$0.025$	$0.022$	$-0.36$	$-0.32$	$10.20$	$10.28$	$10.56$	$10.60$

7. Conclusions

Adaptive decoders are proposed for codes on graphs that can be decoded with message-passing algorithms. Two variants, the parallel WBP and the two-stage decoder, are studied. The parallel decoders search for the best sequence of weights in real time using multiple instances of the WBP decoder running concurrently, while the two-stage neural decoder employs an NN to dynamically determine the weights of WBP for each received word. The performance and complexity of the adaptive and several static decoders are compared for a number of codes over an AWGN and optical fiber channel. The simulations show that significant improvements in BER can be obtained using adaptive decoders, depending on the channel, SNR, the code and its parameters. Future work could explore further reducing the computational complexity of the online learning, and applying adaptive decoders to other types of codes and wireless channels.

Appendix A.1. Construction for the Single-Edge Case

A single-edge PB QC-LDPC code $\mathcal{C}$ is constructed in two steps. First, a base matrix $\mathrm{B}\in {\mathbb{F}}_2^{\lambda \times \omega }$ is constructed, where $\lambda ,\omega \in \mathbb{N}$, $\lambda \le \omega$. Then, $\mathrm{B}$ is expanded to the PCM of $\mathcal{C}$ by replacing each zero in $\mathrm{B}$ with the all-zero matrix $\mathbf{0}\in {\mathbb{F}}_2^{M\times M}$, where $M\in \mathbb{N}$ is the lifting factor, and a one in row i and column j with a sparse circulant matrix ${\mathrm{H}}_{i,j}\in {\mathbb{F}}_2^{M\times M}$.

Let $\mathrm{P}\in {[-1:M-1]}^{\lambda \times \omega }$ be the exponent matrix of the code, with the entries denoted by $p_{i,j}$. The matrices ${\mathrm{H}}_{i,j}$ (and $\mathrm{B}$) can be obtained from $\mathrm{P}$ as follows. Denote by ${\mathrm{I}}^n\in {\mathbb{F}}_2^{M\times M}$, $n\in [-1:M-1]$ the circulant permutation matrix obtained by cyclic-shifting of each row of the $M\times M$ identity matrix n positions to the right, with the convention that ${\mathrm{I}}^{-1}$ is the all-zero matrix. Then, ${\mathrm{H}}_{i,j}={\mathrm{I}}^{p_{i,j}}$. The PCM $\mathrm{H}:=\mathrm{H}(\lambda ,\omega ,M,\mathrm{P})\in {\mathbb{F}}_2^{\omega M\times \lambda M}$ of this QC-LDPC code is

$$\begin{array}{c}\hfill \mathrm{H}=\left(\begin{array}{cccc}{\mathrm{I}}^{p_{1,1}}& {\mathrm{I}}^{p_{1,2}}& \dots & {\mathrm{I}}^{p_{1,\omega }}\\ {\mathrm{I}}^{p_{2,1}}& {\mathrm{I}}^{p_{2,2}}& \dots & {\mathrm{I}}^{p_{2,\omega }}\\ ⋮& ⋮& \ddots & ⋮\\ {\mathrm{I}}^{p_{\lambda ,1}}& {\mathrm{I}}^{p_{\lambda ,2}}& \dots & {\mathrm{I}}^{p_{\lambda ,\omega }}\end{array}\right).\end{array}$$

(A1)

This code has a length $n=\omega M$, $k=\omega M-r$, and a rate $r=1-r/\left(\omega M\right)$, where $r\le \lambda M$ is the rank of $\mathrm{H}$. If $\mathrm{H}$ is of a full rank, $r=1-\lambda /\omega$. The base matrix is also obtained from $\mathrm{P}$, as ${\mathrm{B}}_{i,j}=0$ if $p_{i,j}=-1$ and ${\mathrm{B}}_{i,j}=1$ if $p_{i,j}\ne -1$.

Denote the Tanner graph of $\mathcal{C}$ by $T_{\mathcal{C}}$, and let $d_c$ and $d_v$ be, respectively, the degree of the check node c and the variable node v in $T_{\mathcal{C}}$. If $\mathrm{B}$ is regular (i.e., its rows have the same Hamming weight), then $\mathrm{H}$ is regular, and the check and variable nodes of $\mathcal{C}$ have the same degrees $d_c$ and $d_v$, respectively. In this case, $\mathcal{C}$ is said to be $\left(d_c,d_v\right)$-regular. More generally, a variable node’s degree distribution polynomial can be defined as ${\rm Y}\left(x\right)={\sum }_d{{\rm Y}}_d{x}^{d-1}$, where ${{\rm Y}}_d$ is the fraction of variable nodes of degree d, and a check node degree distribution $\mathsf{\Lambda }\left(x\right)={\sum }_d{\mathsf{\Lambda }}_d{x}^{d-1}$, where ${\mathsf{\Lambda }}_d$ is the fraction of check nodes of degree d.

The parameter matrices $\mathrm{B}$ and $\mathrm{P}$ can be obtained so as to maximize the girth of $T_{\mathcal{C}}$ using search-based methods such as the PEG algorithm [86,94], algebraic methods ([69] Section 10), or a combination of them [95].

Example A1. 

Consider $\lambda =3$, $\omega =5$, and the base matrix $\mathrm{B}\in {\mathbb{F}}_2^{3\times 5}$

$$\begin{array}{ccc}\hfill \mathrm{B}& =\hfill & \hfill \left(\begin{array}{ccccc}0& 1& 1& 1& 0\\ 1& 1& 0& 1& 1\\ 1& 0& 1& 1& 1\end{array}\right).\end{array}$$

For any exponent matrix $\mathrm{P}$, $\mathrm{H}\in {\mathbb{F}}_2^{3M\times 5M}$ is

$$\begin{array}{ccc}\hfill \mathrm{H}& =\hfill & \hfill \left(\begin{array}{ccccc}{\mathrm{I}}^{-1}& {\mathrm{I}}^{p_{1,2}}& {\mathrm{I}}^{p_{1,3}}& {\mathrm{I}}^{p_{1,4}}& {\mathrm{I}}^{-1}\\ {\mathrm{I}}^{p_{2,1}}& {\mathrm{I}}^{p_{2,2}}& {\mathrm{I}}^{-1}& {\mathrm{I}}^{p_{2,4}}& {\mathrm{I}}^{p_{2,5}}\\ {\mathrm{I}}^{p_{3,1}}& {\mathrm{I}}^{-1}& {\mathrm{I}}^{p_{3,3}}& {\mathrm{I}}^{p_{3,4}}& {\mathrm{I}}^{p_{3,5}}\end{array}\right).\square \end{array}$$

Appendix A.2. Construction for the Multi-Edge Case

The above construction can be extended to multi-edge PB QC-LDPC codes. Here, ${\mathrm{B}}_{i,j}$, instead of a binary number, is a sequence of length $D_{i,j}$ with entries in ${\mathbb{F}}_2$. Likewise, $p_{i,j}$ is a sequence of length $D_{i,j}$ with entries $p_{i,j}^d\in [-1:M-1]$, $d\in \left[D_{i,j}\right]$. Then, ${\mathrm{H}}_{i,j}={\sum }_{d=1}^{D_{i,j}}{\mathrm{I}}^{p_{i,j}^d}$. This code is represented by a Tanner graph where there are multiple edges of different types between the variable and the check nodes. We say this code has type $D={max}_{i,j}{D}_{i,j}\ge 1$. A single-edge PB QC-LDPC code is type 1.

Example A2. 

Consider $\lambda =2$, $\omega =3$,

$$\begin{array}{cccccc}\hfill \mathrm{B}& =\hfill & \hfill \left(\begin{array}{ccc}0& (1,1)& 1\\ 1& 0& (1,1,1)\end{array}\right)\mathit{and}& \mathrm{P}\hfill & \hfill =& \left(\begin{array}{ccc}-1& \left(p_{1,2}^1,p_{1,2}^2\right)& p_{1,3}\\ p_{2,1}& -1& \left(p_{2,3}^1,p_{2,3}^2,p_{2,3}^3\right)\end{array}\right).\hfill \end{array}$$

Then

$$\begin{array}{ccc}\hfill \mathrm{H}& =\hfill & \hfill \left(\begin{array}{ccc}{\mathrm{I}}^{-1}& {\mathrm{I}}^{p_{1,2}^1}+{\mathrm{I}}^{p_{1,2}^2}& {\mathrm{I}}^{p_{1,3}}\\ {\mathrm{I}}^{p_{2,1}}& {\mathrm{I}}^{-1}& {\mathrm{I}}^{p_{2,3}^1}+{\mathrm{I}}^{p_{2,3}^2}+{\mathrm{I}}^{p_{2,3}^3}\end{array}\right).\square \end{array}$$

Appendix A.3. Construction for Non-Binary Codes

In the non-binary codes, the entries of codewords, parity-check, and generator matrices are in a finite field ${\mathbb{F}}_q$, where q is a power of a prime. There are several ways to construct non-binary PB QC-LDPC codes. The base matrix $\mathrm{B}\in {\mathbb{F}}_2^{\lambda \times \omega }$ typically remains binary, as defined as in Appendix A.1. We use the unconstrained and random assignment strategy in [80] to extend $\mathrm{B}$ to a PCM.

There is significant flexibility in selecting the edge weights for constructing non-binary QC-LDPC codes, which can be classified as constrained or unconstrained ([80] Section II). In this work, we focus on an unconstrained and random assignment strategy, where each edge in $T_{\mathcal{C}}$ corresponding to a 1 in the binary matrix B is replaced with a coefficient $h_{i,j}\in {\mathbb{F}}_q$. Alternatively, these coefficients could be selected based on predefined rules to ensure appropriate edge-weight diversity, which can lead to enhanced performance. This methodology allows non-binary codes to retain the structural benefits of binary QC-LDPC codes while extending their functionality to finite fields, offering improved error-correcting capabilities for larger q values.

Appendix A.4. Encoder and Decoder

The generator matrix of the code is obtained by applying Gaussian elimination in the binary field to (A1). The encoder is then implemented efficiently using shift registers [96].

The QC-LDPC codes are typically decoded using belief propagation (BP), as described in Section 4.

Appendix B.1. Construction

The SC-QC-LDPC codes in this paper are constructed based on the edge spreading process [97]. Denote the PCM of the constituent PB QC-LDPC code by $\mathrm{H}(\lambda ,\omega ,M,\mathrm{P})$. Denote the PCM of the corresponding SC-QC-LDPC code by ${\mathrm{H}}_{\mathrm{SC}}:={\mathrm{H}}_{\mathrm{SC}}\left(\lambda ,\omega ,M,\mathrm{P},m_s,L,\overline{\mathrm{B}}\right)$, with the additional parameters of the syndrome memory $m_s\in \mathbb{N}$, coupling length $L\in \mathbb{N}$, and the spreading matrix $\overline{\mathrm{B}}\in {[-1:m_s]}^{\lambda \times \omega }$. Then, ${\mathrm{H}}_{\mathrm{SC}}\in {\mathbb{F}}_2^{\lambda M(m_s+L+1)\times \omega ML}$ is given by in which ${\mathrm{H}}_t\left(\mathcal{l}\right),\mathbf{0}\in {\mathbb{F}}_2^{\lambda M\times \omega M}$ are $\lambda \times \omega$ block matrices, $t\in [0:m_s]$, $\mathcal{l}\in [0,L-1]$ and ${\mathrm{H}}_t\left(\mathcal{l}\right)$ is obtained from $\overline{\mathrm{B}}$ as

$$\begin{array}{ccc}\hfill {\mathrm{H}}_t\left(\mathcal{l}\right)\mathrm{at}\mathrm{row}\text{-}\mathrm{block}i\mathrm{and}\mathrm{column}\text{-}\mathrm{block}j& =\hfill & \hfill \left\{\begin{array}{cc}{\mathrm{I}}^{p_{i,j}},\hfill & \mathrm{if}{\overline{\mathrm{B}}}_{i,j}=t,\hfill \\ \mathbf{0},\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(A2)

where ${\mathrm{I}}^{p_{i,j}},\mathbf{0}\in {\mathbb{F}}_2^{M\times M}$.

If ${\mathrm{H}}_t\left({\mathcal{l}}_1\right)={\mathrm{H}}_t\left({\mathcal{l}}_2\right)$, $\forall t\in [0,m_s]$ and $\forall {\mathcal{l}}_1,{\mathcal{l}}_2\in [0,L-1]$, the code is time-invariant. If $\mathrm{H}$ and ${\mathrm{H}}_{\mathrm{SC}}$ are full-rank, then the rate of SC-QC-LDPC code is $r=r_{\mathrm{QC}}-{\displaystyle \frac{m_s\lambda }{L\omega }}$, where $r_{\mathrm{QC}}=1-{\displaystyle \frac{\lambda }{\omega }}$ is the rate of the component QC-LDPC code. For a fixed $m_s$, as $L\to \infty$, then $r\to r_{\mathrm{QC}}$. Thus, the rate loss in SC-LDPC codes can be reduced by increasing the coupling length.

Example A3. 

Consider any QC-LDPC code, $m_s=2$, $L=4$ and the spreading matrix

$$\begin{array}{c}\hfill \overline{\mathrm{B}}=\left(\begin{array}{ccccc}-1& 1& 0& 2& -1\\ 1& -1& 1& 0& -1\\ 1& 2& -1& 0& 2\end{array}\right)\end{array}$$

Then, ${\mathrm{H}}_2\left(\mathcal{l}\right)$ is obtained by replacing each entry of $\overline{\mathrm{B}}$ that is 2 at row i and column j with ${\mathrm{I}}^{p_{i,j}}$, and other entries with $\mathbf{0}$. Thus, for all $\mathcal{l}\in [0:3]$

$$\begin{array}{ccc}\hfill {\mathrm{H}}_2\left(\mathcal{l}\right)& =\hfill & \hfill \left(\begin{array}{ccccc}\mathbf{0}& \mathbf{0}& \mathbf{0}& {\mathrm{I}}^{p_{1,4}}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& {\mathrm{I}}^{p_{3,2}}& \mathbf{0}& \mathbf{0}& {\mathrm{I}}^{p_{3,5}}\end{array}\right).\end{array}$$

In a similar manner,

$$\begin{array}{cccccc}\hfill {\mathrm{H}}_0\left(\mathcal{l}\right)& =\hfill & \hfill \left(\begin{array}{ccccc}\mathbf{0}& \mathbf{0}& {\mathrm{I}}^{p_{1,3}}& \mathbf{0}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& {\mathrm{I}}^{p_{2,4}}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& {\mathrm{I}}^{p_{3,4}}& \mathbf{0}\end{array}\right),& {\mathrm{H}}_1\left(\mathcal{l}\right)\hfill & \hfill =& \left(\begin{array}{ccccc}\mathbf{0}& {\mathrm{I}}^{p_{1,2}}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ {\mathrm{I}}^{p_{2,1}}& \mathbf{0}& {\mathrm{I}}^{p_{2,3}}& \mathbf{0}& \mathbf{0}\\ {\mathrm{I}}^{p_{3,1}}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}\end{array}\right).\hfill \end{array}$$

If ${\mathrm{H}}_{\mathrm{SC}}$ is full-rank, then $r=r_{\mathrm{QC}}-{\displaystyle \frac{2·3}{4·5}}$. If ${\mathrm{H}}_t\left(\mathcal{l}\right)$ is also full-rank, then $r_{\mathrm{QC}}=0.4$ and $r=0.1$. □

The SC-LDPC code is efficiently encoded sequentially [92] so that at each spatial position ℓ, $(\omega -\lambda )M$ information bits are encoded out of $(\omega -\lambda )ML$.

If an entry of $\mathrm{B}$ corresponding to an ${\mathrm{H}}_i\left(\mathcal{l}\right)$ is a sequence, the corresponding entry in $\overline{\mathrm{B}}$ is also a sequence. Thus, if ${\mathrm{H}}_i\left(\mathcal{l}\right)$ represents a multi-edge code for some i or ℓ, so does ${\mathrm{H}}_{\mathrm{SC}}$.

Figure A1. (a) A window in SWD is a $m_w\times m_w$ block, each block with size $\lambda M\times \omega M$. The window slides diagonally by one block. The variable nodes in the block at the block-position $i\times j$ are denoted by $\mathrm{A}\left(i\right)$, and the check nodes by $\mathrm{B}\left(j\right)$. Here, $m_s=2$ and $m_w=3$. (b) The message update for the current window ℓ is denoted by the solid rectangle. The variable nodes in the previous windows ${\mathcal{l}}^{\prime }<\mathcal{l}$ and not in the current window ℓ, denoted in green, send fixed messages ${\mathrm{L}}_{v2c}^{(T_{\mathcal{l}-1},\mathcal{l}-1)}$ to the check nodes in the current window in any iterations t. The blue variable nodes in the current and a previous window send ${\mathrm{L}}_{v2c}^{(T_{\mathcal{l}-1},\mathcal{l}-1)}$ to the check nodes in the current window at iteration $t=1$, which would be updated according to (9) and (10) for $t\ge 2$. The red variable nodes in the current window, but not in any previous window, send $\mathrm{L}\left(\mathbf{y}\right)$ to the check nodes in the current window at $t=1$, which would be updated at $t\ge 2$. The edges from the gray check nodes in windows ${\mathcal{l}}^{\prime }>\mathcal{l}$ are discarded for the decoding at position ℓ.

Appendix B.2. Encoder

The SC-LDPC codes are encoded with the sequential encoder [92].

Appendix B.3. Sliding Window Decoder

Consider the SC LDPC code ${\mathrm{H}}_{\mathrm{SC}}(\lambda ,\omega ,M,\mathrm{P},m_s,L,\overline{\mathrm{B}})$ in Appendix B.1. Note that any two variable-nodes in the Tanner graph of the code whose corresponding columns in ${\mathrm{H}}_{\mathrm{SC}}$ are at least $\left(m_s+1\right)M\omega$ columns apart do not share any common check-nodes. Thus, they are not involved in the same parity-check equation. The SWD uses this property and runs a local BP decoder on windows of ${\mathrm{H}}_{\mathrm{SC}}$ shown in Figure A1.

SWD works through a sequence of spatial iterations ℓ, where a rectangular window slides from the top-left to the bottom-right side of ${\mathrm{H}}_{\mathrm{SC}}$. In general, a window matrix ${\mathrm{H}}_w$ of size $m_w$ consists of $m_w\lambda M$ consecutive rows and $m_w\omega M$ consecutive columns in ${\mathrm{H}}_{\mathrm{SC}}$. At each iteration ℓ, it moves $\lambda M$ rows down and $\omega M$ columns to the right in ${\mathrm{H}}_{\mathrm{SC}}$. Thus, a window is an $m_w\times m_w$ block, starting from the top left and moving diagonally one block down per iteration. There is a special case where the window reaches the boundary. The way the windows near the boundary are terminated impacts performance [98,99]. Our setup for window termination at boundary is early termination, which is discussed in section III-B1 [99].

Denote the variable nodes in window ℓ by

$$V\left(\mathcal{l}\right)=\left\{v_i\in V:i=(\mathcal{l}-1)\omega M,\cdots ,(m_w+\mathcal{l}-1)\omega M-1\right\}.$$

The check-nodes directly connected to $V\left(\mathcal{l}\right)$ are $C\left(\mathcal{l}\right)$. Define $\tilde{V}\left(\mathcal{l}\right)={\cup }_{{\mathcal{l}}^{\prime }<\mathcal{l}}\left\{v\in V\left({\mathcal{l}}^{\prime }\right)\setminus V\left(\mathcal{l}\right),v\mathrm{connected}\mathrm{to}C\left(\mathcal{l}\right)\right\}$ (the variable nodes in the previous windows not in the current window, shown in green in Figure A1) $\overline{V}\left(\mathcal{l}\right)={\cup }_{{\mathcal{l}}^{\prime }<\mathcal{l}}\left\{v\in V\left({\mathcal{l}}^{\prime }\right)\cap V\left(\mathcal{l}\right),v\mathrm{connected}\mathrm{to}C\left(\mathcal{l}\right)\right\}$ (the variable nodes in the current window and any previous window, shown in blue in Figure A1), and $\widehat{V}\left(\mathcal{l}\right)={\cup }_{{\mathcal{l}}^{\prime }<\mathcal{l}}\{v\in V\left(\mathcal{l}\right)\setminus V\left({\mathcal{l}}^{\prime }\right),$$v\mathrm{connected}\mathrm{to}C\left(\mathcal{l}\right)\}$ (the variables nodes in the current window not in any previous window, shown in red in Figure A1). Let ${\mathrm{L}}_{v2c}^{(t,\mathcal{l})}$ and ${\mathrm{L}}_{c2v}^{(t,\mathcal{l})}$ be LLRs in window ℓ and iteration $t\in \left[T_{\mathcal{l}}\right]$ in BP. At $t=1$, the BP is initialized as

$$\begin{array}{c}\hfill {\mathrm{L}}^{(1,\mathcal{l})}\left(v\right)=\left\{\begin{array}{cc}{\mathrm{L}}^{(T_{\mathcal{l}-1},\mathcal{l}-1)}\left(v\right),\hfill & v\in \tilde{V}\left(\mathcal{l}\right)\cup \overline{V}\left(\mathcal{l}\right),\hfill \\ \mathrm{L}\left(\mathbf{y}\right),\hfill & v\in \widehat{V}\left(\mathcal{l}\right)\hfill \end{array}\right.\end{array}$$

(A3)

The update equation for the variable node is

$$\begin{array}{c}\hfill {\mathrm{L}}_{v2c}^{(t,\mathcal{l})}=\left\{\begin{array}{cc}{\mathrm{L}}_{v2c}^{(T_{\mathcal{l}-1},\mathcal{l}-1)},\hfill & v\in \tilde{V}\left(\mathcal{l}\right),\hfill \\ \mathrm{L}\left(\mathbf{y}\right)+{\displaystyle \sum _{c^{\prime }\in {\mathrm{C}}_v\setminus \left\{c\right\}}}{\mathrm{L}}_{c^{\prime }2v}^{(t-1),\mathcal{l}},\hfill & v\in \overline{V}\left(\mathcal{l}\right)\hfill \\ \mathrm{L}\left(\mathbf{y}\right),\hfill & v\in \widehat{V}\left(\mathcal{l}\right).\hfill \end{array}\right.\end{array}$$

The update relation for ${\mathrm{L}}_{c2v}^{(t,\mathcal{l})}$ is given by (9), with no weights, applied for $c\in C\left(\mathcal{l}\right)$ and $v\in V\left(\mathcal{l}\right)$. After $T_{\mathcal{l}}$ iterations, the variables in the window ℓ, called target symbols, are decoded. The SWD is illustrated in Figure A1.