CrossMPM: Cross-Selective Scan Message-Passing Mamba for Short-Length LDPC Codes

Abstract

Error-correcting codes (ECCs) are fundamental to reliable communication over noisy channels. Transformer-based neural decoders achieve strong ECC decoding performance, but attention operations cause computational complexity to grow substantially with code length. We propose Cross-Selective Scan Message-Passing Mamba (CrossMPM), a low-complexity Mamba-based neural decoder, to address this issue. CrossMPM replaces cross-attention-based information exchange between magnitude and syndrome vectors in Cross-Attention Message-Passing Transformer (CrossMPT) with Tanner-graph projection and Cross-Selective Scan State-Space Model (Cross-Selective Scan SSM). Tanner-graph projection aggregates connected node representations according to the parity-check matrix to incorporate code structure, while Cross-Selective Scan SSM dynamically generates selection parameters from the aggregated representations to control state updates and outputs. CrossMPM thus models local parity constraints and long-range dependencies without attention operations and reduces computational complexity. Simulation results show that CrossMPM achieves competitive decoding performance with lower computational complexity than CrossMPT and Hybrid Mamba-Transformer Decoder (ECCM), and this performance is maintained at longer code lengths. These results show that CrossMPM provides a favorable trade-off between error correction performance and computational efficiency.

1. Introduction

Error-correcting codes (ECCs) are a fundamental technology for ensuring reliable data transmission by detecting and correcting errors that occur over noisy communication channels. Iterative decoding algorithms such as belief propagation (BP) and min-sum (MS) have been widely used as conventional methods that directly exploit code structure for decoding [1,2]. Along with these algorithmic approaches, deep learning-based ECC decoders that directly learn decoding rules from data have been actively studied in recent years [3,4,5,6]. Neural decoders can be broadly classified into model-based approaches, which introduce trainable parameters into the operations of existing decoding algorithms [3,4,6,7,8,9], and model-free approaches, which learn the decoding process using general neural network architectures without relying on a specific decoding algorithm [5,10,11].

Model-free neural decoders may suffer from overfitting because the number of possible codewords increases exponentially with code length, resulting in degraded generalization performance for codewords unseen during training [10]. To alleviate this problem, Bennatan et al. introduced syndrome-based preprocessing, which uses the magnitude and syndrome of the received signal as inputs to the neural decoder [10]. The neural decoder predicts multiplicative noise from these two inputs, and this preprocessing allows a model trained only on the all-zero codeword to be applied to the decoding of other transmitted codewords.

Transformer-based ECC decoders using syndrome-based preprocessing have been developed and have shown effective decoding performance across various linear block codes. The Error Correction Code Transformer (ECCT) [12] introduced the Transformer architecture to ECC decoding by combining the magnitude and syndrome into a single sequence and applying a mask generated from the parity-check matrix to self-attention to incorporate the structural relationships of the code. Subsequently, the Cross-Attention Message-Passing Transformer (CrossMPT) [13] was proposed to maintain the magnitude and syndrome as separate representations and iteratively update them through masked cross-attention. However, the cross-attention in CrossMPT computes attention scores between the magnitude and syndrome, resulting in high computational and memory costs when processing longer codewords. This computational burden can limit the scalability of Transformer-based decoders to longer codes.

Several approaches have been proposed to alleviate the computational burden of Transformer-based decoders. Accelerating ECCT (AECCT) [14] reduces the computational and memory costs of ECCT through ternary weight quantization and head partitioning self-attention (HPSA). Efficient Message-Passing Transformer (EfficientMPT) [15] retains the iterative update structure for the magnitude and syndrome in CrossMPT while introducing Efficient Error-Correcting (EEC) attention, which replaces conventional attention matrix multiplication with element-wise operations using a global query.

Along with these approaches to reducing the computational complexity of attention, studies have also explored the use of Mamba [16], which is based on a selective scan state-space model (SSM). SSMs model long-range dependencies by accumulating and propagating sequence information through hidden states. Mamba dynamically generates input-dependent selection parameters to control how new information is incorporated into the hidden state and how existing information is retained. Through these sequential state updates, Mamba models long-range dependencies without computing pairwise attention scores and has linear computational complexity with respect to sequence length. To exploit these advantages of Mamba for ECC decoding, the Hybrid Mamba-Transformer Decoder (ECCM) [17] was proposed, employing a hybrid architecture composed of Mamba and Transformer blocks and applying masks generated from the parity-check matrix to the SSM and self-attention operations.

However, ECCM does not maintain the magnitude and syndrome as separate representations despite their distinct roles in decoding but instead combines them into a single sequence of length $2n-k$. Because Mamba blocks and Transformer blocks are alternately applied to this sequence, the increased sequence length resulting from combining the magnitude and syndrome further increases the computational burden of self-attention. Consequently, the efficient sequence modeling capability of Mamba does not directly translate into a reduction in overall computational cost, and in our experiments, ECCM exhibited higher floating-point operations (FLOPs) than CrossMPT despite its Mamba-based hybrid architecture. Taken together, these architectural and computational limitations highlight the need for a Mamba-based decoder architecture that enables effective structural interaction between the magnitude and syndrome without relying on Transformer blocks.

Therefore, in this paper, we propose Cross-Selective Scan Message-Passing Mamba (CrossMPM), a Mamba-based ECC decoder. CrossMPM maintains the magnitude and syndrome as separate representations of lengths n and $n-k$, respectively, and replaces the information exchange between the two representations performed by the cross-attention in CrossMPT with Tanner-graph projection and Cross-Selective Scan State-Space Model (Cross-Selective Scan SSM). Specifically, Tanner-graph projection aggregates the representations of connected nodes according to the connections between variable nodes (VNs) and check nodes (CNs) defined by the parity-check matrix, projecting the syndrome representation into the VN domain and the magnitude representation into the CN domain to incorporate the structural relationships between the two representations. Cross-Selective Scan SSM then generates selection parameters conditioned on the projected representations of connected nodes to dynamically control the state updates and outputs of each representation.

This design enables CrossMPM to effectively exploit the complementary information in the magnitude and syndrome and achieve competitive decoding performance with lower computational complexity than CrossMPT and ECCM.

The main contributions of this paper are as follows:

• We propose CrossMPM, a Mamba-based ECC decoder that maintains the magnitude and syndrome as separate representations and iteratively updates them.
• We propose Tanner-graph projection, which, based on Tanner-graph connectivity, projects the representations of connected nodes into the domain of the nodes being updated. We also propose Cross-Selective Scan SSM, which dynamically controls the state updates and outputs of the representations being updated based on the projected information from connected nodes.
• Through experiments on various low-density parity-check (LDPC) codes with code lengths ranging from tens to hundreds of bits, we demonstrate that CrossMPM achieves competitive decoding performance with low computational complexity. Specifically, for LDPC $(121,60)$, CrossMPM reduced FLOPs and peak training GPU memory consumption by 65.91% and 15.22%, respectively, compared with ECCM. For the longer LDPC $(529,440)$, the corresponding reductions were 48.90% and 54.94%, respectively, compared with CrossMPT.

2. Background

2.1. Error-Correcting Codes

A binary linear block code C is defined by a generator matrix $\mathbf{G}\in {\{0,1\}}^{k\times n}$ and a parity-check matrix $\mathbf{H}\in {\{0,1\}}^{(n-k)\times n}$, and the two matrices satisfy $\mathbf{G}{\mathbf{H}}^{\mathsf{T}}=\mathbf{0}$ under modulo-2 operations. A message $\mathbf{m}\in {\{0,1\}}^k$ of length k is encoded into a codeword $\mathbf{x}\in C\subseteq {\{0,1\}}^n$ of length n through $\mathbf{x}=\mathbf{m}\mathbf{G}$. The codeword $\mathbf{x}$ is then mapped to ${\mathbf{x}}_s=1-2\mathbf{x}\in {\{-1,+1\}}^n$ through binary phase-shift keying (BPSK) modulation. The modulated signal is transmitted over an additive white Gaussian noise (AWGN) channel, and the received signal is expressed as follows:

$$\mathbf{y}={\mathbf{x}}_s+\mathbf{z},\mathbf{z}\sim \mathcal{N}\left(\mathbf{0},{\sigma }^2{\mathbf{I}}_n\right),$$

(1)

where $\mathbf{y}\in {\mathbb{R}}^n$ is the received signal vector, $\mathbf{z}\in {\mathbb{R}}^n$ is the AWGN vector, ${\sigma }^2$ is the channel noise variance, and ${\mathbf{I}}_n$ is the $n\times n$ identity matrix. The objective of the decoder is to recover the transmitted codeword $\mathbf{x}$ from the received signal $\mathbf{y}$. To this end, a hard decision is applied to the received signal to obtain the following binary vector.

$${\mathbf{y}}_b=\mathrm{bin}\left(\mathrm{sign}\left(\mathbf{y}\right)\right)\in {\{0,1\}}^n,$$

(2)

where the function $\mathrm{sign}\left(a\right)$ returns $+1$ if $a\ge 0$ and $-1$ otherwise, while $\mathrm{bin}(+1)=0$ and $\mathrm{bin}(-1)=1$. The syndrome is then computed using ${\mathbf{y}}_b$ and the parity-check matrix $\mathbf{H}$ as follows:

$$\mathbf{s}\left(\mathbf{y}\right)=\mathbf{H}{\mathbf{y}}_b^{\mathsf{T}}\left(\mathrm{mod}2\right)\in {\{0,1\}}^{n-k}.$$

(3)

If $\mathbf{s}\left(\mathbf{y}\right)=\mathbf{0}$, then ${\mathbf{y}}_b$ is a valid codeword that satisfies all parity-check equations of the code. In contrast, if $\mathbf{s}\left(\mathbf{y}\right)\ne \mathbf{0}$, then one or more parity-check equations are violated.

In this paper, we use syndrome-based preprocessing [10], and the decoder input consists of the magnitude $\left|\mathbf{y}\right|$ of the received signal $\mathbf{y}$ and the syndrome $\mathbf{s}\left(\mathbf{y}\right)$. The continuous multiplicative noise ${\tilde{\mathbf{z}}}_s\in {\mathbb{R}}^n$ is defined as follows:

$$\mathbf{y}={\mathbf{x}}_s+\mathbf{z}={\mathbf{x}}_s\odot {\tilde{\mathbf{z}}}_s,$$

(4)

where ⊙ denotes element-wise multiplication. The sign of ${\tilde{\mathbf{z}}}_s$ is then used to define the binary multiplicative noise target for training.

2.2. State Space Models

SSMs accumulate and propagate sequence information through hidden states, enabling efficient modeling of long-range dependencies. The Structured State Space Sequence (S4) model [18], a well-known structured SSM, processes sequence information in a recurrent or convolutional manner based on linear dynamical systems and the High-order Polynomial Projection Operators (HiPPO) framework [19]. However, conventional structured SSMs are linear time-invariant (LTI) systems that use the same parameters at all sequence positions, which limits their ability to selectively incorporate information based on the input content.

To address this limitation, Mamba [16] introduces a selective scan SSM that dynamically generates the selection parameters based on the input. The selective scan SSM incorporates and retains relevant information in the hidden state based on the input content at each position while selectively suppressing unnecessary information. Unlike self-attention, which computes pairwise interactions among all positions, the selective scan SSM sequentially updates the hidden state along the sequence and therefore has linear computational complexity with respect to sequence length. Based on these characteristics, Mamba demonstrated up to five times the inference throughput of Transformers and achieved state-of-the-art performance across various domains, including language, audio, and genomics. Accordingly, Mamba has emerged as an efficient alternative to Transformers for processing long sequences.

3. Methods

CrossMPM maintains the magnitude, which represents the reliability of the received signal, and the syndrome, which represents the parity-check results, as separate representations and exchanges information between the two representations through Tanner-graph projection and Cross-Selective Scan SSM. The overall architecture of CrossMPM is shown in Figure 1. The proposed model generates the initial magnitude and syndrome representations from the received signal and then predicts bit-wise multiplicative noise through N decoder blocks. Each decoder block sequentially updates the magnitude and syndrome, and each update process consists of Tanner-graph projection, a Cross-Mamba block that applies Cross-Selective Scan SSM, a gated residual connection, and a feed-forward network (FFN).

Figure 1. Architecture of the proposed CrossMPM.

3.1. Initial Embedding Layer

The input representations $\mathbf{m}=\left|\mathbf{y}\right|\in {\mathbb{R}}^n$ and $\mathbf{s}=\mathbf{s}\left(\mathbf{y}\right)\in {\{0,1\}}^{n-k}$ are embedded element-wise into d-dimensional representations through node-specific trainable parameters. Each element of the magnitude and syndrome is projected into a d-dimensional representation through a node-specific trainable embedding vector and bias. Accordingly, the initial magnitude representation ${\mathbf{M}}_i^{\left(0\right)}$ of the ith VN and the initial syndrome representation ${\mathbf{S}}_j^{\left(0\right)}$ of the jth CN are defined, respectively, as follows:

$$\begin{array}{cccc}\hfill {\mathbf{M}}_i^{\left(0\right)}& =m_i{\mathbf{E}}_i^M+{\mathbf{b}}_i^M,\hfill & \hfill & i=1,\dots ,n,\hfill \\ \hfill {\mathbf{S}}_j^{\left(0\right)}& =s_j{\mathbf{E}}_j^S+{\mathbf{b}}_j^S,\hfill & \hfill & j=1,\dots ,n-k,\hfill \end{array}$$

(5)

where ${\mathbf{E}}_i^M,{\mathbf{b}}_i^M\in {\mathbb{R}}^{1\times d}$ and ${\mathbf{E}}_j^S,{\mathbf{b}}_j^S\in {\mathbb{R}}^{1\times d}$ are the trainable embedding vectors and biases for each node, and d denotes the embedding dimension. By concatenating the representations of all nodes, the initial magnitude and syndrome representations are given as follows:

$$\begin{array}{cc}\hfill {\mathbf{M}}^{\left(0\right)}& =\left[{\mathbf{M}}_1^{\left(0\right)};\dots ;{\mathbf{M}}_n^{\left(0\right)}\right]\in {\mathbb{R}}^{n\times d},\hfill \\ \hfill {\mathbf{S}}^{\left(0\right)}& =\left[{\mathbf{S}}_1^{\left(0\right)};\dots ;{\mathbf{S}}_{n-k}^{\left(0\right)}\right]\in {\mathbb{R}}^{(n-k)\times d},\hfill \end{array}$$

(6)

where $[·;·]$ denotes concatenation and combines the representations of the nodes along the node dimension. The resulting ${\mathbf{M}}^{\left(0\right)}$ and ${\mathbf{S}}^{\left(0\right)}$ pass through N decoder blocks and are sequentially and iteratively updated.

3.2. Decoder Block

3.2.1. Tanner-Graph Projection

Tanner-graph projection aggregates the representations of the connected nodes for each node by taking their mean according to the connections defined by the parity-check matrix $\mathbf{H}$ and projects them into the representation domain of the target nodes. In the magnitude update of the lth decoder block, the syndrome representations of the CNs connected to each VN are aggregated. The set of CNs connected to the ith VN is defined as follows:

$${\mathcal{N}}_M\left(i\right)=\left\{j\in \{1,\dots ,n-k\}\mid H_{j,i}=1\right\}.$$

(7)

The syndrome representation passed to the corresponding VN is given as follows:

$${\tilde{\mathbf{S}}}_i^{\left(l\right)}={\displaystyle \frac{1}{\left|{\mathcal{N}}_M\left(i\right)\right|}}\sum _{j\in {\mathcal{N}}_M\left(i\right)}{\mathbf{S}}_j^{(l-1)},i=1,\dots ,n,$$

(8)

where $\left|{\mathcal{N}}_M\left(i\right)\right|$ is the number of CNs connected to the ith VN, and ${\mathbf{S}}_j^{(l-1)}$ is the syndrome representation of the jth CN input to the current decoder block. Applying this process to all VNs yields the syndrome representation ${\tilde{\mathbf{S}}}^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$, which has the same size as the magnitude representation ${\mathbf{M}}^{(l-1)}$.

In the syndrome update, the previously updated magnitude representation ${\mathbf{M}}^{\left(l\right)}$ is used to aggregate information from the VNs connected to each CN. The set of VNs connected to the jth CN is defined as follows:

$${\mathcal{N}}_S\left(j\right)=\left\{i\in \{1,\dots ,n\}\mid H_{j,i}=1\right\}.$$

(9)

The magnitude representation passed to the corresponding CN is computed as follows:

$${\tilde{\mathbf{M}}}_j^{\left(l\right)}={\displaystyle \frac{1}{\left|{\mathcal{N}}_S\left(j\right)\right|}}\sum _{i\in {\mathcal{N}}_S\left(j\right)}{\mathbf{M}}_i^{\left(l\right)},j=1,\dots ,n-k,$$

(10)

where $\left|{\mathcal{N}}_S\left(j\right)\right|$ is the number of VNs connected to the jth CN, and ${\mathbf{M}}_i^{\left(l\right)}$ is the updated magnitude representation of the ith VN in the current decoder block. Applying this process to all CNs yields the magnitude representation ${\tilde{\mathbf{M}}}^{\left(l\right)}\in {\mathbb{R}}^{(n-k)\times d}$, which has the same size as the syndrome representation ${\mathbf{S}}^{(l-1)}$.

Since each CN corresponds to a parity-check constraint defined by the parity-check matrix $\mathbf{H}$, aggregating the syndrome representations of CNs connected to each VN provides information related to the local parity checks involving that VN. Similarly, aggregating the magnitude representations of VNs connected to each CN provides information about the variables involved in its corresponding parity-check constraint. Thus, Tanner-graph projection makes structural information associated with local parity-check constraints available for the subsequent updates of the magnitude and syndrome representations.

3.2.2. Cross-Mamba Block

The Cross-Mamba block consists of an SSM path and a gate path that process the current node representation, and the proposed Cross-Selective Scan SSM is applied to the SSM path. Since Tanner graphs do not have a natural sequential ordering, the magnitude and syndrome sequences are arranged according to the variable node indices corresponding to the columns of the parity-check matrix $\mathbf{H}$ and the check node indices corresponding to its rows, respectively. To incorporate bidirectional information from each sequence, the Cross-Mamba block performs forward and backward Selective Scans in ascending and descending node-index order, respectively. Figure 2 illustrates the architecture of the Cross-Mamba block that updates the magnitude representation.

Figure 2. Architecture of the Cross-Mamba block.

First, in the SSM path, normalization and linear projection are applied to the current magnitude representation ${\mathbf{M}}^{(l-1)}$, and the SSM input is then generated through one-dimensional (1D) convolution and the sigmoid linear unit (SiLU) activation function.

$${\mathbf{U}}_M^{\left(l\right)}=\mathrm{SiLU}\left(\mathrm{Conv1D}\left(\mathrm{Norm}\left({\mathbf{M}}^{(l-1)}\right){\mathbf{W}}_u^{M,\left(l\right)}\right)\right),$$

(11)

where ${\mathbf{W}}_u^{M,\left(l\right)}\in {\mathbb{R}}^{d\times d}$ is a trainable projection matrix, and ${\mathbf{U}}_M^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ is the SSM input for the magnitude representation. The 1D convolution extracts local features from the magnitude sequence, and the subsequent SSM propagates this information across the entire sequence. The proposed Cross-Selective Scan SSM uses ${\mathbf{U}}_M^{\left(l\right)}$, generated from the current magnitude representation, as the SSM input and generates the selection parameters ${\mathbf{\Delta }}_M^{\left(l\right)}$, ${\mathbf{B}}_M^{\left(l\right)}$, and ${\mathbf{C}}_M^{\left(l\right)}$ from the projected syndrome representation ${\tilde{\mathbf{S}}}^{\left(l\right)}$ obtained through Tanner-graph projection. Each parameter is computed as follows:

$$\begin{array}{cc}\hfill {\mathit{\delta }}_M^{\left(l\right)}& =\mathrm{Norm}\left({\tilde{\mathbf{S}}}^{\left(l\right)}\right){\mathbf{W}}_{\delta }^{M,\left(l\right)},\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\mathbf{\Delta }}_M^{\left(l\right)}& =\mathrm{softplus}\left({\mathit{\delta }}_M^{\left(l\right)}{\mathbf{W}}_{\Delta }^{M,\left(l\right)}+{\mathbf{b}}_{\Delta }^{M,\left(l\right)}\right),\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill {\mathbf{B}}_M^{\left(l\right)}& =\mathrm{Norm}\left({\tilde{\mathbf{S}}}^{\left(l\right)}\right){\mathbf{W}}_B^{M,\left(l\right)},\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill {\mathbf{C}}_M^{\left(l\right)}& =\mathrm{Norm}\left({\tilde{\mathbf{S}}}^{\left(l\right)}\right){\mathbf{W}}_C^{M,\left(l\right)},\hfill \end{array}$$

(15)

where ${\mathbf{W}}_{\delta }^{M,\left(l\right)}\in {\mathbb{R}}^{d\times r}$, ${\mathbf{W}}_{\Delta }^{M,\left(l\right)}\in {\mathbb{R}}^{r\times d}$, ${\mathbf{W}}_B^{M,\left(l\right)},{\mathbf{W}}_C^{M,\left(l\right)}\in {\mathbb{R}}^{d\times d_{\mathrm{state}}}$, and ${\mathbf{b}}_{\Delta }^{M,\left(l\right)}\in {\mathbb{R}}^d$ are trainable parameters. r denotes the intermediate projection dimension for generating ${\mathbf{\Delta }}_M^{\left(l\right)}$, and $d_{\mathrm{state}}$ denotes the state dimension of the SSM. ${\mathit{\delta }}_M^{\left(l\right)}\in {\mathbb{R}}^{n\times r}$ is an intermediate representation for generating ${\mathbf{\Delta }}_M^{\left(l\right)}$, and ${\mathbf{\Delta }}_M^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ denotes the discretization time step of the SSM. In addition, ${\mathbf{B}}_M^{\left(l\right)},{\mathbf{C}}_M^{\left(l\right)}\in {\mathbb{R}}^{n\times d_{\mathrm{state}}}$ denote the input matrix and output matrix of the SSM, respectively. Following the selective SSM discretization in Mamba [16], using ${\mathbf{\Delta }}_{M,t}^{\left(l\right)}$ generated at position t, ${\mathbf{A}}_M^{\left(l\right)}$ and ${\mathbf{B}}_{M,t}^{\left(l\right)}$ are discretized as follows:

$${\overline{\mathbf{A}}}_{M,t}^{\left(l\right)}=\exp \left(\mathrm{diag}\left({\mathbf{\Delta }}_{M,t}^{\left(l\right)}\right){\mathbf{A}}_M^{\left(l\right)}\right),t=1,\dots ,n,$$

(16)

$${\overline{\mathbf{B}}}_{M,t}^{\left(l\right)}={\left({\mathbf{\Delta }}_{M,t}^{\left(l\right)}\right)}^{\mathsf{T}}{\mathbf{B}}_{M,t}^{\left(l\right)},$$

(17)

where ${\mathbf{\Delta }}_{M,t}^{\left(l\right)}\in {\mathbb{R}}^{1\times d}$ and ${\mathbf{B}}_{M,t}^{\left(l\right)}\in {\mathbb{R}}^{1\times d_{\mathrm{state}}}$ denote the corresponding row vectors at position t, and ${\mathbf{A}}_M^{\left(l\right)}\in {\mathbb{R}}^{d\times d_{\mathrm{state}}}$ is a trainable state transition parameter. Accordingly, ${\overline{\mathbf{A}}}_{M,t}^{\left(l\right)},{\overline{\mathbf{B}}}_{M,t}^{\left(l\right)}\in {\mathbb{R}}^{d\times d_{\mathrm{state}}}$. The operator $\mathrm{diag}(·)$ constructs a diagonal matrix from a vector, and $\exp (·)$ is applied element-wise.

Using the discretized parameters, the SSM operation is performed as follows:

$${\mathbf{h}}_{M,t}^{\left(l\right)}={\overline{\mathbf{A}}}_{M,t}^{\left(l\right)}\odot {\mathbf{h}}_{M,t-1}^{\left(l\right)}+\mathrm{diag}\left({\mathbf{U}}_{M,t}^{\left(l\right)}\right){\overline{\mathbf{B}}}_{M,t}^{\left(l\right)},$$

(18)

$${\mathbf{V}}_{M,t}^{\left(l\right)}={\mathbf{C}}_{M,t}^{\left(l\right)}{\left({\mathbf{h}}_{M,t}^{\left(l\right)}\right)}^{\mathsf{T}}+{\mathbf{D}}_M^{\left(l\right)}\odot {\mathbf{U}}_{M,t}^{\left(l\right)},$$

(19)

where ${\mathbf{h}}_{M,t}^{\left(l\right)}\in {\mathbb{R}}^{d\times d_{\mathrm{state}}}$ is the hidden state at the tth VN, and ${\mathbf{V}}_{M,t}^{\left(l\right)}\in {\mathbb{R}}^d$ is the SSM output generated at that position. ${\mathbf{C}}_{M,t}^{\left(l\right)}\in {\mathbb{R}}^{1\times d_{\mathrm{state}}}$ is the output parameter at position t, and ${\mathbf{D}}_M^{\left(l\right)}\in {\mathbb{R}}^d$ is a trainable parameter that directly incorporates the SSM input into the output. The initial hidden state is set to ${\mathbf{h}}_{M,0}^{\left(l\right)}=\mathbf{0}$. The Cross-Selective Scan SSM sequentially processes the sequence arranged according to node indices, updating the hidden state at each position using the previous hidden state and the current SSM input. The selection parameters generated from the syndrome representation aggregated through Tanner-graph projection regulate the state updates and output generation by incorporating information related to local parity-check constraints. In particular, the position-dependent selection parameters control the retention of previously accumulated information and the incorporation of new input information, while the updated hidden state is propagated to subsequent positions. This enables the SSM to model dependencies between distant positions in the sequence.

Meanwhile, in the gate path, the gate representation is generated from the current magnitude representation ${\mathbf{M}}^{(l-1)}$ as follows:

$${\mathbf{Z}}_M^{\left(l\right)}=\mathrm{SiLU}\left(\mathrm{Norm}\left({\mathbf{M}}^{(l-1)}\right){\mathbf{W}}_z^{M,\left(l\right)}\right),$$

(20)

where ${\mathbf{W}}_z^{M,\left(l\right)}\in {\mathbb{R}}^{d\times d}$ is a trainable projection matrix. The gate representation ${\mathbf{Z}}_M^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ and the previously obtained SSM output ${\mathbf{V}}_M^{\left(l\right)}$ undergo element-wise multiplication, followed by output projection to generate the Mamba output.

$${\mathbf{Y}}_M^{\left(l\right)}=\left({\mathbf{V}}_M^{\left(l\right)}\odot {\mathbf{Z}}_M^{\left(l\right)}\right){\mathbf{W}}_o^{M,\left(l\right)},$$

(21)

where ${\mathbf{W}}_o^{M,\left(l\right)}\in {\mathbb{R}}^{d\times d}$ is a trainable output projection matrix, and ${\mathbf{Y}}_M^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ is the Mamba output. The above operations are performed in both the forward and backward directions using separate model parameters. The backward output is restored to the original node order, and the final output of the Cross-Mamba block is generated by summing the outputs from both directions as follows:

$${\mathbf{O}}_M^{\left(l\right)}={\mathbf{Y}}_{M,\to }^{\left(l\right)}+{\mathbf{Y}}_{M,\leftarrow }^{\left(l\right)},$$

(22)

where ${\mathbf{Y}}_{M,\to }^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ and ${\mathbf{Y}}_{M,\leftarrow }^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ are the forward and backward Mamba outputs, respectively, and ${\mathbf{O}}_M^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ is the final output that combines information from both directions.

The same Cross-Mamba block is also applied to the syndrome representation. In this case, the SSM input and gate representation are generated from the current syndrome representation ${\mathbf{S}}^{(l-1)}$, while ${\mathbf{\Delta }}_S^{\left(l\right)}$, ${\mathbf{B}}_S^{\left(l\right)}$, and ${\mathbf{C}}_S^{\left(l\right)}$ are generated from the magnitude representation ${\tilde{\mathbf{M}}}^{\left(l\right)}$ obtained through Tanner-graph projection. Using these parameters, the SSM operation for the syndrome representation is performed conditioned on the magnitude representation, and the forward and backward outputs are summed to generate the final Mamba output ${\mathbf{O}}_S^{\left(l\right)}$.

3.2.3. Gated Residual Connection and Feed-Forward Network

The output generated by the Cross-Mamba block is selectively incorporated into the existing node representation through a gated residual connection. For the magnitude representation, a gate is first generated using the previous magnitude representation ${\mathbf{M}}^{(l-1)}$ and the output ${\mathbf{O}}_M^{\left(l\right)}$ of the Cross-Mamba block as follows:

$${\mathbf{G}}_M^{\left(l\right)}=\sigma \left(\left[\mathrm{Norm}\left({\mathbf{M}}^{(l-1)}\right);{\mathbf{O}}_M^{\left(l\right)}\right]{\mathbf{W}}_g^{M,\left(l\right)}+{\mathbf{b}}_g^{M,\left(l\right)}\right),$$

(23)

where the concatenation is performed along the feature dimension, and $\sigma (·)$ is the sigmoid function. ${\mathbf{W}}_g^{M,\left(l\right)}\in {\mathbb{R}}^{2d\times d}$ and ${\mathbf{b}}_g^{M,\left(l\right)}\in {\mathbb{R}}^d$ are trainable parameters for generating the gate, and ${\mathbf{G}}_M^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ is the gate for the magnitude representation. The generated gate selectively controls the contribution of the Cross-Mamba block output to the existing magnitude representation. The gated residual connection is performed as follows:

$${\widehat{\mathbf{M}}}^{\left(l\right)}={\mathbf{M}}^{(l-1)}+{\mathbf{G}}_M^{\left(l\right)}\odot {\mathbf{O}}_M^{\left(l\right)}.$$

(24)

This preserves the magnitude information passed from the previous decoder block while selectively incorporating the information generated by the Cross-Mamba block, resulting in ${\widehat{\mathbf{M}}}^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$.

Subsequently, an FFN and a residual connection are applied to ${\widehat{\mathbf{M}}}^{\left(l\right)}$ to generate the final magnitude representation ${\mathbf{M}}^{\left(l\right)}\in {\mathbb{R}}^{n\times d}$ of the lth decoder block.

$${\mathbf{M}}^{\left(l\right)}={\widehat{\mathbf{M}}}^{\left(l\right)}+{\mathrm{FFN}}_M^{\left(l\right)}\left(\mathrm{Norm}\left({\widehat{\mathbf{M}}}^{\left(l\right)}\right)\right).$$

(25)

The same gated residual connection and FFN are applied to the syndrome representation. A gate is generated from the previous syndrome representation ${\mathbf{S}}^{(l-1)}$ and the output ${\mathbf{O}}_S^{\left(l\right)}$ of the Cross-Mamba block, and the gated residual connection and FFN are then sequentially applied to generate the syndrome representation ${\mathbf{S}}^{\left(l\right)}$ of the lth decoder block.

$$\begin{array}{cc}\hfill {\mathbf{G}}_S^{\left(l\right)}& =\sigma \left(\left[\mathrm{Norm}\left({\mathbf{S}}^{(l-1)}\right);{\mathbf{O}}_S^{\left(l\right)}\right]{\mathbf{W}}_g^{S,\left(l\right)}+{\mathbf{b}}_g^{S,\left(l\right)}\right),\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill {\widehat{\mathbf{S}}}^{\left(l\right)}& ={\mathbf{S}}^{(l-1)}+{\mathbf{G}}_S^{\left(l\right)}\odot {\mathbf{O}}_S^{\left(l\right)},\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill {\mathbf{S}}^{\left(l\right)}& ={\widehat{\mathbf{S}}}^{\left(l\right)}+{\mathrm{FFN}}_S^{\left(l\right)}\left(\mathrm{Norm}\left({\widehat{\mathbf{S}}}^{\left(l\right)}\right)\right).\hfill \end{array}$$

(28)

The resulting ${\mathbf{M}}^{\left(l\right)}$ and ${\mathbf{S}}^{\left(l\right)}$ are used as the outputs of the lth decoder block and are passed to the next decoder block as inputs.

3.3. Output Prediction and Training Objective

CrossMPM predicts multiplicative noise using the magnitude and syndrome representations updated in each decoder block. The two representations of the lth decoder block are concatenated along the node dimension as follows:

$${\mathbf{Y}}^{\left(l\right)}=\left[{\mathbf{M}}^{\left(l\right)};{\mathbf{S}}^{\left(l\right)}\right]\in {\mathbb{R}}^{(2n-k)\times d},l=1,\dots ,N.$$

(29)

Normalization and two fully connected (FC) layers are sequentially applied to the concatenated representation. The first FC layer reduces the d-dimensional representation of each node to a scalar, and the second FC layer transforms the information from $2n-k$ nodes into an n-dimensional multiplicative noise prediction. The prediction of the lth decoder block is computed as follows:

$${\mathbf{o}}^{\left(l\right)}={\mathbf{W}}_s\left(\mathrm{Norm}\left({\mathbf{Y}}^{\left(l\right)}\right){\mathbf{w}}_r+b_r\right)+{\mathbf{b}}_s\in {\mathbb{R}}^n,$$

(30)

where ${\mathbf{w}}_r\in {\mathbb{R}}^d$ and $b_r\in \mathbb{R}$ are the trainable parameters of the first FC layer, and ${\mathbf{W}}_s\in {\mathbb{R}}^{n\times (2n-k)}$ and ${\mathbf{b}}_s\in {\mathbb{R}}^n$ are the trainable parameters of the second FC layer.

During training, the same output prediction layer is applied to all decoder blocks to generate the predictions ${\mathbf{o}}^{\left(l\right)}$. During inference, ${\mathbf{o}}^{\left(N\right)}$ generated by the final decoder block is used as the final multiplicative noise prediction, and the decoded codeword is computed as follows:

$$\widehat{\mathbf{x}}={\displaystyle \frac{1-\mathrm{sign}\left(-{\mathbf{o}}^{\left(N\right)}\odot \mathrm{sign}\left(\mathbf{y}\right)\right)}{2}}.$$

(31)

The binary multiplicative noise target used for training is obtained from the sign of the multiplicative noise as follows:

$$\tilde{\mathbf{z}}=\mathrm{bin}\left(\mathrm{sign}\left({\tilde{\mathbf{z}}}_s\right)\right)=\mathrm{bin}\left(\mathrm{sign}\left(\mathbf{y}\odot {\mathbf{x}}_s\right)\right)\in {\{0,1\}}^n.$$

(32)

The proposed model applies a layer-wise loss to the prediction of every decoder block. The binary cross-entropy loss between the prediction ${\mathbf{o}}^{\left(l\right)}$ generated by the lth decoder block and the target multiplicative noise $\tilde{\mathbf{z}}$ is defined as follows:

$${\mathcal{L}}^{\left(l\right)}=-{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}\left[{\tilde{z}}_i\log \left(\sigma \left(o_i^{\left(l\right)}\right)\right)+\left(1-{\tilde{z}}_i\right)\log \left(1-\sigma \left(o_i^{\left(l\right)}\right)\right)\right].$$

(33)

The losses computed at each decoder block are summed across the layers to construct the final training objective of the proposed model as follows:

$${\mathcal{L}}_{\mathrm{total}}={\displaystyle \sum _{l=1}^N}{\mathcal{L}}^{\left(l\right)}.$$

(34)

This provides a direct training signal to every decoder block, allowing the magnitude and syndrome representations to be progressively refined at each stage.

4. Experimental Results

To evaluate the performance of the proposed model, we followed the same training protocol as CrossMPT [13]. We trained the model for a total of 1000 epochs using the Adam optimizer [20], where each epoch consisted of 1000 minibatches with 128 samples per minibatch. The initial learning rate was set to ${10}^{-4}$ and decayed to $5\times {10}^{-7}$ over 1000 epochs using a cosine decay scheduler, with the learning rate updated at the end of each epoch and no warmup applied. No separate validation set or early stopping was used. The received signals used for training were generated based on the all-zero codeword with AWGN sampled over an $E_b/N_0$ range from 3 dB to 7 dB. Evaluation was performed on randomly generated codewords, and the simulation was repeated until at least 500 frame errors were observed at each $E_b/N_0$ of 4, 5, and 6 dB. As the performance metric, we used the negative natural logarithm of the bit error rate (BER), $-\ln \left(\mathrm{BER}\right)$, where a higher value indicates better decoding performance. Our experiments were conducted using an NVIDIA GeForce RTX 5090 GPU and an AMD Ryzen Threadripper PRO 5955WX 16-Core Processor.

4.1. Decoding Performance Evaluation

We selected LDPC codes because their sparse parity-check matrices make them well suited for examining the information exchange between nodes through Tanner-graph projection. Table 1 compares the decoding performance of BP, autoregressive belief propagation (AR BP), ECCT, CrossMPT, ECCM, and the proposed CrossMPM on various LDPC codes. The results of the BP-based decoder were obtained using 50 decoding iterations. To minimize the effect of model size, ECCT, CrossMPT, ECCM, and CrossMPM were compared with approximately $1.2\mathrm{M}$ trainable parameters each. The best results are shown in bold, and a dash (–) indicates that the corresponding result was not reported in the original study. The decoding performance results of ECCM were taken from the original study rather than obtained by retraining it under the same training protocol as CrossMPM. In the original study, ECCM was trained over an $E_b/N_0$ range of 2 dB to 7 dB using the Adam optimizer and a cosine learning-rate schedule, with the learning rate decreasing from $2.5\times {10}^{-4}$ to ${10}^{-10}$.

Table 1. Comparison of decoding performance.

Codes	$(\mathit{n},\mathit{k})$	BP [1]			AR BP [9]			ECCT [12]			CrossMPT [13]			ECCM [17]			CrossMPM
		4	5	6	4	5	6	4	5	6	4	5	6	4	5	6	4	5	6
LDPC	$(49,24)$	6.23	8.19	11.72	6.58	9.39	12.39	6.13	8.71	12.10	6.68	9.52	13.19	6.71	9.55	13.25	6.77	9.59	13.40
	$(121,60)$	4.82	7.21	10.87	5.22	8.31	13.07	5.17	8.31	13.30	5.74	9.26	14.78	5.49	8.87	14.23	5.75	9.51	15.20
	$(121,70)$	5.88	8.76	13.04	6.45	10.01	14.77	6.40	10.21	16.11	7.06	11.39	17.52	–	–	–	7.26	11.60	17.54
	$(121,80)$	6.66	9.82	13.98	7.22	11.03	15.90	7.41	11.51	16.44	7.99	12.75	18.15	7.81	12.34	18.35	8.21	12.81	18.21

CrossMPM achieved better decoding performance than CrossMPT across all codes and $E_b/N_0$ conditions. Compared with ECCM, CrossMPM also achieved higher $-\ln \left(\mathrm{BER}\right)$ values under most conditions. For a more intuitive comparison, Figure 3 presents the BER curves corresponding to the results in Table 1 for four LDPC codes at $E_b/N_0$ values of 4, 5, and 6 dB.

Figure 3. Comparison of BER performance. (a) LDPC (49, 24). (b) LDPC (121, 60). (c) LDPC (121, 70). (d) LDPC (121, 80).

To evaluate the decoding performance of CrossMPM on longer codes, experiments were conducted on wireless regional area network (WRAN) LDPC (384, 320) and LDPC (529, 440). For a fair comparison, the numbers of trainable parameters of CrossMPT and CrossMPM were set to approximately 0.26 M for WRAN LDPC (384, 320) and approximately 0.43 M for LDPC (529, 440). As shown in Table 2, CrossMPM achieved higher $-\ln \left(\mathrm{BER}\right)$ values than CrossMPT for both codes. Specifically, for WRAN LDPC (384, 320), CrossMPM achieved higher $-\ln \left(\mathrm{BER}\right)$ values than CrossMPT by 0.03, 0.11, and 0.06 at 4, 5, and 6 dB, respectively. For LDPC (529, 440), larger differences of 0.38, 1.86, and 2.32 were observed at the corresponding $E_b/N_0$ values. These results indicate that the decoding performance improvements of CrossMPM are not limited to short LDPC codes but are also maintained for the two longer LDPC codes evaluated in this study.

Table 2. Comparison of decoding performance for longer LDPC codes.

Codes	$(\mathit{n},\mathit{k})$	CrossMPT [13]			CrossMPM
		4	5	6	4	5	6
WRAN LDPC	$(384,320)$	7.04	11.70	16.00	7.07	11.81	16.06
LDPC	$(529,440)$	6.53	10.94	16.68	6.91	12.80	19.00

4.2. Computational and Memory Efficiency Analysis

Following the decoding performance analysis, we compared computational and memory efficiency. Table 3 and Table 4 present the number of trainable parameters, FLOPs, and peak training GPU memory consumption for short and longer LDPC codes, respectively.

Table 3. Comparison of the number of parameters, FLOPs, and peak training GPU memory.

Codes	$(\mathit{n},\mathit{k})$	CrossMPT [13]			ECCM [17]			CrossMPM
		Parameters	FLOPs	Memory	Parameters	FLOPs	Memory	Parameters	FLOPs	Memory
		(M)	(M)	(MiB)	(M)	(M)	(MiB)	(M)	(M)	(MiB)
LDPC	$(49,24)$	1.20	95.70	39.48	1.21	207.51	40.12	1.20	72.21	39.93
	$(121,60)$	1.24	246.85	47.18	1.24	514.46	56.95	1.23	175.37	48.28
	$(121,70)$	1.23	229.68	45.50	1.24	483.19	54.78	1.23	164.99	47.45
	$(121,80)$	1.23	212.52	43.84	1.24	452.05	52.72	1.23	154.62	46.50

Table 4. Comparison of the number of parameters, FLOPs, and peak training GPU memory for longer LDPC codes.

Codes	$(\mathit{n},\mathit{k})$	CrossMPT [13]			CrossMPM
		Parameters (M)	FLOPs (M)	Memory (MiB)	Parameters (M)	FLOPs (M)	Memory (MiB)
WRAN LDPC	$(384,320)$	0.26	55.98	47.77	0.26	33.19	28.38
LDPC	$(529,440)$	0.43	90.31	75.85	0.43	46.15	34.18

First, Table 3 compares the three models on four short LDPC codes, with approximately 1.2 M trainable parameters for each model. The results show that CrossMPM achieved lower FLOPs than CrossMPT and ECCM across all codes, with reductions of 28.96% and 65.91%, respectively, for LDPC (121, 60). In terms of peak training GPU memory consumption, CrossMPM used less memory than ECCM across all codes but slightly more than CrossMPT. This may be attributed to the fact that a reduction in FLOPs does not necessarily lead to lower GPU memory consumption, as GPU memory usage during training can also be affected by various factors beyond computational cost, including intermediate activation storage and implementation details.

To further evaluate efficiency on longer LDPC codes, Table 4 compares the complexity of CrossMPT and CrossMPM for WRAN LDPC (384, 320) and LDPC (529, 440). The two models were configured to have similar numbers of parameters, approximately 0.26 M and 0.43 M for the respective codes. The results show that CrossMPM reduced FLOPs and peak training GPU memory consumption by 40.71% and 40.59%, respectively, compared with CrossMPT for WRAN LDPC (384, 320). For LDPC (529, 440), the corresponding reductions were 48.90% and 54.94%, respectively. Unlike the results for short LDPC codes, where CrossMPM exhibited slightly higher GPU memory consumption than CrossMPT, CrossMPM achieved substantial reductions in both computational cost and GPU memory consumption for the longer LDPC codes.

CrossMPM is designed to alleviate computational burden by replacing the cross-attention-based information exchange in CrossMPT with Tanner-graph projection and Cross-Selective Scan SSM. These experimental results indicate that the proposed architecture is effective in improving computational efficiency and suggest that its advantages in terms of computational and memory efficiency may become more pronounced for longer LDPC codes.

4.3. Ablation Study

To investigate the contribution of the core components of CrossMPM to decoding performance, an ablation study was conducted on LDPC (49, 24). We evaluated variants in which Tanner-graph projection was removed, Cross-Selective Scan SSM was replaced with the conventional Selective Scan SSM of Mamba, the backward scan was removed, gated residual connection was replaced with a standard residual connection, or layer-wise loss was removed. Table 5 compares the decoding performance of these variants at $E_b/N_0$ values of 4, 5, and 6 dB.

Table 5. Ablation study of CrossMPM on LDPC (49, 24) in terms of $-\ln \left(\mathrm{BER}\right)$.

Model	${\mathit{E}}_{\mathit{b}}/{\mathit{N}}_0$ (dB)
	4	5	6
CrossMPM (Full Model)	6.77	9.59	13.40
w/o Tanner-graph projection	3.60	4.37	6.53
w/o Cross-Selective Scan SSM	3.89	5.09	6.76
w/o Backward scan	6.61	9.32	13.20
w/o Gated residual connection	6.74	9.59	13.36
w/o Layer-wise loss	6.60	9.36	13.10

The results show that removing Tanner-graph projection or replacing Cross-Selective Scan SSM with the conventional Selective Scan SSM of Mamba substantially degraded decoding performance. In particular, at 6 dB, the full CrossMPM achieved a $-\ln \left(\mathrm{BER}\right)$ value of 13.40, whereas the variants without Tanner-graph projection and Cross-Selective Scan SSM achieved values of 6.53 and 6.76, respectively. These results highlight the importance of information exchange based on Tanner-graph connectivity and the Cross-Selective Scan SSM for decoding performance. Removing the backward scan or layer-wise loss also resulted in lower decoding performance across all $E_b/N_0$ conditions. In contrast, replacing gated residual connection with a standard residual connection resulted in relatively small performance differences. Overall, these findings demonstrate the contributions of the core components of CrossMPM, particularly Tanner-graph projection and Cross-Selective Scan SSM, to its decoding performance.

5. Conclusions

In this paper, we proposed CrossMPM, a low-complexity Mamba-based neural decoder. The proposed model is designed to effectively exchange information between the magnitude and syndrome while reducing the computational burden associated with attention in existing Transformer-based decoders. CrossMPM maintains the two representations separately and sequentially updates them, exchanging information through Tanner-graph projection and Cross-Selective Scan SSM. Tanner-graph projection aggregates information from connected nodes according to the connections defined by the parity-check matrix, thereby incorporating code structure into the information exchange between the magnitude and syndrome. Cross-Selective Scan SSM uses the current representation as the input for state updates and generates selection parameters from the projected representations of connected nodes to dynamically control the state update at each position.

In experiments on various LDPC codes, CrossMPM achieved better decoding performance than CrossMPT under all evaluation conditions with similar numbers of parameters and also achieved higher performance than ECCM under most conditions. CrossMPM achieved better performance than CrossMPT on WRAN LDPC $(384,320)$ and LDPC $(529,440)$, confirming that the performance improvement is not limited to short codes. In terms of computational and memory efficiency, CrossMPM reduced FLOPs and peak training GPU memory consumption by 65.91% and 15.22%, respectively, compared with ECCM for LDPC $(121,60)$. For the longer LDPC $(529,440)$, the corresponding reductions were 48.90% and 54.94%, respectively, compared with CrossMPT. These results show that the proposed design provides a favorable trade-off between decoding performance and computational efficiency.

In future work, we will evaluate the decoding performance of CrossMPM on even longer LDPC codes and a wider range of code rates, and further investigate its scalability, generalization capability, and computational and memory efficiency across different code lengths. Although this study evaluated only LDPC codes, Tanner-graph projection uses the VN–CN connectivity defined by a parity-check matrix, which suggests that CrossMPM may be applicable to other linear block codes. We will investigate its decoding performance and computational efficiency on Bose–Chaudhuri–Hocquenghem (BCH) and polar codes, taking into account differences in their parity-check matrix structures and corresponding model configurations. We will also extend the evaluation to non-AWGN channels, including Rayleigh fading. To this end, we will investigate input representations that account for channel estimates and noise variance and evaluate the applicability of the Cross-Selective Scan SSM using these representations. Furthermore, we plan to investigate the use of signals obtained after active transmission detection and channel estimation as decoding inputs to CrossMPM in multi-user unsourced random access scenarios [21].