Activity Detection and Channel Estimation Based on Correlated Hybrid Message Passing for Grant-Free Massive Random Access

Abstract

Massive machine-type communications (mMTC) in future 6G networks will involve a vast number of devices with sporadic traffic. Grant-free access has emerged as an effective strategy to reduce the access latency and processing overhead by allowing devices to transmit without prior permission, making accurate active user detection and channel estimation (AUDCE) crucial. In this paper, we investigate the joint AUDCE problem in wideband massive access systems. We develop an innovative channel prior model that captures the dual correlation structure of the channel using three state variables: active indication, channel supports, and channel values. By integrating Markov chains with coupled Gaussian distributions, the model effectively describes both the structural and numerical dependencies within the channel. We propose the correlated hybrid message passing (CHMP) algorithm based on Bethe free energy (BFE) minimization, which adaptively updates model parameters without requiring prior knowledge of user sparsity or channel priors. Simulation results show that the CHMP algorithm accurately detects active users and achieves precise channel estimation.

1. Introduction

With the continuous evolution of mobile communication technology and the rapid development of Internet of Things (IoT) applications, future 6G technologies are required to support the massive connectivity of IoT devices. Massive access is integral to emerging fields such as smart homes, smart cities, and industrial automation, with the primary goal of providing ubiquitous connectivity for low-cost, energy-efficient devices [1,2]. However, due to the large scale, sporadic activity, and short data packet transmission characteristic of IoT devices, traditional grant-based random access methods, which rely on negotiation and resource allocation, can lead to heavy signaling overhead and high transmission latency [3]. To address these fundamental challenges, researchers have proposed grant-free access schemes. By allowing devices to transmit data without prior authorization, this paradigm significantly improves the communication efficiency and is considered a cornerstone technology for future massive access scenarios [4].

The successful implementation of grant-free access relies on a critical task at the base station (BS): the ability to efficiently perform joint active user detection and channel estimation (AUDCE) from a single snapshot of superimposed signals [5]. Leveraging the sporadic nature of user activity, this problem can be elegantly formulated as a sparse signal recovery task. Early and foundational approaches were largely based on traditional compressed sensing (CS) algorithms, such as orthogonal matching pursuit (OMP), which provided initial solutions but often exhibited limited performance, especially in low signal-to-noise-ratio (SNR) regimes or when the channel statistics are complex [6,7]. To achieve higher accuracy and robustness, the research paradigm has increasingly shifted towards model-based Bayesian inference, which offers a principled way to incorporate rich prior knowledge of the system’s structure and statistics. Within this paradigm, advanced message passing (MP) algorithms, including those derived from sparse Bayesian learning (SBL) and generalized approximate message passing (GAMP), have emerged as powerful and computationally efficient frameworks for AUDCE, demonstrating significant performance gains over their CS counterparts [8,9,10].

To further push the performance boundaries, a pivotal research direction has been the exploitation of inherent correlation structures within the massive access channel. This body of work can be broadly summarized by the dimension of correlation explored. The core strategy in the time domain is to leverage the temporal dependency of user activity, where a user tends to remain active or inactive over consecutive time slots. Representative works, such as [11,12], have captured this slow-varying behavior using methods like message passing with temporally correlated priors or Bernoulli–Gaussian–Markov chain models. Beyond the time domain, the primary approach in the frequency domain is to mitigate the effects of frequency-selective fading by transforming the channel into a domain where it exhibits a more compact, sparse representation. For instance, the work in [13] effectively exploits channel sparsity in the discrete cosine transform (DCT) domain using a hybrid message passing algorithm. Furthermore, the spatial domain offers yet another layer of rich structural information, where researchers have capitalized on large antenna arrays from two main perspectives. One line of inquiry leverages the channel’s spatial correlation, where works like [14] have incorporated the channel’s spatial covariance matrix into the compressed sensing framework as prior information to enhance the estimation accuracy. Another powerful approach, adopted in [15,16], is to exploit the inherent channel sparsity that emerges when the channel is characterized in the angle domain, which is a direct consequence of the clustered nature of physical scatterers. This research direction has also been successfully extended to next-generation architectures like cell-free massive MIMO systems [17].

Parallel to the advancements in correlation modeling, the research community has explored the AUDCE problem from a diverse set of methodological perspectives, reflecting the vibrancy of this field. A significant recent trend is the rise of data-driven methods; for example, a deep learning-based approach was introduced in [18], which combines a preamble detection neural network and a data detection neural network. Another type of advanced signal processing, namely tensor-based methods, has been utilized to handle more complex, high-dimensional scenarios, such as those involving high user mobility [19]. Furthermore, some works have expanded the problem’s scope to include the subsequent data decoding stage, tackling the entire processing chain from detection to decoding within a unified framework [20]. Concurrently, innovation continues in transmission schemes and problem formulations. Novel techniques like differential modulation have been proposed to simplify detection [21], while others have tackled the more challenging blind access problem in demanding mmWave settings [22] or considered broader system-level requirements such as multi-service provisioning [23].

While the aforementioned works have made significant progress by exploiting individual channel correlations, their focus on a single facet of the channel’s complex statistical topology may be insufficient to provide the full descriptive power needed for the challenging conditions of wideband massive MIMO-OFDM systems. Therefore, a notable avenue for performance enhancement lies in more comprehensive modeling of the channel’s statistical properties. The primary limitation of existing models is their partial exploitation of these properties. They typically focus on the correlation within only one of the channel’s components, such as its structure or its values, while treating the other with a simpler, often independent prior. By not simultaneously modeling both the intricate structural patterns and the nuanced value dependencies, these approaches fail to leverage the full statistical richness of the physical channel. The core challenge, therefore, is to develop a unified probabilistic framework that can cohesively model this dual correlation, which involves capturing both the structured correlation within the channel support and the value correlation within the channel gains, all under a single comprehensive prior. In this paper, we address this critical gap by proposing a novel framework for joint AUDCE under the massive MIMO-OFDM system. The main contributions of our work can be summarized as follows:

• To characterize the sparse properties of the channel in the user–angle–delay domain, we construct a novel hierarchical probabilistic structure that includes three state variables: active indication, channel supports, and channel values. The innovation of this model lies in its layered structure, which provides a principled way to factorize the joint prior distribution and cohesively model sparsity at different levels. This model accurately captures the user-domain sparsity through the active indication variable and, critically, utilizes the channel support and channel values variables to enable the modeling of what we term the dual correlation of user channels. This dual correlation refers to the modeling of two distinct statistical phenomena under a unified prior: the structural correlation, which captures the clustered nature of non-zero channel paths using a Markov chain, and the value correlation, which describes the statistical dependency between the gains of adjacent patterns using a coupled Gaussian distribution.
• Based on the proposed system and probability model, the joint active user detection and channel estimation problem is formulated as a Bethe free energy (BFE) minimization problem under hybrid constraints. Through the optimization and reconstruction of the constraint conditions, we propose the correlated hybrid message passing (CHMP) algorithm. The hybrid nature of the algorithm lies in its tailored message passing schedule, which applies exact inference for discrete state variables (active indication, channel supports) and efficient, moment-based approximate inference for continuous variables (channel values). This approach, derived rigorously under our hybrid constraint framework, achieves a favorable balance between estimation accuracy and computational complexity. Furthermore, this algorithm can adaptively update model parameters without prior knowledge of user sparsity or channel prior information.
• Numerical simulations demonstrate that the proposed joint correlation modeling significantly enhances performance. The reason is that, by exploiting these dual correlations, our algorithm gains powerful prior information that regularizes the ill-posed estimation problem. The structural correlation helps to more accurately locate clusters of active channel taps, while the value correlation helps to refine their coefficient estimates, leading to more robust and precise joint detection and estimation.

To provide a clear overview and roadmap for the reader, we summarize the structure of our proposed solution here. We begin by establishing the system model for a grant-free massive MIMO-OFDM network. Then, to capture the channel’s inherent structure, we develop a novel hierarchical probabilistic prior model that characterizes the dual correlations of channel support via Markov chains and channel values via coupled Gaussian distributions. Based on this model, we formulate the joint estimation problem as a BFE minimization task. Finally, we derive the CHMP algorithm as an efficient iterative solution to this problem, which jointly yields the active user set and their corresponding channels.

The remainder of this paper is organized as follows. In Section 2, the system model is presented for grant-free massive random access in massive MIMO-OFDM systems. In Section 3, the probability model is developed by utilizing three types of state variables—active indicators, channel supports, and channel values—to model the equivalent angle–delay domain channel. Section 4 formulates the joint active user detection and channel estimation problem as a constrained BFE minimization problem. Section 5 provides the detailed theoretical foundation of the CHMP algorithm by solving the constrained BFE minimization problem. Section 6 describes the proposed algorithm, including the procedure details and computational complexity analysis, while Section 7 presents the simulation experiment results. Section 8 concludes this paper.

Notation: The boldface lowercase (uppercase) letters denote vectors (matrices). ${\left(·\right)}^{*}$, ${\left(·\right)}^{\mathsf{T}}$, and ${\left(·\right)}^{\mathsf{H}}$ denote the matrix conjugate, transpose, and conjugate transpose, respectively. $⌊·⌋$, $\ln \left(·\right)$, and $\mathrm{Re}\left\{·\right\}$ denote floor, natural logarithm, and real part operations, respectively. $\delta \left(·\right)$ and $\delta \left[·\right]$ denote continuous and discrete delta functions, respectively. $\mathrm{E}\left[·\right]$, $\mathrm{H}\left[·\right]$, and $\mathrm{D}\left[·\right]$ denote the statistical expectation, entropy, and relative entropy, respectively. ⊗ denotes the Kronecker product, ${∥·∥}_2$ denotes the ${\ell }_2$ norm, ∝ denotes the scalar proportionality between two real-valued functions, and $\mathcal{CN}(x;\mu ,\upsilon )$ denotes the complex Gaussian distribution with mean $\mu$ and variance $\upsilon$. For the reader’s convenience, we summarize the key notations used throughout this paper in

Table 1. Table of notations.

Symbol	Definition
System Parameters
M	Number of antennas at the BS
K	Number of potential users
$K_a$	Number of active users
N	Number of effective subcarriers
G	Number of OFDM symbols for access
P	Number of pilot subcarriers
$D,L$	Number of grid points in angle and delay domains
Channel and Signal Representations
${\alpha }_k$	Activity indicator for user k
${\mathbf{h}}_{n,k}^{\mathrm{SF}}$	Space–frequency domain channel vector for user k on subcarrier n
${\mathbf{H}}_k^{\mathrm{SF}}$	Space–frequency domain channel matrix for user k
${\mathbf{H}}_k^{\mathrm{AD}}$	Angle–delay domain channel matrix for user k
${\mathbf{H}}_k^{\mathrm{sf}},{\mathbf{H}}_k^{\mathrm{ad}}$	Equivalent channels (multiplied by activity indicator ${\alpha }_k$)
$\mathbf{y},\mathbf{Y}$	Received vectorized signal and matrix signal
$\mathbf{h}$	Vectorized equivalent angle–delay domain channel for all users
$\mathbf{\Phi }$	Sensing matrix
Probabilistic Model Variables
${\mathbf{S}}_k$	Binary channel support matrix for user k
${\mathbf{G}}_k$	Channel value matrix for user k
${\rho }_k$	Activity probability of user k
${\rho }_{l,k}^{0,1},{\rho }_{l,k}^{1,0}$	Transition probabilities of the Markov chain for channel support
${\omega }_{d,l,k}$	Latent precision of the coupled Gaussian distribution
${\mathbf{\Omega }}_k$	Latent precision matrix for user k
$\gamma ,\tau ,\eta ,\kappa ,\mu$	Lagrange multipliers for different constraints

.

2. System Model

Consider the uplink grant-free random access in a single-cell massive MIMO-OFDM system, as shown in Figure 1. The BS is equipped with a uniform linear array consisting of M antennas and serves K single-antenna users within the cell. Due to the sporadic transmission characteristic of IoT devices, only a small number of users in the potential user set $\mathcal{K}=\left\{1,2,\cdots ,K\right\}$ remain active during uplink access, while the remaining users remain dormant, i.e., $\left|{\mathcal{K}}_{\mathrm{a}}\right|=K_{\mathrm{a}}\ll K$, where ${\mathcal{K}}_{\mathrm{a}}$ denotes the set of active users and ${\mathcal{K}}_{\mathrm{a}}\subseteq \mathcal{K}$ [24]. This section will first present the channel model for the grant-free massive access system and then provide the received signal model based on the sparse channel representation in the transform domain.

2.1. Channel Model

Assume that there are $I_k$ propagation paths from user k to the BS in the wireless scattering environment. The system employs OFDM modulation with $N_{\mathrm{c}}$ subcarriers, where the spacing between adjacent subcarriers is ${\Delta }_f$, and signal transmission occurs over N effective subcarriers. The space–frequency domain channel vector for user k on the n-th effective subcarrier can be expressed as

$$\begin{array}{c}\hfill {\mathbf{h}}_{n,k}^{\mathrm{SF}}=\sum _{i=1}^{I_k}{\beta }_{i,k}\mathbf{a}\left({\varphi }_{i,k}\right)e^{-j2\pi f_n{\tau }_{i,k}},\end{array}$$

(1)

where ${\beta }_{i,k}$ and ${\tau }_{i,k}$ represent the small-scale complex channel gain and propagation delay of the i-th path for user k, respectively, and ${\varphi }_{i,k}=sin{\theta }_{i,k}$, where ${\theta }_{i,k}$ is the angle of arrival of the i-th path for user k. $f_n=\left(n-1-{\displaystyle \frac{N}{2}}\right){\Delta }_f$ denotes the relative frequency of the n-th effective subcarrier with respect to the carrier frequency $f_{\mathrm{c}}$, where $n=1,\cdots ,N$. The array response vector $\mathbf{a}\left({\varphi }_{i,k}\right)\in {\mathbb{C}}^{M\times 1}$ is given by

$$\mathbf{a}\left({\varphi }_{i,k}\right)={\left[1,e^{-j2\pi {\displaystyle \frac{d_{\mathrm{s}}}{\lambda }}{\varphi }_{i,k}},\cdots ,e^{-j2\pi \left(M-1\right){\displaystyle \frac{d_{\mathrm{s}}}{\lambda }}{\varphi }_{i,k}}\right]}^{\mathsf{T}},$$

(2)

where $\lambda$ represents the wavelength of the carrier, and $d_{\mathrm{s}}={\displaystyle \frac{\lambda }{2}}$ denotes the half-wavelength antenna spacing. Thus, the space–frequency domain channel matrix for user k can be constructed from its space–frequency domain channel vectors of the effective subcarriers as follows:

$$\begin{array}{cc}\hfill {\mathbf{H}}_k^{\mathrm{SF}}& =\left[{\mathbf{h}}_{1,k}^{\mathrm{SF}},{\mathbf{h}}_{2,k}^{\mathrm{SF}},\cdots ,{\mathbf{h}}_{N,k}^{\mathrm{SF}}\right]\in {\mathbb{C}}^{M\times N}\hfill \\ \hfill & =\sum _{i=1}^{I_k}{\beta }_{i,k}\mathbf{a}\left({\varphi }_{i,k}\right)\mathbf{b}{\left({\tau }_{i,k}\right)}^{\mathsf{T}},\hfill \end{array}$$

(3)

where the delay response vector $\mathbf{b}\left({\tau }_{i,k}\right)\in {\mathbb{C}}^{N\times 1}$ is given by   

$$\mathbf{b}\left({\tau }_{i,k}\right)={\left[e^{-j2\pi f_1{\tau }_{i,k}},e^{-j2\pi f_2{\tau }_{i,k}},\cdots ,e^{-j2\pi f_N{\tau }_{i,k}}\right]}^{\mathsf{T}}.$$

(4)

Then, by implementing quantization sampling in the angle and delay domains, we can establish the transformation relationship between the space–frequency domain channel and the angle–delay domain channel. Since ${\varphi }_{i,k}=sin{\theta }_{i,k}$, its value ranges within $\left[-1, 1\right]$. Assuming that the cyclic prefix (CP) length of the OFDM system is $N_{\mathrm{cp}}$ system sampling intervals, which exceeds the maximum path delay, the value range of ${\tau }_{i,k}$ is $\left[0, N_{\mathrm{cp}}{T}_{\mathrm{s}}\right]$, where $T_{\mathrm{s}}={\displaystyle \frac{1}{N_{\mathrm{c}}{\Delta }_f}}$ is the system sampling interval. Consider uniformly sampling the value ranges of ${\varphi }_{i,k}$ and ${\tau }_{i,k}$ using D and L grid points, respectively. To ensure sufficient quantization accuracy, the conditions $D\ge M$ and $L\ge N_{\mathrm{cp}}$ should be satisfied. Therefore, the sampled angle ${\varphi }_d$ and sampled delay ${\tau }_l$ can be expressed as

$$\begin{array}{cc}\hfill & {\varphi }_d={\displaystyle \frac{2}{D}}\left(d-1\right)-1,d=1,2,\cdots ,D,\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill & {\tau }_l={\displaystyle \frac{N_{\mathrm{cp}}}{L{N}_{\mathrm{c}}{\Delta }_f}}\left(l-1\right),l=1,2,\cdots ,L.\hfill \end{array}$$

(6)

Next, we can construct the angle domain sampling matrix $\mathbf{A}$ and the delay domain sampling matrix $\mathbf{B}$, which are composed of their respective sampling response vectors and are specifically given by

$$\begin{array}{cc}\hfill & \mathbf{A}=\left[\mathbf{a}\left({\varphi }_1\right),\mathbf{a}\left({\varphi }_2\right),\cdots ,\mathbf{a}\left({\varphi }_D\right)\right]\in {\mathbb{C}}^{M\times D},\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill & \mathbf{B}=\left[\mathbf{b}\left({\tau }_1\right),\mathbf{b}\left({\tau }_2\right),\cdots ,\mathbf{b}\left({\tau }_L\right)\right]\in {\mathbb{C}}^{N\times L}.\hfill \end{array}$$

(8)

With the angle and delay domain sampling matrices $\mathbf{A}$ and $\mathbf{B}$ now defined, we can express the physical space–frequency channel ${\mathbf{H}}_k^{\mathrm{SF}}$ in terms of its angle–delay domain counterpart, which we denote as ${\mathbf{H}}_k^{\mathrm{AD}}\in {\mathbb{C}}^{D\times L}$. Therefore, the space–frequency domain channel matrix ${\mathbf{H}}_k^{\mathrm{SF}}$ in (3) can be re-expressed as

$$\begin{array}{c}\hfill {\mathbf{H}}_k^{\mathrm{SF}}={\mathbf{AH}}_k^{\mathrm{AD}}{\mathbf{B}}^{\mathsf{T}}.\end{array}$$

(9)

2.2. Received Signal Model

Define the active indicator factor ${\alpha }_k\in \left\{0,1\right\}$ to denote the active status of the user k. When $k\in {\mathcal{K}}_a$, we have ${\alpha }_k=1$, indicating that the user is active; conversely, when $k\in \mathcal{K}\setminus {\mathcal{K}}_{\mathrm{a}}$, we have ${\alpha }_k=0$, indicating that the user is dormant. Thus, the received signal at the BS for the t-th OFDM symbol can be given by

$$\begin{array}{cc}\hfill {\tilde{\mathbf{Y}}}_t& =\sum _{k=1}^K{\alpha }_k\sqrt{p_k{\chi }_k}{\mathbf{H}}_k^{\mathrm{SF}}{\tilde{\mathbf{X}}}_{k,t}+{\tilde{\mathbf{N}}}_t\hfill \\ \hfill & =\sum _{k=1}^K\sqrt{{\tilde{\chi }}_k}{\mathbf{H}}_k^{\mathrm{sf}}{\tilde{\mathbf{X}}}_{k,t}+{\tilde{\mathbf{N}}}_t,\hfill \end{array}$$

(10)

where ${\tilde{\mathbf{Y}}}_t\in {\mathbb{C}}^{M\times N}$, ${\mathbf{H}}_k^{\mathrm{sf}}={\alpha }_k{\mathbf{H}}_k^{\mathrm{SF}}$ represents the equivalent space–frequency domain channel for user k; $p_k$ and ${\chi }_k$ denote the signal transmission power and large-scale fading factor of user k when active, respectively; and ${\tilde{\chi }}_k=p_k{\chi }_k$ represents the equivalent large-scale fading factor for user k. ${\tilde{\mathbf{X}}}_{k,t}=\mathrm{diag}\left({\tilde{\mathbf{x}}}_{k,t}\right)\in {\mathbb{C}}^{N\times N}$ denotes the unit energy transmitted signal of the user k, satisfying the power constraint $\mathrm{E}\left[{\tilde{\mathbf{x}}}_{k,t}{\tilde{\mathbf{x}}}_{k,t}^{\mathsf{H}}\right]=\mathbf{I}$, and ${\tilde{\mathbf{N}}}_t$ represents the additive white Gaussian noise matrix with zero mean and variance $\sigma$. Assuming effective power control, each user’s signal transmission power can compensate for channel large-scale fading, resulting in equal equivalent large-scale factors for all users [11,25], i.e., ${\tilde{\chi }}_k=\chi$, $\forall k$. To simplify the discussion, let $\chi =1$. The small-scale channel matrix ${\mathbf{H}}_k^{\mathrm{SF}}$ is assumed to have the same power, i.e., $\mathrm{E}\left[\parallel {\mathbf{H}}_k^{\mathrm{SF}}{\parallel }_{\mathrm{F}}^2\right]=\xi$, $\forall k$. This ensures that the total received signal power from each active user in the BS is equal, avoiding the impact of the near–far effect on the users [26]. Finally, the equivalent space–frequency domain channel matrix for all users is defined as ${\mathbf{H}}^{\mathrm{sf}}=\left[{\mathbf{H}}_1^{\mathrm{sf}},{\mathbf{H}}_2^{\mathrm{sf}},\cdots ,{\mathbf{H}}_K^{\mathrm{sf}}\right]\in {\mathbb{C}}^{M\times NK}$.

Assume that all users transmit uplink access pilots using the same subcarrier set $\mathcal{P}$ when active, with the number of pilot subcarriers being P, embedded uniformly in the N effective subcarriers. The system assigns a unique Gaussian access pilot sequence $\left\{{\mathbf{x}}_{k,t}|t\in \mathcal{T}\right\}$ to each user, where $\mathcal{T}=\left\{t_1,t_2,\cdots ,t_G\right\}$ represents the set of G OFDM symbols used for access, and ${\mathbf{x}}_{k,t}\in {\mathbb{C}}^{P\times 1}$, with elements $x_{p,k,t}$ following the standard complex Gaussian distribution $\mathcal{CN}\left(x_{p,k,t};0,1\right)$. Then, the received pilot signal at the BS for the t-th OFDM symbol, $t\in \mathcal{T}$, can be expressed as

$$\begin{array}{cc}\hfill {\mathbf{Y}}_t& =\sum _{k=1}^K{\mathbf{H}}_k^{\mathrm{sf}}{\tilde{\mathbf{X}}}_{k,t}\mathbf{\Xi }+{\mathbf{N}}_t\hfill \\ \hfill & =\mathbf{A}\sum _{k=1}^K{\mathbf{H}}_k^{\mathrm{ad}}{\mathbf{B}}^{\mathsf{T}}\mathbf{\Xi }{\mathbf{X}}_{k,t}+{\mathbf{N}}_t\hfill \\ \hfill & =\mathbf{A}{\mathbf{H}}^{\mathrm{ad}}\left({\mathbf{I}}_K\otimes \left({\mathbf{B}}^{\mathsf{T}}\mathbf{\Xi }\right)\right){\mathbf{X}}_t+{\mathbf{N}}_t,\hfill \end{array}$$

(11)

where $\mathbf{\Xi }\in {\left\{0,1\right\}}^{N\times P}$ is the index matrix for the pilot subcarriers, composed of columns from ${\mathbf{I}}_N$ corresponding to $\mathcal{P}$. ${\mathbf{Y}}_t={\tilde{\mathbf{Y}}}_t\mathbf{\Xi }\in {\mathbb{C}}^{M\times P}$ and ${\mathbf{N}}_t={\tilde{\mathbf{N}}}_t\mathbf{\Xi }\in {\mathbb{C}}^{M\times P}$ represent the received pilot signal and noise matrix, respectively. ${\mathbf{X}}_{k,t}=\mathrm{diag}\left({\mathbf{x}}_{k,t}\right)\in {\mathbb{C}}^{P\times P}$, and ${\tilde{\mathbf{X}}}_{k,t}\mathbf{\Xi }=\mathbf{\Xi }{\mathbf{X}}_{k,t}$. ${\mathbf{H}}_k^{\mathrm{ad}}={\alpha }_k{\mathbf{H}}_k^{\mathrm{AD}}$ is the equivalent angle–delay domain channel matrix for user k, ${\mathbf{H}}^{\mathrm{ad}}=\left[{\mathbf{H}}_1^{\mathrm{ad}},{\mathbf{H}}_2^{\mathrm{ad}},\cdots ,{\mathbf{H}}_K^{\mathrm{ad}}\right]\in {\mathbb{C}}^{D\times LK}$ is the equivalent angle–delay domain channel matrix for all users, and ${\mathbf{X}}_t={\left[{\mathbf{X}}_{1,t}^{\mathsf{T}},{\mathbf{X}}_{2,t}^{\mathsf{T}},\cdots ,{\mathbf{X}}_{K,t}^{\mathsf{T}}\right]}^{\mathsf{T}}\in {\mathbb{C}}^{PK\times P}$ is the pilot matrix for the t-th OFDM symbol. Therefore, the overall received pilot signal at the BS can be formulated as

$$\begin{array}{cc}\hfill \mathbf{Y}& =\mathbf{A}{\mathbf{H}}^{\mathrm{ad}}\left({\mathbf{I}}_K\otimes \left({\mathbf{B}}^{\mathsf{T}}\mathbf{\Xi }\right)\right)\mathbf{X}+\mathbf{N}\hfill \\ \hfill & =\mathbf{A}{\mathbf{H}}^{\mathrm{ad}}\tilde{\mathbf{B}}+\mathbf{N},\hfill \end{array}$$

(12)

where $\mathbf{Y}=\left[{\mathbf{Y}}_{t_1},{\mathbf{Y}}_{t_2},\cdots ,{\mathbf{Y}}_{t_G}\right]\in {\mathbb{C}}^{M\times PG}$, $\mathbf{X}=\left[{\mathbf{X}}_{t_1},{\mathbf{X}}_{t_2},\cdots ,{\mathbf{X}}_{t_G}\right]\in {\mathbb{C}}^{PK\times PG}$, $\mathbf{N}=\left[{\mathbf{N}}_{t_1},{\mathbf{N}}_{t_2},\cdots ,{\mathbf{N}}_{t_G}\right]\in {\mathbb{C}}^{M\times PG}$, and $\tilde{\mathbf{B}}=\left({\mathbf{I}}_K\otimes \left({\mathbf{B}}^{\mathsf{T}}\mathbf{\Xi }\right)\right)\mathbf{X}\in {\mathbb{C}}^{LK\times PG}$. By vectorizing the received signal $\mathbf{Y}$, (12) can be further transformed into the following matrix–vector multiplication form:

$$\begin{array}{c}\hfill \mathbf{y}=\left({\tilde{\mathbf{B}}}^{\mathsf{T}}\otimes \mathbf{A}\right)\mathbf{h}+\mathbf{n}=\mathbf{\Phi }\mathbf{h}+\mathbf{n},\end{array}$$

(13)

where $\mathbf{y}=\mathrm{vec}\left(\mathbf{Y}\right)\in {\mathbb{C}}^{R\times 1}$, $\mathbf{h}=\mathrm{vec}\left({\mathbf{H}}^{\mathrm{ad}}\right)\in {\mathbb{C}}^{J\times 1}$, $\mathbf{n}=\mathrm{vec}\left(\mathbf{N}\right)\in {\mathbb{C}}^{R\times 1}$, and $\mathbf{\Phi }={\tilde{\mathbf{B}}}^{\mathsf{T}}\otimes \mathbf{A}\in {\mathbb{C}}^{R\times J}$ are the sensing matrices, where $R=MPG$ and $J=DLK$. To simplify subsequent notations, we define $r=\left(g-1\right)MP+\left(p-1\right)M+m$ and $j=\left(k-1\right)DL+\left(l-1\right)D+d$ and also define the subscript ${\left(·\right)}_i={\left(·\right)}_{m,p,g}$ and ${\left(·\right)}_j={\left(·\right)}_{d,l,k}$. It is important to note that, while we assume unique Gaussian pilots, this choice serves as a standard model for the non-orthogonal pilot scenario common in massive access systems [20,23,27] and is widely adopted for its analytical tractability. The specific properties of the pilot sequences are entirely captured within the sensing matrix $\mathbf{\Phi }$. Therefore, the algorithm subsequently proposed in this paper is a general framework that is not restricted to a specific pilot type.

The goal of this paper is to jointly detect the activity of all users $\left\{{\alpha }_k|k\in \mathcal{K}\right\}$ and estimate the space–frequency domain channels of active users $\left\{{\mathbf{H}}_k^{\mathrm{SF}}|k\in {\mathcal{K}}_{\mathrm{a}}\right\}$ using the pilot reception signal $\mathbf{y}$ from the BS and the received signal model (13). Since the user’s activity status is embedded in the equivalent channel, this target is equivalent to estimating the equivalent space–frequency domain channels of all users $\left\{{\mathbf{H}}_k^{\mathrm{sf}}|k\in \mathcal{K}\right\}$ [9]. Furthermore, based on the domain transformation relationship of the channel (9), the space–frequency domain channels $\left\{{\mathbf{H}}_k^{\mathrm{sf}}|k\in \mathcal{K}\right\}$ can be reconstructed by obtaining the equivalent angle–delay domain channels $\left\{{\mathbf{H}}_k^{\mathrm{ad}}|k\in \mathcal{K}\right\}$.

3. Probability Model

This section establishes a probability model to characterize the active sparsity of the equivalent channel in the user dimension and the sparse correlation in the angle–delay domain. Based on the previous system model, we first provide a probability representation of the received signal model. Then, the prior probabilities of the equivalent angle–delay domain channel are modeled using three types of state variables: active indicators, channel support, and channel values. The sparse correlation between channel support and channel values is detailed using a combination of Markov chains and coupled Gaussian distributions. Finally, the corresponding posterior probability representation is constructed for the joint problem of active user detection and channel estimation.

3.1. Received Signal Probability Formulation

Based on the received signal model (13), we begin by expressing the problem in a probabilistic form. The likelihood of the received signal given the channel can be written as a marginalization over an auxiliary noiseless signal variable $\mathbf{z}$:

$$\begin{array}{c}\hfill p\left(\mathbf{y}|\mathbf{h}\right)=\int p\left(\mathbf{y}|\mathbf{z}\right)p\left(\mathbf{z}|\mathbf{h}\right)\mathrm{d}\mathbf{z},\end{array}$$

(14)

where the auxiliary vector $\mathbf{z}=\mathbf{\Phi }\mathbf{h}\in {\mathbb{C}}^{R\times 1}$ represents the BS’s received pilot signal in the absence of noise. Specifically, the conditional probability density $p\left(\mathbf{y}|\mathbf{z}\right)$ can be expressed as

$$\begin{array}{c}\hfill p\left(\mathbf{y}|\mathbf{z}\right)=\prod _{r=1}^R\underset{\stackrel{\Delta }{=}{f}_{Y,r}\left(z_r\right)}{\underbrace{p\left(y_r|z_r\right)}}=\prod _{r=1}^R\mathcal{CN}\left(y_r;z_r,\sigma \right),\end{array}$$

(15)

where $y_r$ and $z_r$ represent the r-th elements of $\mathbf{y}$ and $\mathbf{z}$, respectively, and $f_{Y,r}\left(z_r\right)$ denotes the corresponding factor function form, provided here for ease of subsequent discussion. The conditional probability density $p\left(\mathbf{z}|\mathbf{h}\right)$ is specifically given by

$$\begin{array}{c}\hfill p\left(\mathbf{z}|\mathbf{h}\right)=\prod _{r=1}^R\underset{\stackrel{\Delta }{=}{f}_{Z,r}\left(z_r,\mathbf{h}\right)}{\underbrace{p\left(z_r|\mathbf{h}\right)}}=\prod _{r=1}^R\delta \left(z_r-\sum _{j=1}^J{\phi }_{r,j}{h}_j\right),\end{array}$$

(16)

where ${\phi }_{r,j}$ denotes the $(r,j)$-th element of $\mathbf{\Phi }$, $h_j$ denotes the j-th element of $\mathbf{h}$, and $\delta \left(·\right)$ denotes the continuous Dirac delta function. This representation is a crucial modeling choice with a threefold purpose. Fundamentally, it embeds the deterministic linear constraint $\mathbf{z}=\mathbf{\Phi }\mathbf{h}$ into the probabilistic factor graph. Structurally, this factorization allows the global problem to be decomposed, which facilitates the modular derivation of our algorithm. Most importantly, it is a prerequisite for the subsequent constraint reconstruction within the BFE framework, where the belief associated with this factor is relaxed.

3.2. Prior Model of the Equivalent Channel

To reveal the sparsity characteristic of the equivalent angle–delay domain channel $\left\{{\mathbf{H}}_k^{\mathrm{ad}}={\alpha }_k{\mathbf{H}}_k^{\mathrm{AD}},k\in \mathcal{K}\right\}$ in terms of users, angles, and delays, we further construct the corresponding prior probability models. To model the channel’s inherent sparse structure in the angle–delay domain, we decompose the physical channel matrix into its structural and numerical components using two types of latent state matrices:

$$\begin{array}{c}\hfill {\mathbf{H}}_k^{\mathrm{AD}}={\mathbf{S}}_k\odot {\mathbf{G}}_k,\end{array}$$

(17)

where the channel support ${\mathbf{S}}_k\in {\left\{0,1\right\}}^{D\times L}$ is a binary matrix designed to describe the sparse structure of the channel; the channel value ${\mathbf{G}}_k\in {\mathbb{C}}^{D\times L}$ is a complex matrix representing the numerical response of the channel. Specifically, the element $s_{d,l,k}$ of ${\mathbf{S}}_k$ indicates whether a particular channel element in ${\mathbf{H}}_k^{\mathrm{AD}}$ is zero, while the element $g_{d,l,k}$ of ${\mathbf{G}}_k$ is a complex-valued random variable representing the channel gain of the corresponding non-zero path. Here, the indices $(d,l,k)$ correspond to the d-th angle grid point, the l-th delay grid point, and the k-th user, respectively. Thus, given ${\alpha }_k$, ${\mathbf{S}}_k$, and ${\mathbf{G}}_k$, the conditional probability density of the equivalent channel ${\mathbf{H}}_k^{\mathrm{ad}}$ is expressed as

$$\begin{array}{cc}\hfill p\left({\mathbf{H}}_k^{\mathrm{ad}}|{\alpha }_k,{\mathbf{S}}_k,{\mathbf{G}}_k\right)& =\prod _{d=1}^D\prod _{l=1}^L\underset{\stackrel{\Delta }{=}{f}_{H,d,l,k}\left(h_{d,l,k},{\alpha }_k,s_{d,l,k},g_{d,l,k}\right)}{\underbrace{p\left(h_{d,l,k}|{\alpha }_k,s_{d,l,k},g_{d,l,k}\right)}}\hfill \\ \hfill & =\prod _{d=1}^D\prod _{l=1}^L\delta \left(h_{d,l,k}-{\alpha }_k{s}_{d,l,k}{g}_{d,l,k}\right),\hfill \end{array}$$

(18)

in which ${\mathbf{G}}_k$ and ${\mathbf{S}}_k$ are mutually independent. Next, we will establish probability models for these latent variables separately to fully leverage their sparse characteristics.

The user-domain sparsity is captured by the activity indicator ${\alpha }_k$, which is modeled as a Bernoulli random variable with the following prior probability:

$$\begin{array}{c}\hfill p\left({\alpha }_k;{\rho }_k\right)\stackrel{\Delta }{=}{f}_{A,k}\left({\alpha }_k\right)={{\rho }_k}^{{\alpha }_k}{\left(1-{\rho }_k\right)}^{1-{\alpha }_k},\end{array}$$

(19)

where ${\rho }_k$ denotes the activity probability of user k. Considering the practical finite scattering environment, the angle–delay domain channel exhibits a clustered sparse structure, with the non-zero elements of ${\mathbf{S}}_k$ primarily concentrated in specific angles corresponding to a limited number of scattering paths. Therefore, a Markov chain can be used to capture the clustered characteristics in the angle domain for each path delay. The prior distribution of the channel support ${\mathbf{S}}_k$ can be given by

$$\begin{array}{c}\hfill p\left({\mathbf{S}}_k;{\mathit{\rho }}_k^{0,1},{\mathit{\rho }}_k^{1,0}\right)=\prod _{l=1}^L\left(\underset{\stackrel{\Delta }{=}{f}_{S,1,l,k}\left(s_{1,l,k}\right)}{\underbrace{p\left(s_{1,l,k};{\rho }_{l,k}^{0,1},{\rho }_{l,k}^{1,0}\right)}}\prod _{d=2}^D\underset{\stackrel{\Delta }{=}{f}_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)}{\underbrace{p\left(s_{d,l,k}|s_{d-1,l,k};{\rho }_{l,k}^{0,1},{\rho }_{l,k}^{1,0}\right)}}\right),\end{array}$$

(20)

where ${\rho }_{l,k}^{0,1}$ denotes the transition probability from 1 to 0, and ${\rho }_{l,k}^{1,0}$ denotes the transition probability from 0 to 1. ${\mathit{\rho }}_k^{0,1}={\left[{\rho }_{1,k}^{0,1},{\rho }_{2,k}^{0,1},\cdots ,{\rho }_{L,k}^{0,1}\right]}^{\mathsf{T}}$ and ${\mathit{\rho }}_k^{1,0}={\left[{\rho }_{1,k}^{1,0},{\rho }_{2,k}^{1,0},\cdots ,{\rho }_{L,k}^{1,0}\right]}^{\mathsf{T}}$. For $d=2,\cdots ,D$, the conditional probability distribution $p\left(s_{d,l,k}|s_{d-1,l,k};{\rho }_{l,k}^{0,1},{\rho }_{l,k}^{1,0}\right)$ is specifically written as

$$\begin{array}{cc}\hfill p\left(s_{d,l,k}|s_{d-1,l,k};{\rho }_{l,k}^{0,1},{\rho }_{l,k}^{1,0}\right)& ={\left(1-{\rho }_{l,k}^{0,1}\right)}^{s_{d,l,k}{s}_{d-1,l,k}}{\left({\rho }_{l,k}^{0,1}\right)}^{\left(1-s_{d,l,k}\right)s_{d-1,l,k}}\hfill \\ \hfill & \times {\left({\rho }_{l,k}^{1,0}\right)}^{s_{d,l,k}\left(1-s_{d-1,l,k}\right)}{\left(1-{\rho }_{l,k}^{1,0}\right)}^{\left(1-s_{d,l,k}\right)\left(1-s_{d-1,l,k}\right)}.\hfill \end{array}$$

(21)

When $d=1$, the initial probability distribution $p\left(s_{1,l,k};{\rho }_{l,k}^{0,1},{\rho }_{l,k}^{1,0}\right)$ is modeled as the steady-state distribution of the Markov chain:

$$\begin{array}{c}\hfill p\left(s_{1,l,k};{\rho }_{l,k}^{0,1},{\rho }_{l,k}^{1,0}\right)={\displaystyle \frac{1}{{\rho }_{l,k}^{0,1}+{\rho }_{l,k}^{1,0}}}{\left({\rho }_{l,k}^{1,0}\right)}^{s_{1,l,k}}{\left({\rho }_{l,k}^{0,1}\right)}^{1-s_{1,l,k}}.\end{array}$$

(22)

Finally, to describe the correlation characteristics of adjacent elements in the channel values ${\mathbf{G}}_k$, a coupled Gaussian distribution is used for modeling, which is given by

$$\begin{array}{cc}\hfill p\left({\mathbf{G}}_k|{\mathbf{\Omega }}_k\right)& =\prod _{d=1}^D\prod _{l=1}^L\underset{\stackrel{\Delta }{=}{f}_{G,d,l,k}\left(g_{d,l,k},{\mathit{\omega }}_{d,l,k}\right)}{\underbrace{p\left(g_{d,l,k}|{\mathit{\omega }}_{d,l,k}\right)}}\hfill \\ \hfill & =\prod _{d=1}^D\prod _{l=1}^L\mathcal{CN}\left(g_{d,l,k};0,{\left({\sum }_{(d^{\prime },l^{\prime },k)\in {\mathcal{N}}_{d,l,k}}{\omega }_{d^{\prime },l^{\prime },k}\right)}^{-1}\right),\hfill \end{array}$$

(23)

where ${\omega }_{d,l,k}$ represents the latent precision of the coupled Gaussian distribution, and ${\mathbf{\Omega }}_k\in {\mathbb{R}}^{D\times L}$ is a latent precision matrix aligned with the element positions of ${\mathbf{G}}_k$. The set ${\mathcal{N}}_{d,l,k}$ defines the nearest neighbors and the index of ${\omega }_{d,l,k}$ in the angle–delay domain, and ${\mathit{\omega }}_{d,l,k}$ represents the vector consisting of ${\omega }_{d,l,k}$ and its nearest neighbors. Specifically, when the coordinate of ${\omega }_{d,l,k}$ is inside the angle–delay domain, i.e., $2\le d\le D-1$ and $2\le l\le L-1$, we have ${\mathcal{N}}_{d,l,k}=\left\{\left(d,l,k\right),\left(d-1,l,k\right),\left(d+1,l,k\right),\left(d,l-1,k\right),\left(d,l+1,k\right)\right\}$ and ${\mathit{\omega }}_{d,l,k}={\left[{\omega }_{d,l,k},{\omega }_{d-1,l,k},{\omega }_{d+1,l,k},{\omega }_{d,l-1,k},{\omega }_{d,l+1,k}\right]}^{\mathsf{T}}$. When the coordinate of ${\omega }_{d,l,k}$ is on the edge or corner of the angle–delay domain, the definitions of ${\mathcal{N}}_{d,l,k}$ and ${\mathit{\omega }}_{d,l,k}$ need to be adjusted accordingly. This indicates that the channel value $g_{d,l,k}$ shares the latent precision ${\omega }_{d,l,k}$ with its neighboring elements. The coupling connection of the latent precision illustrates the statistical correlation between channel value responses. Furthermore, the latent precision matrix ${\mathbf{\Omega }}_k$ is further modeled as the following Gamma distribution:

$$\begin{array}{cc}\hfill p\left({\mathbf{\Omega }}_k\right)& =\prod _{d=1}^D\prod _{l=1}^L\underset{\stackrel{\Delta }{=}{f}_{\Omega ,d,l,k}\left({\omega }_{d,l,k}\right)}{\underbrace{p\left({\omega }_{d,l,k}\right)}}=\prod _{d=1}^D\prod _{l=1}^L\mathrm{Ga}\left({\omega }_{d,l,k};a,b\right)\hfill \\ \hfill & =\prod _{d=1}^D\prod _{l=1}^L{\displaystyle \frac{b^a}{\Gamma \left(a\right)}}{\omega }_{d,l,k}^{a-1}{e}^{-b{\omega }_{d,l,k}},\hfill \end{array}$$

(24)

where $\Gamma (·)$ denotes the Gamma function, and a and b are preset hyperparameters corresponding to the shape parameter and rate parameter, respectively. When prior knowledge about random variables is lacking, a and b are often set to very small positive numbers close to zero, such as ${10}^{-20}$, to ensure that the prior does not contain any information [28].

We note that a similar probabilistic structure, which also employs a Bernoulli–Gaussian process and a Markov chain, has been considered in [12] for grant-free NOMA systems. However, there are several fundamental distinctions between our proposed dual correlation model and the model presented in that work. First, our model is tailored to the massive MIMO-OFDM system and thus operates in the user–angle–delay domain, whereas the previous work was focused on a single-antenna BS system. Second, and more critically, the application of the Markov chain is fundamentally different. We employ it to capture the spatial structural correlation of the channel supports across the angular dimension for each delay tap. In contrast, the referenced work uses it to model the temporal correlation of the binary user activity state across time slots. Third, our model introduces a second, distinct layer of correlation by capturing the intrinsic dependency between adjacent channel patterns via a coupled Gaussian distribution, a component that is absent in the model from the other work. Fourth, our primary objective is the fundamental problem of AUDCE, while the objective in the referenced work is data detection. In summary, by jointly modeling both spatial structural and numerical value correlations for a distinct problem, our proposed framework provides a more comprehensive and accurate prior for the complex channels found in wideband massive MIMO systems.

3.3. Probability Representation of the Target Problem

Based on the probability models provided above, the joint posterior probability density function of the equivalent channel, activity indicator, channel support, and channel value can be decomposed as follows:

$$\begin{array}{cc}\hfill & p\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }|\mathbf{y};\mathcal{G}\right)=Z^{-1}p\left(\mathbf{y}|\mathbf{z}\right)p\left(\mathbf{z}|\mathbf{h}\right)\hfill \\ \hfill & \times \prod _{k=1}^K\left(p\left({\mathbf{H}}_k|{\alpha }_k,{\mathbf{S}}_k,{\mathbf{G}}_k\right)p\left({\alpha }_k;{\rho }_k\right)p\left({\mathbf{S}}_k;{\mathit{\rho }}_k^{0,1},{\mathit{\rho }}_k^{1,0}\right)p\left({\mathbf{G}}_k|{\mathbf{\Omega }}_k\right)p\left({\mathbf{\Omega }}_k\right)\right),\hfill \end{array}$$

(25)

where Z is a normalization constant; $\mathit{\alpha }={\left[{\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_K\right]}^{\mathsf{T}}$, $\mathbf{s}=\mathrm{vec}\left(\left[{\mathbf{S}}_1,{\mathbf{S}}_2,\cdots ,{\mathbf{S}}_K\right]\right)$, $\mathbf{g}=\mathrm{vec}\left(\left[{\mathbf{G}}_1,{\mathbf{G}}_2,\cdots ,{\mathbf{G}}_K\right]\right)$, $\mathit{\omega }=\mathrm{vec}\left(\left[{\mathbf{\Omega }}_1,{\mathbf{\Omega }}_2,\cdots ,{\mathbf{\Omega }}_K\right]\right)$, and $\mathcal{G}=\left\{{\rho }_k,{\mathit{\rho }}_k^{0,1},{\mathit{\rho }}_k^{1,0}\mid k\in \mathcal{K}\right\}$ represent the set of model parameters. The joint active user detection and channel estimation can be approached by obtaining the equivalent channel. Based on (25), the minimum mean square error (MMSE) estimator of the equivalent channel $h_{d,l,k}$ can be given by

$$\begin{array}{c}\hfill {\widehat{h}}_{d,l,k}=\int h_{d,l,k}p\left(h_{d,l,k}|\mathbf{y};\mathcal{G}\right)\mathrm{d}{h}_{d,l,k},\end{array}$$

(26)

and the marginal posterior probability density $p\left(h_{d,l,k}|\mathbf{y};\mathcal{G}\right)$ can be specifically expressed as

$$\begin{array}{c}\hfill p\left(h_{d,l,k}|\mathbf{y};\mathcal{G}\right)=\int \sum _{\mathit{\alpha },\mathbf{s}}p\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }|\mathbf{y};\mathcal{G}\right)\mathrm{d}\mathbf{z}\mathrm{d}\mathbf{g}\mathrm{d}\mathit{\omega }\mathrm{d}{\mathbf{h}}_{\setminus h_{d,l,k}}.\end{array}$$

(27)

To clearly demonstrate the relationships between the probabilities, we visualize the factorization of the joint posterior probability $p\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }|\mathbf{y};\mathcal{G}\right)$ in (25), as shown in Figure 2, where blue squares represent factor nodes, while orange circles represent variable nodes. The factor graph in Figure 2 comprises five main modules, corresponding to signal transmission, equivalent channels, activity indications, channel support, and channel values.

Due to the large number of potential users, the factor graph exhibits a complex multi-loop structure, requiring that the exact calculation of the MMSE estimator of the equivalent channel $h_{d,l,k}$ in (26) involves high-dimensional integrals, which are difficult to handle. The next section will transform this complex probability density function integral into a solvable optimization problem based on the constrained BFE minimization theory, combining Bethe approximation and constraint reconstruction methods, thereby laying a solid theoretical foundation for subsequent algorithm design.

4. Constrained BFE Minimization Problem

This section will transform the high-dimensional probability density integral calculation into a constrained BFE minimization problem, laying the theoretical foundation for the hybrid message passing algorithm. First, the original BFE expression for the proposed probability model will be established using the Bethe approximation method. Next, a series of constraints on the beliefs of the free energy will be reconstructed to facilitate algorithm design. Finally, the constrained BFE minimization problem will be formulated.

4.1. Free Energy and Bethe Approximation

To calculate the marginal posterior probability density $p\left(h_{d,l,k}|\mathbf{y};\mathcal{G}\right)$, we first consider the joint posterior probability density $p\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }|\mathbf{y};\mathcal{G}\right)$. To solve for the marginal posterior probabilities, we turn to the framework of variational inference. According to the definition provided in [29,30,31], the variational free energy is given by

$$\begin{array}{cc}\hfill {\mathcal{F}}_{\mathrm{V}}\left(b\right)& =\int \sum _{\mathit{\alpha },\mathbf{s}}b\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }\right)ln{\displaystyle \frac{b\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }\right)}{p\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega },\mathbf{y};\mathcal{G}\right)}}\mathrm{d}\mathbf{z}\mathrm{d}\mathbf{h}\mathrm{d}\mathbf{g}\mathrm{d}\mathit{\omega }\hfill \\ \hfill & =\mathrm{D}\left[b\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }\right)\parallel p\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }|\mathbf{y};\mathcal{G}\right)\right]-lnZ,\hfill \end{array}$$

(28)

where $lnZ$ is the normalization constant. Directly minimizing this free energy is intractable. Therefore, we approximate the true belief with the Bethe approximation, which factorizes the belief $b\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }\right)$ according to the structure of the factor graph:

$$\begin{array}{cc}\hfill b\left(\mathbf{z},\mathbf{h},\mathit{\alpha },\mathbf{s},\mathbf{g},\mathit{\omega }\right)=\underset{\mathrm{signal}\mathrm{transmission}}{\underbrace{\prod _{r=1}^R{\displaystyle \frac{b_{Y,r}\left(z_r\right)b_{Z,r}\left(z_r,\mathbf{h}\right)}{q_{Z,r}\left(z_r\right)}}}}\underset{\mathrm{equivalent}\mathrm{channel}}{\underbrace{\prod _{j=1}^J{\displaystyle \frac{b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)}{{\left(q_{H,j}\left(h_j\right)\right)}^R}}}}\underset{\mathrm{activity}\mathrm{indicator}}{\underbrace{\prod _{k=1}^K{\displaystyle \frac{b_{A,k}\left({\alpha }_k\right)}{{\left(q_{A,k}\left({\alpha }_k\right)\right)}^{DL}}}}}& \hfill \\ \hfill \times \underset{\mathrm{channel}\mathrm{support}}{\underbrace{\prod _{l=1}^L\prod _{k=1}^K{\displaystyle \frac{b_{S,1,l,k}\left(s_{1,l,k}\right){\prod }_{d=2}^D{b}_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)}{q_{S,D,l,k}\left(s_{D,l,k}\right){\prod }_{d=1}^{D-1}{\left(q_{S,d,l,k}\left(s_{d,l,k}\right)\right)}^2}}}}\underset{\mathrm{channel}\mathrm{value}}{\underbrace{\prod _{j=1}^J{\displaystyle \frac{b_{G,j}\left(g_j,{\mathit{\omega }}_j\right)b_{\Omega ,j}\left({\omega }_j\right)}{q_{G,j}\left(g_j\right){\left(q_{\Omega ,j}\left({\omega }_j\right)\right)}^{\left|{\mathcal{N}}_j\right|}}}}}& ,\hfill \end{array}$$

(29)

where $\left|{\mathcal{N}}_j\right|$ denotes the cardinality of the set ${\mathcal{N}}_j$. The factor beliefs and variable beliefs can also be divided into five categories, corresponding to signal transmission, equivalent channels, activity indicators, channel support, and channel values, as listed in

Table 2. Factor beliefs and variable beliefs.

Factor Function	Factor Belief	Variable	Variable Belief
$f_{Y,r}\left(z_r\right)$	$b_{Y,r}\left(z_r\right)$	$z_r$	$q_{Z,r}\left(z_r\right)$
$f_{Z,r}\left(z_r,\mathbf{h}\right)$	$b_{Z,r}\left(z_r,\mathbf{h}\right)$	$h_j$	$q_{H,j}\left(h_j\right)$
$f_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$	$b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$	$h_j$	$q_{H,j}\left(h_j\right)$
$f_{A,k}\left({\alpha }_k\right)$	$b_{A,k}\left({\alpha }_k\right)$	${\alpha }_k$	$q_{A,k}\left({\alpha }_k\right)$
$f_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$	$b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$	$s_{d,l,k}$	$q_{S,d,l,k}\left(s_{d,l,k}\right)$
$f_{G,j}\left(g_j,{\mathit{\omega }}_j\right)$	$b_{G,j}\left(g_j,{\mathit{\omega }}_j\right)$	$g_j$	$q_{G,j}\left(g_j\right)$
$f_{\Omega ,j}\left({\omega }_j\right)$	$b_{\Omega ,j}\left({\omega }_j\right)$	${\omega }_j$	$q_{\Omega ,j}\left({\omega }_j\right)$

.

Moreover, the Bethe approximation requires that the factor beliefs and variable beliefs satisfy the following marginal consistency constraints:

$$\begin{array}{cc}\hfill & q_{Z,r}\left(z_r\right)=b_{Y,r}\left(z_r\right)=\int b_{Z,r}\left(z_r,\mathbf{h}\right)\mathrm{d}\mathbf{h},\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill & q_{H,j}\left(h_j\right)=\int b_{Z,r}\left(z_r,\mathbf{h}\right)\mathrm{d}{z}_r\mathrm{d}{\mathbf{h}}_{\setminus h_j}=\int \sum _{{\alpha }_k,s_j}{b}_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)\mathrm{d}{g}_j,\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill & q_{A,k}\left({\alpha }_k\right)=b_{A,k}\left({\alpha }_k\right)=\int \sum _{s_j}{b}_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)\mathrm{d}{h}_j\mathrm{d}{g}_j,\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill & q_{G,j}\left(g_j\right)=\int b_{G,j}\left(g_j,{\mathit{\omega }}_j\right)\mathrm{d}{\mathit{\omega }}_j=\int \sum _{{\alpha }_k,s_j}{b}_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)\mathrm{d}{h}_j,\hfill \end{array}$$

(33)

$$\begin{array}{cc}\hfill & q_{\Omega ,j}\left({\omega }_j\right)=b_{\Omega ,j}\left({\omega }_j\right)=\int b_{G,j^{\prime }}\left(g_{j^{\prime }},{\mathit{\omega }}_{j^{\prime }}\right)\mathrm{d}{g}_{j^{\prime }}\mathrm{d}{\mathit{\omega }}_{j^{\prime }\setminus j},j^{\prime }\in {\mathcal{N}}_j,\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill q_{S,d,l,k}\left(s_{d,l,k}\right)& =\int \sum _{{\alpha }_k}{b}_{H,d,l,k}\left(h_{d,l,k},{\alpha }_k,s_{d,l,k},g_{d,l,k}\right)\mathrm{d}{h}_{d,l,k}\mathrm{d}{g}_{d,l,k}\hfill \\ & =\sum _{s_{d+1,l,k}}{b}_{S,d+1,l,k}\left(s_{d+1,l,k},s_{d,l,k}\right),d=1,\cdots ,D-1,\hfill \\ & =\left\{\begin{array}{c}{b}_{S,1,l,k}\left(s_{1,l,k}\right),\\ \sum _{s_{d-1,l,k}}{b}_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right),d=2,\cdots ,D,\end{array}\right.\hfill \end{array}$$

(35)

where the constraint (35) uses three-dimensional coordinates $\left(d,l,k\right)$ to clearly demonstrate the relationship between the beliefs. Additionally, the factor beliefs and variable beliefs need to satisfy the following normalization constraints:

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{b_{Y,r}}\left[1\right]={\mathrm{E}}_{q_{Z,r}}\left[1\right]={\mathrm{E}}_{b_{Z,r}}\left[1\right]={\mathrm{E}}_{q_{H,j}}\left[1\right]={\mathrm{E}}_{b_{H,j}}\left[1\right]={\mathrm{E}}_{q_{A,k}}\left[1\right]\hfill \\ \hfill =& {\mathrm{E}}_{b_{A,k}}\left[1\right]={\mathrm{E}}_{q_{S,j}}\left[1\right]={\mathrm{E}}_{b_{S,j}}\left[1\right]={\mathrm{E}}_{q_{G,j}}\left[1\right]={\mathrm{E}}_{b_{\Omega ,j}}\left[1\right]=1,\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{b_{G,j}}\left[1\right]={\mathrm{E}}_{q_{\Omega ,j}}\left[1\right]=1,\hfill \end{array}$$

(37)

where (37) is listed separately to facilitate the discussion in the next subsection. Since the marginal consistency constraints and normalization constraints ensure that the non-negativity constraints of the beliefs are always satisfied, their non-negativity constraints can be omitted. Finally, by substituting the Bethe approximation (29) into the variational free energy expression (28), the specific BFE expression can be obtained as   

$$\begin{array}{cc}\hfill & {\mathcal{F}}_{\mathrm{B}}\left(b,q\right)=\sum _{r=1}^R\left\{\mathrm{D}\left[b_{Y,r}\parallel f_{Y,r}\right]+\mathrm{D}\left[b_{Z,r}\parallel f_{Z,r}\right]+\mathrm{H}\left[q_{Z,r}\right]\right\}+\sum _{j=1}^J\left\{\mathrm{D}\left[b_{H,j}\parallel f_{H,j}\right]+R\mathrm{H}\left[q_{H,j}\right]\right\}\hfill \\ \hfill & +\sum _{k=1}^K\left\{\mathrm{D}\left[b_{A,k}\parallel f_{A,k}\right]+DL\mathrm{H}\left[q_{A,k}\right]\right\}+\sum _{j=1}^J\left\{\mathrm{D}\left[b_{S,j}\parallel f_{S,j}\right]+\left(2-\delta \left[mod\left(j/D\right)\right]\right)\mathrm{H}\left[q_{S,j}\right]\right\}\hfill \\ \hfill & +\sum _{j=1}^J\left\{\mathrm{D}\left[b_{G,j}\parallel f_{G,j}\right]+\mathrm{H}\left[q_{G,j}\right]+\mathrm{D}\left[b_{\Omega ,j}\parallel f_{\Omega ,j}\right]+\left|{\mathcal{N}}_j\right|\mathrm{H}\left[q_{\Omega ,j}\right]\right\}.\hfill \end{array}$$

(38)

Thus, the original BFE minimization problem can be formulated as

$$\begin{array}{c}\hfill \underset{b,q}{min}{\mathcal{F}}_{\mathrm{B}}\left(b,q\right),\mathrm{s}.\mathrm{t}.\left(30\right)–\left(37\right).\end{array}$$

(39)

By solving the problem (39), the optimal solutions for the factor beliefs and variable beliefs can be obtained. Using the optimal variable belief $q_{H,j}^{\star }\left(h_j\right)$, the marginal posterior probability density $p\left(h_j|\mathbf{y};\mathcal{G}\right)$ can be well approximated.

4.2. Constraint Reconstruction

The Bethe approximation introduces auxiliary beliefs through factor relationships in the joint posterior probability density, aiming to maintain the connections between probability factors and to avoid the complex integrals involved in computing the marginal posterior probability densities. However, since the original marginal consistency constraints still contain some high-dimensional integrals, these constraints need to be reconstructed into a more manageable form to reduce the computational burden.

To simplify the factor belief $b_{G,j}\left(g_j,{\mathit{\omega }}_j\right)$, the latent precision ${\omega }_j$ can be assumed to be independent of other variables, which allows it to be separated from $b_{G,j}\left(g_j,{\mathit{\omega }}_j\right)$ through a factorization constraint as

$$\begin{array}{c}\hfill b_{G,j}\left(g_j,{\mathit{\omega }}_j\right)=b_{\tilde{G},j}\left(g_j\right)\prod _{j^{\prime }\in {\mathcal{N}}_j}{q}_{\Omega ,j}\left({\omega }_{j^{\prime }}\right),\end{array}$$

(40)

where $b_{\tilde{G},j}\left(g_j\right)$ is the factor belief after separation. It can be observed that $b_{\tilde{G},j}\left(g_j\right)$ still satisfies the normalization constraint, which is given by

$$\begin{array}{c}\hfill {\mathrm{E}}_{b_{\tilde{G},j}}\left[1\right]=\int b_{\tilde{G},j}\left(g_j\right)\mathrm{d}{g}_j=1.\end{array}$$

(41)

Further, by substituting the factorization constraint (40) into the KL divergence $\mathrm{D}\left[b_{G,j}\parallel f_{G,j}\right]$, we obtain

$$\begin{array}{cc}\hfill & \mathrm{D}\left[b_{G,j}\parallel f_{G,j}\right]=\int b_{\tilde{G},j}\left(g_j\right)\prod _{j^{\prime }\in {\mathcal{N}}_j}{q}_{\Omega ,j^{\prime }}\left({\omega }_{j^{\prime }}\right)ln{\displaystyle \frac{b_{\tilde{G},j}\left(g_j\right){\prod }_{j^{\prime }\in {\mathcal{N}}_j}{q}_{\Omega ,j^{\prime }}\left({\omega }_{j^{\prime }}\right)}{f_{G,j}\left(g_j,{\mathit{\omega }}_j\right)}}\mathrm{d}{g}_j\mathrm{d}{\mathit{\omega }}_j\hfill \\ \hfill & =-\int b_{\tilde{G},j}\left(g_j\right)\prod _{j^{\prime }\in {\mathcal{N}}_j}{q}_{\Omega ,j^{\prime }}\left({\omega }_{j^{\prime }}\right)ln{f}_{G,j}\left(g_j,{\mathit{\omega }}_j\right)\mathrm{d}{g}_j\mathrm{d}{\mathit{\omega }}_j-\mathrm{H}\left[b_{\tilde{G},j}\right]-\sum _{j^{\prime }\in {\mathcal{N}}_j}\mathrm{H}\left[q_{\Omega ,j^{\prime }}\right].\hfill \end{array}$$

(42)

By substituting the marginal consistency constraint (34) and the factorization constraint (40) back into the BFE ${\mathcal{F}}_{\mathrm{B}}\left(b,q\right)$ in (38), the channel value part in the last line of (38) can be rewritten as

$$\begin{array}{cc}\hfill {\mathcal{F}}_{\mathrm{cv}}\left(b,q\right)& =\sum _{j=1}^J\left\{\mathrm{D}\left[b_{G,j}\parallel f_{G,j}\right]+\mathrm{H}\left[q_{G,j}\right]+\mathrm{D}\left[b_{\Omega ,j}\parallel f_{\Omega ,j}\right]+\left|{\mathcal{N}}_j\right|\mathrm{H}\left[q_{\Omega ,j}\right]\right\}\hfill \\ \hfill & =\sum _{j=1}^J\left\{\mathrm{H}\left[q_{G,j}\right]+\mathrm{D}\left[b_{\Omega ,j}\parallel f_{\Omega ,j}\right]-\mathrm{H}\left[b_{\tilde{G},j}\right]\right\}-\sum _{j=1}^J\int b_{\tilde{G},j}\left(g_j\right)\prod _{j^{\prime }\in {\mathcal{N}}_j}{b}_{\Omega ,j^{\prime }}\left({\omega }_{j^{\prime }}\right)ln{f}_{G,j}\left(g_j,{\mathit{\omega }}_j\right)\mathrm{d}{g}_j\mathrm{d}{\mathit{\omega }}_j,\hfill \end{array}$$

(43)

Then, by replacing the BFE ${\mathcal{F}}_{\mathrm{B}}\left(b,q\right)$ in (38) with the expression in (43), it is denoted as ${\mathcal{F}}_{\tilde{\mathrm{B}}}\left(b,q\right)$. For further simplification, following the idea of expectation propagation approximation, the marginal consistency constraint (30) can be relaxed to the following mean and variance consistency constraints:

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{q_{Z,r}}\left[z_r\right]={\mathrm{E}}_{b_{Y,r}}\left[z_r\right]={\mathrm{E}}_{b_{Z,r}}\left[z_r\right],\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill & {\mathrm{Var}}_{q_{Z,r}}\left[z_r\right]={\mathrm{Var}}_{b_{Y,r}}\left[z_r\right]={\mathrm{Var}}_{b_{Z,r}}\left[z_r\right].\hfill \end{array}$$

(45)

For the constraint (31), considering that the factor belief $b_{Z,r}\left(z_r,\mathbf{h}\right)$ under any r includes the variable $h_j$, an average variance consistency constraint can be used to further reduce the complexity. To ensure estimation accuracy, the mean consistency constraint remains unchanged, yielding   

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{q_{H,j}}\left[h_j\right]={\mathrm{E}}_{b_{Z,r}}\left[h_j\right]={\mathrm{E}}_{b_{H,j}}\left[h_j\right],\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill & {\mathrm{Var}}_{q_{H,j}}\left[h_j\right]={\displaystyle \frac{1}{R}}\sum _{r=1}^R{\mathrm{Var}}_{b_{Z,r}}\left[h_j\right]={\mathrm{Var}}_{b_{H,j}}\left[h_j\right],\hfill \end{array}$$

(47)

Similarly, since $g_j$ is a continuous variable, the constraint (33) can be relaxed by applying mean and variance consistency constraints as

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{q_{G,j}}\left[g_j\right]={\mathrm{E}}_{b_{H,j}}\left[g_j\right]={\mathrm{E}}_{b_{\tilde{G},j}}\left[g_j\right],\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill & {\mathrm{Var}}_{q_{G,j}}\left[g_j\right]={\mathrm{Var}}_{b_{H,j}}\left[g_j\right]={\mathrm{Var}}_{b_{\tilde{G},j}}\left[g_j\right].\hfill \end{array}$$

(49)

Since the integration of binary variables is relatively simple, there is no need to relax the constraints (32) and (35). Combining the reconstruction design above, the BFE minimization problem under hybrid constraints can be finally formulated as

$$\begin{array}{cc}\hfill & \underset{b,q,\mathcal{G}}{min}{\mathcal{F}}_{\tilde{\mathrm{B}}}\left(b,q;\mathcal{G}\right),\mathrm{s}.\mathrm{t}.\hfill \\ \hfill & \mathrm{marginal}\mathrm{consistency}:\left(32\right),\left(35\right),\hfill \\ \hfill & \mathrm{normalization}:\left(36\right),\left(41\right),\hfill \\ \hfill & \mathrm{mean}\mathrm{and}\mathrm{variance}\mathrm{consistency}:\left(44\right)–\left(49\right).\hfill \end{array}$$

(50)

where $\mathcal{G}$ is treated as an unknown model parameter and is optimized together with the factor beliefs and variable beliefs in problem (50).

5. Theoretical Foundation of Correlated Hybrid Message Passing

This section aims to solve the aforementioned constrained BFE minimization problem to develop the hybrid message passing algorithm for joint active user detection and channel estimation. In the algorithm design, Lagrange multipliers are first used to weight the constraints, constructing the Lagrangian function. Then, the first-order partial derivatives of the Lagrangian function with respect to the beliefs, model parameters, and Lagrange multipliers are set to zero, yielding a set of stationary point equations. These equations include expressions for the beliefs, model parameters, and hybrid constraints. Finally, by solving the stationary point equations, update rules for the Lagrange multipliers or auxiliary variables can be obtained.

5.1. Lagrangian Function Construction

This subsection introduces the corresponding Lagrange multipliers for the hybrid constraints in problem (50) and constructs the required Lagrangian function accordingly. To solve the constrained minimization problem, we construct the Lagrangian function by introducing Lagrange multipliers for each of the hybrid constraints. The complete Lagrangian function can be decomposed into the following six parts:

$$\begin{array}{c}\hfill {\mathcal{L}}_{\mathrm{B}}={\mathcal{F}}_{\tilde{\mathrm{B}}}+{\mathcal{L}}_{\mathrm{Z}}+{\mathcal{L}}_{\mathrm{H}}+{\mathcal{L}}_{\mathrm{A}}+{\mathcal{L}}_{\mathrm{S}}+{\mathcal{L}}_{\mathrm{G}},\end{array}$$

(51)

where ${\mathcal{F}}_{\tilde{\mathrm{B}}}$ represents the BFE, and ${\mathcal{L}}_{\mathrm{Z}}$, ${\mathcal{L}}_{\mathrm{H}}$, ${\mathcal{L}}_{\mathrm{A}}$, ${\mathcal{L}}_{\mathrm{S}}$, and ${\mathcal{L}}_{\mathrm{G}}$ denote the Lagrangian function components corresponding to the five types of beliefs in (29). Specifically, ${\mathcal{L}}_{\mathrm{Z}}$ corresponds to the signal transmission-type beliefs $b_{Y,r}\left(z_r\right)$, $q_{Z,r}\left(z_r\right)$, and $b_{Z,r}\left(z_r\right)$ with respect to the constraints (44), (45) and the relevant parts of constraint (36), expressed as   

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{Z}}& =\sum _{r=1}^R\left\{2\mathrm{Re}\left\{{\left({\tau }_r^{z,b_Y}\right)}^{*}\left({\mathrm{E}}_{b_{Y,r}}\left[z_r\right]-{\mathrm{E}}_{q_{Z,r}}\left[z_r\right]\right)\right\}+{\eta }_r^{z,b_Y}\left({\mathrm{Var}}_{q_{Z,r}}\left[z_r\right]-{\mathrm{Var}}_{b_{Y,r}}\left[z_r\right]\right)\right\}\hfill \\ \hfill & +\sum _{r=1}^R\left\{2\mathrm{Re}\left\{{\left({\tau }_r^{z,b_Z}\right)}^{*}\left({\mathrm{E}}_{b_{Z,r}}\left[z_r\right]-{\mathrm{E}}_{q_{Z,r}}\left[z_r\right]\right)\right\}+{\eta }_r^{z,b_Z}\left({\mathrm{Var}}_{q_{Z,r}}\left[z_r\right]-{\mathrm{Var}}_{b_{Z,r}}\left[z_r\right]\right)\right\}\hfill \\ \hfill & +\sum _{r=1}^R\left\{{\kappa }_r^{q_Z}\left({\mathrm{E}}_{q_{Z,r}}\left[1\right]-1\right)+{\kappa }_r^{b_Y}\left({\mathrm{E}}_{b_{Y,r}}\left[1\right]-1\right)+{\kappa }_r^{b_Z}\left({\mathrm{E}}_{b_{Z,r}}\left[1\right]-1\right)\right\}.\hfill \end{array}$$

(52)

Here, the symbol $\tau$ represents the Lagrange multipliers for the mean consistency constraints, $\eta$ represents the Lagrange multipliers for the variance consistency constraints, and $\kappa$ represents the Lagrange multipliers for the normalization constraints. Then, ${\mathcal{L}}_{\mathrm{H}}$ corresponds to the constraints on the equivalent channel-type beliefs $q_{H,j}\left(h_j\right)$ and $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ in the relevant parts of constraints (46), (47), and (36), which is given by

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{H}}& =\sum _{j=1}^{J}2\mathrm{Re}\left\{{\left({\tau }_{r,j}^{h,b_Z}\right)}^{*}\left({\mathrm{E}}_{b_{Z,r}}\left[h_j\right]-{\mathrm{E}}_{q_{H,j}}\left[h_j\right]\right)\right\}+\sum _{j=1}^J{\eta }_j^{h,b_Y}\left({\mathrm{RVar}}_{q_{H,j}}\left[h_j\right]-\sum _{r=1}^R{\mathrm{Var}}_{b_{Z,r}}\left[h_j\right]\right)\hfill \\ \hfill & +\sum _{j=1}^{J}2\mathrm{Re}\left\{{\left({\tau }_j^{h,b_H}\right)}^{*}\left({\mathrm{E}}_{b_{H,j}}\left[h_j\right]-{\mathrm{E}}_{q_{H,j}}\left[h_j\right]\right)\right\}+\sum _{j=1}^J{\eta }_j^{h,b_H}\left({\mathrm{Var}}_{q_{H,j}}\left[h_j\right]-{\mathrm{Var}}_{b_{H,j}}\left[h_j\right]\right)\hfill \\ \hfill & +\sum _{j=1}^J\left\{{\kappa }_j^{q_H}\left({\mathrm{E}}_{q_{H,j}}\left[1\right]-1\right)+{\kappa }_j^{b_H}\left({\mathrm{E}}_{b_{H,j}}\left[1\right]-1\right)\right\}.\hfill \end{array}$$

(53)

Next, ${\mathcal{L}}_{\mathrm{A}}$ corresponds to the constraints on the activity indicator-type beliefs $q_{A,k}\left({\alpha }_k\right)$ and $b_{A,k}\left({\alpha }_k\right)$ in the relevant parts of constraints (32) and (36), expressed as

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{A}}& =\sum _{j=1}^J\sum _{{\alpha }_k}{\mu }_j^{\alpha ,b_H}\left({\alpha }_k\right)\left(q_{A,k}\left({\alpha }_k\right)-\int \sum _{s_j}{b}_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)\mathrm{d}{h}_j\mathrm{d}{g}_j\right)\hfill \\ \hfill & +\sum _{k=1}^K\sum _{{\alpha }_k}{\mu }_k^{\alpha ,b_A}\left({\alpha }_k\right)\left(q_{A,k}\left({\alpha }_k\right)-b_{A,k}\left({\alpha }_k\right)\right)+\sum _{k=1}^K\left\{{\kappa }_k^{q_A}\left({\mathrm{E}}_{q_{A,k}}\left[1\right]-1\right)+{\kappa }_k^{b_A}\left({\mathrm{E}}_{b_{A,k}}\left[1\right]-1\right)\right\},\hfill \end{array}$$

(54)

where the symbol $\mu$ represents the Lagrange multipliers for the marginal consistency constraints. Further, ${\mathcal{L}}_{\mathrm{S}}$ corresponds to the constraints on the channel support beliefs $q_{S,d,l,k}\left(s_{d,l,k}\right)$ and $b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$ in the relevant parts of constraints (35) and (36), which can be specifically written as   

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{S}}& =\sum _{d,l,k}\sum _{s_{d,l,k}}{\mu }_{d,l,k}^{s,b_H}\left(s_{d,l,k}\right)\left(q_{S,d,l,k}\left(s_{d,l,k}\right)-\int \sum _{{\alpha }_k}{b}_{H,d,l,k}\mathrm{d}{h}_{d,l,k}\mathrm{d}{g}_{d,l,k}\right)\hfill \\ \hfill & +\sum _{l,k}\sum _{s_{1,l,k}}{\mu }_{1,l,k}^{s,b_S}\left(s_{1,l,k}\right)\left(q_{S,1,l,k}\left(s_{1,l,k}\right)-b_{S,1,l,k}\left(s_{1,l,k}\right)\right)\hfill \\ \hfill & +\sum _{l,k}\sum _{d=2}^D\sum _{s_{d,l,k}}{\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)\left(q_{S,d,l,k}\left(s_{d,l,k}\right)-\sum _{s_{d-1,l,k}}{b}_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)\right)\hfill \\ \hfill & +\sum _{l,k}\sum _{d=1}^{D-1}\sum _{s_{d,l,k}}{\mu }_{d,l,k}^{s,b_S+}\left(s_{d,l,k}\right)\left(q_{S,d,l,k}\left(s_{d,l,k}\right)-\sum _{s_{d+1,l,k}}{b}_{S,d+1,l,k}\left(s_{d+1,l,k},s_{d,l,k}\right)\right)\hfill \\ \hfill & +\sum _{d,l,k}\left\{{\kappa }_{d,l,k}^{q_S}\left({\mathrm{E}}_{q_{S,d,l,k}}\left[1\right]-1\right)+{\kappa }_{d,l,k}^{b_S}\left({\mathrm{E}}_{b_{S,d,l,k}}\left[1\right]-1\right)\right\}.\hfill \end{array}$$

(55)

Finally, ${\mathcal{L}}_{\mathrm{G}}$ corresponds to the constraints on the channel value-type beliefs $q_{G,j}\left(g_j\right)$, $b_{\tilde{G},j}\left(g_j\right)$, and $q_{\Omega ,j}\left({\omega }_j\right)$ in the relevant parts of constraints (41), (46), (47), and (36), given in   

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{G}}& =\sum _{j=1}^J\left\{2\mathrm{Re}\left\{{\left({\tau }_j^{g,b_H}\right)}^{*}\left({\mathrm{E}}_{q_{G,j}}\left[g_j\right]-{\mathrm{E}}_{b_{H,j}}\left[g_j\right]\right)\right\}+{\eta }_j^{g,b_H}\left({\mathrm{Var}}_{b_{H,j}}\left[g_j\right]-{\mathrm{Var}}_{q_{G,j}}\left[g_j\right]\right)\right\}\hfill \\ \hfill & =\sum _{j=1}^J\left\{2\mathrm{Re}\left\{{\left({\tau }_j^{g,b_{\tilde{G}}}\right)}^{*}\left({\mathrm{E}}_{q_{G,j}}\left[g_j\right]-{\mathrm{E}}_{b_{\tilde{G},j}}\left[g_j\right]\right)\right\}+{\eta }_j^{g,b_{\tilde{G}}}\left({\mathrm{Var}}_{b_{\tilde{G},j}}\left[g_j\right]-{\mathrm{Var}}_{q_{G,j}}\left[g_j\right]\right)\right\}\hfill \\ \hfill & +\sum _{j=1}^J\left\{{\kappa }_j^{q_G}\left({\mathrm{E}}_{q_{G,j}}\left[1\right]-1\right)+{\kappa }_j^{b_{\tilde{G}}}\left({\mathrm{E}}_{b_{\tilde{G},j}}\left[1\right]-1\right)+{\kappa }_j^{b_{\Omega }}\left({\mathrm{E}}_{q_{\Omega ,j}}\left[1\right]-1\right)\right\}.\hfill \end{array}$$

(56)

5.2. Belief Representation

For the Lagrangian function (51), by setting the first-order partial derivatives with respect to the auxiliary beliefs and model parameters to zero and solving, we derive the expressions for the factor beliefs, variable beliefs, and model parameters. By combining these expressions with the hybrid constraints, the stationary point equations can be formulated. These equations can be solved according to the modular structure of the factor graph in Figure 2, leading to the hybrid message passing algorithm. Due to the connections between probabilities, some beliefs are shared among modules. For ease of reference, we next list the specific representations of the factor beliefs and variable beliefs.

First, the specific expressions for the signal transmission-type beliefs $b_{Y,r}\left(z_r\right)$, $q_{Z,r}\left(z_r\right)$, and $b_{Z,r}\left(z_r\right)$ are provided. The factor belief $b_{Y,r}\left(z_r\right)$ is given by

$$\begin{array}{c}\hfill b_{Y,r}\left(z_r\right)\propto f_{Y,r}\left(z_r\right)\mathcal{CN}\left(z_r;\underset{\stackrel{\Delta }{=}{\xi }_r^{z,b_Y}}{\underbrace{{\mathrm{E}}_{b_{Y,r}}\left[z_r\right]+{\displaystyle \frac{{\tau }_r^{z,b_Y}}{{\eta }_r^{z,b_Y}}}}},{\displaystyle \frac{1}{{\eta }_r^{z,b_Y}}}\right),\end{array}$$

(57)

and the variable belief $q_{Z,r}\left(z_r\right)$ is expressed as

$$\begin{array}{c}\hfill q_{Z,r}\left(z_r\right)=\mathcal{CN}\left(z_r;{\mathrm{E}}_{q_{Z,r}}\left[z_r\right]+{\displaystyle \frac{{\tau }_r^{z,b_Y}+{\tau }_r^{z,b_Z}}{{\eta }_r^{z,b_Y}+{\eta }_r^{z,b_Z}}},{\displaystyle \frac{1}{{\eta }_r^{z,b_Y}+{\eta }_r^{z,b_Z}}}\right),\end{array}$$

(58)

while the factor belief $b_{Z,r}\left(z_r,\mathbf{h}\right)$ is expressed as

$$\begin{array}{cc}\hfill b_{Z,r}\left(z_r,\mathbf{h}\right)& \propto f_{Z,r}\left(z_r,\mathbf{h}\right)\mathcal{CN}\left(z_r;\underset{\stackrel{\Delta }{=}{\xi }_r^{z,b_Z}}{\underbrace{{\mathrm{E}}_{b_{Z,r}}\left[z_r\right]+{\displaystyle \frac{{\tau }_r^{z,b_Z}}{{\eta }_r^{z,b_Z}}}}},{\displaystyle \frac{1}{{\eta }_r^{z,b_Z}}}\right)\mathcal{CN}\left(h_j;\underset{\stackrel{\Delta }{=}{\xi }_{r,j}^{h,b_Z}}{\underbrace{{\mathrm{E}}_{b_{Z,r}}\left[h_j\right]+{\displaystyle \frac{{\tau }_{r,j}^{h,b_Z}}{{\eta }_j^{h,b_Z}}}}},{\displaystyle \frac{1}{{\eta }_j^{h,b_Z}}}\right).\hfill \end{array}$$

(59)

Next, the specific expressions for the equivalent channel-type beliefs $q_{H,j}\left(h_j\right)$ and $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ are shown below. The variable belief $q_{H,j}\left(h_j\right)$ is given by

$$\begin{array}{c}\hfill q_{H,j}\left(h_j\right)=\mathcal{CN}\left(h_j;{\mathrm{E}}_{q_{H,j}}\left[h_j\right]+{\displaystyle \frac{{\sum }_{r=1}^R{\tau }_{r,j}^{h,b_Z}+{\tau }_j^{h,b_H}}{R{\eta }_j^{h,b_Z}+{\eta }_j^{h,b_H}}},{\displaystyle \frac{R}{R{\eta }_j^{h,b_Z}+{\eta }_j^{h,b_H}}}\right),\end{array}$$

(60)

and the factor belief $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ is given by

$$\begin{array}{cc}\hfill b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)& \propto f_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)\mathcal{CN}\left(h_j;\underset{\stackrel{\Delta }{=}{\xi }_j^{h,b_H}}{\underbrace{{\mathrm{E}}_{b_{H,j}}\left[h_j\right]+{\displaystyle \frac{{\tau }_j^{h,b_H}}{{\eta }_j^{h,b_H}}}}},{\displaystyle \frac{1}{{\eta }_j^{h,b_H}}}\right)\hfill \\ \hfill & \times exp\left({\mu }_j^{\alpha ,b_H}\left({\alpha }_k\right)+{\mu }_j^{s,b_H}\left(s_j\right)\right)\mathcal{CN}\left(g_j;\underset{\stackrel{\Delta }{=}{\xi }_j^{g,b_H}}{\underbrace{{\mathrm{E}}_{b_{H,j}}\left[g_j\right]+{\displaystyle \frac{{\tau }_j^{g,b_H}}{{\eta }_j^{g,b_H}}}}},{\displaystyle \frac{1}{{\eta }_j^{g,b_H}}}\right).\hfill \end{array}$$

(61)

Then, we present the specific expressions for the activity indicator beliefs $q_{A,k}\left({\alpha }_k\right)$ and $b_{A,k}\left({\alpha }_k\right)$. The variable belief $q_{A,k}\left({\alpha }_k\right)$ is given by

$$\begin{array}{c}\hfill q_{A,k}\left({\alpha }_k\right)\propto exp\left({\displaystyle \frac{1}{DL}}\left({\mu }_k^{\alpha ,b_A}\left({\alpha }_k\right)+\sum _{d,l}{\mu }_{d,l,k}^{\alpha ,b_H}\left({\alpha }_k\right)\right)\right),\end{array}$$

(62)

and the factor belief $b_{A,k}\left({\alpha }_k\right)$ is expressed as

$$\begin{array}{c}\hfill b_{A,k}\left({\alpha }_k\right)\propto f_{A,k}\left({\alpha }_k\right)exp\left({\mu }_k^{\alpha ,b_A}\left({\alpha }_k\right)\right).\end{array}$$

(63)

Furthermore, the specific expressions for the channel support-type beliefs $q_{S,d,l,k}\left(s_{d,l,k}\right)$ and $b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$ are provided. The variable belief $q_{S,d,l,k}\left(s_{d,l,k}\right)$ is given by

$$\begin{array}{c}\hfill q_{S,d,l,k}\left(s_{d,l,k}\right)\propto \left\{\begin{array}{c}exp\left({\displaystyle \frac{1}{2}}\left({\mu }_{d,l,k}^{s,b_H}\left(s_{d,l,k}\right)+{\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)+{\mu }_{d,l,k}^{s,b_S+}\left(s_{d,l,k}\right)\right)\right),\hfill \\ \hspace{1em} d=1,\cdots ,D-1,\hfill \\ exp\left({\mu }_{d,l,k}^{s,b_H}\left(s_{d,l,k}\right)+{\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)\right),d=D.\hfill \end{array}\right.\end{array}$$

(64)

When $d=2,\cdots ,D$, the factor belief $b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$ is written as

$$\begin{array}{c}\hfill b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)\propto f_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)exp\left({\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)+{\mu }_{d-1,l,k}^{s,b_S+}\left(s_{d-1,l,k}\right)\right).\end{array}$$

(65)

When $d=1$, the factor belief $b_{S,1,l,k}\left(s_{1,l,k}\right)$ is given by

$$\begin{array}{c}\hfill b_{S,1,l,k}\left(s_{1,l,k}\right)\propto f_{S,1,l,k}\left(s_{1,l,k}\right)exp\left({\mu }_{1,l,k}^{s,b_S}\left(s_{1,l,k}\right)\right).\end{array}$$

(66)

Finally, the specific expressions for the channel value beliefs $q_{G,j}\left(g_j\right)$, $b_{G,j}^{\prime }\left(g_j\right)$, and $b_{\Omega ,j}\left({\omega }_j\right)$ are given. The variable belief $q_{G,j}\left(g_j\right)$ is expressed as

$$\begin{array}{c}\hfill q_{G,j}\left(g_j\right)=\mathcal{CN}\left(g_j;{\mathrm{E}}_{q_{G,j}}\left[g_j\right]+{\displaystyle \frac{{\tau }_j^{g,b_H}+{\tau }_j^{g,b_{\tilde{G}}}}{{\eta }_j^{g,b_H}+{\eta }_j^{g,b_{\tilde{G}}}}},{\displaystyle \frac{1}{{\eta }_j^{g,b_H}+{\eta }_j^{g,b_{\tilde{G}}}}}\right),\end{array}$$

(67)

the factor belief $b_{\tilde{G},j}\left(g_j\right)$ is given by   

$$\begin{array}{c}\hfill b_{\tilde{G},j}\left(g_j\right)\propto \mathcal{CN}\left(g_j;\underset{\stackrel{\Delta }{=}{\xi }_j^{g,b_{\tilde{G}}}}{\underbrace{{\mathrm{E}}_{b_{\tilde{G},j}}\left[g_j\right]+{\displaystyle \frac{{\tau }_j^{g,b_{\tilde{G}}}}{{\eta }_j^{g,b_{\tilde{G}}}}}}},{\displaystyle \frac{1}{{\eta }_j^{g,b_{\tilde{G}}}}}\right)exp\left(\int \prod _{j^{\prime }\in {\mathcal{N}}_j}{b}_{\Omega ,j^{\prime }}\left({\omega }_{j^{\prime }}\right)ln{f}_{G,j}\left(g_j,{\mathit{\omega }}_j\right)\mathrm{d}{\mathit{\omega }}_j\right),\end{array}$$

(68)

and the factor belief $b_{\Omega ,j}\left({\omega }_j\right)$ is given by   

$$\begin{array}{c}\hfill b_{\Omega ,j}\left({\omega }_j\right)\propto f_{\Omega ,j}\left({\omega }_j\right)\prod _{j^{\prime }\in {\mathcal{N}}_j}exp\left(\int b_{\tilde{G},j^{\prime }}\left(g_{j^{\prime }}\right)\prod _{j^{\prime \prime }\in {\mathcal{N}}_{j^{\prime }\setminus j}}{b}_{\Omega ,j^{\prime \prime }}\left({\omega }_{j^{\prime \prime }}\right)ln{f}_{G,j^{\prime }}\left(g_{j^{\prime }},{\mathit{\omega }}_{j^{\prime }}\right)\mathrm{d}{g}_{j^{\prime }}\mathrm{d}{\mathit{\omega }}_{j^{\prime }\setminus j}\right).\end{array}$$

(69)

5.3. Signal Transmission Module

This subsection will detail the message passing process of the signal transmission module. By simplifying the mean expression of the factor belief $b_{Y,r}\left(z_r\right)$ in (57), we can obtain

$$\begin{array}{c}\hfill {\mathrm{E}}_{b_{Y,r}}\left[z_r\right]=y_r+\sigma {\tau }_r^{z,b_Y}.\end{array}$$

(70)

Based on the mean expression of the variable belief $q_{Z,r}\left(z_r\right)$ in (58), we can obtain ${\tau }_r^{z,b_Y}=-{\tau }_r^{z,b_Z}$. By substituting the factor belief $b_{Y,r}\left(z_r\right)$ from (57) and the variable belief $q_{Z,r}\left(z_r\right)$ from (58) into the variance consistency constraint ${\mathrm{Var}}_{q_{Z,r}}\left[z_r\right]={\mathrm{Var}}_{b_{Y,r}}\left[z_r\right]$ in (45), we get ${\eta }_r^{z,b_Z}=1/\sigma$. Combining the equations ${\tau }_r^{z,b_Y}=-{\tau }_r^{z,b_Z}$, ${\eta }_r^{z,b_Z}=1/\sigma$, and (70), the auxiliary mean ${\xi }_r^{z,b_Z}$ in (59) can be rewritten as

$$\begin{array}{c}\hfill {\xi }_r^{z,b_Z}=-{\tau }_r^{z,b_Y}\sigma +{\mathrm{E}}_{b_{Y,r}}\left[z_r\right]=y_r.\end{array}$$

(71)

Then, based on the variance expression of the variable belief $q_{H,j}\left(h_j\right)$ in (60), the variance consistency constraint ${\mathrm{Var}}_{q_{H,j}}\left[h_j\right]={\mathrm{Var}}_{b_{H,j}}\left[h_j\right]$ in (47) can be re-expressed as

$$\begin{array}{c}\hfill {\eta }_j^{h,b_Z}=1/{\mathrm{Var}}_{b_{H,j}}\left[h_j\right]-{\displaystyle \frac{{\eta }_j^{h,b_H}}{R}}.\end{array}$$

(72)

By substituting the variable belief $q_{Z,r}\left(z_r\right)$ from (58) and the factor belief $b_{Z,r}\left(z_r,\mathbf{h}\right)$ from (59) into the variance consistency constraint ${\mathrm{Var}}_{q_{Z,r}}\left[z_r\right]={\mathrm{Var}}_{b_{Z,r}}\left[z_r\right]$ in (45), and further simplifying this constraint, we can obtain

$$\begin{array}{c}\hfill 1/{\eta }_r^{z,b_Y}=\sum _{j=1}^J{\left|{\phi }_{r,j}\right|}^2/{\eta }_j^{h,b_Z}.\end{array}$$

(73)

Next, based on the mean consistency constraints (44) and (46), the auxiliary mean ${\xi }_r^{z,b_Y}$ in (57) can be given by

$$\begin{array}{c}\hfill {\xi }_r^{z,b_Y}={\displaystyle \frac{{\tau }_r^{z,b_Y}}{{\eta }_r^{z,b_Y}}}+\sum _{j=1}^J{\phi }_{r,j}{\mathrm{E}}_{b_{H,j}}\left[h_j\right].\end{array}$$

(74)

According to the definitions of the auxiliary mean ${\xi }_r^{z,b_Y}$ in Equation (57) and the auxiliary mean ${\xi }_r^{z,b_Z}$ in (59), we have

$$\begin{array}{c}\hfill {\tau }_r^{z,b_Y}={\displaystyle \frac{{\xi }_r^{z,b_Y}-{\xi }_r^{z,b_Z}}{1/{\eta }_r^{z,b_Y}+1/{\eta }_r^{z,b_Z}}}={\displaystyle \frac{{\xi }_r^{z,b_Y}-y_r}{1/{\eta }_r^{z,b_Y}+\sigma }}.\end{array}$$

(75)

Then, based on the factor belief $b_{Z,r}\left(z_r,\mathbf{h}\right)$ in Equation (59), we calculate the mean expressions ${\mathrm{E}}_{b_{Y,r}}\left[z_r\right]$ and ${\mathrm{E}}_{b_{Y,r}}\left[h_j\right]$. By connecting and simplifying these means, the relationship between the Lagrange multipliers ${\tau }_{r,j}^{h,b_Z}$ and ${\tau }_r^{z,b_Y}$ can be derived as follows:

$$\begin{array}{c}\hfill {\tau }_{r,j}^{h,b_Z}=-{\phi }_{r,j}^{*}{\tau }_r^{z,b_Z}={\phi }_{r,j}^{*}{\tau }_r^{z,b_Y},\end{array}$$

(76)

Furthermore, by substituting the factor belief $b_{Z,r}\left(z_r,\mathbf{h}\right)$ from (59) and the variable belief $q_{H,j}\left(h_j\right)$ from (60) into the variance consistency constraint ${\mathrm{Var}}_{q_{H,j}}\left[h_j\right]={\displaystyle \frac{1}{R}}{\sum }_{r=1}^R{\mathrm{Var}}_{b_{Z,r}}\left[h_j\right]$ in (47) and simplifying, we derive

$$\begin{array}{c}\hfill 1/{\eta }_j^{h,b_H}=1/\left(\sum _{r=1}^R{\displaystyle \frac{{\left|{\phi }_{r,j}\right|}^2}{1/{\eta }_r^{z,b_Y}+\sigma }}\right)-{\displaystyle \frac{1}{R{\eta }_j^{h,b_Z}}},\end{array}$$

(77)

Finally, based on the mean expression of the variable belief $q_{H,j}\left(h_j\right)$ in Equation (60), we obtain ${\tau }_j^{h,b_H}=-{\sum }_{r=1}^R{\tau }_{r,j}^{h,b_Z}$. Combining this with (76), the auxiliary mean ${\xi }_j^{h,b_H}$ in (61) can be reconstructed as

$$\begin{array}{c}\hfill {\xi }_j^{h,b_H}={\mathrm{E}}_{b_{H,j}}\left[h_j\right]-{\displaystyle \frac{1}{{\eta }_j^{h,b_H}}}\sum _{r=1}^R{\phi }_{r,j}^{*}{\tau }_r^{z,b_Y}.\end{array}$$

(78)

5.4. Equivalent Channel Module

This subsection will detail the message passing process of the equivalent channel module. Based on the factor belief $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ in (61), the  auxiliary messages are introduced as follows:

$$\begin{array}{cc}\hfill & v_j^{\alpha ,b_H}\left({\alpha }_k\right)\propto exp\left({\mu }_j^{\alpha ,b_H}\left({\alpha }_k\right)\right)\stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp\left(-{\gamma }_j^{\alpha ,b_H}\right)}}\delta \left[{\alpha }_k-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_j^{\alpha ,b_H}\right)}}\delta \left[{\alpha }_k\right],\hfill \end{array}$$

(79)

$$\begin{array}{cc}\hfill & v_j^{s,b_H}\left(s_j\right)\propto exp\left({\mu }_j^{s,b_H}\left(s_j\right)\right)\stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp\left(-{\gamma }_j^{s,b_H}\right)}}\delta \left[s_j-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_j^{s,b_H}\right)}}\delta \left[s_j\right],\hfill \end{array}$$

(80)

where ${\gamma }_j^{\alpha ,b_H}=ln{\displaystyle \frac{v_j^{\alpha ,b_H}\left({\alpha }_k=1\right)}{v_j^{\alpha ,b_H}\left({\alpha }_k=0\right)}}$ and ${\gamma }_j^{s,b_H}=ln{\displaystyle \frac{v_j^{s,b_H}\left(s_j=1\right)}{v_j^{s,b_H}\left(s_j=0\right)}}$ are the log-likelihood ratios of $v_j^{\alpha ,b_H}\left({\alpha }_k\right)$ and $v_j^{s,b_H}\left(s_j\right)$, respectively. Here, the subscript ${\left(·\right)}_j$ is equivalent to the subscript ${\left(·\right)}_{d,l,k}$. With these definitions, the factor belief $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ in (61) can be reformulated as

$$\begin{array}{cc}\hfill & b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)\hfill \\ \hfill & \propto \delta \left(h_j-{\alpha }_k{s}_j{g}_j\right)\mathcal{C}\mathcal{N}\left(h_j;{\xi }_j^{h,b_H},1/{\eta }_j^{h,b_H}\right)v_j^{\alpha ,b_H}\left({\alpha }_k\right)v_j^{s,b_H}\left(s_j\right)\mathcal{C}\mathcal{N}\left(g_j;{\xi }_j^{g,b_H},1/{\eta }_j^{g,b_H}\right).\hfill \end{array}$$

(81)

First, we present the derivation related to $h_j$ in the equivalent channel module. By integrating the factor belief $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ in (81) over all variables except $h_j$, this yields the following equation:

$$\begin{array}{c}\hfill \int \sum _{{\alpha }_k,s_j}{b}_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)\mathrm{d}{g}_j={\tilde{\rho }}_j^{b_H}\mathcal{CN}\left(h_j;{\tilde{\xi }}_j^{b_H},{\tilde{\lambda }}_j^{b_H}\right)+\left(1-{\tilde{\rho }}_j^{b_H}\right)\delta \left(h_j\right),\end{array}$$

(82)

where ${\tilde{\rho }}_j^{b_H}$, ${\tilde{\xi }}_j^{b_H}$, and ${\tilde{\lambda }}_j^{b_H}$ are expressed as   

$$\begin{array}{cc}\hfill {\tilde{\rho }}_j^{b_H}& =1/\{1+\left(exp\left(-{\gamma }_j^{\alpha ,b_H}\right)+exp\left(-{\gamma }_j^{s,b_H}\right)+exp\left(-{\gamma }_j^{\alpha ,b_H}-{\gamma }_j^{s,b_H}\right)\right)\mathcal{C}\mathcal{N}\left(0;{\xi }_j^{h,b_H},1/{\eta }_j^{h,b_H}\right)\hfill \\ & /\mathcal{C}\mathcal{N}\left(0;{\xi }_j^{h,b_H}-{\xi }_j^{g,b_H},1/{\eta }_j^{h,b_H}+1/{\eta }_j^{g,b_H}\right)\},\hfill \end{array}$$

(83)

$$\begin{array}{cc}\hfill & {\tilde{\xi }}_j^{b_H}={\displaystyle \frac{{\xi }_j^{h,b_H}{\eta }_j^{h,b_H}+{\xi }_j^{g,b_H}{\eta }_j^{g,b_H}}{{\eta }_j^{h,b_H}+{\eta }_j^{g,b_H}}},\hfill \end{array}$$

(84)

$$\begin{array}{cc}\hfill & {\tilde{\lambda }}_j^{b_H}=1/\left({\eta }_j^{h,b_H}+{\eta }_j^{g,b_H}\right).\hfill \end{array}$$

(85)

Thus, the mean ${\mathrm{E}}_{b_{H,j}}\left[h_j\right]$ and the variance ${\mathrm{Var}}_{b_{H,j}}\left[h_j\right]$ can be calculated as

$$\begin{array}{c}{\mathrm{E}}_{b_{H,j}}\left[h_j\right]={\tilde{\rho }}_j^{b_H}{\tilde{\xi }}_j^{b_H},\end{array}$$

(86)

$$\begin{array}{c}{\mathrm{Var}}_{b_{H,j}}\left[h_j\right]={\tilde{\rho }}_j^{b_H}\left(1-{\tilde{\rho }}_j^{b_H}\right){\left|{\tilde{\xi }}_j^{b_H}\right|}^2+{\tilde{\rho }}_j^{b_H}{\tilde{\lambda }}_j^{b_H}.\end{array}$$

(87)

Then, we provide the derivation related to ${\alpha }_k$ in the equivalent channel module. Based on (62) for the variable belief $q_{A,k}\left({\alpha }_k\right)$, the auxiliary message $v_j^{b_H,\alpha }\left({\alpha }_k\right)$ is defined as

$$\begin{array}{cc}\hfill & v_j^{b_H,\alpha }\left({\alpha }_k\right)\hfill \\ \hfill & \stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp\left(-{\gamma }_j^{b_H,\alpha }\right)}}\delta \left[{\alpha }_k-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_j^{b_H,\alpha }\right)}}\delta \left[{\alpha }_k\right]\hfill \\ \hfill & \propto exp\left({\displaystyle \frac{1}{DL}}\left({\mu }_k^{\alpha ,b_A}\left({\alpha }_k\right)+\sum _{d^{\prime },l^{\prime }}{\mu }_{d^{\prime },l^{\prime },k}^{\alpha ,b_H}\left({\alpha }_k\right)\right)-{\mu }_{d,l,k}^{\alpha ,b_H}\left({\alpha }_k\right)\right),\hfill \end{array}$$

(88)

where ${\gamma }_j^{b_H,\alpha }=ln{\displaystyle \frac{v_j^{b_H,\alpha }\left({\alpha }_k=1\right)}{v_j^{b_H,\alpha }\left({\alpha }_k=0\right)}}$ is the log-likelihood ratio of $v_j^{b_H,\alpha }\left({\alpha }_k\right)$. By substituting the variable belief $q_{A,k}\left({\alpha }_k\right)$ in (62) and the factor belief $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ in (81) into the marginal consistency constraint in (32) and simplifying this constraint, we can obtain

$$\begin{array}{cc}\hfill & {\gamma }_j^{b_H,\alpha }=ln({\displaystyle \frac{1}{1+exp\left({\gamma }_j^{s,b_H}\right)}}\hfill \\ \hfill & +{\displaystyle \frac{\mathcal{C}\mathcal{N}\left(0;{\xi }_j^{h,b_H}-{\xi }_j^{g,b_H},1/{\eta }_j^{h,b_H}+1/{\eta }_j^{g,b_H}\right)}{\left(1+exp\left(-{\gamma }_j^{s,b_H}\right)\right)\mathcal{C}\mathcal{N}\left(0;{\xi }_j^{h,b_H},1/{\eta }_j^{h,b_H}\right)}}).\hfill \end{array}$$

(89)

Next, the following derivation concerns the part related to the channel support $s_j$. Based on the variable belief $q_{S,j}\left(s_j\right)$ in (64), the auxiliary message $v_j^{b_H,s}\left(s_j\right)$ is defined as

$$\begin{array}{cc}\hfill & v_j^{b_H,s}\left(s_j\right)\hfill \\ \hfill & \stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp\left(-{\gamma }_j^{b_H,s}\right)}}\delta \left[s_j-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_j^{b_H,s}\right)}}\delta \left[s_j\right]\hfill \\ \hfill & \propto \left\{\begin{array}{c}exp\left({\displaystyle \frac{1}{2}}\right(-{\mu }_{d,l,k}^{s,b_H}\left(s_{d,l,k}\right)+{\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)\hfill \\ +{\mu }_{d,l,k}^{s,b_S+}\left(s_{d,l,k}\right)\left)\right),d=1,\cdots ,D-1,\hfill \\ exp\left({\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)\right),d=D,\hfill \end{array}\right.\hfill \end{array}$$

(90)

where ${\gamma }_j^{b_H,s}=ln{\displaystyle \frac{v_j^{b_H,s}\left(s_j=1\right)}{v_j^{b_H,s}\left(s_j=0\right)}}$ represents the log-likelihood ratio of $v_j^{b_H,s}\left(s_j\right)$. By substituting the variable belief $q_{S,j}\left(s_j\right)$ in (64) and the factor belief $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ in (81) into the marginal consistency constraint in (35) and simplifying this constraint, we have   

$$\begin{array}{cc}\hfill & {\gamma }_j^{b_H,s}=ln({\displaystyle \frac{1}{1+exp\left({\gamma }_j^{\alpha ,b_H}\right)}}\hfill \\ \hfill & +{\displaystyle \frac{\mathcal{C}\mathcal{N}\left(0;{\xi }_j^{h,b_H}-{\xi }_j^{g,b_H},1/{\eta }_j^{h,b_H}+{\xi }_j^{g,b_H}\right)}{\left(1+exp\left(-{\gamma }_j^{\alpha ,b_H}\right)\right)\mathcal{C}\mathcal{N}\left(0;{\xi }_j^{h,b_H},1/{\eta }_j^{h,b_H}\right)}}).\hfill \end{array}$$

(91)

Finally, the derivation related to $g_j$ is presented. By marginalizing the factor belief $b_{H,j}(h_j,{\alpha }_k,s_j,g_j)$ in (81) over all variables except $g_j$, we can obtain its marginal belief as

$$\begin{array}{cc}\hfill & \int \sum _{{\alpha }_k,s_j}{b}_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)\mathrm{d}{h}_j={\tilde{\rho }}_j^{b_H}\mathcal{C}\mathcal{N}\left(h_j;{\tilde{\xi }}_j^{b_H},{\tilde{\lambda }}_j^{b_H}\right)\hfill \\ \hfill & +\left(1-{\tilde{\rho }}_j^{b_H}\right)\mathcal{C}\mathcal{N}\left(h_j;{\xi }_j^{g,b_H},1/{\eta }_j^{g,b_H}\right),\hfill \end{array}$$

(92)

where ${\tilde{\rho }}_j^{b_H}$, $\tilde{\xi }{j}^{b_H}$, and $\tilde{\lambda }{j}^{b_H}$ are given in (83), (84), and (85), respectively. Therefore, ${\mathrm{E}}_{b_{H,j}}\left[{\left|g_j\right|}^2\right]$ can be calculated as

$$\begin{array}{cc}\hfill {\mathrm{E}}_{b_{H,j}}\left[{\left|g_j\right|}^2\right]& ={\tilde{\rho }}_j^{b_H}\left({\left|{\tilde{\xi }}_j^{b_H}\right|}^2+{\tilde{\lambda }}_j^{b_H}\right)\hfill \\ \hfill & +\left(1-{\tilde{\rho }}_j^{b_H}\right)\left({\left|{\xi }_j^{g,b_H}\right|}^2+1/{\eta }_j^{g,b_H}\right).\hfill \end{array}$$

(93)

5.5. Activity Indicator Module

This subsection will detail the message passing process of the activity indicator module. Based on the variable belief $b_{A,k}\left({\alpha }_k\right)$ in (63), an auxiliary message $v_k^{\alpha ,b_H}\left({\alpha }_k\right)$ is introduced, which is expressed as

$$\begin{array}{cc}\hfill & v_k^{\alpha ,b_A}\left({\alpha }_k\right)\propto exp\left({\mu }_k^{\alpha ,b_A}\left({\alpha }_k\right)\right)\hfill \\ \hfill & \stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp\left(-{\gamma }_k^{\alpha ,b_A}\right)}}\delta \left[{\alpha }_k-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_k^{\alpha ,b_A}\right)}}\delta \left[{\alpha }_k\right],\hfill \end{array}$$

(94)

where ${\gamma }_k^{\alpha ,b_A}=ln{\displaystyle \frac{v_k^{\alpha ,b_A}\left({\alpha }_k=1\right)}{v_k^{\alpha ,b_A}\left({\alpha }_k=0\right)}}$ is the log-likelihood ratio of $v_k^{\alpha ,b_A}\left({\alpha }_k\right)$. By substituting the factor belief $b_{H,j}\left(h_j,{\alpha }_k,s_j,g_j\right)$ in (61) and the variable belief $q_{A,k}\left({\alpha }_k\right)$ in (62) into the marginal consistency constraint in (32) and simplifying, and by combining the definitions of the auxiliary messages $v_j^{b_H,\alpha }\left({\alpha }_k\right)$ in (88) and $v_k^{\alpha ,b_A}\left({\alpha }_k\right)$ in (94), the following result is obtained:

$$\begin{array}{c}\hfill {\gamma }_k^{\alpha ,b_A}=\sum _{d=1}^D\sum _{l=1}^L{\gamma }_{d,l,k}^{b_H,\alpha }.\end{array}$$

(95)

Then, by substituting the variable belief $q_{A,k}\left({\alpha }_k\right)$ from (62) and the factor belief $b_{A,k}\left({\alpha }_k\right)$ from (63) into the marginal consistency constraint in (32) and simplifying it, and then combining this with the definitions of the auxiliary messages $v_j^{\alpha ,b_H}\left({\alpha }_k\right)$ in (79) and $v_j^{b_H,\alpha }\left({\alpha }_k\right)$ in (88), we can get

$$\begin{array}{cc}\hfill {\gamma }_{d,l,k}^{\alpha ,b_H}& =\sum _{d^{\prime }=1}^D\sum _{l^{\prime }=1}^L{\gamma }_{d^{\prime },l^{\prime },k}^{b_H,\alpha }-{\gamma }_{d,l,k}^{b_H,\alpha }+ln{\displaystyle \frac{{\rho }_k}{1-{\rho }_k}}\hfill \\ \hfill & ={\gamma }_k^{\alpha ,b_A}-{\gamma }_{d,l,k}^{b_H,\alpha }+{\gamma }_k^{b_A},\hfill \end{array}$$

(96)

where ${\gamma }_k^{b_A}=ln{\displaystyle \frac{{\rho }_k}{1-{\rho }_k}}$ represents the log-likelihood ratio of the active probability ${\rho }_k$. Next, by taking the first partial derivative of the Lagrangian function (51) with respect to the active probability ${\rho }_k$ and setting it to zero, the update rule for the active probability ${\rho }_k$ can be derived as

$$\begin{array}{cc}\hfill {\rho }_k& ={\mathrm{E}}_{b_{A,k}}\left[{\alpha }_k\right]\hfill \\ \hfill & ={\displaystyle \frac{{\rho }_k^{\left(\mathrm{old}\right)}{\rho }_k^{\alpha ,b_A}}{{\rho }_k^{\left(\mathrm{old}\right)}{\rho }_k^{\alpha ,b_A}+\left(1-{\rho }_k^{\left(\mathrm{old}\right)}\right)(1-{\rho }_k^{\alpha ,b_A})}},\hfill \end{array}$$

(97)

where the mean ${\mathrm{E}}_{b_{A,k}}\left[{\alpha }_k\right]$ can be calculated according to (63). The expression ${\rho }_k^{\alpha ,b_A}=1/\left(1+exp\left(-{\gamma }_k^{\alpha ,b_A}\right)\right)$ represents the updated active probability, where ${\rho }_k^{\left(\mathrm{old}\right)}$ denotes the active probability before the update. To avoid confusion, the superscript ${\left(·\right)}^{\left(\mathrm{old}\right)}$ is used for distinction throughout this paper. Finally, (97) can be rewritten in the following equivalent form of the log-likelihood ratio:

$$\begin{array}{c}\hfill {\gamma }_k^{b_A}=ln{\displaystyle \frac{{\rho }_k}{1-{\rho }_k}}={\gamma }_k^{b_A\left(\mathrm{old}\right)}+{\gamma }_k^{\alpha ,b_A}.\end{array}$$

(98)

5.6. Channel Support Module

This subsection will provide a detailed description of the message passing process in the channel support module. For ease of notation, the subscript ${\left(·\right)}_j$ will be rewritten as ${\left(·\right)}_{d,l,k}$ in this subsection. Referring to the auxiliary message $v_{d,l,k}^{s,b_H}\left(s_{d,l,k}\right)$ in (80), and based on the factor belief $b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$ in (65), the following auxiliary messages are introduced:

$$\begin{array}{cc}\hfill & v_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)\hfill \\ \hfill & \stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp(-{\gamma }_{d,l,k}^{s,b_S})}}\delta \left[s_{d,l,k}-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_{d,l,k}^{s,b_S}\right)}}\delta \left[s_{d,l,k}\right]\hfill \\ \hfill & \propto exp\left({\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)\right),d=1,\cdots ,D,\hfill \end{array}$$

(99)

$$\begin{array}{cc}\hfill & v_{d,l,k}^{s,b_S+}\left(s_{d,l,k}\right)\hfill \\ \hfill & \stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp(-{\gamma }_{d,l,k}^{s,b_S+})}}\delta \left[s_{d,l,k}-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_{d,l,k}^{s,b_S+}\right)}}\delta \left[s_{d,l,k}\right]\hfill \\ \hfill & \propto exp\left({\mu }_{d,l,k}^{s,b_S+}\left(s_{d,l,k}\right)\right),d=1,\cdots ,D-1,\hfill \end{array}$$

(100)

where ${\gamma }_{d,l,k}^{s,b_S}=ln{\displaystyle \frac{v_{d,l,k}^{s,b_S}\left(s_{d,l,k}=1\right)}{v_{d,l,k}^{s,b_S}\left(s_{d,l,k}=0\right)}}$ and ${\gamma }_{d,l,k}^{s,b_S+}=ln{\displaystyle \frac{v_{d,l,k}^{s,b_S+}\left(s_{d,l,k}=1\right)}{v_{d,l,k}^{s,b_S+}\left(s_{d,l,k}=0\right)}}$ are the log-likelihood ratios of $v_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)$ and $v_{d,l,k}^{s,b_S+}\left(s_{d,l,k}\right)$, respectively. Referring to (90), the auxiliary message $v_{d,l,k}^{b_S,s}\left(s_{d,l,k}\right)$ is defined as

$$\begin{array}{cc}\hfill & v_{d,l,k}^{b_S,s}\left(s_{d,l,k}\right)\hfill \\ \hfill & \stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp(-{\gamma }_{d,l,k}^{b_S,s})}}\delta \left[s_{d,l,k}-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_{d,l,k}^{b_S,s}\right)}}\delta \left[s_{d,l,k}\right]\hfill \\ \hfill & \propto \left\{\begin{array}{c}exp\left({\displaystyle \frac{1}{2}}\left({\mu }_{d,l,k}^{s,b_H}\left(s_{d,l,k}\right)-{\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)+{\mu }_{d,l,k}^{s,b_S+}\left(s_{d,l,k}\right)\right)\right),\hfill \\ d=1,\cdots ,D-1,\hfill \\ exp\left({\mu }_{d,l,k}^{s,b_H}\left(s_{d,l,k}\right)\right),d=D,\hfill \end{array}\right.\hfill \end{array}$$

(101)

where ${\gamma }_{d,l,k}^{b_S,s}=ln{\displaystyle \frac{v_{d,l,k}^{b_S,s}\left(s_{d,l,k}=1\right)}{v_{d,l,k}^{b_S,s}\left(s_{d,l,k}=0\right)}}$ is the log-likelihood ratio of $v_{d,l,k}^{b_S,s}\left(s_{d,l,k}\right)$. Similarly, define the auxiliary message $v_{d,l,k}^{b_S+,s}\left(s_{d,l,k}\right)$ as   

$$\begin{array}{cc}\hfill & v_{d,l,k}^{b_S+,s}\left(s_{d,l,k}\right)d=1,\cdots ,D-1,\hfill \\ \hfill & \stackrel{\Delta }{=}{\displaystyle \frac{1}{1+exp(-{\gamma }_{d,l,k}^{b_S+,s})}}\delta \left[s_{d,l,k}-1\right]+{\displaystyle \frac{1}{1+exp\left({\gamma }_{d,l,k}^{b_S+,s}\right)}}\delta \left[s_{d,l,k}\right]\hfill \\ \hfill & \propto exp\left({\displaystyle \frac{1}{2}}\left({\mu }_{d,l,k}^{s,b_H}\left(s_{d,l,k}\right)+{\mu }_{d,l,k}^{s,b_S}\left(s_{d,l,k}\right)-{\mu }_{d,l,k}^{s,b_S+}\left(s_{d,l,k}\right)\right)\right),\hfill \end{array}$$

(102)

where ${\gamma }_{d,l,k}^{b_S+,s}=ln{\displaystyle \frac{v_{d,l,k}^{b_S+,s}\left(s_{d,l,k}=1\right)}{v_{d,l,k}^{b_S+,s}\left(s_{d,l,k}=0\right)}}$ is the log-likelihood ratio of $v_{d,l,k}^{b_S+,s}\left(s_{d,l,k}\right)$.

Based on the definitions in Equations (80), (101), and (102), the following log-likelihood ratio relationship can be derived:

$$\begin{array}{c}\hfill {\gamma }_{d,l,k}^{s,b_H}=\left\{\begin{array}{c}{\gamma }_{d,l,k}^{b_S,s}+{\gamma }_{d,l,k}^{b_S+,s},d=1,\cdots ,D-1,\hfill \\ {\gamma }_{d,l,k}^{b_S,s},d=D.\hfill \end{array}\right.\end{array}$$

(103)

Then, using the definitions from (90), (99), and (102), the corresponding log-likelihood ratio relationship is expressed as

$$\begin{array}{c}\hfill {\gamma }_{d,l,k}^{s,b_S}=\left\{\begin{array}{c}{\gamma }_{d,l,k}^{b_H,s}+{\gamma }_{d,l,k}^{b_S+,s},d=1,\cdots ,D-1,\hfill \\ {\gamma }_{d,l,k}^{b_H,s},d=D.\hfill \end{array}\right.\end{array}$$

(104)

Based on Equations (90), (100), and (101), the following log-likelihood ratio relationship can be further given:

$$\begin{array}{c}\hfill {\gamma }_{d,l,k}^{s,b_S+}={\gamma }_{d,l,k}^{b_H,s}+{\gamma }_{d,l,k}^{b_S,s},d=1,\cdots ,D-1.\end{array}$$

(105)

Following this, the marginal consistency constraint involving the variable belief $q_{S,d,l,k}\left(s_{d,l,k}\right)$ and the factor belief $b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$ or $b_{S,1,l,k}\left(s_{1,l,k}\right)$ in (35) can be restated as

$$\begin{array}{cc}\hfill & {\gamma }_{d,l,k}^{b_S,s}\hfill \\ \hfill & =\left\{\begin{array}{c}ln{\displaystyle \frac{{\rho }_{l,k}^{1,0}}{{\rho }_{l,k}^{0,1}}},d=1,\hfill \\ ln{\displaystyle \frac{{\rho }_{l,k}^{1,0}+\left(1-{\rho }_{l,k}^{0,1}\right)exp\left({\gamma }_{d-1,l,k}^{s,b_S+}\right)}{1-{\rho }_{l,k}^{1,0}+{\rho }_{l,k}^{0,1}exp\left({\gamma }_{d-1,l,k}^{s,b_S+}\right)}},d=2,\cdots ,D.\hfill \end{array}\right.\hfill \end{array}$$

(106)

Moreover, when $d=1,\cdots ,D-1$, the marginal consistency constraint involving the beliefs $q_{S,d,l,k}\left(s_{d,l,k}\right)$ and $b_{S,d+1,l,k}\left(s_{d+1,l,k},s_{d,l,k}\right)$ in (35) can be rewritten as

$$\begin{array}{c}\hfill {\gamma }_{d,l,k}^{b_S+,s}=ln{\displaystyle \frac{{\rho }_{l,k}^{0,1}+\left(1-{\rho }_{l,k}^{0,1}\right)exp\left({\gamma }_{d+1,l,k}^{s,b_S}\right)}{1-{\rho }_{l,k}^{1,0}+{\rho }_{l,k}^{1,0}exp\left({\gamma }_{d+1,l,k}^{s,b_S}\right)}}.\end{array}$$

(107)

Subsequently, by taking the first-order partial derivative of the Lagrangian function (51) with respect to the transition probability ${\rho }_{l,k}^{0,1}$ and setting it to zero, ${\rho }_{l,k}^{0,1}$ can be expressed as the solution to the following quadratic equation:

$$\begin{array}{c}\hfill {\mathrm{A}}_{l,k}^{0,1}{\left({\rho }_{l,k}^{0,1}\right)}^2+{\mathrm{B}}_{l,k}^{0,1}{\rho }_{l,k}^{0,1}+{\mathrm{C}}_{l,k}^{0,1}=0,\end{array}$$

(108)

where ${\mathrm{A}}_{l,k}^{0,1}$, ${\mathrm{B}}_{l,k}^{0,1}$, and ${\mathrm{C}}_{l,k}^{0,1}$ are, respectively, given by   

$$\begin{array}{cc}\hfill & {\mathrm{A}}_{l,k}^{0,1}={\mathrm{E}}_{b_{S,1,l,k}}\left[s_{1,l,k}\right]-\sum _{d=2}^D{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d-1,l,k}\right],\hfill \end{array}$$

(109)

$$\begin{array}{cc}\hfill & {\mathrm{B}}_{l,k}^{0,1}=\left({\rho }_{l,k}^{1,0}-1\right){\mathrm{E}}_{b_{S,1,l,k}}\left[s_{1,l,k}\right]-{\rho }_{l,k}^{1,0}+\sum _{d=2}^D\{\left(1-{\rho }_{l,k}^{1,0}\right)\hfill \\ \hfill & ·{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d-1,l,k}\right]-{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}{s}_{d-1,l,k}\right]\},\hfill \end{array}$$

(110)

$$\begin{array}{cc}\hfill & {\mathrm{C}}_{l,k}^{0,1}={\rho }_{l,k}^{1,0}\left(1-{\mathrm{E}}_{b_{S,1,l,k}}\left[s_{1,l,k}\right]\right)\hfill \\ \hfill & +{\rho }_{l,k}^{1,0}\sum _{d=2}^D\left\{{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d-1,l,k}\right]-{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}{s}_{d-1,l,k}\right]\right\}.\hfill \end{array}$$

(111)

Specifically, the solution for ${\rho }_{l,k}^{0,1}$ in (108) is expressed as

$$\begin{array}{c}\hfill {\rho }_{l,k}^{0,1}={\displaystyle \frac{-{\mathrm{B}}_{l,k}^{0,1}-\sqrt{{\left({\mathrm{B}}_{l,k}^{0,1}\right)}^2-4{\mathrm{A}}_{l,k}^{0,1}{\mathrm{C}}_{l,k}^{0,1}}}{2{\mathrm{A}}_{l,k}^{0,1}}}.\end{array}$$

(112)

Similarly, by taking the first-order partial derivative of the Lagrangian function (51) with respect to the transition probability ${\rho }_{l,k}^{1,0}$ and setting it to zero, ${\rho }_{l,k}^{1,0}$ can be expressed as the solution to the following quadratic equation:

$$\begin{array}{c}\hfill {\mathrm{A}}_{l,k}^{1,0}{\left({\rho }_{l,k}^{1,0}\right)}^2+{\mathrm{B}}_{l,k}^{1,0}{\rho }_{l,k}^{1,0}+{\mathrm{C}}_{l,k}^{1,0}=0,\end{array}$$

(113)

where ${\mathrm{A}}_{l,k}^{1,0}$, ${\mathrm{B}}_{l,k}^{1,0}$, and ${\mathrm{C}}_{l,k}^{1,0}$ are, respectively, given by

$$\begin{array}{cc}\hfill {\mathrm{A}}_{l,k}^{1,0}& =2-D-{\mathrm{E}}_{b_{S,1,l,k}}\left[s_{1,l,k}\right]+\sum _{d=2}^D{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d-1,l,k}\right],\hfill \end{array}$$

(114)

$$\begin{array}{cc}\hfill {\mathrm{B}}_{l,k}^{1,0}& =\left(1-{\rho }_{l,k}^{0,1}\right){\mathrm{E}}_{b_{S,1,l,k}}\left[s_{1,l,k}\right]-1-{\rho }_{l,k}^{0,1}\left(D-1\right)\hfill \\ \hfill & +\sum _{d=2}^D\{{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}\right]-{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}{s}_{d-1,l,k}\right]\hfill \\ \hfill & +{\rho }_{l,k}^{0,1}{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d-1,l,k}\right]\},\hfill \end{array}$$

(115)

$$\begin{array}{cc}\hfill {\mathrm{C}}_{l,k}^{1,0}& ={\rho }_{l,k}^{0,1}{\mathrm{E}}_{b_{S,1,l,k}}\left[s_{1,l,k}\right]+{\rho }_{l,k}^{0,1}\hfill \\ \hfill & ·\sum _{d=2}^D\left\{{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}\right]-{\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}{s}_{d-1,l,k}\right]\right\},\hfill \end{array}$$

(116)

Specifically, the solution for ${\rho }_{l,k}^{1,0}$ in (113) can be calculated as

$$\begin{array}{c}\hfill {\rho }_{l,k}^{1,0}={\displaystyle \frac{-{\mathrm{B}}_{l,k}^{1,0}-\sqrt{{\left({\mathrm{B}}_{l,k}^{1,0}\right)}^2-4{\mathrm{A}}_{l,k}^{1,0}{\mathrm{C}}_{l,k}^{1,0}}}{2{\mathrm{A}}_{l,k}^{1,0}}}.\end{array}$$

(117)

Finally, considering the expression for the transition probability ${\rho }_{0,1}$ in (112) and the expression for the transition probability ${\rho }_{1,0}$ in (117), both are related to the mean of the factor belief $b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$. When $d=2,\cdots ,D$, based on the factor belief $b_{S,d,l,k}\left(s_{d,l,k},s_{d-1,l,k}\right)$ in (65), we can derive the following series of mean expressions:   

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}\right]={\displaystyle \frac{{\psi }_{d,l,k}^{1,0}+{\psi }_{d,l,k}^{1,1}}{{\psi }_{d,l,k}^{0,0}+{\psi }_{d,l,k}^{0,1}+{\psi }_{d,l,k}^{1,0}+{\psi }_{d,l,k}^{1,1}}},\hfill \end{array}$$

(118)

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d-1,l,k}\right]={\displaystyle \frac{{\psi }_{d,l,k}^{0,1}+{\psi }_{d,l,k}^{1,1}}{{\psi }_{d,l,k}^{0,0}+{\psi }_{d,l,k}^{0,1}+{\psi }_{d,l,k}^{1,0}+{\psi }_{d,l,k}^{1,1}}},\hfill \end{array}$$

(119)

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}{s}_{d-1,l,k}\right]={\displaystyle \frac{{\psi }_{d,l,k}^{1,1}}{{\psi }_{d,l,k}^{0,0}+{\psi }_{d,l,k}^{0,1}+{\psi }_{d,l,k}^{1,0}+{\psi }_{d,l,k}^{1,1}}},\hfill \end{array}$$

(120)

where ${\psi }_{d,l,k}^{0,0}\stackrel{\Delta }{=}\left(1-{\rho }_{l,k}^{1,0}\right)\left(1-{\rho }_{d,l,k}^{s,b_S}\right)\left(1-{\rho }_{d-1,l,k}^{s,b_S+}\right)$, ${\psi }_{d,l,k}^{0,1}\stackrel{\Delta }{=}\left({\rho }_{l,k}^{0,1}\right)\left(1-{\rho }_{d,l,k}^{s,b_S}\right)\left({\rho }_{d-1,l,k}^{s,b_S+}\right)$, ${\psi }_{d,l,k}^{1,0}\stackrel{\Delta }{=}\left({\rho }_{l,k}^{1,0}\right)\left({\rho }_{d,l,k}^{s,b_S}\right)\left(1-{\rho }_{d-1,l,k}^{s,b_S+}\right)$, ${\psi }_{d,l,k}^{1,0}\stackrel{\Delta }{=}\left(1-{\rho }_{l,k}^{0,1}\right)\left({\rho }_{d,l,k}^{s,b_S}\right)\left({\rho }_{d-1,l,k}^{s,b_S+}\right)$, ${\rho }_{d,l,k}^{s,b_S}=1/\left(1+exp\left(-{\gamma }_{d,l,k}^{s,b_S}\right)\right)$, and ${\rho }_{d,l,k}^{s,b_S+}=1/\left(1+exp\left(-{\gamma }_{d,l,k}^{s,b_S+}\right)\right)$. When $d=1$, based on the factor belief $b_{S,1,l,k}\left(s_{1,l,k}\right)$ in (66), the mean expression can be obtained as

$$\begin{array}{c}\hfill {\mathrm{E}}_{b_{S,1,l,k}}\left[s_{1,l,k}\right]={\displaystyle \frac{{\rho }_{l,k}^{1,0}{\rho }_{1,l,k}^{s,b_S}}{{\rho }_{l,k}^{1,0}{\rho }_{1,l,k}^{s,b_S}+{\rho }_{l,k}^{0,1}\left(1-{\rho }_{1,l,k}^{s,b_S}\right)}},\end{array}$$

(121)

where ${\rho }_{1,l,k}^{s,b_S}=1/\left(1+exp\left(-{\gamma }_{1,l,k}^{s,b_S}\right)\right)$ denotes the probability that the variable $s_{1,l,k}$ equals 1, with the corresponding log-likelihood ratio given by ${\gamma }_{1,l,k}^{s,b_S}$.

5.7. Channel Value Module

This subsection will provide a detailed description of the message passing process in the channel value module. Based on the mean expression for the variable belief $q_{G,j}\left(g_j\right)$ in (67), we have ${\tau }_j^{g,b_H}=-{\tau }_j^{g,b_G}$. For the factor belief $b_{\tilde{G},j}\left(g_j\right)$ in (68), by expanding and simplifying the term $f_{G,j}\left(g_j,{\mathit{\omega }}_j\right)$, the following can be obtained:

$$\begin{array}{cc}\hfill b_{\tilde{G},j}\left(g_j\right)& \propto \mathcal{CN}\left(g_j;{\xi }_j^{g,b_G},1/{\eta }_j^{g,b_{\tilde{G}}}\right)\mathcal{CN}\left(g_j;0,1/{\eta }_j^{g,b_{\Omega }}\right)\hfill \\ \hfill & \propto \mathcal{CN}\left(g_j;{\displaystyle \frac{{\xi }_j^{g,b_{\tilde{G}}}{\eta }_j^{g,b_{\tilde{G}}}}{{\eta }_j^{g,b_{\tilde{G}}}+{\eta }_j^{g,b_{\Omega }}}},{\displaystyle \frac{1}{{\eta }_j^{g,b_{\tilde{G}}}+{\eta }_j^{g,b_{\Omega }}}}\right),\hfill \end{array}$$

(122)

where ${\eta }_j^{g,b_{\Omega }}={\sum }_{j^{\prime }\in {\mathcal{N}}_j}{\mathrm{E}}_{b_{\Omega ,j^{\prime }}}\left[{\omega }_{j^{\prime }}\right]$. By substituting the variable belief $q_{G,j}\left(g_j\right)$ from (67) and the factor belief $b_{\tilde{G},j}\left(g_j\right)$ from (122) into the variance consistency constraint ${\mathrm{Var}}_{q_{G,j}}\left[g_j\right]={\mathrm{Var}}_{b_{\tilde{G},j}}\left[g_j\right]$ in (49), we have

$$\begin{array}{c}\hfill {\eta }_j^{g,b_H}={\eta }_j^{g,b_{\Omega }}=\sum _{j^{\prime }\in {\mathcal{N}}_j}{\mathrm{E}}_{b_{\Omega ,j^{\prime }}}\left[{\omega }_{j^{\prime }}\right].\end{array}$$

(123)

By then substituting ${\tau }_j^{g,b_H}=-{\tau }_j^{g,b_{\tilde{G}}}$ and ${\eta }_j^{g,b_H}={\eta }_j^{g,b_{\Omega }}$ into (122), and combining the mean consistency constraint ${\mathrm{E}}_{b_{H,j}}\left[g_j\right]={\mathrm{E}}_{b_{\tilde{G},j}}\left[g_j\right]$ from (48) with the definition of the auxiliary mean ${\xi }_j^{g,b_H}$ in (61), after simplification, it follows that the mean of the factor belief $b_{\tilde{G},j}\left(g_j\right)$ is given by ${\mathrm{E}}_{b_{H,j}}\left[g_j\right]=-{\tau }_j^{g,b_H}/{\eta }_j^{g,b_H}$. Consequently, the auxiliary mean ${\xi }_j^{g,b_H}$ can be expressed as

$$\begin{array}{c}\hfill {\xi }_j^{g,b_H}={\mathrm{E}}_{b_{\tilde{G},j}}\left[g_j\right]+{\displaystyle \frac{{\tau }_j^{g,b_H}}{{\eta }_j^{g,b_H}}}=0.\end{array}$$

(124)

Next, by expanding and simplifying the term $f_{G,j}\left(g_j,{\mathit{\omega }}_j\right)$ in the factor belief $b_{\Omega ,j}\left({\omega }_j\right)$ from (69), the following expression can be obtained:

$$\begin{array}{cc}\hfill & b_{\Omega ,j}\left({\omega }_j\right)\propto f_{\Omega ,j}\left({\omega }_j\right)\hfill \\ \hfill & ·exp\left(\varpi \left({\omega }_j\right)-{\omega }_j\left(\sum _{j^{\prime }\in {\mathcal{N}}_j}{\mathrm{E}}_{b_{\tilde{G},j^{\prime }}}\left[{\left|g_{j^{\prime }}\right|}^2\right]\right)\right),\hfill \end{array}$$

(125)

where ${\varpi }_j\left({\omega }_j\right)$ is given by

$$\begin{array}{c}\hfill \varpi \left({\omega }_j\right)=\sum _{j^{\prime }\in {\mathcal{N}}_j}\int ln\left({\omega }_j+\sum _{j^{\prime \prime }\in {\mathcal{N}}_{j^{\prime }\setminus j}}{\omega }_{j^{\prime \prime }}\right)\mathrm{d}{\mathit{\omega }}_{j^{\prime }\setminus j}.\end{array}$$

(126)

As stated in (24), we have $f_{\Omega ,j}\left({\omega }_j\right)=\mathrm{Ga}\left({\omega }_j;a,b\right)$. To simplify the calculation, it is desirable for the $exp\left(·\right)$ function in (125) to be proportional to a Gamma distribution so that $b_{\Omega ,j}\left({\omega }_j\right)$ also follows a Gamma distribution, which would facilitate the computation of the mean ${\mathrm{E}}_{b_{\Omega ,j}}\left[{\omega }_j\right]$. However, due to the presence of variable coupling in the $ln\left(·\right)$ function, obtaining a closed-form solution for $\varpi \left({\omega }_j\right)$ in (126) is very challenging. Notably, the $exp\left(·\right)$ function in (125) contains both linear and nonlinear parts in ${\omega }_j$, which are coupled with other variables. We refer to this as the fully coupled model. Next, consider an alternative scenario where all ${\omega }_{j^{\prime }}$ in the prior (23) are replaced by $\omega j$, leading to $f_{G,j}\left(g_j,{\omega }_j\right)=\mathcal{CN}\left(g_j;0,{\left({\sum }_{j^{\prime }\in {\mathcal{N}}_j}{\omega }_j\right)}^{-1}\right)=\mathcal{CN}\left(g_j;0,{\left(\left|{\mathcal{N}}_j\right|{\omega }_j\right)}^{-1}\right)$. In this case, we can derive

$$\begin{array}{cc}\hfill & b_{\Omega ,j}\left({\omega }_j\right)\propto f_{\Omega ,j}\left({\omega }_j\right)\hfill \\ \hfill & ·exp\left(ln\left(\left|{\mathcal{N}}_j\right|{\omega }_j\right)-{\omega }_j\left|{\mathcal{N}}_j\right|{\mathrm{E}}_{b_{G,j}}\left[{\left|g_j\right|}^2\right]\right),\hfill \end{array}$$

(127)

which we refer to as the decoupled model. To strike a balance between computational convenience and variable correlation, we can retain the linear part of the $exp\left(·\right)$ function in $b_{\Omega ,j}\left({\omega }_j\right)$ under the fully coupled mode, while replacing the nonlinear part ${\varpi }_j\left({\omega }_j\right)$ with the nonlinear part $ln\left(\left|{\mathcal{N}}_j\right|{\omega }_j\right)$ from the uncoupled mode, thus obtaining

$$\begin{array}{cc}\hfill & b_{\Omega ,j}\left({\omega }_j\right)\propto f_{\Omega ,j}\left({\omega }_j\right)\hfill \\ \hfill & ·exp\left(ln\left(\left|{\mathcal{N}}_j\right|{\omega }_j\right)-{\omega }_j\left(\sum _{j^{\prime }\in {\mathcal{N}}_j}{\mathrm{E}}_{b_{\tilde{G},j^{\prime }}}\left[{\left|g_{j^{\prime }}\right|}^2\right]\right)\right)\hfill \\ \hfill & \propto \mathrm{Ga}\left({\omega }_j;a+1,b+\sum _{j^{\prime }\in {\mathcal{N}}_j}{\mathrm{E}}_{b_{\tilde{G},j^{\prime }}}\left[{\left|g_{j^{\prime }}\right|}^2\right]\right).\hfill \end{array}$$

(128)

where the factor belief $b_{\Omega ,j}\left({\omega }_j\right)$ after the replacement is referred to as the semi-coupled model. The mean of this semi-coupled factor belief can be computed as follows:

$$\begin{array}{cc}\hfill {\mathrm{E}}_{b_{\Omega ,j}}\left[{\omega }_j\right]& ={\displaystyle \frac{a+1}{b+{\sum }_{j^{\prime }\in {\mathcal{N}}_j}{\mathrm{E}}_{b_{\tilde{G},j^{\prime }}}\left[{\left|g_{j^{\prime }}\right|}^2\right]}}\hfill \\ \hfill & ={\displaystyle \frac{a+1}{b+{\sum }_{j^{\prime }\in {\mathcal{N}}_j}{\mathrm{E}}_{b_{H,j^{\prime }}}\left[{\left|g_{j^{\prime }}\right|}^2\right]}},\hfill \end{array}$$

(129)

where the second equality arises due to the consistency constraint between the mean and variance, which implies that ${\mathrm{E}}_{b_{H,j}}\left[{\left|g_j\right|}^2\right]={\mathrm{E}}_{b_{\tilde{G},j}}\left[{\left|g_j\right|}^2\right]$, and ${\mathrm{E}}_{b_{H,j}}\left[{\left|g_j\right|}^2\right]$ is given by (93).

5.8. Active User Detection Strategy

This section will provide a detailed description of the active user detection strategy. After obtaining the estimated values of the effective angle–delay domain channel ${\widehat{h}}_j^{\mathrm{ad}}={\mathrm{E}}_{b_{H,j}}\left[h_j\right],\forall j$ (i.e., ${\widehat{h}}_{d,l,k}^{\mathrm{ad}}={\mathrm{E}}_{b_{H,d,l,k}}\left[h_{d,l,k}\right],\forall d,l,k$), the equivalent space–frequency domain channel estimate for user k can be derived based on the channel transformation relationship in (9) as

$$\begin{array}{c}\hfill {\widehat{\mathbf{H}}}_k^{\mathrm{sf}}=\mathbf{A}{\widehat{\mathbf{H}}}_k^{\mathrm{ad}}{\mathbf{B}}^{\mathsf{T}},\end{array}$$

(130)

where ${\widehat{h}}_{d,l,k}^{\mathrm{ad}}$ represents the $(d,l)$-th element of the matrix ${\widehat{\mathbf{H}}}_k^{\mathrm{ad}}$. The algorithm determines the active status of user k using a detector based on the user’s activity log-likelihood ratio, as described in [32,33,34], and this is expressed as

$$\begin{array}{c}\hfill {\widehat{\alpha }}_k=\left\{\begin{array}{c}1,{\gamma }_k^{b_A}>0,\hfill \\ 0,{\gamma }_k^{b_A}\le 0,\hfill \end{array}\right.\end{array}$$

(131)

where the update for ${\gamma }_k^{b_A}$ is given in (98). Then, based on the activity detection metric ${\widehat{\alpha }}_k$, the equivalent space–frequency domain channel estimation for user k is adjusted as

$$\begin{array}{c}\hfill {\widehat{\mathbf{H}}}_k^{\mathrm{sf}}=\left\{\begin{array}{c}{\widehat{\mathbf{H}}}_k^{\mathrm{sf}},{\alpha }_k=1,\hfill \\ \mathbf{0},{\alpha }_k=0.\hfill \end{array}\right.\end{array}$$

(132)

Finally, the equivalent space–frequency domain channel estimation matrix for all users can be obtained as ${\widehat{\mathbf{H}}}^{\mathrm{sf}}=\left[{\widehat{\mathbf{H}}}_1^{\mathrm{sf}},{\widehat{\mathbf{H}}}_2^{\mathrm{sf}},\cdots ,{\widehat{\mathbf{H}}}_K^{\mathrm{sf}}\right]$, along with the activity detection vector $\widehat{\alpha }={\left[{\widehat{\alpha }}_1,{\widehat{\alpha }}_2,\cdots ,{\widehat{\alpha }}_K\right]}^{\mathsf{T}}$.

6. Algorithm Description

6.1. Procedure Details

The detailed procedure of the proposed CHMP algorithm is summarized in Algorithm 1. In this algorithm, the equivalent channel module, serving as a connecting component, is divided into its corresponding modules. The Markov chain of the channel support module corresponds to Steps 17 to 24 of the algorithm, while the coupled Gaussian distribution in the channel value module corresponds to Steps 32 and 33. The algorithm will iterate according to the rules until convergence is achieved.

The CHMP algorithm adopts an EM-like iterative update strategy and utilizes the damping technique to enhance the convergence stability [35,36,37]. Every ${\Delta }_T$ iterations, the algorithm executes the update steps corresponding to the aforementioned correlated probability model. In the remaining iterations, the channel support module executes the update Step 26, which corresponds to the i.i.d. Bernoulli prior

$$\begin{array}{c}\hfill {\gamma }_{d,l,k}^{s,b_H}={\gamma }_{d,l,k}^{b_H,s}+{\gamma }_{d,l,k}^{s,b_H\left(\mathrm{old}\right)},\end{array}$$

(133)

The channel value module executes Step 35, which corresponds to the i.i.d. zero-mean complex Gaussian prior

$$\begin{array}{c}\hfill {\eta }_j^{g,b_H}=1/{\mathrm{E}}_{b_{H,j}}\left[{\left|g_j\right|}^2\right].\end{array}$$

(134)

6.2. Computational Complexity

Considering that Step 6 of the CHMP algorithm involves a matrix–vector product of the form ${\mathbf{\Phi }}_{\circ i}\mathbf{u}$, where ${\mathbf{\Phi }}_{\circ i}\in {\mathbb{C}}^{R\times J}$ (with $R=MPG$ and $J=DLK$), and i can take the value 1 or 2, we define ${\mathbf{\Phi }}_{\circ 1}=\mathbf{\Phi }$ and ${\mathbf{\Phi }}_{\circ 2}$ as the matrix obtained by squaring the modulus of each element in $\mathbf{\Phi }$. Storing the matrix ${\mathbf{\Phi }}_{\circ i}$ is computationally expensive, and calculating this product ${\mathbf{\Phi }}_{\circ i}\mathbf{u}$ requires $MPGDLK$ multiplications and $MPG(DLK-1)$ additions. According to (13), we have ${\mathbf{\Phi }}_{\circ i}={\tilde{\mathbf{B}}}_{\circ i}^{\mathsf{T}}\otimes {\mathbf{A}}_{\circ i}$, where ${\mathbf{A}}_{\circ i}\in {\mathbb{C}}^{M\times D}$ and ${\tilde{\mathbf{B}}}_{\circ i}\in {\mathbb{C}}^{LK\times PG}$. Using this relationship, the matrix–vector product can be rewritten as

$$\begin{array}{c}\hfill {\mathbf{\Phi }}_{\circ i}\mathbf{u}=\mathrm{vec}\left({\mathbf{A}}_{\circ i}\mathbf{U}{\tilde{\mathbf{B}}}_{\circ i}\right),\end{array}$$

(135)

where $\mathbf{U}\in {\mathbb{C}}^{D\times LK}$, and $\mathbf{u}=\mathrm{vec}\left(\mathbf{U}\right)$. By left-multiplying $\mathbf{U}$ with ${\mathbf{A}}_{\circ i}$ and then right-multiplying with ${\tilde{\mathbf{B}}}_{\circ i}$, the computational complexity is reduced to $M\left(PG+D\right)LK$ multiplications and $\left(M\left(PG+D-1\right)LK-MPG\right)$ additions. Alternatively, by  right-multiplying $\mathbf{U}$ with ${\tilde{\mathbf{B}}}_{\circ i}$ and then left-multiplying with ${\mathbf{A}}_{\circ i}$, the computational complexity is reduced to $D\left(LK+M\right)PG$ multiplications and $\left(D\left(LK+M-1\right)PG-MPG\right)$ additions. Therefore, depending on the specific system configuration, the optimal implementation method can be selected to minimize the computational cost.

Additionally, Step 8 involves a matrix–vector product of the form ${\mathbf{\Phi }}_{\circ i}^{\mathsf{H}}\mathbf{v}$, which can similarly be computed using the following relation:

$$\begin{array}{c}\hfill {\mathbf{\Phi }}_{\circ i}^{\mathsf{H}}\mathbf{v}=\mathrm{vec}\left({\mathbf{A}}_{\circ i}^{\mathsf{H}}\mathbf{V}{\tilde{\mathbf{B}}}_{\circ i}^{\mathsf{H}}\right),\end{array}$$

(136)

where $\mathbf{V}\in {\mathbb{C}}^{M\times PG}$, and $\mathbf{v}=\mathrm{vec}\left(\mathbf{V}\right)$. If we first right-multiply $\mathbf{V}$ by ${\tilde{\mathbf{B}}}_{\circ i}^{\mathsf{H}}$ and then left-multiply by ${\mathbf{A}}_{\circ i}^{\mathsf{H}}$, the computational complexity is reduced to $M\left(PG+D\right)LK$ multiplications and $\left(M\left(PG+D-1\right)LK-DLK\right)$ additions. Alternatively, if we first left-multiply $\mathbf{V}$ by ${\mathbf{A}}_{\circ i}^{\mathsf{H}}$ and then right-multiply by ${\tilde{\mathbf{B}}}_{\circ i}^{\mathsf{H}}$, the computational complexity is reduced to $D\left(LK+M\right)PG$ multiplications and $\left(D\left(LK+M-1\right)PG-DLK\right)$ additions. Thus, depending on the specific system configuration, the implementation method with the minimal computational cost can be chosen.

Next, the computational complexity is assessed using real-valued floating-point operations (FLOPs), where the FLOP cost is determined by the type of operation. Specifically, addition, subtraction, multiplication, division, and the square root of two real numbers each require one FLOP; the addition and subtraction of two complex numbers each require two FLOPs; the multiplication and division of a complex number by a real number each require two FLOPs; the square of the modulus of a complex number requires three FLOPs; and multiplying two complex numbers requires six FLOPs (excluding complex conjugation). Additionally, the $exp\left(·\right)$ and $ln\left(·\right)$ functions for real numbers can be implemented using lookup tables, which does not consume computational resources [38,39]. Consequently, the total computational complexity of the CHMP algorithm is $\mathcal{O}\left(T·min\left\{M\left(PG+D\right)LK,D\left(LK+M\right)PG\right\}\right)$.

Algorithm 1 CHMP

1: Input: $\mathbf{A}$, $\mathbf{B}$, $\tilde{\mathbf{B}}$, $\mathbf{y}$, $\sigma$, $a=b={10}^{-20}$, ${\Delta }_T=2$, $\vartheta =0.7$.   2: Initialize: ${\tau }_r^{z,b_Y}={\eta }_j^{h,b_H}={\gamma }_j^{\alpha ,b_H}={\gamma }_j^{s,b_H}={\gamma }_j^{b_S,s}={\gamma }_j^{b_S+,s}=0$, ${\rho }_k=0.5$, ${\rho }_{l,k}^{0,1}={\rho }_{l,k}^{1,0}=0.5$, ${\mathrm{E}}_{b_{H,j}}\left[h_j\right]=0$, ${\mathrm{Var}}_{b_{H,j}}\left[h_j\right]={\displaystyle \frac{{\rho }_{l,k}^{0,1}{\rho }_k\left({\parallel \mathbf{y}\parallel }_{\mathrm{F}}^2-R\sigma \right)}{{\parallel \mathbf{A}\parallel }_{\mathrm{F}}^2{\parallel \tilde{\mathbf{B}}\parallel }_{\mathrm{F}}^2}}$, $\forall l,k,r,j$.   3: for  $t=1,\cdots ,T$  do   4:      // Signal Transmission   5:     Update the auxiliary precision ${\eta }_j^{h,b_Z}$ according to (72), $\forall j$.   6:     Update the auxiliary precision ${\eta }_r^{z,b_Y}$ and the auxiliary mean ${\xi }_r^{z,b_Y}$ according to (73) and (74), $\forall r$.   7:     Update the auxiliary multiplier ${\tau }_r^{z,b_Y}$ according to (75), $\forall r$.   8:     Update the auxiliary precision ${\eta }_j^{h,b_H}$ and the auxiliary mean ${\xi }_j^{h,b_H}$ according to (77) and (78), $\forall j$.   9:      // Activity Indicator 10:     Update the log-likelihood ratio ${\gamma }_j^{b_H,\alpha }$ according to (89), $\forall j$. 11:     Update the log-likelihood ratio ${\gamma }_k^{\alpha ,b_A}$ according to (95), $\forall k$. 12:     Update the user activity log-likelihood ratio ${\gamma }_k^{b_A}$ according to (98), $\forall k$. 13:     Update the log-likelihood ratio ${\gamma }_{d,l,k}^{\alpha ,b_H}$ using (96), $\forall d,l,k$. 14:      // Channel Support 15:     Update the log-likelihood ratio ${\gamma }_{d,l,k}^{b_H,s}$ using (91), $\forall d,l,k$. 16:     if t is an integer multiple of ${\Delta }_T$ then 17:         Update the log-likelihood ratios ${\gamma }_{d,l,k}^{s,b_S}$ and ${\gamma }_{d,l,k}^{s,b_S+}$ according to (104) and (105), $\forall d,l,k$. 18:         Update the mean ${\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}\right]$ according to (118) and (121), $\forall d,l,k$. 19:         Update the mean ${\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d-1,l,k}\right]$ according to (119), $d=2,\cdots ,D$, $\forall l,k$. 20:         Update the mean ${\mathrm{E}}_{b_{S,d,l,k}}\left[s_{d,l,k}{s}_{d-1,l,k}\right]$ according to (120), $d=2,\cdots ,D$, $\forall l,k$. 21:         Update the probability ${\rho }_{l,k}^{0,1}$ using (109)–(112), $\forall l,k$. 22:         Update the probability ${\rho }_{l,k}^{1,0}$ using (114)–(117), $\forall l,k$. 23:         Update the log-likelihood ratios ${\gamma }_{d,l,k}^{b_S,s}$ and ${\gamma }_{d,l,k}^{b_S+,s}$ according to (106) and (107), $\forall d,l,k$. 24:         Update the log-likelihood ratio ${\gamma }_{d,l,k}^{s,b_H}$ using (103), $\forall d,l,k$. 25:     else 26:         Update the log-likelihood ratio ${\gamma }_{d,l,k}^{s,b_H}$ using (133), $\forall d,l,k$. 27:     end if 28:      // Channel Value 29:     Update the non-zero probability ${\tilde{\rho }}_j^{b_H}$, the Gaussian mean ${\tilde{\xi }}_j^{b_H}$, and the Gaussian variance ${\tilde{\lambda }}_j^{b_H}$ according to (83)–(85), $\forall j$. 30:     Update the mean square ${\mathrm{E}}_{b_{H,j}}\left[{\left|g_j\right|}^2\right]$ according to (93), $\forall j$. 31:     if t is an integer multiple of ${\Delta }_T$ then 32:         Update the mean ${\mathrm{E}}_{b_{\Omega ,j}}\left[{\omega }_j\right]$ according to (129), $\forall j$. 33:         Update the precision ${\eta }_j^{g,b_H}$ according to (123), $\forall j$. 34:     else 35:         Update the precision ${\eta }_j^{g,b_H}$ according to (134), $\forall j$. 36:     end if 37:     Update the mean ${\mathrm{E}}_{b_{H,j}}\left[h_j\right]$ and the variance ${\mathrm{Var}}_{b_{H,j}}\left[h_j\right]$ according to (86) and (87), $\forall j$. 38: end for 39: Detect the user activity ${\widehat{\alpha }}_k$ and obtain the equivalent space–frequency domain channel ${\widehat{\mathbf{H}}}_k^{\mathrm{sf}}$ according to (130)–(132), $\forall k$. 40: Output: The equivalent space–frequency domain channel estimation ${\widehat{\mathbf{H}}}^{\mathrm{sf}}$ and the activity detection $\widehat{\alpha }$ for all users.

7. Simulation Results

This section presents the simulation results and analysis of the CHMP algorithm. The channel parameters in the simulation were generated using the QuaDRiGa platform [40], with the simulation scenario set to ‘3GPP_38.901_UMa_NLOS’ [41]. The simulation parameters are shown in

Table 3. Simulation parameter settings.

Parameter	Value	Parameter	Value
Carrier Frequency	2.6 GHz	OFDM Symbols G	3
Subcarrier Spacing ${\Delta }_f$	15 kHz	CP Length $N_{\mathrm{cp}}$	36
Subcarriers $N_{\mathrm{c}}$	512	Delay Grids L	36
Active Subcarriers N	300	Angle Grids D	2 $\times M$
Pilot Subcarriers P	50	Potential Users K	500

. This work employs the Normalized Mean Square Error (NMSE) and Activity Error Rate (AER) as performance metrics to evaluate the accuracy of channel estimation and active user detection, respectively. The NMSE and AER are defined as follows:

$$\begin{array}{cc}\hfill \mathrm{NMSE}& ={\displaystyle \frac{1}{M_{\mathrm{c}}}}\sum _{m_{\mathrm{c}}=1}^{M_{\mathrm{c}}}{\displaystyle \frac{\parallel {\widehat{\mathbf{H}}}_{\left(m_{\mathrm{c}}\right)}^{\mathrm{sf}}-{\mathbf{H}}_{\left(m_{\mathrm{c}}\right)}^{\mathrm{sf}}{\parallel }_{\mathrm{F}}^2}{\parallel {\mathbf{H}}_{\left(m_{\mathrm{c}}\right)}^{\mathrm{sf}}{\parallel }_{\mathrm{F}}^2}},\hfill \end{array}$$

(137)

$$\begin{array}{cc}\hfill \mathrm{AER}& ={\displaystyle \frac{1}{M_{\mathrm{c}}}}\sum _{m_{\mathrm{c}}=1}^{M_{\mathrm{c}}}{\displaystyle \frac{{\parallel {\widehat{\alpha }}_{\left(m_{\mathrm{c}}\right)}-{\alpha }_{\left(m_{\mathrm{c}}\right)}\parallel }_1}{K}},\hfill \end{array}$$

(138)

where ${∥·∥}_{\mathrm{F}}$ and ${∥·∥}_1$ denote the Frobenius norm and ${\ell }_1$ norm, respectively, and $M\mathrm{c}$ represents the number of Monte Carlo trials. To ensure a high degree of statistical reliability and to minimize the impact of random fluctuations, all subsequent simulation results are averaged over 1000 independent Monte Carlo trials, which is consistent with [14,19]. This large number of repetitions is standard practice for performance evaluation in this field and ensures that the presented curves are stable and robust estimates of the true performance.

The baseline algorithms compared in the simulation include (1) the CHMP algorithm with channel support correlation only, which does not consider channel value correlation, where channel values $g_j$ follow a complex Gaussian distribution with mean 0 and variance ${\omega }_j^{-1}$; (2) the CHMP algorithm with channel value correlation only, which does not consider channel support correlation, where channel support $s_j$ follows a Bernoulli distribution with $Pr\left(s_j=1\right)={\rho }_j$; (3) the CHMP algorithm without correlation, which does not consider either channel support or value correlation; (4) the GGAMP-SBL algorithm in [42], which combines SBL with the Gaussian GAMP framework to achieve robust performance; (5) the hybrid EM-VMP-EP variant algorithm in [43], which uses expectation–maximization (EM) to learn model parameters, variational message passing (VMP) to decouple the estimation of parameters, and an EP variant to perform sparse signal recovery; (6) the HMP-DCT algorithm in [13], which combats frequency-selective fading by first employing the discrete cosine transform (DCT) to create a sparse representation of the channel and then performs joint UAD and CE using an efficient hybrid message passing (HMP) algorithm, where the prior of the DCT coefficients is modeled with a Cauchy distribution and its unknown hyperparameters are learned via a gradient descent method.

7.1. Impact of SNR

Figure 3a,b illustrate the variation in the NMSE and AER with respect to the SNR for different algorithms, with the number of BS antennas M set to 64 and the number of active users $K_{\mathrm{a}}$ set to 60. It can be observed that the NMSE and AER of all algorithms decrease as the SNR increases. Specifically, the rate of NMSE reduction for all algorithms gradually slows as the SNR increases, with all curves exhibiting smooth and consistent downward trends. For the AER, starting from 0 dB, the values initially decrease rapidly with increasing SNRs, but, beyond 10 or 12 dB, further increases in the SNR result in only marginal reductions in the AER. From the performance comparison of the algorithms, it is evident that the CHMP algorithm, which simultaneously considers both channel support and value correlations, achieves the best performance in both channel estimation and active user detection.

In the NMSE curves shown in Figure 3a, compared to the CHMP algorithm without correlation, incorporating the correlation of channel values using a coupled Gaussian distribution and modeling channel support correlation with a Markov chain both enhance algorithm performance. The performance improvement from channel support correlation is relatively modest, while the channel value correlation yields a more substantial improvement. The joint utilization of both correlations results in the greatest performance gain. Among the baseline algorithms, HMP-DCT-Cauchy-GD demonstrates strong performance, consistently outperforming the general-purpose GGAMP-SBL and hybrid EM-VMP-EP variant across the entire SNR range. This highlights the effectiveness of using the discrete cosine transform to exploit channel sparsity. However, our proposed CHMP algorithm and the CHMP with only value correlation still maintain a clear performance gap regarding HMP-DCT-Cauchy-GD. This underscores the substantial benefit of our proposed dual-correlation channel model, which provides a more accurate channel prior than the models implicitly or explicitly used in these advanced algorithms.

In the AER curves shown in Figure 3b, the CHMP algorithm again demonstrates the best performance. Notably, in the low-to-medium SNR range (0 dB to 8 dB), the proposed CHMP algorithm demonstrates clear and substantial superiority in the AER. In this range, HMP-DCT-Cauchy-GD achieves competitive AER performance, which is better than that of the hybrid EM-VMP-EP variant but slightly worse than that of the CHMP algorithm with only support correlation. GGAMP-SBL and the hybrid EM-VMP-EP variant, in turn, outperform CHMP with only value correlation, but their performance is worse than that of CHMP with only support correlation and CHMP without correlation. As the SNR increases into the high regime (above 10 dB), the performance improvement for all algorithms becomes marginal. Here, the performance of GGAMP-SBL becomes similar to that of CHMP with only value correlation and CHMP without correlation, while the hybrid EM-VMP-EP variant performs comparably to CHMP with only support correlation. However, all these algorithms remain slightly inferior to the proposed CHMP algorithm, which maintains the lowest AER. Furthermore, CHMP with only value correlation shows less competitive AER performance, primarily because the coupled Gaussian distribution affects the judgment of activity indicators. Furthermore, it can be seen that the Markov chain mitigates the negative impact of the coupled Gaussian distribution on the AER. Their joint modeling effectively captures the internal structure of channel support and the inherent correlation of channel values, leading to the most significant performance improvement.

7.2. Impact of the Number of Active Users

Figure 4a,b illustrate the NMSE and AER performance of various algorithms as a function of the number of active users, with the number of BS antennas M set to 64 and the SNR set to 8 dB. It is observed that the performance of all algorithms degrades as the number of active users grows, increasing the difficulty of the sparse recovery problem.

Specifically, in the NMSE curves shown in Figure 4a, compared to the CHMP algorithm without considering correlations, utilizing channel value correlation yields a more significant improvement than channel support correlation, and the joint utilization of both results in the greatest performance enhancement. Notably, the proposed CHMP algorithm maintains a clear performance advantage over all baselines. Among these, HMP-DCT-Cauchy-GD delivers a better NMSE than both GGAMP-SBL and the hybrid EM-VMP-EP variant across the entire range of user loads. In the AER curves shown in Figure 4b, the superiority of the proposed CHMP algorithm is even more pronounced. The HMP-DCT-Cauchy-GD algorithm’s AER performance is competitive, surpassing the hybrid EM-VMP-EP variant and performing just below CHMP with only support correlation. The curves for the algorithm without correlations and the one considering only channel support correlation are quite similar. GGAMP-SBL and the hybrid EM-VMP-EP variant are superior to CHMP with only value correlation but are clearly outperformed by the other three CHMP algorithms. The coupled Gaussian distribution, which is used in CHMP with only value correlation, has a negative impact on the AER. The proposed CHMP algorithm, which jointly utilizes both support and value correlations, consistently maintains the lowest NMSE and AER across all tested user loads. This demonstrates not only the effectiveness but also the robustness of our method, as its performance advantage is sustained over all baselines, even under the increasingly difficult conditions of a crowded massive access scenario. This again suggests that our tailored dual-correlation model provides a more accurate representation of the channel’s physical structure than the general-purpose sparsity assumption exploited by the HMP-DCT-Cauchy-GD algorithm.

7.3. Impact of the Number of BS Antennas

Figure 5a,b illustrate the NMSE and AER performance as a function of the number of BS antennas, with the number of active users $K_{\mathrm{a}}$ set to 60 and the SNR set to 6 dB. As expected, the performance of all algorithms improves as the number of antennas increases, owing to the enhanced spatial resolution and array gain.

In the NMSE curves, the performance hierarchy remains consistent, with the full CHMP algorithm and the CHMP algorithm with only value correction leading. The HMP-DCT-Cauchy-GD algorithm performs well, showing a better NMSE than the other baselines. However, the performance gap between our proposed CHMP algorithm and all other methods, including HMP-DCT-Cauchy-GD, remains significant and even appears to widen slightly as the number of antennas grows. This trend is particularly revealing. In the AER curves shown in Figure 5b, the proposed CHMP algorithm again shows a significant advantage over all other methods. The HMP-DCT-Cauchy-GD algorithm’s AER is competitive and better than that of the hybrid EM-VMP-EP variant, while being slightly outperformed by CHMP with only support correlation and CHMP without correlation. This again highlights that joint correlation modeling effectively captures the internal structure of channel support and the inherent relationships of channel values. The widening performance gap demonstrates that our angle–delay domain dual-correlation model is exceptionally effective at leveraging the additional spatial information afforded by larger massive MIMO arrays. While HMP-DCT-Cauchy-GD effectively handles frequency-domain structures, our approach’s explicit modeling of the spatial channel characteristics allows it to better capitalize on the increase in antenna elements, leading to more accurate detection and estimation.

8. Conclusions

This paper investigates the joint active user detection and channel estimation problem in the uplink grant-free random access scenario of massive MIMO-OFDM systems. First, we propose an effective probability model that accurately characterizes the sparse nature of the massive random access channel in the user–angle–delay domain by introducing three types of state variables: active indicators, channel supports, and channel values. The proposed probability model not only captures the sparsity of the channel in the user domain but also leverages Markov chains and coupled Gaussian distributions to deeply explore the correlation properties of the channel in the angle–delay domain. Then, based on this probability model, the joint active user detection and channel estimation problem is formulated as a BFE minimization problem under hybrid constraints. To address this problem effectively, we further propose the CHMP algorithm, which adaptively adjusts the model parameters under unknown user sparsity and channel prior information. Numerical simulations validate the advantages of the proposed method in active user detection and channel estimation. While this work provides a robust framework for joint detection and estimation, we acknowledge that our analysis is based on a quasi-static channel model. In practical grant-free scenarios, impairments such as channel aging due to user mobility and imperfect synchronization can affect performance. Extending the proposed CHMP algorithm to account for these dynamic, non-ideal conditions and evaluating its robustness is an important and valuable direction for future research.