Variational Sparse Bayesian Learning for Estimation of Gaussian Mixture Distributed Wireless Channels

Abstract

In this paper, variational sparse Bayesian learning is utilized to estimate the multipath parameters for wireless channels. Due to its flexibility to fit any probability density function (PDF), the Gaussian mixture model (GMM) is introduced to represent the complicated fading phenomena in various communication scenarios. First, the expectation-maximization (EM) algorithm is applied to the parameter initialization. Then, the variational update scheme is proposed and implemented for the channel parameters’ posterior PDF approximation. Finally, in order to prevent the derived channel model from overfitting, an effective pruning criterion is designed to eliminate the virtual multipath components. The numerical results show that the proposed method outperforms the variational Bayesian scheme with Gaussian prior in terms of root mean squared error (RMSE) and selection accuracy of model order.

1. Introduction

Channel modeling plays a pivotal role in the design and evaluation of wireless communication networks. In the fifth-generation (5G) mobile communication systems, the diversity of the communication scenarios leads to the urgent need for advanced channel modeling techniques with higher accuracy and lower implementation complexity. In the existing channel measurement campaigns, the channel sounding technique based on pseudo-random (PN) sequences has widely been adopted due to its higher processing gain and delay resolution. As for the estimation of the multipath characteristics such as power delay profile, Doppler power spectrum, and frequency response from the obtained channel responses, super-resolution algorithms such as space-alternating generalized expectation-maximization (SAGE) [1] and subtractive deconvolution technique (CLEAN algorithm) [2] are widely applied. However, since the SAGE and CLEAN algorithms are based on the maximum likelihood criterion, the selection of model order, i.e., the number of multipath, is a nontrivial task. The estimated order of the channel model tends to be overestimated because of the contamination of noise. The unnecessary “virtual paths” will not only affect the accuracy of the channel coefficients estimation but also increase the computational cost of the algorithm. To solve this problem, different techniques such as negative log-evidence (NLE) [3] and Bayesian information criterion (BIC) [4] have been proposed to estimate the model order. However, the estimation performance of these model estimation schemes is far from ideal, especially when considering the low signal-to-noise ratio scenarios [5]. Since considerable computation is needed to find the optimal model order value, the existing techniques also suffer from prohibitive complexity.

Due to the fact that the multipath components (MPC) are sparse in channel impulse responses (CIR) [6], the joint estimation problem of channel parameters and channel model order can be addressed by sparse Bayesian learning (SBL). SBL is a typical machine learning algorithm that was first proposed by Tipping in 2001 [7] and has been developed into an important branch of compressed sensing reconstruction algorithms. The basic idea of SBL is to introduce the sparsity parameter to establish a hierarchical Bayesian model. By defining the priors on the sparse vector, SBL can automatically determine the position and amount of the non-zero elements in the reconstructed signal without pre-specifying the sparsity number. SBL is a promising method for the joint estimation of channel model order and the channel response coefficients. However, even after the hierarchical Bayesian model is established, the posterior PDF cannot be directly derived due to an intractable integral of the joint probability density. Thus, the variational inference is introduced to obtain an approximation of the parameter posterior in the SBL framework. This method, also known as variational sparse Bayesian learning (VSBL) [8], has been widely applied in online spectrum estimation [9], orthogonal time-frequency space (OTFS) detector [10], multi-user joint decoding [11], joint signal detection [12], and high-resolution radar imaging [13], etc. For channel parameter estimation, the idea of complete hidden data is introduced to speed up the convergence in [5]. The simulation result shows that the VSBL scheme outperforms the BIC criterion in both root mean squared error (RMSE) and model order selection. In [14], the delay of MPCs is modeled as the Poisson distribution, which incorporates the Fast VSBL based channel estimation as a priori information and significantly improves the bit error ratio (BER) performance.

Several channel measurement campaigns conducted in high-speed railways [15], airport grounds [16,17], and inland rivers [18] have shown that Weibull and Nakagami-m distributions are more suitable for modeling channel characteristics compared with Rayleigh and Rice distribution. This phenomenon originates from the uneven distribution of scatterers in the mobile communication environment. However, in most VSBL models, the channel taps were assumed to be Gaussian distributed [19,20], which is not appropriate for parameter learning. In this paper, the Gaussian mixture model (GMM) is adopted to describe the statistical characteristics of the MPCs. A GMM is linearly composed of $K$ Gaussian PDFs with corresponding means and covariances. By using sufficient Gaussian components, the GMM can be flexible to approximate any given PDF [21], which is suitable for establishing various fading channels models.

The contributions of this paper are summarized as follows:

• The Gaussian mixture model as a powerful method for sparse parameter learning for wireless channels is introduced to the estimation problem of wireless channel parameters under the VSBL framework. The flexibility of GMM is capable of describing the statistical characteristics of both theoretical general channels and complex real-world channels.
• A new variational Bayesian inference scheme for the Gaussian mixture model (VB-GMM) is developed based on multiple channel observations. The corresponding graphical model is given, and the closed-form updates of the model variables are derived. By setting a pruning criterion on the sparsity priors, the joint estimation of channel parameters and model order is achieved with low complexity.
• The simulation results demonstrate that the performance of VB-GMM is superior to the existing algorithms in terms of the estimation error, the convergence rate, and the model order selection accuracy in most non-Gaussian channels.

Throughout the paper, we use the following notations. ${\left(·\right)}^H$ and ${\left(·\right)}^{-1}$ denote the Hermitian transpose and the inverse of matrices, respectively. The expression $\mathcal{C}\mathcal{N}\left(\mu ,\Sigma \right)$ represents the multivariate complex Gaussian distribution with mean vector $\mu$ and covariance matrix $\Sigma$. $Ga\left(\alpha |a,b\right)$ denotes the Gamma distribution $p\left(\alpha \right)=\frac{b^a}{{\Gamma }\left(\alpha \right)}\cdot {\alpha }^{a-1}{e}^{-b\alpha },$ $\alpha >0$ in which $a$, $b$ and ${\Gamma }\left(·\right)$ represent shape parameter, scale parameter and Gamma function, respectively. We use calligraphic uppercase letters to represent sets, e.g., $\mathcal{I}$ denotes the index set of MPCs and $\overline{\mathcal{I}(l)}$ is the complement set of $l\in \mathcal{I}$. The remainder of the paper is organized as follows. In Section 2, the signal model is introduced, and the graphical model of VB-GMM is presented. In Section 3, the derivations of the VB-GMM estimators are given followed by the initialization algorithm and the pruning criterion. Finally, in Section 4, the numerical results are presented to demonstrate the effectiveness of the proposed scheme.

2. System Model

2.1. Signal Model

Consider a single-input-single-output (SISO) wireless channel measurement system, the sounding signal is denoted as $s\left(t\right)={\displaystyle {\sum }_{i=0}^{N_C-1}{p}_{i}b(t-i{T}_c)}$, in which the pseudo-random sequence $\left\{p_i\right\},i=0,\dots ,N_C-1$ has $N_C$ chips and the shaped pulse $b(t)$ has the duration $T_c$. Assume that $N_S$ copies of $s\left(t\right)$ are transmitted periodically to obtain the time-varying channel responses. After passing through the multipath channel, the signal at the receiver is made of $L$ duplications of $s\left(t\right)$ with additive Gaussian white noise $\eta \left(t\right)$, and the received signal corresponding to the n-th $s\left(t\right)$ can be expressed as

$$y_n\left(t\right) ={\displaystyle \sum _{l=1}^L{w}_{ln}s\left(t,{\theta }_{\ln }\right) + \eta \left(t\right)},n=1,\dots ,N_S$$

(1)

where $w_{ln}$ is the complex coefficient of the $l$-th multipath and ${\theta }_{ln}$ denotes the dispersion parameter vector including multipath delay, DoA, DoD, etc. We define $y_n={\left[y_n\left(0\right),y_n\left(T_s\right),\dots ,y_n\left(\left(D-1\right)T_s\right)\right]}^T$ as the discrete samples of the n-th received signal in which $T_s$ is the sampling interval at the receiver and $D$ represents the number of discrete samples, i.e., $D{T}_s=N_c{T}_c$. The discrete vector $y_n$ is considered as an observation of the measured channel. In our proposed scheme, an Bayesian probabilistic model is established to infer the posterior probabilities of the channel parameters $\left\{w_{ln},{\theta }_{ln},L\right\}$ based on multiple observations.

Since the proposed algorithm is developed based on time-varying channels, it is crucial to determine the wide-sense stationary (WSS) region in which the statistical characteristics of channel parameters are assumed invariant. For vehicle-to-vehicle channel, the spatial distance of 20~40 times the wavelength is considered to be WSS [22]. For convenience, we denote the multiple observation matrix as $y={[y_1,y_2,\dots ,y_N]}_{D\times N}$, where $N$ represents the number of channel responses obtained in a WSS region. If the spatial distance of the WSS is chosen properly according to the criterion, the dispersion parameter vector is assumed to be constant for the observations, i.e., ${\theta }_{l1}=\dots ={\theta }_{lN}={\theta }_l,n=1,\dots ,N$. Thus Equation (1) can be rewritten in a discrete form as:

$$y_n={\displaystyle \sum _{l=1}^L{w}_{ln}s({\theta }_l)+\eta }$$

(2)

in which $\eta \text{~}\mathcal{C}\mathcal{N}\left(0,\Sigma \right)$ is the Gaussian white noise vector.

In the SAGE algorithm, the complete parameter set $\Omega$ is divided into several subsets ${\Omega }_{\mathrm{sub}}$ so that the joint estimation for $\Omega$ is decomposed into optimizations for ${\Omega }_{\mathrm{sub}}$ in a sequential manner. Let $x_{\mathrm{sub}}$ denote the admissible hidden data with respect to ${\Omega }_{\mathrm{sub}}$, when the equation $p\left(y\left|x_{\mathrm{sub}},\Omega \right.\right)=p\left(y\left|x_{\mathrm{sub}},{\Omega }_{\overline{\mathrm{sub}}}\right.\right)p\left(x_{\mathrm{sub}}\left|\Omega \right.\right)$ is satisfied, the likelihood corresponding to the estimation monotonically increases and may converge to the local maximum [23]. In this paper, the concept of admissible hidden data ${\mathrm{x}}_l$ is also adopted. Consider an MPC $l$ we have

$${\mathrm{x}}_l=w_{l}s({\theta }_l)+{\eta }_l$$

(3)

in which ${\eta }_l$ represents the noise that the l-th MPC contribute to the total noise $\eta$ with $E\left\{{\eta }_l{\eta }_l^H\right\}={\Sigma }_l=\beta \Sigma$, $0\le \beta \le 1$, and the likelihood $p(x_l|{\theta }_l,w_l)=\mathcal{C}\mathcal{N}(x_l;w_{l}s({\theta }_l),{\Sigma }_l)$. Then the covariance of the noise that is not related to the $l$-th component is ${{\Delta }}_l=\left(1-\beta \right)\Sigma$, which can also affect the estimation of $x_l$ when observations are obtained and fixed. The noise covariance has a significant impact on the estimation results of channel parameters. Since the length of the channel sounding frame is generally much longer than the channel delay spread, most of the channel response samples are considered to be measurement noise. In this paper, the noise covariance is computed by the tail of CIRs.

2.2. The GMM Model of Channel Coefficients

Denote the $N$ channel coefficients of the $l$-th multipath as $w_l={[w_{l1},w_{l2},\dots ,w_{lN}]}_{1\times N}$. Assume that $w_l$ obeys the Gaussian mixture distribution with $K$ components:

$$p\left(w_l\left|{\pi }_l,{\mu }_l,{\Lambda }_l\right.\right)={\displaystyle \sum _{k=1}^K{\pi }_{lk}\mathcal{C}\mathcal{N}\left(w_l|{\mu }_{lk},{\Lambda }_{lk}\right)}$$

(4)

where ${\pi }_l=[{\pi }_{l1},{\pi }_{l2},\dots ,{\pi }_{lK}],{\displaystyle {\sum }_{k=1}^K{\pi }_{lk}=1}$ is defined as mixing coefficients. ${\mu }_{lk}$ and ${\Lambda }_{lk}$ represent the complex mean and covariance matrix of the $k$-th Gaussian distribution, respectively. For $w_l$, each sample can only be derived from one component of the Gaussian mixture distribution. Therefore, the discrete hidden variable $c_l$ is introduced. $c_l$ is a 0–1 matrix in which ${\mathrm{c}}_{lnk}=1$ means the channel coefficient $w_{ln}$ corresponds to the $k$-th Gaussian component. Given the mixing coefficients, the probability of hidden variables takes a product form:

$$p\left(c_l\left|{\pi }_l\right.\right)={\displaystyle \prod _{n=1}^N{\displaystyle \prod _{k=1}^K{\pi }_{lk}^{c_{lnk}}}}$$

(5)

For the variable ${\pi }_l$ a Dirichlet distribution prior is applied:

$$p({\pi }_l|{\gamma }_l)=Dir({\pi }_l|{\gamma }_l)=C({\gamma }_l){\displaystyle \prod _{k=1}^K{\pi }_{lk}^{{\gamma }_{lk}-1}}$$

(6)

where $C({\gamma }_l)=\frac{{\Gamma }\left({\gamma }_{l0}\right)}{{\displaystyle {\prod }_{k=1}^K{\Gamma }\left({\gamma }_{lk}\right)}}, {\gamma }_{l0}={\displaystyle \sum _{k=1}^K{\gamma }_{lk}}$, and ${\gamma }_l=\left[{\gamma }_{l1},\dots ,{\gamma }_{lK}\right]$ is assumed non-informative, i.e., ${\gamma }_{l1}={\gamma }_{l2}=\cdots ={\gamma }_{lK}$. The Dirichlet distribution is introduced here as the conjugate prior of the polynomial likelihood function $p\left(c_l\left|{\pi }_l\right.\right)$, which means the posterior of $c_l$ has the same form of Dirichlet distribution. In variational inference, the application of conjugate priors makes it convenient to merely update the parameters of the density functions with their form unchanged.

The graphical model for the variational Bayesian-based GMM is presented in Figure 1. Denote the parameter set as $\Omega =\left\{\theta ,w,\mu ,c,\alpha ,\pi \right\}$ the joint probability density can be rewritten according to the structure of the graph:

$$p\left(y,\Omega \right)=p\left(y\left|w,\theta ,\eta \right.\right)p\left(w\left|\alpha \right.,c,\mu \right)p\left(c,\pi \right)p\left(\alpha \right)p\left(\mu \right)$$

(7)

in which

$$p\left(y\left|w,\theta ,\eta \right.\right)={\displaystyle {\prod }_{n=1}^N\mathcal{C}\mathcal{N}(y_n;{\displaystyle {\sum }_{l=1}^L{w}_{ln}s({\theta }_l)},\eta )}$$

(8)

$$p\left(w\left|\alpha \right.,c,\mu \right)={\displaystyle {\prod }_{l=1}^{L}p\left(w_l\left|{\alpha }_l,c_l,{\mu }_l\right.\right)}$$

(9)

The sparsity prior of $l$-th path $p\left(w_l\left|{\alpha }_l,c_l,{\mu }_l\right.\right)$ is a key to restrain the gain of MPCs and implement model order selection. The sparsity prior is usually chosen as Laplace [24] or Gaussian distributed [7]. The Laplace prior leads to a $l_1$-type sparsity and Gaussian prior a $l_2$-type sparsity [5]. In the proposed scheme each component of the GMM is specified with an independent Gaussian sparsity prior, i.e., $p\left(w_{ln}\left|{\alpha }_{lk};c_{lnk}=1\right.\right)=\mathcal{C}\mathcal{N}({\mu }_{lk},{\alpha }_{lk}^{-1})$. Thus consider all the observations for the $l$-th path, we define

$$p\left(w_l\left|{\alpha }_l,c_l,{\mu }_l\right.\right)={\displaystyle \prod _{n=1}^N{\displaystyle \prod _{k=1}^K{\left\{\frac{1}{\sqrt{2\pi {\alpha }_{lk}^{-1}}}\exp \left(-\frac{{\left(w_{ln}-{\mu }_{lk}\right)}^2}{2{\alpha }_{lk}^{-1}}\right)\right\}}^{c_{lnk}}}}$$

(10)

The nonnegative sparsity parameter ${\alpha }_{lk}$ is inversely proportional to the width of the Gaussian function. Consider a situation that ${\alpha }_{lk}\to \infty$, the Gaussian function will be concentrated at the mean value ${\mu }_{lk}$, which renders the MAP estimations of channel coefficients to be constant. Due to the time-varying nature of wireless channels, the channel coefficients cannot be a constant value over time, unless they are concentrated at the value of zero, i.e., the $l$-th MPC is considered to be the measurement noise and should be removed. By setting a large threshold for ${\alpha }_l$, the irrelevant MPCs are pruned and the sparsity is obtained. The prior of the sparsity parameter $p\left({\alpha }_{lk}\right)$ is set to be a Gamma distribution $p\left({\alpha }_{lk}\right)=Ga\left({\alpha }_{lk}|a_{lk},b_{lk}\right)$ with a non-informative hyperprior for all branches, i.e., $a_{lk}=b_{lk}={10}^{-7}$, $k=1,2,\dots ,K$. Such choice is proper when no prior information is available, and it has been widely applied in various Bayesian hierarchical models, e.g., [5,7,8].

3. GMM-Based Variational Bayesian Learning

In this section, the VSBL is applied to the estimation problem of the Gaussian mixture channel model. First, the basic principle of variational Bayesian inference is introduced. Then the update expression of each variable in the graphical model is derived. Lastly, the appropriate initialization algorithm and corresponding parameter settings are given.

3.1. Variational Bayesian Inference

Consider a probabilistic model defined by observed variables $\mathcal{O}$, hidden variable sets $\mathcal{H}$, and the corresponding joint probability density $p\left(\mathcal{H},\mathcal{O}\right)$. Our target is to obtain the posterior $p\left(\mathcal{H}\left|\mathcal{O}\right.\right)=p\left(\mathcal{H},\mathcal{O}\right)/p\left(\mathcal{O}\right)$. However, the computation of the marginal likelihood function $p\left(\mathcal{O}\right)$ is intractable since it involves a multi-dimensional and complicated integral ${\displaystyle \int p\left(\mathcal{H},\mathcal{O}\right)d\mathcal{H}}$. Therefore, a PDF with a simpler structure is introduced to approximate the posterior distribution, i.e.,

$$q\left(\mathcal{H}\right)\approx p\left(\mathcal{H}\left|\mathcal{O}\right.\right)$$

(11)

where $q\left(\mathcal{H}\right)$ is the variational posterior distribution (VPD). In the variational Bayesian framework, the Kullback-Leibler (KL) divergence is defined as

$$\mathrm{KL}(q‖p)={\displaystyle \int q\left(\mathcal{H}\right)}\ln \left\{\frac{q\left(\mathcal{H}\right)}{p\left(\mathcal{H}\left|\mathcal{O}\right.\right)}\right\}d\mathcal{H}$$

(12)

To obtain a deterministic approximation of the posterior, (12) should be minimized, and the optimal VPD $q^{\ast }\left(\mathcal{H}\right)$ is achieved only if $\mathrm{KL}(q‖p)=0$. Using Bayes theorem to replace $p\left(\mathcal{H}\left|\mathcal{O}\right.\right)$, it makes the derivation of VPD a tractable optimization problem:

$$q^{*}\left(\mathcal{H}\right)=\arg \underset{q\left(\mathcal{H}\right)}{\max }\int q\left(\mathcal{H}\right)\ln \left\{\frac{p\left(\mathcal{H},\mathcal{O}\right)}{q\left(\mathcal{H}\right)}\right\}d\mathcal{H}$$

(13)

For simplicity, we adopt the mean-field theory and assume the joint PDF factorizes as

$$q\left(\mathcal{H}\right)={\displaystyle \prod _{s=1}^S{q}_s\left({\mathcal{H}}_s\right)}$$

(14)

For most of the variables including $\left\{w,\alpha ,c,\pi ,x\right\}$, (13) is optimized over the exponential family of distributions. In this case, the KL divergence vanishes, and the optimal solution of (13) is assumed to be achievable. For $\mu$ and $\theta$, the optimization on $q_s\left({\mathcal{H}}_s\right)$ is restricted on Dirac measures to obtain point estimates. By optimizing (13), the solution that minimizes the KL divergence is:

$$q_s^{*}\left({\mathcal{H}}_s\right)\propto \exp \left(E_{q\left(\mathcal{M}\mathcal{B}\left\{{\mathcal{H}}_s\right\}\right)}\ln \left\{p\left(\mathcal{H},\mathcal{O}\right)\right\}\right)$$

(15)

In (15), $E_{q\left(x\right)}\left(·\right)$ denotes the expectation with respect to the distribution $q\left(x\right)$,$\mathcal{M}\mathcal{B}\left\{{\mathcal{H}}_s\right\}$ represents the Markov blanket [25] of the variable ${\mathcal{H}}_s$. In the Bayesian network, the Markov Blanket of a node is the set of the nodes comprised of the node’s parents, its children, and its co-parents. I.e., $\mathcal{M}\mathcal{B}\left\{{\mathcal{H}}_s\right\}$ is the minimal set around ${\mathcal{H}}_s$ which makes ${\mathcal{H}}_s$ independent of the rest of the variables.

3.2. Update of the Estimation Expressions

As discussed in Section 2, a hierarchical Bayesian model is set up to find sparse MPCs and infer the parameter sets $\Omega =\left\{\theta ,w,\mu ,c\right\}$ from the measurements $y$. A detailed derivation of the VPDs for each factor is given below. The VPDs can be updated in any order during the iteration process.

1.

Estimation of$q(x_l)$: From the graphical model in Figure 1 and (15), the VPD of $x_l$ is expressed as $q\left(x_l\right)\propto \exp \left(E_{q\left(w_{\mathcal{I}}\right)q\left({\theta }_{\mathcal{I}}\right)}\left(\ln \left\{p(x_l|y,{\theta }_{\mathcal{I}},w_{\mathcal{I}})\right\}\right)\right)$, in which

$$p(x_l|y,{\theta }_{\mathcal{I}},w_{\mathcal{I}})=p(x_l|{\theta }_l,w_l)p(x_l|y,{\theta }_{\overline{\mathcal{I}\left(l\right)}},w_{\overline{\mathcal{I}\left(l\right)}})$$

(16)

In (16), $\mathcal{I}$ is the set of all MPCs. Note that scattering parameters ${\theta }_l$ should be determined for a specific MPC, thus, we define $q({\theta }_l)$ to be a point estimation, i.e., $q({\theta }_l)=\delta ({\theta }_l- {\hat{\theta }}_l)$, where ${\hat{\theta }}_l$ denotes the estimated value for ${\theta }_l$ in a former update iteration. Based on the Gaussian noise assumption in Section 2.1, we have

$$p(x_l|y,{\theta }_{\overline{\mathcal{I}(l)}},w_{\overline{\mathcal{I}(l)}})=\mathcal{C}\mathcal{N}(x_l|y-{\displaystyle {\sum }_{p\in \overline{\mathcal{I}(l)}}{w}_{p}s({\theta }_p)},{{\Delta }}_l)$$

(17)

By substituting $p(x_l|{\theta }_l,w_l)$ and (17) into (16), the admissible hidden data of the $n$-th observation obeys a Gaussian distribution with the mean and covariance matrix as follows:

$${\hat{x}}_{ln}^{\prime }=\left(1-\beta \right){\widehat{w}}_{ln}s({\hat{\theta }}_l)+\beta \left(y_n-{\displaystyle \sum _{p\in \overline{\mathcal{I}(l)}}{\widehat{w}}_{pn}s({\hat{\theta }}_p)}\right)$$

(18)

$${\hat{S}}_l^{\prime }={\left({\Delta }_l^{-1}+{\Sigma }_l^{-1}\right)}^{-1}$$

(19)

2.

Estimation of$q(c_l)$: Evaluating (15) with (5) and (10), the VPD of $c_l$ takes the form

$$q({\mathrm{c}}_l)\propto {\displaystyle \prod _{n=1}^N{\displaystyle \prod _{k=1}^K{r}_{lnk}^{c_{lnk}}}}$$

(20)

in which

$$r_{lnk}=\exp \left(\mathsf{\psi }({\hat{\gamma }}_{lk})-\mathsf{\psi }({\displaystyle \sum _{k=1}^K{\hat{\gamma }}_{lk}})+\frac{1}{2}\left[\mathsf{\psi }({\widehat{a}}_{lk})-\ln ({\widehat{b}}_{lk})\right]-\frac{{\widehat{\alpha }}_{lk}}{2}\left|{\widehat{w}}_{ln}^2+{\widehat{\Phi }}_{ln}^{\prime }-2{\widehat{\mu }}_{lk}{\widehat{w}}_{ln}+{\widehat{\mu }}_{lk}^2\right|\right)$$

(21)

and $\mathsf{\psi }\left(·\right)$ represents the di-gamma function. Since $c_l$ is a 0–1 matrix with a discrete VPD, the posterior expectation of $c_{lnk}$ is $E_{q({\mathrm{c}}_l)}\left\{c_{lnk}\right\}=q(c_{lnk}=0)\times 0+q(c_{lnk}=1)\times 1=r_{lnk}$.

3.

Estimation of$q(w_l)$: From the graph the Markov blanket of $w_l$ is $\left\{x_l,{\alpha }_l,{\theta }_l,c_l,{\mu }_l\right\}$, thus

$$q(w_l)\propto \exp \left(E_{q\left(ℳℬ\left\{w_l\right\}\right)}\left(\ln \left\{p(x_l|{\theta }_l,w_l)p(w_l|{\alpha }_l,c_l,{\mu }_l)\right\}\right)\right)$$

(22)

By integrating the factors in $\mathcal{M}\mathcal{B}\left\{w_l\right\}$, the VPD can be written as $q(w_l)={\displaystyle {\mathbf{\prod }}_{n=1}^{N}q(w_{ln})}={\displaystyle {\mathbf{\prod }}_{n=1}^N\mathcal{C}\mathcal{N}\left({\widehat{w}}_{ln}^{\prime }, {\hat{\Phi }}_{ln}^{\prime }\right)}$ with

$${\widehat{w}}_{ln}^{\prime }={\widehat{\Phi }}_{ln}^{\prime }\left(s{({\widehat{\theta }}_l)}^H{\Sigma }_l^{-1} {\hat{x}}_{ln}+{\displaystyle \sum _{k=1}^K{\hat{\mu }}_{lk}{\widehat{r}}_{lnk}{\widehat{\alpha }}_{lk}}\right)$$

(23)

$${\widehat{\Phi }}_{ln}^{\prime }={\left({\displaystyle \sum _{k=1}^K{\widehat{r}}_{lnk}}{\widehat{\alpha }}_{lk}+\mathrm{s}{({\hat{\theta }}_l)}^H{\Sigma }_l^{-1}s({\hat{\theta }}_l)\right)}^{-1}$$

(24)

4.

Estimation of$q({\theta }_l)$: Similarly, evaluating (15) leads to $\tilde{p}({\theta }_l)\propto$ $p({\theta }_l)\exp \left(E_{q(x_l)}{}_{q(w_l)}\left(\ln \left\{p(x_l|w_l,{\theta }_l)\right\}\right)\right)$. As mentioned above, the proxy PDF is defined as $q({\theta }_l)=\delta ({\theta }_l- {\hat{\theta }}_l)$ in order to obtain a point estimation. Therefore, we have

$${\hat{\theta }}_l^{\prime }=\arg \underset{{\theta }_l}{\max }\left(\ln p({\theta }_l)+{\displaystyle {\sum }_{n=1}^N\left|{\widehat{w}}_{ln}s({\widehat{\theta }}_l){\Sigma }^{-1}{\hat{x}}_{ln}^H\right|}\right)$$

(25)

5.

Estimation of$q({\alpha }_l)$: From the graphic model the Markov blanket of ${\alpha }_l$ is $\mathcal{M}\mathcal{B}\left\{{\alpha }_l\right\}=\left\{w_l,c_l,{\mu }_l\right\}$ and $q({\alpha }_l)\propto p({\alpha }_l)\exp \left(E_{q(w_l,c_l,{\mu }_l)}\left(\ln \left\{p(w_l|{\alpha }_l,c_l,{\mu }_l)\right\}\right)\right)$. Due to the gamma hyperprior $p({\alpha }_l)=Ga({\alpha }_{lk}|a_{lk},b_{lk})$ and the nature of conjugate prior, the VPD also satisfies the Gamma distribution, i.e., $q({\alpha }_{lk})\propto Ga({\alpha }_{lk}|{\widehat{a}}_{lk}^{\prime },{\widehat{b}}_{lk}^{\prime })$ with

$${\widehat{a}}_{lk}^{\prime }=a_{lk}+\frac{1}{2}{\displaystyle \sum _{n=1}^N{\widehat{r}}_{lnk}}$$

(26)

$${\widehat{b}}_{lk}^{\prime }=b_{lk}+{\displaystyle \sum _{n=1}^N{\widehat{r}}_{lnk}\frac{\left|{\widehat{w}}_{ln}^2+{\widehat{\Phi }}_{ln}-2{\hat{\mu }}_{lk}{\hat{w}}_{ln}+{\hat{\mu }}_{lk}^2\right|}{2}}$$

(27)

6.

Estimation of$q({\pi }_l)$: The only variable related to ${\pi }_l$ is $c_l$, thus, $q({\pi }_l)\propto p({\pi }_l)\exp \left(E_{q(c_l)}\left\{\ln p(c_l|{\pi }_l)\right\}\right)$. By substituting (5) and (6) into (15) we obtain

$$q({\pi }_l)\propto {\displaystyle \prod _{k=1}^K{\pi }_k^{{\gamma }_{lk}+{\displaystyle \sum _{n=1}^N{r}_{lnk}}-1}}$$

(28)

which indicates that the VPD also satisfies the Dirichlet distribution. The parameter of the VPD can be updated by

$${\hat{\gamma }}_{lk}^{\prime }={\gamma }_{lk}+{\displaystyle {\sum }_{n=1}^N{r}_{lnk}}$$

(29)

7.

Estimation of$q({\mu }_l)$: The Markov blanket of ${\mu }_l$ is $\mathcal{M}\mathcal{B}\left\{{\mu }_l\right\}=\left\{w_l,c_l,{\alpha }_l\right\}$, so that

$$\tilde{p}({\mu }_l)\propto p({\mu }_l)\exp \left(E_{q(w_l,c_l,{\alpha }_l)}\ln \left\{p(w_l|{\alpha }_l,c_l,{\mu }_l)\right\}\right)$$

(30)

For simplicity, the prior of the mean vector ${\mu }_l$ is set to be flat, and a delta VPD $q({\mu }_l)=\delta ({\mu }_l- {\hat{\mu }}_l)$ is also applied. By maximizing $\tilde{p}({\mu }_l)$, a point estimation can be obtained as follows:

$${\hat{\mu }}_{lk}^{\prime }=\frac{{\displaystyle {\sum }_{n=1}^N{\widehat{r}}_{lnk}{\widehat{w}}_{ln}}}{{\displaystyle {\sum }_{n=1}^N{\widehat{r}}_{lnk}}}$$

(31)

3.3. Initialization Algorithm

The initialization algorithm is proposed to infer the initial variational variables, including the preliminary estimation of the parameter set $\Omega$ and the GMM model parameters. As shown in Algorithm 1, the estimation for multipath parameters $w_l$ and ${\theta }_l$ is obtained by cyclically finding and removing the corresponding template function $s({\theta }_l)$ from the remaining observations. The estimation process stops when the number of multipath reaches the preset amount $L_0$. This method will inevitably result in duplicated estimations with the same scatter parameter vectors. Therefore, the channel taps with the same ${\theta }_l$ are checked and combined to be one. After obtaining the multipath coefficients $w_l$, we use the expectation-maximization (EM) algorithm [21] to estimate the initial parameters of $K$ Gaussian clusters. In the E-step, the probability that $w_{ln}$ is classified as the $k$-th Gaussian component, i.e., $p(c_{lnk}=1|w_{ln})$, is computed by the relative weight of the Gaussian likelihoods. In the M-step, ${\mu }_{lk}$ and ${\Lambda }_{lk}$ are updated by the statistics of observations. When the EM process converges, we obtain $r_{lnk}=p(c_{lnk}=1|w_{ln})$. Notice that when initializing the EM algorithm, improper mean and variance values may cause the estimation of $p(c_{lnk}=1|w_{ln})$ failing to converge. Thus in this paper, the K-means algorithm [26] is applied to estimate the initial mean and variance of each cluster.

Algorithm 1: Initialization

$$\begin{gathered} \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{Set}L = 1, {\alpha }_l = 0, r_{nk}=0, \mathcal{I} =Ø;{\mathrm{x}}_l\leftarrow \mathrm{z} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathbf{for} l = 1\to L_0 \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{Initialize} q({\theta }_l) \mathrm{from} \left(25\right) \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{Initialize} q(w_l) \mathrm{from} \left(23\right) \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{Initialize} q({\alpha }_l) \mathrm{from} \left(26\right) \mathrm{and} \left(27\right) \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{\hat{x}}_{ln}\leftarrow y_n-{\displaystyle {\sum }_{l=1}^L{\widehat{w}}_{ln}s({\hat{\theta }}_l)} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathcal{I}=\mathcal{I}\cup l \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}L=L+1 \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathbf{end} \mathbf{for} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathit{//remove} \mathit{duplicate} \mathit{estimates} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathbf{for} l\in \mathcal{I} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathbf{if} \forall d,{\theta }_d={\theta }_l \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{\mathrm{w}}_l={\mathrm{w}}_l+{\mathrm{w}}_d \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathcal{I}=\mathcal{I}-d \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}L=L-1 \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathbf{end} \mathbf{if} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathbf{end} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathit{//initialize\; GMM\; parameters} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathbf{for} l\in \mathcal{I} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{initialize} {\mu }_l^0, {\Lambda }_l^0 \mathrm{from} w_l \mathrm{by} \mathrm{K-means} \mathrm{algorithm} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathrm{estimate} {\mu }_l, {\Lambda }_l, p(c_{lnk}=1|w_{ln}) \mathrm{by} \mathrm{EM} \mathrm{algorithm} \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}{r}_{lnk}=p(c_{lnk}=1|w_{ln}) \\ \hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\mathbf{end} \end{gathered}$$

3.4. Pruning and Convergence Condition

A close examination of the sparsity prior (10) for the $l$-th multipath reveals that the sparsity parameter ${\alpha }_{lk}$ controls the width of the Gaussian prior. When ${\alpha }_{lk}\to \infty$ the cluster will be concentrated around the center which tends to be zero for a fading channel. Thus, large values of ${\alpha }_l$ will make the component “irrelevant” and be removed from the estimation. In the proposed method, we set ${\displaystyle {\sum }_{k=1}^K{\alpha }_{lk}}>T{H}_p$ as a pruning condition where $T{H}_p$ is a preset threshold. In order to halt the iterative variational process, another preset threshold ${{\Delta }}_0$ is used to check whether the estimation converges. Define the update rate for the i-th iteration

$${{\Delta }}^{(i)}=\frac{{\displaystyle {\sum }_{l=1}^L{\displaystyle {\sum }_{n=1}^N\left|w_{ln}{}^{(i)}-w_{ln}{}^{(i-1)}\right|}}}{{\displaystyle {\sum }_{l=1}^L{\displaystyle {\sum }_{n=1}^N\left|w_{ln}{}^{(i)}\right|}}}$$

(32)

The iteration process stops when ${{\Delta }}^{(i)}\le {{\Delta }}_0$. The general flow chart of the VB-GMM algorithm is shown in Figure 2.

3.5. Computational Complexity

The computational complexity of the proposed algorithm is discussed in this section. For the benchmark comparison, we introduce the Gaussian prior-based variational Bayesian estimation scheme proposed in [5] (VB-G). The complex multiplication times (CMT) is used as the comparison metric for the evaluation of the computational complexity in VB-G and VB-GMM. Note that the multiplication of a $L\times M$ complex matrix and a $M\times N$ complex matrix demands the CMT of $M\times L\times N$. And the CMT required by computing the inversion of a $N\times N$ complex matrix is $4N^3$ [27]. The computational complexity of each procedure is given in

Table 1. The CMT required by VB-G and VB-GMM algorithms.

Procedure	VB-G	VB-GMM
Initialization	$\left(6D^3+4D^2+\frac{1}{2}D{L}_0\right)N{L}_0$	$\left(ND{\left(D+1\right)}^2+4N^{2}K+\frac{1}{2}D{L}_0\right)L_0+\left(D^2+K\right)N{L}_0{T}_2+\mathcal{O}\left(T_{3}NK\right)$
Variational inference	${\displaystyle \sum _{t=1}^{T_1}ND{L}_t+\left(6D^3+3D^2+3D\right)N{T}_1}$	${\displaystyle \sum _{t=1}^{T_4}ND{L}_t+\left(ND{\left(D+1\right)}^2+N\left(K^2+2K+4\right)+\left(4N^2+2\right)K\right)T_4}$

, in which $T_1\text{~}{T}_4$ represents the number of iterations taken by the variational updates in VB-G, the EM process, the K-means algorithm, and the variational updates of the VB-GMM, respectively. Compared with VB-G, the initialization process of VB-GMM has greater complexity due to an iterative estimation for GMM parameters, in which the computational complexity of the K-means algorithm is $\mathcal{O}\left(T_{3}NK\right)$ [28]. In the variational inference procedure, the coefficient of the main complexity term $N{D}^3$ in VB-GMM is much smaller, resulting in the running time of about 1/5 of the VB-G in numerical simulations.

4. Simulation Results

In the numerical simulation, the setup of the sounding parameters is given below. We use the constant amplitude zero auto-correlation sequence as the sounding sequence with 256 chips, and the interval is $T_c=5 {\mu }\mathrm{s}$. The sample rate at the receiver is set as $R_s=1\mathrm{MHz}$ which implies that the number of discrete samples per CIR is $D=1280$. The number of observations per WSS region is assumed as $N=500$. For the multipath channel, a 4-tap model is adopted in which the mean power of each tap is set as 0, −3, −5, −10 dB, and the multipath delay equals 10, 12, 15, 20 μs, respectively. For testing the performance of VB-GMM in various complex environments, three different fading types are considered. In the first scenario, the coefficients of the MPC are generated from Gaussian mixture distributions with $K=3$. The real and imaginary parts of Gaussian means, covariances, and the mixing coefficients are randomly selected from the interval $[-10,10]$,$[0,1]$ and $[0,1]$, respectively. In the second scenario, all four multipaths are modeled as Weibull distribution with the shape parameter set as 3, 4, 5, 6, respectively. In the third scenario, we define the channel taps to be Nakagami-m distributed, and the m-parameters are set as 2, 3, 4, 5, respectively, to simulate different fading depths.

The numerical setup for the VB-GMM algorithm is as follows. The preset multipath amount is $L_0=20$, and the pruning threshold is set as $T{H}_p={10}^6$ for all SNR conditions. In the VBE-step, the initial parameters are: ${\alpha }_l=0$, $\beta =1$ and a flat prior probability $p\left({\theta }_l\right)$. In the VBM-step, the initial values of the parameters are: $a_{lk}=b_{lk}={10}^{-7}$, ${\gamma }_{lk}=1/K$. Note that all the numerical simulations are based on over 5000 observations, and the iteration process is terminated when the update rate ${{\Delta }}^{(i)}$ is smaller than 0.001.

Figure 3a shows a scatter plot of the estimated Gaussian mixture channel taps with the noise level $\mathrm{SNR}=-2 \mathrm{dB}$, in which the circles represent cluster centroids and the dots represent the estimated channel taps. The graph indicates that when the channel coefficients are Gaussian mixture distributed, the proposed VB-GMM is capable of classifying channel tap samples that come from different Gaussian components. Meanwhile, the cluster centroids can be precisely located. In Figure 3b,c, we examined the classification capability of the proposed algorithm when the channel taps are assumed to be Weibull and Nakagami distributed, respectively. As shown in Figure 3b,c, the channel coefficients in Weibull and Nakagami scenarios with random phases are clearly clustered into three regions, which indicates that the proposed method can learn and fit the distribution of random channels.

Figure 4a represents the RMSE between the synthetic and reconstructed CIRs versus SNR in different scenarios. Notice that the VB-G scheme adopts a fixed sensitivity level of $SNR\prime =10 \mathrm{dB}$. For low SNR of −5 dB, the VB-GMM exhibits comparable performance with the VB-G. As SNR increases, the VB-GMM clearly outperforms the VB-G in all three scenarios due to the flexibility in fitting non-Gaussian data. The reason is that for VB-SAGE the estimation errors of ${\theta }_l$, i.e., the delay of MPCs in part of the observations will result in a significant RMSE between the synthetic and reconstructed channel responses. While for VB-GMM, the multipath observations make the estimate of ${\theta }_l$ more robust because ${\theta }_l$ is assumed invariant in the WSS region. Figure 4b shows the curves of the estimated mean model order versus SNR. In the SNR range of −5 dB to 4 dB, the value of sparsity parameters tends to diverge, resulting in an obvious suppression effect on the MPCs. Under most SNR conditions, the VB-GMM shows accurate model selection for the 4-tap multipath channel, while the VB-G has a positive model order bias. Because for some of the observations if ${\theta }_l$ is wrongly estimated, there will be residual in the received signal after removing the template function, which leads to the falsely identified components. From the results of Figure 3 and Figure 4, the proposed method demonstrates its superiority in terms of model complexity and estimation accuracy. In Figure 5 we demonstrate curves of RMSEs that vary with the increasing iteration number in different scenarios for an SNR of 10 dB. The proposed method performs a similar convergence rate to the Gaussian prior-based scheme with smaller RMSEs in all the scenarios.

To study the influence of the number of Gaussian components $K$, we generated over 30,000 channel responses from the Weibull and Nakagami channels and the performance of the proposed VB-GMM is given in Figure 6. In the simulation, the SNR is fixed at 10 dB. As depicted in Figure 6a, the RMSEs of the reconstructed channel responses have no significant change as $K$ increases in both Weibull and Nakagami channels. From Figure 6b, it can be seen that the estimated model orders are accurate when $K\ge 2$. Consider that a large $K$ cannot bring considerate improvement and may increase the complexity of GMM, we propose to set $K=3$ for most channel conditions.

5. Conclusions

In this paper, we presented a variational Bayesian-based channel parameter estimation approach for Gaussian mixture distributed wireless channels, which combined the variational Bayesian estimation [5] and the variational GMM training [21]. Compared with the existing Bayesian estimation schemes, the proposed scheme can exploit the sparse solutions of the multipath channel and represent the complicated distribution patterns by the application of GMMs. Experiment results showed that the VB-GMM method can obtain the channel estimates with a small reconstruction RMSE and determine the optimal number of MPCs through a fixed pruning criterion under various SNR conditions. For future works, we will further investigate factors that may affect the algorithm performance in terms of the sparsity and the implementation complexity.