Neural Network Based AMP Method for Multi-User Detection in Massive Machine-Type Communication

: In massive machine-type communications (mMTC) scenarios, grant-free non-orthogonal multiple access becomes crucial due to the small transmission latency, limited signaling overhead and the ability to support massive connectivity. In a multi-user detection (MUD) problem, the base station (BS) is unaware of the active users and needs to detect active devices. With sporadic devices transmitting signals at any moment, the MUD problem can be formulated as a multiple measurement vector (MMV) sparse recovery problem. Through the Khatri–Rao product, we prove that the MMV problem is transformed into a single measurement vector (SMV) problem. Based on the basis pursuit de-noising approximate message passing (BPDN-AMP) algorithm, a novel learning AMP network (LAMPnet) algorithm is proposed, which is designed to reduce the false alarm probability when the required detection probability is high. Simulation results show that when the required detection probablity is high, the AMP algorithm based on LAMPnet noticeably outperforms the traditional algorithms with acceptable computational complexity.


Introduction
With the rapid growth in the number of wireless connected machines, massive machine-type communications (mMTC) [1,2] have been listed as one of the three main use cases in the fifth-generation (5G) cellular technologies road map [3]. The current grant based method for uplink transmission in the fourth-generation system requires a procedure to ask for the grant of the base station (BS), bringing attendant large transmission latency and excessive signaling overhead [4]. The mechanism disables grant based access from being adopted in the 5G system. Consequently, a grant-free non-orthogonal multiple access is proposed [5][6][7]. However, the BS is unaware of the active users without the granting process and needs to detect active devices among large number of potential devices, which is named multi-user detection (MUD) [8].
Multi-user detector includes an optimal detector and a sub-optimal detector. An optimal detector is actually a maximum likelihood sequence detector and can only be used with amplitudes and phases of all users as prior information. Besides, exponential complexity of an optimal detector prevents it from practical application [9]. A sub-optimal detector includes a linear detector and a nonlinear detector [10,11]. A linear detector uses a linear transformation on the output of the correlator before making a judgement. Decorrelating linear MUD and minimum mean square MUD are typical linear detectors. A nonlinear detector mainly focuses on interference cancellation. Parallel interference cancellers and serial interference cancellers are typical nonlinear detectors.
A crucial feature of mMTC is that only a small fraction of the potentially devices are active at any moment, thus MUD problem can be formulated as a sparse recovery problem [12]. Studies have been made to address MUD problem with compressed sensing (CS) in the literature. Orthogonal matching pursuit (OMP) [13][14][15] is widely used and modified in the field. In [16], a greedy algorithm based on OMP for MUD is proposed. A dynamic active user detection based on OMP is proposed in [17] to realize both user activity and data detection in several continuous time slots. For the BS with multiple antennas, OMP for multiple measurement vector (MMV) is proposed and analyzed in [18]. However, the computational complexity of OMP based algorithms increases rapidly when more active users are connected. Comparatively, computationally more efficient approximate message passing (AMP) algorithm is used for MUD in [19][20][21].
AMP is proposed by Donoho and others [22,23] and is a simplification of belief propagation in effect. When applied in different circumstances, AMP transforms to different types [23]. Basis pursuit AMP (BP-AMP) and basis pursuit de-noising AMP (BPDN-AMP) [24] are designed for basis pursuit problem like MUD. Assuming that the distribution of original signal is unknown, BP-AMP and BPDN-AMP adopt soft threshold denoiser. BP-AMP is a special case of BPDN-AMP without noise and they differ in the updating to the threshold. Thouless-Anderson-Palmer AMP (TAP-AMP) and Bayesian optimal AMP (BAMP) [25] are deduced with distribution of original signal as prior information and the denoiser is designed based on Bayesian methodology. The main idea of AMP is to simplify the belief propagation process with central limit theorem and Taylor expansion. Through assuming the messages obey Gaussian distribution, the computational complexity is greatly reduced.
In this paper, since AMP can only be applied in single measurement vector (SMV) problem, we formulate the original MMV problem as a SMV problem through Khatri-Rao product. Besides, the learning AMP network (LAMPnet) is proposed to reduce the false alarm probability and optimize the detection performance, especially when a high detection probablity is required. The LAMPnet is designed to study the updating of the threshold in BPDN-AMP and the LAMPnet-AMP returns the detected active users together with their reliabilities. Through setting a threshold of the realibility, the active users can be ultimately determined.
The rest of the paper is organized as follows. In Section 2, we establish uplink grant-free transmission system model and convert the MUD problem into a SMV problem through Khatri-Rao product. In Section 3, AMP is introduced. Then the proposed algorithm LAMPnet is shown in Section 4. Section 5 is devoted to comparison among LAMPnet-AMP, BPDN-AMP and MMV-OMP. The conclusion is drawn in Section 6. Notations of symbols are shown in Table 1.

System Model
Consider an uplink grant-free transmission system with a BS and N potentially active devices (see Figure 1). Assume that the BS possess M antennas for signal transmission and reception. Then the received signal at the i-th antenna is where y i ∈ C L×1 is the received signal at the i-th antenna and α u ∈ {1, 0} represents whether the u-th device is active or not. h ui is the fading channel gain between the u-th device and the i-th antenna, with the channel assumed to be a Rayleigh flat channel. s u ∈ R L×1 is denoted as the pilot sequence for the u-th device, which is unique in order to distinguish different users. w i ∈ C L×1 is an additive white Gaussian noise. S [s 1 , Then the model can be built as where y [y 1 , When the model is built under a multipath channel, slight revisions need to be made. Considering that the time delays in different paths differ, a cyclic shift matrixS k is defined as whereS k ∈ R L×L . Rows inS k represent the pilot sequence of user k with different time delays. Then the received signal at the i-th antenna is y i ∈ C L×1 is the received signal at the i-th antenna. u ik ∈ C L×1 is the channel gain between the k-th device and the i-th antenna and the entries in u ik correspond to different time delays.
The complete multipath channel model can be built as where y [y 1 , y 2 · · · y M ] ∈ C L×M , u [u 1 , u 2 · · · u M ] ∈ C LN×M . Obviously, the difference between the model under multipath channel and the model under single path channel is mainly about the size of the sparse signal. However, the difference in size is negligible in the following process. Thus we adopt the model under single path channel in the rest of this paper. The MUD problem aims at detecting all the K active users u 1 , u 2 · · · u K with α uj = 1, j = 1, 2 · · · K, i.e., identifying indices of the K non-zero rows in x. The length of pilot sequence L is much smaller than the number of devices N due to the limited coherence time in a practical mMTC scenario. Additionally, number of the non-zero rows in x is small since only sporadic devices transmit signals at any moment. Therefore, the MUD problem is viewed as a MMV problem of CS [26].
Theoretically, the self covariance matrix of y i is By applying Khatri-Rao product, it can be formulated as the following SMV problem.
S * is the conjugate matrix of S. R can be estimated based on the received signals in M antennas.
Since (x i x H i ) and S are both real matrixes, R is a real matrix as well. Thus the imagine part inR, which is generated due to errors, should be ignored. Define Y real(vec( 1 M ∑ M i=1 y i y H i )), Φ S * S and the MUD problem is reformulated as The non-zero entries in x represnet the active users and the target is to estimate x from Y. The estimation of x is equivalent to solving an underdetermined equation. It will be hard to obtain an accurate solution without using sparse property of x. Thus, an l 0 -norm minimization can be applied as The optimization is a l 0 problem, a classical NP hard problem. If Φ meets restricted isometry property (RIP), it can be transferred to l 1 problem. RIP is defined as follows.
For each integer s = 1, 2, ... , define the isometry constant δ s of a matrix Φ as the smallest number such that holds for all s-sparse vectors. Candès also explains the relation between δ 2s and the solution of the l 0 -norm minimization problem [27]: • If δ 2s < 1, the l 0 problem has a unique s-sparse solution; • If δ 2s < √ 2 − 1,the solution of the l 1 problem is that of the l 0 problem.
A matrix S with δ 2s < 1 guarantees that any s-sparse solution is unique. Actually, any 2s columns in Φ are linearly independent when δ 2s < 1, which means that it is impossible for different s-sparse signals to be mapped to one observation signal y. When δ 2s < √ 2 − 1, it can be ensured that the solution to l 1 problem is that of l 0 problem. The conclusion is important because l 0 problem can be solved by convex optimization and many sparse signal recovery algorithms are base on it. Tao and Candès also prove that a normal random matrix can be used as observation matrix [28]. In our paper, the pilots are generated from Gaussian distributions with zero mean and variance 1 L , thus Φ meets RIP and we only need to solve a l 1 problem instead min x 1 , Y = Φx. (12) As mentioned ahead, BP-AMP and BPDN-AMP are both designed for basis pursuit problem and we will introduce them in the next part. Another kind of recovery algorithms like OMP are greedy algorithms, and are basically realized by iterations. In each iteration, one column in the over-complete dictionary matrix is selected as the one most related to the current residuals. Least mean square (LMS) method is used to recover sparse signals based on the columns selected in previous iterations, and residuals will be updated afterwards. When residuals are smaller than a threshold or the number of non-zero values in recovered signal is equal to sparsity, it stops iteration.

AMP for MUD
AMP and MP are both proposed based on belief propagation and iteration threshold method. MP algorithm updates all the signals between nodes in factor graph, which makes it complicated, and AMP algorithm takes approximations using central limit theorem and Taylor expansion to reduce the complexity. The general form of AMP is as follows.
where η is the denoiser function and η is its first order derivative. x t is the recovered signal in t-th generation. z t is the residual in t-th generation. λ is the threshold level. δ is the ratio of the number of rows to the number of columns of Φ. Different types of AMP algorithms mainly differ in the design of the denoiser η [29][30][31]. When AMP is used for basis pursuit problem, the soft threshold function η s is used as the denoiser, which is defined as Since noise exists in transmission, BPDN-AMP is adopted here, which can be expressed as follows.
γ t is the threshold level in t-th generation and λ is the original threshold level. AMP iteratively produces a signal with noise x t +Φ T z t and uses the denoiser to minimize the mean square error (MSE). Device n is declared to be active if x n in the output is non-zero and declared to be inactive otherwise.

AMP Based on LAMPnet for MUD
The LAMPnet is proposed to study the updating of the threshold level in BPDN-AMP and LAMPnet based AMP uses LAMPnet to update the threshold in each iteration. After training D LAMPnets, AMP based on the D LAMPnets totally returns D sets of detected users. When different neural networks are trained with the same training set, they are still not all the same due to chances in the training process such as initialization of weights and thresholds, especially when the structure of network is simple. Thus the D sets of users are similar but different. The frequency of an user being declared to be active in all the D sets represents the reliability of its activation. Through setting a frequency threshold, reliable active users can be detected ultimately.

Studying the Update to Threshold Level
In each iteration of AMP algorithm, the update to γ is a polynomial which contains η' x t + Φ T z t ; λ + γ t . Such update is possible to be studied by neural network and the result confirms it. Training a neural network to study the update to γ and using it to update γ instead of γ t+1 = λ + γ t δ η' x t + Φ T z t ; λ + γ t proves to recover signals successfully with its RMSE close to the result of AMP. Besides, when different neural networks with the same structure, inputs and outputs are trained to complete the recovery, outputs of them have slight differences as expected. Such differences are obvious when structure of the network is simple. Figure 2 shows the algorithm of the back propagation networks used in this paper. The input of the neural network is x t + Φ T z t in each iteration of BPDN-AMP and the expected output of the neural network is the corresponding threshold γ t+1 . Through recording x t + Φ T z t and γ t+1 in iterations of BPDN-AMP, adequate samples can be obtained as training dataset. With only 1 hidden layer and 6 neurons in the hidden layer, the network is trained quite quickly. Linear transfer function is used as activation function for output layer and Tan-sigmoid function is used for hidden layer. Mean square error is used as loss function. Figure 3 shows the training phase, validation phase and test phase of a LAMPnet. After 21 epochs, it reaches best validation performance.

Detecting Active Users
When D different LAMPnets are used, D signals will be recovered. In order to eliminate the mistakenly detected users, we adopt a hard threshold function η h with an adaptive threshold to filter the small values in the recovered signals.
where b is defined as the average absolute value of the non-zero entries in the recovered x. While the filtering causes missed detection, the losses can be corrected by using more different LAMPnets. The output of the D recovered signals converts to D sets of estimated active users. Reliability of user activation can be expressed by the frequency of users appearing in the sets. Through setting a frequency threshold θ, the users whose appearance frequency are over θ become the ultimately determined active users. The detailed algorithm of AMP-NN is listed in Algorithm 1.

Algorithm 1: AMP based on LAMPnets.
Input: Y, Φ Output: Active users l 1 , l 2 · · · l K , estimated channel x Step 1. Initialize Step 2. z t = y − Φx t ; Step 3. Use the n-th network to estimate γ t+1 with x t +Φ T z t as its input; Step 4. x t+1 = η s x t +Φ T z t ; λ + γ t+1 ; Step 5. If t < T BPDN−AMP , then t = t + 1 and go back to Step 2; Step 6. Record the recovered signal; Step 7. If n < D, then n = n + 1, t = 1 and go back to Step 2; Step 8. User i is declared to be possibly active if η h (x i ) > 0 and declared to be inactive otherwise. Apply the criterion to all the recorded D recovered signals and get D sets of possible active users; Step 9. Count the occurrences f of the users in all the L sets. User i is declared to be active if The computational complexity of KR-BPDN-AMP, OMP-MMV and KR-LAMPnet-AMP are shown in Table 2.

Channel Estimation
Additionally, to further show the performance of KR-LAMPnet-AMP, we adopt least mean square (LMS) to estimate the channel gain based on the result of MUD. With the active users detected, channel estimation can be completed by solving an underdetermined equation, in which the basis matrix Φ new is composed of columns in Φ corresponding to the selected users.
where x new is composed of the channel gain of the active users, equivalent to the non-zero entries in x.

LMS is applied herex
x new corresponds to the non-zero entries in x. Thus the channel estimation is done through LMS and the result of MUD.

Simulation Results and Analysis
To verify the effectiveness of LAMPnet based AMP, we simulate MUD algorithms including KR-BPDN-AMP, OMP-MMV and KR-LAMPnet-AMP. The prefix KR means that Khatri-Rao product is used in the algorithm to transfer the original MUD problem into a SMV problem. The parameters of the tests corresponding to the figures are showm in Table 3. When the detection probability of BPDN-AMP is over a threshold, even little promotion in detection probability is made at the expense of a substantial rise of false alarm probability. Comparitively, the rise of false alarm probability of KR-LAMPnet-AMP is much smoother and steadier. MMV-OMP performs the best when K = 10, but worse than KR-LAMPnet-AMP when K = 20, 30. The performance of OMP algorithm is seriously affected by the number of active users since the correlations between pilots of different users are not zero. The influence of the non-zero correlation becomes more serious with the increase of active users. According to the simulation, the weakening of OMP-MMV is the most obvious among KR-BPDN-AMP, OMP-MMV and KR-LAMPnet-AMP. Comprehensively, the detection performance of LAMPnet based AMP outperforms KR-BPDN-AMP and OMP-MMV, especially when the required detection probability is high.
With M larger, (vec( 1 M ∑ M i=1 y i y H i )) gets closer to R and the estimation is more accurate. Figure 5a shows the ROC curve with different number of antennas. K is set as 10 and other parameters remain unchanged. Figure 5b is done with different L. Both figures prove that the detection ability of KR-LAMPnet-AMP is better than KR-BPDN-AMP when the required detection ability is high. Figure 6a is done with SNR = 10 dB and other parameters unchanged. The increase in the number of used nets brings better performance but the computational complexity rises meanwhile. Figure 6b shows the performance under different signal to noise ratio (SNR) and the superiority of KR-LAMPnet-AMP is clear.

Conclusions
In this paper, we have proved that the MUD problem with multiple antennas can be transformed into a SMV problem through applying Khatri-Rao product in the mMTC model. A novel LAMPnet based AMP has been proposed to estimate active users with low error rate and the simulation proves that when the required detection probability is high, LAMPnet based AMP reaches a lower false alarm probability than BPDN-AMP with the detection probability the same. When the number of active users is large, LAMPnet-AMP also outperforms MMV-OMP. The advantage of LAMPnet-AMP mainly concentrates in the lower false alarm probability with high detection probability compared with BPDN-AMP. When the requirement of the detection probability is not hard, LAMPnet-AMP does not outperform BPDN-AMP. Therefore, LAMPnet-AMP is suitable for the conditions with high required detection probability and tolerance for high complexity, which is brought by multiple LAMPnets used in the algorithm. Besides, the threshold of the hard threshold function requires dynamically adjusting according to different circumstances. When the channel environments are noiseless, setting the threshold as the average absolute value of the non-zero entries will exclude active users. Further optimization on the algorithm will follow.