Performance Analysis of Regularized Convex Relaxation for Complex-Valued Data Detection

In this work, we study complex-valued data detection performance in massive multiple-input multiple-output (MIMO) systems. We focus on the problem of recovering an $n$-dimensional signal whose entries are drawn from an arbitrary constellation $\mathcal{K} \subset \mathbb{C}$ from $m$ noisy linear measurements, with an independent and identically distributed (i.i.d.) complex Gaussian channel. Since the optimal maximum likelihood (ML) detector is computationally prohibitive for large dimensions, many convex relaxation heuristic methods have been proposed to solve the detection problem. In this paper, we consider a regularized version of this convex relaxation that we call the regularized convex relaxation (RCR) detector and sharply derive asymptotic expressions for its mean square error and symbol error probability. Monte-Carlo simulations are provided to validate the derived analytical results.


I. INTRODUCTION
D ETECTION of complex-valued data from noisy linear measurements appears often in many communication applications, such as massive multiple-input multiple-output (MIMO) signal detection [1], multiuser detection [2], etc. In this work, we consider the problem of recovering an ndimensional complex-valued signal whose entries are drawn from an arbitrary constellation K ⊂ C from m noisy linear measurements in a massive MIMO application. Although the maximum likelihood (ML) detector can achieve excellent performance, its computational complexity becomes prohibitive as the problem size increases [2]. To achieve an acceptable performance with a low computing complexity, various convex optimization-based heuristics have been developed. One such heuristic method is to relax the discrete set K to a convex and continuous set V, and then solve the detection problem using a regularized convex optimization program followed by hard-thresholding. We call this method the regularized convex relaxation (RCR) detector, and we sharply analyze its asymptotic performance in terms of its mean square error (MSE) and symbol error probability (SEP) in the limit of m, n → ∞ with a fixed rate. We assume an independent and identically distributed (i.i.d.) complex Gaussian channel matrix and additive white Gaussian noise. The technical tool used in the analysis is the convex Gaussian min-max theorem (CGMT) framework [3], [4]. A The CGMT has been used to analyze the error performance of many optimization problems [3], [5]- [10]. However, prior performance analysis was only established for real-valued constellations such as BPSK and PAM using the box-relaxation method [5]. For the complex-valued constellations, to the best of our knowledge, the CGMT was only applied in one work [11] which considered convex-relaxation performance, but without regularization.
Our main contribution is to extend these performance results to complex-valued constellations with regularized detectors, and show the additional performance gains attained by adding the convex relaxation constraint. Our results also enable us to select the optimal regularization factor to further improve the detection performance. As a concrete example, we focus our attention on studying the performance of the RCR detector for phase-shift keying (PSK) and quadrature amplitude modulation (QAM) constellations.

A. Notation
The basic notations used throughout this article are gathered here. For a complex scalar z ∈ C, z R , and z I represent the real and imaginary parts of z, respectively and |z| = z 2 R + z 2 I . We use the letter j to denote the complex unit, i.e., j 2 = −1. A real Gaussian distribution with mean µ and variance σ 2 is denoted by N (µ, σ 2 ). Similarly, CN (µ, σ 2 ) denotes a complex Gaussian distribution with real and imaginary parts drawn independently from N (µ R , σ 2 /2) and N (µ I , σ 2 /2) respectively. X ∼ p X implies that the random variable X has a density p X . Bold lower-case letters are reserved for vectors, e.g., x, with x i denoting its i-th entry. The Euclidean norm of a vector is denoted by · . Matrices are represented by bold upper-case letters, e.g., A, while (·) ⊤ represents the transpose operator. We reserve the letters G and Z to denote real standard normal random variables. Similarly, G c is reserved to denote a complex CN (0, 2) normal random variable. E[·] and P[·] denote the expectation and probability operators, respectively. Notations " d =" and " P −→" are used to denote equivalence in distribution, and convergence in probability, respectively. Finally, for a closed and nonempty convex set V ⊂ C, and for any vector x ∈ C n , we define its distance and projection functions, respectively, as follows

B. Problem Setup
We need to recover an n-dimensional complex-valued transmit vector s 0 ∈ K n ⊂ C n , where K is the discrete transmit modulation constellation (e.g., PSK, QAM, etc.). The received signal vector r ∈ C m is given by where H ∈ C m×n is the MIMO channel matrix that has CN (0, 1 n ) i.i.d. entries and v ∈ C m is the noise vector with CN (0, σ 2 ) i.i.d. entries. Under the current setup, the signalto-noise ratio (SNR) is SNR = 1/σ 2 . Detector: The optimum detector that minimizes the probability of error is the maximum likelihood (ML) detector which solves This is computationally prohibitive in our massive MIMO setup; due to the discreteness nature of the constraints set K. Instead, in this paper, we consider the regularized convex relaxation (RCR) detector. The RCR recovers s following the next two steps: where in the first step (5a), the discrete set K is relaxed to a convex set V, and then we solve a regularized version of this relaxed problem with ζ ≥ 0 being the regularization factor. In the second step (5b), each entry of s is mapped to its closest point in K to produce the final estimate s ⋆ .

C. Performance Measures
In this paper, we provide sharp performance analysis of the RCR detector as a function of the problem parameters such as SNR, m, n, K and V. We will consider two different performance measures, namely the MSE and the SEP discussed next. Mean Square Error (MSE): This measures the performance of the estimation step of the detector (first step in (5a)) and is defined as: Another important performance metric is the symbol error probability. Symbol Error Probability (SEP): the symbol error rate (SER) characterizes the performance of the detection process (second step (5b)) and is defined as: where 1 {·} represents the indicator function.
In relation to the SER is the symbol error probability (SEP) which is defined as the expectation of the SER averaged over the noise, the channel and the constellation. Formally, the symbol error probability denoted by SEP is given by: Next, we introduce the notation V x for x ∈ K, as the set of all points in V that will be mapped to x in (5b). Equivalently With this notation at hand, we can rewrite the SEP in (8) as where s i is a minimizer of (5a).

D. Technical Assumptions
We assume that the entries of s 0 are sampled i.i.d. from a density p s0 , with Furthermore, we assume that s 0 is normalized to have zeromean and unit-variance, i.e., E[S 2 0 ] = 1. The convex set V is assumed to be symmetric, i.e., if (b 1 + jb 2 ) ∈ V, then (b 2 +jb 1 ) ∈ V as well. Finally, we assume a high-dimensional regime in which m, n → ∞ with a proportional rate κ := m n ∈ (0, ∞).

A. Main Results
This section provides our main results on the performance evaluation of the RCR detector in the considered high dimensional setting.
Theorem 1 (Asymptotics of the RCR). Let MSE, and SEP be the mean square error and symbol error probability of the RCR detector in (5), respectively, for an unknown signal s 0 ∈ K n with entries sampled i.i.d. from a distribution p s0 . Let V be a convex relaxation of K that satisfies the assumption of Section II-D. For fixed ζ ≥ 0, and κ > 0, if following optimization problem has a unique solution (α * , β * ), then it holds in probability that lim and where the expectation and probability in the above expressions are taken over S 0 ∼ p s0 and G c ∼ CN (0, 2).
Proof. See Section V.
Theorem 1 provides high dimensional asymptotic expressions to calculate the MSE/SEP of the RCR detector under an arbitrary complex-valued constellation. (11) is convex-concave and only involves scalar variables, thus α * and β * can be efficiently numerically calculated using the first-order optimality conditions.

B. Modulation Schemes
Using Theorem 1, we can sharply characterize the performance of the RCR detector for a general complex-valued constellation K, which can be relaxed to an arbitrary convex set V. To better understand our result and to show how to apply it to different schemes, we will focus on two conventional schemes; PSK and QAM constellations.
1) M -PSK Constellation: In an M -PSK constellation, where M = 2 k , for some k ∈ Z + , each entry of s 0 is randomly drawn from the set The elements of K are uniformly distributed over the unit circle in the complex plane, and therefore we suggest the use of the so-called circular relaxation (CR), where we choose the set V CR = {x ∈ C : |x| ≤ 1} as the convex relaxation set in (5a). The projection function on this set has the following form: x |x| , otherwise. Due to the symmetric nature of the M -PSK constellation, the asymptotic SEP can be derived in the following closed-form: where Z and G are i.i.d. N (0, 1) random variables.
2) M -QAM Constellation: In this paper, we will only consider square QAM modulation constellations, where M = 2 2k , such as 16-QAM, 64-QAM, etc. Then, the constellation set is given by where we normalize the constellation points by E avg := 2(M−1) 3 ; to have unit average power. 2 The convex relaxation that is often used for this modulation is known as the Boxrelaxation (BR) [5] which is given as Similar to the previous section, In order to use Theorem 1, we need to form the projection and distance functions of V BR which is straightforward for a box set. Then, SEP can be calculated using (13). Here, unlike the M -PSK case, the SEP of the recovery is not the same for different symbols in K; since an M -QAM constellation has different types of points, namely inner, edge and corner points.

IV. NUMERICAL RESULTS
In Fig. 1, we plotted the MSE and SEP performances as functions of the regularizer ζ for a 16-QAM constellation with BR. These figures verify the accuracy of the prediction of Theorem 1 when compared to Monte-Carlo (MC) simulations. In addition, from those figures we can see a clear minimum value of ζ that gives the best MSE/SEP performance, thus Theorem 1 can be used to select the optimum value of the regularizer.
Furthermore, Fig. 2 verifies the accuracy of the MSE/SEP predictions of Theorem 1 as functions of the SNR, for a 16-PSK modulation scheme with circular relaxation. Note that although the theorem requires m, n → ∞, it can be observed that the prediction is already accurate for n = 128. In this figure, we also plotted the unconstrained regularized least-squares (RLS) (without convex relaxation, i.e., V = R). Besides, it can be seen that the RCR detector outperforms the RLS.
Finally, under the box relaxation, we apply Theorem 1 to sharply characterize the MSE and SEP of a 16-QAM modulated system as functions of the SNR. The result is shown in Fig. 3, which again illustrates the high accuracy of our results, as well as that RCR detector outperforms the RLS.

V. SKETCH OF THE PROOF
In this section, we provide a proof sketch of Theorem 1. For the reader's convenient, we summarize the main tool of our analysis, namely the CGMT, in the next subsection.

A. Analysis Tool: CGMT
We first need to state the key ingredient of the analysis which is the CGMT. Here, we just recall the statement of the theorem, and we refer the reader to [3] for the complete technical requirements. Consider the following two min-max problems, which we refer to as the Primal Optimization (PO) and the Auxiliary Optimization (AO) problems: ψ(g 1 , g 2 ) := min where C ∈ Rm ×ñ , g 1 ∈ Rm, g 2 ∈ Rñ, S a ⊂ Rñ, S b ⊂ Rm and T : Rñ × Rm → R. Moreover, the function T is assumed to be independent of the matrix C. Denote by a Ψ := a Ψ (C), and a ψ := a ψ (g 1 , g 2 ) any optimal minimizers of (15a) and (15b), respectively. Further let S a , S b be convex and compact sets, T (a, b) is convex-concave continuous on S a × S b , and C, g 1 and g 2 all have i.i.d. standard normal entries. The equivalence between the PO and AO is formally described in the following theorem, the proof of which can be found in [3].
Theorem 2. Let S be any arbitrary open subset of S a , and S c = S a \ S. Denote ψ S c (g 1 , g 2 ) the optimal cost of the optimization in (15b), when the minimization over a is constrained over a ∈ S c . Suppose that there exist constants η < δ, such that ψ(g 1 , g 2 ) P −→ η, and ψ S c (g 1 , g 2 ) P −→ δ. Then, limñ →∞ P[a ψ ∈ S] = 1, and limñ →∞ P[a Ψ ∈ S] = 1.

B. Asymptotic Analysis
In this part, we provide an outline of the ideas used to prove our main results based on the CGMT framework. We start by rewriting (5a) by a change of variable to the error vector w := s − s 0 , to get: where H R , v R (H I , v I ) are the real (imaginary) parts of H and v, respectively. With this, and normalizing (16) by 1/n we get w := arg min w∈R 2n wi+j wi+n∈V−s0,i 1 2n where s 0 = s 0,R s 0,I ∈ R 2n . Because of the dependence between the entries of H, the above optimization is difficult to analyze and the CGMT framework cannot be used directly here. However, as discussed in [11], one can use Lindeberg methods as in [12] to replace H with a Gaussian matrix that has i.i.d. entries without affecting the asymptotic performance, then we get w = arg min w∈R 2n wi+j wi+n∈V−s0,i 1 2n where A ∈ R 2m×2n has i.i.d. N (0, 1) entries, and v has i.i.d. N (0, σ 2 2 ) elements. Next, we proceed to apply the CGMT by rewriting (18) as the following min-max optimization: One technical requirement of the CGMT is the compactness of the feasibility set over u. This can be handled according to the approach in [3, Appendix A], by introducing a sufficiently large artificial constraint set S u which will not affect the optimization problem with high probability to obtain: The above problem is in the form of a PO of the CGMT, hence we can associate to it the following simplified AO optimization problem: where g 1 ∈ R 2m and g 2 ∈ R 2n have i.i.d. N (0, 1) entries. With some abuse of notation on g 1 , we can see that Fixing β := u √ 2n , the optimization over u simplifies to min w∈R 2n wi+j wi+n∈V−s0,i max β>0 β g 1 √ 2n The square root in the above problem can be expressed using the following identity 3 which yields the following optimization problem Using the weak law of large numbers (WLLN): g 1 √ 2n P −→ √ κ, then the above problem reduces to Defining α := τ √ κ , and by a completion of squares in the minimization over w, and using the WLLN, we obtain the following scalar (deterministic) optimization problem The SEP of w in (18) can be derived in a similar way to the proof of [11] to get SEP P −→ P Π β * 2α * β * 2α * +ζ S 0 − β * β * α * + 2ζ simplifying (27) and (28) concludes the proof of the SEP part of Theorem 1.
The MSE expression can be proven in a similar way by noting that whereτ n is the solution to (26). Hence, usingα n =τ n √ κ , and α n P −→ α * , where α * is the solution of (27), we conclude, by applying the CGMT, that which completes the proof of the MSE part of Theorem 1.

VI. CONCLUSION
In this article, we provided sharp performance analysis of the regularized convex relaxation detector when used in complex-valued data detection. In particular, we studied its MSE and SEP performance in a massive MIMO application with arbitrary constellation schemes such as QAM and PSK. Numerical simulations show a close match to the obtained asymptotic results. In addition, the derived results can be used to optimally select the detector's parameters such as the regularization factor. Furthermore, we showed that this convex relaxation outperforms the unconstrained RLS.