Performance Analysis of Maximum Likelihood Detection in Cooperative DF MIMO Systems with One-Bit ADCs

Abstract

This paper investigates the error performance of cooperative decode-and-forward (DF) multiple-input multiple-output (MIMO) systems employing one-bit analog-to-digital converters (ADCs) over Rayleigh fading channels. In cooperative DF MIMO systems, detection errors at the relay may propagate to the destination, thereby degrading overall detection performance. Although joint maximum likelihood detection can efficiently mitigate error propagation by leveraging probabilistic information from a source-to-relay link, its computational complexity is impractical. To address this issue, an approximate maximum likelihood (AML) detection scheme is introduced, which significantly reduces complexity while maintaining reliable performance. However, its analysis under one-bit ADCs is challenging because of its nonlinearity. The main contributions of this paper are summarized as follows: (1) a tractable upper bound on the pairwise error probability (PEP) of the AML detector is derived using Jensen’s inequality and the Chernoff bound, (2) the asymptotic behavior of the PEP is analyzed to reveal the achievable diversity gain, (3) the analysis shows that full diversity is attained only when symbol pairs in the PEP satisfy a sign-inverted condition and the relay correctly decodes the source symbol, and (4) the simulation results verify the accuracy of the theoretical analysis and demonstrate the effectiveness of the proposed analysis.

1. Introduction

Wireless communications have become a core component of modern mobile systems, enabling high-speed data transmission and reliable connectivity. Over the past two decades, its advancement has been largely driven by the 3rd Generation Partnership Project, which established global standards ranging from 3G and 4G LTE to the current 5G New Radio [1]. These standards introduced key innovations in wireless communications, including orthogonal frequency-division multiplexing, carrier aggregation, and multi-antenna transmission schemes, all of which aim to improve the spectral efficiency, communication reliability, and user experience. Recently, advanced technologies such as extremely high-frequency communication, cognitive networks, integrated sensing and communication, and reconfigurable intelligent surfaces have been actively examined to satisfy the unprecedented demands of the next-generation wireless standards [2,3,4,5].

Multi-input multi-output (MIMO) systems are a fundamental and effective approach for enhancing wireless communication performance by deploying multiple antennas at the transmitter and/or receiver [6,7,8,9]. Multi-antenna configurations have been widely incorporated into modern wireless standards to exploit these advantages. Massive MIMO, which employs a large number of antennas, has gained significant attention because of its ability to improve both spectral and energy efficiency. However, implementing numerous antennas on the receiver side remains a practical challenge in mobile devices because of limitations in the physical form factor and increased hardware complexity.

Cooperative communication has emerged as an effective solution to overcome this limitation without requiring additional antennas at the user end. Among cooperative strategies, the decode-and-forward (DF) protocol is attractive because of its favorable performance and relatively low implementation complexity [10,11,12]. However, DF systems are inherently susceptible to error propagation when a relay node incorrectly decodes the transmitted signal. Thus far, various detection strategies have been proposed [13,14,15], with joint maximum likelihood (JML) detection being known for achieving full diversity gain to mitigate this effect. However, the computational burden of JML detection at the destination limits its practical applicability in resource-constrained systems.

Alternative approaches such as cooperative maximum ratio combining (MRC) [13] and pseudo-linear combiners [14] have been introduced; however, their applications are often restricted to single-antenna systems or orthogonal space-time block coded schemes. To address these limitations, a pairwise error probability (PEP)-based maximum likelihood (ML) detection method was proposed in [15], offering a good trade-off between performance and complexity. However, these studies assumed the availability of ideal or high-resolution analog-to-digital converters (ADCs), often being impractical for power- and cost-constrained devices.

In this context, low-resolution analog-to-digital converters (ADCs) have emerged as a promising solution for reducing power consumption in future wireless systems characterized by wide bandwidths and large-scale antenna arrays [16,17,18]. Among these, one-bit ADCs provide the most significant power savings by quantizing received signals based only on their sign information. Although this extreme quantization leads to considerable energy efficiency, it introduces substantial nonlinear distortion [19,20]. Research on one-bit ADCs has gained momentum, and their application in cooperative communication systems remains relatively underexplored. Recent studies have started addressing this gap [21,22,23,24,25]. For example, the studies in [21,22] examined the capacity limits of multipair massive MIMO systems under one-bit quantization, while those in [23,24] investigated multi-hop multi-user MIMO systems in which the adverse effects of one-bit ADCs were mitigated through supervised-learning-based techniques. However, such deep-learning approaches impose significant computational demands, which limits their practical deployment in energy- and resource-constrained devices.

More recently, an approximate ML (AML) detection scheme tailored for cooperative DF MIMO systems with one-bit ADCs was introduced [26]; it focuses on alleviating error propagation effects that arise in such configurations. The AML scheme is sufficiently versatile to be extended to one-way and two-way DF relay systems and has been shown to outperform direct communication in terms of error probability. Despite these advances, a rigorous analytical characterization of DF MIMO systems with one-bit ADCs, especially in the presence of fading channels, remains lacking. In the absence of such an analysis, it is difficult to gain theoretical insights into the error performance and achievable diversity gains of the system. Therefore, there is a clear need for analytical tools that can quantify the performance limits of cooperative MIMO systems with one-bit ADCs.

This paper presents a detailed performance analysis that focuses on the PEP of the AML detection over Rayleigh fading channels for addressing the performance limitations of cooperative DF MIMO systems with one-bit ADCs. Obtaining a closed-form expression for the exact PEP is analytically intractable because of the severe nonlinearity induced by one-bit quantization. To overcome this challenge, a tractable upper bound on the PEP is derived by employing Jensen’s inequality and the Chernoff bound. Building on the derived upper bound, the asymptotic behavior of the PEP is further investigated in the high signal-to-noise ratio (SNR) regime. These asymptotic results yield key theoretical insights into the diversity gain achievable in cooperative MIMO systems with one-bit ADCs. A primary finding is that the diversity gain can be enhanced through the DF relay protocol when symbol pairs considered in the PEP analysis form sign-reversed pairs and the relay correctly detects the source symbol. In contrast, the diversity contribution from the source-to-relay link is lost when the relay fails to decode the source symbol correctly and the gain is limited to the source-to-destination link. Extensive Monte Carlo simulations were conducted on Rayleigh fading channels to validate the analytical findings and evaluate their practical relevance. The simulation results closely match the derived upper bounds, demonstrating the accuracy of the analysis. Finally, the proposed analysis provides fundamental insights and design guidelines for cooperative MIMO systems with one-bit ADCs in energy-efficient wireless networks.

The remainder of this paper is organized as follows: Section 2 presents the system model for MIMO communications with one-bit ADCs and introduces the JML detection scheme along with its low-complexity approximation. The AML detection performance is analyzed in Section 3. Section 4 provides simulation results to validate the effectiveness of the proposed method. Finally, Section 5 concludes the paper.

Notation

Notations ${(·)}^T$ and ${(·)}^H$ represent the transpose and Hermitian (conjugate transpose) of a matrix, respectively. The operators $\mathbb{P}(·)$ and $\mathbb{E}(·)$ represent the probability and the expectation of an event, respectively, and $\mathrm{Re}(·)$ and $\mathrm{Im}(·)$ refer to the real and imaginary parts of the complex number, respectively. Function $log(·)$ represents a natural logarithm. The symbols $|·|$ and ${\parallel ·\parallel }_0$ represent the absolute value and zero norm (i.e., the number of nonzero elements), respectively. The indicator function $\mathbb{1}\left\{\mathcal{X}\right\}$ equals to 1 if the event $\mathcal{X}$ is true and is 0 otherwise. The sign function $\mathrm{Sign}\left(x\right)$ is defined as 1 for $x\ge 1$ and $-1$ otherwise. The Q-function is defined as $Q\left(x\right)={\displaystyle \frac{1}{\sqrt{2\pi }}}{\int }_x^{\infty }{e}^{-u^2/2}du$. Finally, $\mathbb{R}$ and $\mathbb{C}$ represent sets of real and complex numbers, respectively.

2. System Model

A signal model is formulated for point-to-point MIMO systems equipped with one-bit ADCs. Based on this model, a likelihood-based decision rule is developed to optimize the detection error probability under quantized observations. Subsequently, a cooperative DF MIMO system that exploits spatial diversity is introduced. The signal model and detection strategy serve as an analytical basis to extend the performance analysis to cooperative relay scenarios.

2.1. Signal Model

Consider a MIMO communication scenario in which transmitter node a is equipped with $N_a$ antennas and the receiver node b possesses $N_b$ antennas. The signal transmitted over a Rayleigh fading channel from node a to node b can be modeled as

$$\begin{array}{cc}\hfill {\overline{\mathbf{r}}}_{ab}& =\sqrt{{\displaystyle \frac{{\gamma }_{ab}}{N_a}}}{\overline{\mathbf{G}}}_{ab}{\overline{\mathbf{x}}}_a+{\overline{\mathbf{n}}}_{ab},\hfill \end{array}$$

(1)

where ${\gamma }_{ab}$ represents the average received SNR. The channel matrix ${\overline{\mathbf{G}}}_{ab}\in {\mathbb{C}}^{N_b\times N_a}$ is assumed to follow a Rayleigh fading distribution, where each element is independently drawn from a circularly symmetric complex Gaussian distribution with zero mean and unit variance. The transmitted signal vector ${\overline{\mathbf{x}}}_a\in {\mathbb{C}}^{N_a\times 1}$ is generated from encoded data symbols drawn from the modulation alphabet ${\overline{\mathcal{A}}}^{N_a}$, where $\overline{\mathcal{A}}$ represents an M-ary quadrature amplitude modulation (QAM) constellation. This signal is normalized such that its average power satisfies ${\displaystyle \frac{1}{N_a}}\mathbb{E}\{|{\overline{\mathbf{x}}}_a{|}^2\}=1$. The received signal is corrupted using additive white Gaussian noise ${\overline{\mathbf{n}}}_{ab}\in {\mathbb{C}}^{N_{ab}\times 1}$, which has independent components with zero mean and unit variance.

To enable symbol detection under one-bit quantization, the complex-valued received signal in (1) is reformulated in an equivalent real-valued representation ${\mathbf{r}}_{ab}={[\mathrm{Re}\left\{{\overline{\mathbf{r}}}_{ab}\right\},\mathrm{Im}\left\{{\overline{\mathbf{r}}}_{ab}\right\}]}^T$ as

$$\begin{array}{cc}\hfill {\mathbf{r}}_{ab}& =\sqrt{{\displaystyle \frac{{\gamma }_{ab}}{N_a}}}{\mathbf{G}}_{ab}{\mathbf{x}}_a+{\mathbf{n}}_{ab}={\mathbf{H}}_{ab}{\mathbf{x}}_a+{\mathbf{n}}_{ab},\hfill \end{array}$$

(2)

where

$$\begin{array}{cc}\hfill {\mathbf{G}}_{ab}& =\left[\begin{array}{cc}\mathrm{Re}\left\{{\overline{\mathbf{G}}}_{ab}\right\}& -\mathrm{Im}\left\{{\overline{\mathbf{G}}}_{ab}\right\}\\ \mathrm{Im}\left\{{\overline{\mathbf{G}}}_{ab}\right\}& \mathrm{Re}\left\{{\overline{\mathbf{G}}}_{ab}\right\}\end{array}\right],{\mathbf{x}}_a=\left[\begin{array}{c}\mathrm{Re}\left\{{\overline{\mathbf{x}}}_a\right\}\\ \mathrm{Im}\left\{{\overline{\mathbf{x}}}_a\right\}\end{array}\right]{\mathbf{n}}_{ab}=\left[\begin{array}{c}\mathrm{Re}\left\{{\overline{\mathbf{n}}}_{ab}\right\}\\ \mathrm{Im}\left\{{\overline{\mathbf{n}}}_{ab}\right\}\end{array}\right],\hfill \end{array}$$

(3)

where the effective channel matrix is defined as ${\mathbf{H}}_{ab}\triangleq \sqrt{{\displaystyle \frac{{\gamma }_{ab}}{N_a}}}{\mathbf{G}}_{ab}\in {\mathbb{R}}^{2N_b\times 2N_a}$. The real-valued input vector ${\mathbf{x}}_a\in {\mathcal{A}}^{2N_a}$ is drawn from the underlying modulation alphabet $\mathcal{A}$, which corresponds to an $\sqrt{M}$-ary pulse-amplitude modulation constellation. The ADC applied at the receiver imposes a hard-limiting operation, which converts each entry of ${\mathbf{r}}_{ab}$ to its sign. The quantized observation corresponding to the i-th dimension is given by

$$\begin{array}{cc}\hfill q_{ab,i}& =\mathrm{sign}\left(r_{ab.i}\right)=\mathrm{sign}({\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a+n_{ab,i}),\hfill \end{array}$$

(4)

where $i\in \mathcal{I}=\{1,\dots ,2N_b\}$ and ${\mathbf{h}}_{ab,i}^T$ represents the i-th row of the matrix ${\mathbf{H}}_{ab}$.

2.2. ML Detection

The ML detection rule is revisited in the context of one-bit quantized MIMO systems inspired by the formulation in [27]. Let $\mathcal{G}=\{i:q_{ab,i}\ge 0,i\in \mathcal{I}\}$ and $\mathcal{L}=\{i:q_{ab,i}<0,i\in \mathcal{I}\}$. Given a candidate signal vector ${\mathbf{x}}_a$, the probability of observing a quantized output ${\mathbf{q}}_{ab}$ is computed under the assumption of statistically independent components as follows:

$$\begin{array}{cc}\hfill \mathbb{P}\{{\mathbf{q}}_{ab}|{\mathbf{x}}_a\}& =\mathbb{P}\{q_{ab,i}\ge 0|\forall i\in \mathcal{G}\}\mathbb{P}\{q_{ab,i}<0|\forall i\in \mathcal{L}\}\hfill \\ \hfill & =\mathbb{P}\{q_{ab,i}{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a\ge -n_{ab,i}|\forall i\in \mathcal{G}\}\mathbb{P}\{q_{ab,i}{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a\ge n_{ab,i}|\forall i\in \mathcal{L}\}\hfill \\ \hfill & \stackrel{\left(a\right)}{=}\mathbb{P}\{q_{ab,i}{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a\ge -n_{ab,i}|\forall i\in \mathcal{G}\}\mathbb{P}\{q_{ab,i}{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a\ge -n_{ab,i}|\forall i\in \mathcal{L}\}\hfill \\ \hfill & \stackrel{\left(b\right)}{=}\prod _{i=1}^{2N_b}1-Q\left(\sqrt{2}{q}_{ab,i}{\mathbf{h}}_{ab,i}^T{x}_a\right),\hfill \end{array}$$

(5)

where step $\left(a\right)$ holds because $n_{ab,i}$ and $-n_{ab,i}$ are identically distributed, and step $\left(b\right)$ holds because ${\mathbf{n}}_{ab}$ follows Gaussian distribution with zero mean and ${\displaystyle \frac{1}{2}}$ variance.

Then, the likelihood probability can be formulated as follows.

$$\begin{array}{cc}\hfill \mathbb{P}\{{\mathbf{q}}_{ab}|{\mathbf{x}}_a\}& =\prod _{q_{ab,i}=c_{ab,i}\left({\mathbf{x}}_a\right)}\left(1-Q\left(\sqrt{2}{q}_{ab,i}{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a\right)\right)\prod _{q_{ab,i}\ne c_{ab,i}\left({\mathbf{x}}_a\right)}\left(1-Q\left(\sqrt{2}{q}_{ab,i}{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a\right)\right)\hfill \\ \hfill & \stackrel{\left(a\right)}{=}\prod _{q_{ab,i}=c_{ab,i}\left({\mathbf{x}}_a\right)}\left(1-Q\left(\sqrt{2}{q}_{ab,i}{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a\right)\right)\prod _{q_{ab,i}\ne c_{ab,i}\left({\mathbf{x}}_a\right)}Q\left(-\sqrt{2}{q}_{ab,i}{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a\right)\hfill \\ \hfill & =\prod _{q_{ab,i}=c_{ab,i}\left({\mathbf{x}}_a\right)}\left(1-Q\left(\sqrt{2}|{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a|\right)\right)\prod _{q_{ab,i}\ne c_{ab,i}\left({\mathbf{x}}_a\right)}Q\left(\sqrt{2}|{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a|\right)\hfill \\ \hfill & =\prod _{i=1}^{2N_b}\mathbb{P}\{q_{ab,i}|c_{ab,i}\left({\mathbf{x}}_a\right)\},\hfill \end{array}$$

(6)

where $c_{ab,i}\left({\mathbf{x}}_a\right)$ represents the i-th entry of the noiseless decision vector ${\mathbf{c}}_{ab}\left({\mathbf{x}}_a\right)\triangleq \mathrm{sign}\left({\mathbf{H}}_{ab}{\mathbf{x}}_a\right)$ in $\left(a\right)$. This vector captures the quantized version of the channel output in the absence of noise. The conditional probability $\mathbb{P}\{q_{ab,i}|c_{ab,i}\left({\mathbf{x}}_a\right)\}$ of an individual quantized symbol $q_{ab,i}$ depends on whether it matches the ideal decision bit $c_{ab,i}$, and it is given by

$$\begin{array}{cc}\hfill \mathbb{P}\{q_{ab,i}|c_{ab,i}\left({\mathbf{x}}_a\right)\}& =\left\{\begin{array}{cc}{p}_{ab,i}\left({\mathbf{x}}_a\right)\hfill & ,q_{ab,i}\ne c_{ab,i}\left({\mathbf{x}}_a\right)\hfill \\ 1-p_{ab,i}\left({\mathbf{x}}_a\right)\hfill & ,q_{ab,i}=c_{ab,i}\left({\mathbf{x}}_a\right),\hfill \end{array}\right.\hfill \end{array}$$

(7)

where the error probability $p_{ab,i}\left({\mathbf{x}}_a\right)$ associated with the i-th bit is characterized using the Q-function $Q\left(\sqrt{2}|{\mathbf{h}}_{ab,i}^T{\mathbf{x}}_a|\right)$.

Taking the logarithm of the likelihood probability in (6), the product of individual conditional probabilities is converted into a sum, which enables a more tractable metric for detection.

$$\begin{array}{cc}\hfill & log\left(\mathbb{P}\{{\mathbf{q}}_{ab}|{\mathbf{x}}_a\}\right)\hfill \\ \hfill & =\sum _{i=1}^{2N_b}\left[log\left(p_{ab,i}\left({\mathbf{x}}_a\right)\right)\parallel q_{ab,i}-c_{ab,i}\left({\mathbf{x}}_a\right){\parallel }_0+log\left(1-p_{ab,i}\left({\mathbf{x}}_a\right)\right)(1-\parallel q_{ab,i}-c_{ab,i}\left({\mathbf{x}}_a\right){\parallel }_0)\right]\hfill \\ \hfill & =-\left(\sum _{i=1}^{2N_b}\left[w_{ab,i}\left({\mathbf{x}}_a\right)\parallel q_{ab,i}-c_{ab,i}\left({\mathbf{x}}_a\right){\parallel }_0+{\tilde{w}}_{ab,i}\left({\mathbf{x}}_a\right)(1-\parallel q_{ab,i}-c_{ab,i}\left({\mathbf{x}}_a\right){\parallel }_0)\right]\right),\hfill \end{array}$$

(8)

where $w_{ab,i}\left({\mathbf{x}}_a\right)=-log\left(p_{ab,i}\left({\mathbf{x}}_a\right)\right)$ and ${\tilde{w}}_{ab,i}\left({\mathbf{x}}_a\right)=-log\left(1-p_{ab,i}\left({\mathbf{x}}_a\right)\right)$. The above expression is simplified to a negative-weighted sum.

$$\begin{array}{cc}\hfill d_w\left({\mathbf{q}}_{ab},{\mathbf{c}}_{ab}\left({\mathbf{x}}_a\right);{\mathbf{w}}_{ab}\left({\mathbf{x}}_a\right),{\tilde{\mathbf{w}}}_{ab}\left({\mathbf{x}}_a\right)\right)& =-log\left(\mathbb{P}\{{\mathbf{q}}_{ab}|{\mathbf{x}}_a\}\right).\hfill \end{array}$$

(9)

This expression motivates the introduction of a weighted Hamming distance function that quantifies the distance between the observed and noise-free quantized outputs using symbol-wise weights. This metric is defined as

$$\begin{array}{cc}\hfill {\widehat{\mathbf{x}}}_a& =\underset{{\mathbf{x}}_a\in {\mathcal{A}}^{2N_a}}{argmax}log\left(\mathbb{P}\{{\mathbf{q}}_{ab}|{\mathbf{x}}_a\}\right)=\underset{{\mathbf{x}}_a\in {\mathcal{A}}^{2N_a}}{argmin}{d}_w\left({\mathbf{q}}_{ab},{\mathbf{c}}_{ab}\left({\mathbf{x}}_a\right);{\mathbf{w}}_{ab}\left({\mathbf{x}}_a\right),{\tilde{\mathbf{w}}}_{ab}\left({\mathbf{x}}_a\right)\right).\hfill \end{array}$$

(10)

The ML decision rule corresponds to the selection of the input vector that yields the smallest weighted Hamming distance in (10).

2.3. JML Detection

A JML detection rule is formulated to enhance detection performance and fully exploit the diversity potential of the system.

The relay protocol operation is illustrated in Figure 1, where the source, relay, and destination nodes are denoted by S, R, and D, respectively. During the first transmission phase (time slot 1), the source broadcasts a signal vector ${\mathbf{x}}_S$ received by both the relay and destination nodes. The transmitted signal can be expressed as

$$\begin{array}{cc}\hfill {\mathbf{q}}_{SR}& =\mathrm{sign}({\mathbf{H}}_{SR}{\mathbf{x}}_S+{\mathbf{n}}_{SR}),{\mathbf{q}}_{SD}=\mathrm{sign}({\mathbf{H}}_{SD}{\mathbf{x}}_S+{\mathbf{n}}_{SD}).\hfill \end{array}$$

(11)

In the direct transmission path, the destination node attempts to recover the transmitted vector ${\mathbf{x}}_S$ based on the quantized observation ${\mathbf{q}}_{SD}$. Meanwhile, on receiving the quantized signal ${\mathbf{q}}_{SR}$, the relay node performs signal detection by applying the ML decision rule to estimate ${\mathbf{x}}_S$.

$$\begin{array}{cc}\hfill {\mathbf{x}}_R& =\underset{{\mathbf{x}}_S\in {\mathcal{A}}^{2N_S}}{argmin}{d}_w\left({\mathbf{q}}_{SR},{\mathbf{c}}_{SR}\left({\mathbf{x}}_S\right);{\mathbf{w}}_{SR}\left({\mathbf{x}}_R\right),{\tilde{\mathbf{w}}}_{SR}\left({\mathbf{x}}_S\right)\right).\hfill \end{array}$$

(12)

During the second transmission phase (time slot 2), the relay forwards the previously detected signal ${\mathbf{x}}_R$ to the destination node. The signal transmitted from the relay can be represented as

$$\begin{array}{cc}\hfill {\mathbf{q}}_{RD}& =\mathrm{sign}({\mathbf{H}}_{RD}{\mathbf{x}}_R+{\mathbf{w}}_{RD}).\hfill \end{array}$$

(13)

At the destination node, joint detection is performed by utilizing the quantized signals received from both transmission phases, namely ${\mathbf{q}}_{SD}$ and ${\mathbf{q}}_{RD}$, to recover the original source signal ${\mathbf{x}}_S$. A widely adopted approach for maximizing diversity in such cooperative systems is the JML detection method, originally proposed by [10], which formulates the decision rule as

$$\begin{array}{cc}\hfill {\widehat{\mathbf{x}}}_S& =\underset{{\mathbf{x}}_S\in {\mathcal{A}}^{2N_S}}{argmax}\mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{x}}_S\}=\underset{{\mathbf{x}}_S\in {\mathcal{A}}^{2N_S}}{argmax}\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{x}}_S\}\times \sum _{{\mathbf{x}}_R\in {\mathcal{A}}^{2N_S}}\mathbb{P}\{{\mathbf{x}}_R|{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{x}}_R\},\hfill \end{array}$$

(14)

where the combined quantized observation at the destination is denoted by ${\mathbf{q}}_{SRD}={[{\mathbf{q}}_{SD}^T,{\mathbf{q}}_{RD}^T]}^T$, which aggregates the information received over both time slots. The probability that the relay decodes $\left({\mathbf{x}}_S\right)$ into an incorrect symbol ${\mathbf{x}}_R$ is represented by the conditional error probability $P\{{\mathbf{x}}_R|{\mathbf{x}}_S\}$.

2.4. AML Detection

The JML detection approach described in (14) entails significant computational complexity because of the exhaustive search over ${|\mathcal{A}|}^{4N_S}$ possible symbol combinations. An approximation detection method based on error propagation at the relay is proposed in [26] to alleviate this burden. This method applies a combination of the max-log and PEP approximations to simplify the original JML formulation.

$$\begin{array}{cc}\hfill {\widehat{\mathbf{x}}}_S& \stackrel{\left(a\right)}{=}\underset{{\mathbf{x}}_S\in {\mathcal{A}}^{2N_S}}{argmax}\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{x}}_S\}\times \underset{{\mathbf{x}}_R\in {\mathcal{A}}^{2N_S}}{max}\mathbb{P}\{{\mathbf{x}}_R|{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{x}}_R\}\hfill \\ \hfill & =\underset{{[{\mathbf{x}}_S^T,{\mathbf{x}}_R^T]}^T\in {\mathcal{A}}^{4N_S}}{argmax}\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{x}}_R|{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{x}}_R\}\hfill \\ \hfill & \stackrel{\left(b\right)}{=}\underset{{[{\mathbf{x}}_S^T,{\mathbf{x}}_R^T]}^T\in {\mathcal{A}}^{4N_S}}{argmax}\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{x}}_R\}.\hfill \end{array}$$

(15)

Step $\left(a\right)$ employs the max-log approximation to simplify the logarithmic sum of the detection metric. In step $\left(b\right)$, the conditional error probability is further approximated using the PEP, which enables a tractable analysis of the detection reliability of the relay.

Term $\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}$ represents the PEP, which indicates the likelihood that the relay erroneously detects ${\mathbf{x}}_R$ given that ${\mathbf{x}}_S$ is originally transmitted, and it is also expressed as the weighted Hamming distance function [26].

$$log\left(\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}\right) \le -d_w\left({\mathbf{c}}_{SR}\left({\mathbf{x}}_S\right),{\mathbf{c}}_{SR}\left({\mathbf{x}}_R\right);{\mathbf{w}}_{SR}({\mathbf{x}}_S,{\tilde{\mathbf{x}}}_R),{\tilde{\mathbf{w}}}_{SR}({\mathbf{x}}_S,{\mathbf{x}}_R)\right).$$

(16)

Since the PEP admits an upper bound that depends on the weighted Hamming distance between two quantized codewords ${\mathbf{c}}_{ab}\left({\mathbf{x}}_a\right)$ and ${\mathbf{c}}_{ab}\left({\tilde{\mathbf{x}}}_a\right)$, the detection rule in (15) can be reformulated in a more compact and computationally efficient form as

$$\begin{array}{cc}\hfill {\widehat{\mathbf{x}}}_S& =\underset{{[{\mathbf{x}}_S^T,{\mathbf{x}}_R^T]}^T\in {\mathcal{A}}^{4N_S}}{argmin}{d}_w\left(\left[\begin{array}{c}{\mathbf{q}}_{SD}\\ {\mathbf{q}}_{RD}\\ {\mathbf{c}}_{SR}\left({\mathbf{x}}_S\right)\end{array}\right],\left[\begin{array}{c}{\mathbf{c}}_{SD}\left({\mathbf{x}}_S\right)\\ {\mathbf{c}}_{RD}\left({\mathbf{x}}_R\right)\\ {\mathbf{c}}_{SR}\left({\mathbf{x}}_R\right)\end{array}\right];\left[\begin{array}{c}{\mathbf{w}}_{SD}\left({\mathbf{x}}_S\right)\\ {\mathbf{w}}_{RD}\left({\mathbf{x}}_R\right)\\ {\mathbf{w}}_{SR}({\mathbf{x}}_S,{\mathbf{x}}_R)\end{array}\right],\left[\begin{array}{c}{\tilde{\mathbf{w}}}_{SD}\left({\mathbf{x}}_S\right)\\ {\tilde{\mathbf{w}}}_{RD}\left({\mathbf{x}}_R\right)\\ {\tilde{\mathbf{w}}}_{SR}({\mathbf{x}}_S,{\mathbf{x}}_R)\end{array}\right]\right).\hfill \end{array}$$

(17)

This unified representation consolidates three separate weighted Hamming distance terms into a single expression, simplifying implementation and reducing computational complexity. The detection strategy in (17) enables the interpretation of a DF relay system with an antenna configuration ($N_S,N_R,N_D$) as an equivalent MIMO system comprising $(N_S+N_R)$ transmitting antennas and $(2N_D+N_R)$ receiving antennas. This structural equivalence enables the low-complexity detection method proposed in [27] to be applied directly to (17). Consequently, AML achieves a substantial reduction in computational complexity compared to the original JML detection rule.

A comparison of the detection schemes is given below:

• ML detection: Optimal decision rule for point-to-point MIMO systems. It maximizes the likelihood probability based on the received signal and provides the lowest bit error rate under perfect channel state information.
• JML detection: An extension of the ML decision rule to cooperative DF MIMO systems. It considers both source-to-destination and relay-to-destination links. However, it suffers from high computational complexity because of an exhaustive search over all possible symbol combinations.
• AML detection: Low-complexity approximation of JML detection. Its decision rule is derived from a tractable PEP expression, which significantly reduces the computational burden while preserving a near-optimal performance.

3. Performance Analysis

Jensen’s inequality and the Chernoff bound are applied to the likelihood probability of AML to derive a tractable expression. Based on the derived upper bound, the asymptotic behavior of PEP is examined to reveal its relationship with the achievable diversity order.

3.1. PEP Analysis

PEP plays a central role in characterizing the diversity performance of MIMO systems because the overall detection error probability can be upper-bounded by a weighted sum of PEPs over all symbol pairs [28]. However, it is difficult to obtain a closed-form expression for the exact PEP because of the inherent nonlinearity introduced by the one-bit quantizer in (4).

To address this issue, an upper bound on the PEP is derived using Jensen’s inequality and the Chernoff bound, which offers analytical tractability while preserving key performance insights. Jensen’s inequality is applied to handle the square root function, whereas the Chernoff bound is employed to bound the Q-function. This combination of techniques is commonly used to derive the mutual information and bit error rate expressions in communication theory. By leveraging these tools, PEP $\mathcal{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S\}$ can be obtained by averaging over the channel, which is given by

$$\mathcal{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S\}=\mathbb{E}\left\{\mathbb{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S|{\mathbf{H}}_{SRD}\}\right\},$$

(18)

where ${\mathbf{H}}_{SRD}={[{\mathbf{H}}_{SD}^T,{\mathbf{H}}_{RD}^T,{\mathbf{H}}_{SR}^T]}^T$. Given a specific channel realization ${\mathbf{H}}_{SRD}$, the PEP that erroneously determines in favor of ${\tilde{\mathbf{x}}}_S$ when ${\mathbf{x}}_S$ is transmitted can be expressed as

$$\begin{array}{cc}\hfill \mathbb{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S|{\mathbf{H}}_{SRD}\}& ={\displaystyle \sum _{{\mathbf{q}}_{SRD}\in {\{-1,1\}}^{4N_D+2N_R}}}\mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{H}}_{SRD},{\mathbf{x}}_S\}\hfill \\ \hfill & \times \mathbb{1}\{\mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{H}}_{SRD},{\mathbf{x}}_S\}\le \mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{H}}_{SRD},{\tilde{\mathbf{x}}}_S\}\}.\hfill \end{array}$$

(19)

The indicator function can be upper bounded using the inequality $\mathbb{1}\{A\le B\}\le {\left({\displaystyle \frac{B}{A}}\right)}^s$, where $s\in [0,1]$ [29]. Applying this bound to (19) with $s=1/2$, the PEP in (19) can be further upper bounded as

$$\begin{array}{cc}\hfill \mathbb{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S|{\mathbf{H}}_{SRD}\}& \le \sum _{{\mathbf{q}}_{SRD}\in {\{-1,1\}}^{4N_D+2N_R}}\mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{H}}_{SRD},{\mathbf{x}}_S\}{\left({\displaystyle \frac{\mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{H}}_{SRD},{\tilde{\mathbf{x}}}_S\}}{\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{H}}_{SRD},{\mathbf{x}}_S\}}}\right)}^{{\displaystyle \frac{1}{2}}}\hfill \\ \hfill & =\sum _{{\mathbf{q}}_{SRD}\in {\{-1,1\}}^{4N_D+2N_R}}\sqrt{\mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{H}}_{SRD},{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{H}}_{SRD},{\tilde{\mathbf{x}}}_S\}}.\hfill \end{array}$$

(20)

The likelihood probability $\mathbb{P}\{{\mathbf{q}}_{SRD}|{\mathbf{H}}_{SRD},{\mathbf{x}}_S\}$ of the AML is further approximated using the max-log and PEP approximations, following the approach in (15), which yields

$$\begin{array}{cc}\hfill & \mathbb{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S|{\mathbf{H}}_{SRD}\}\hfill \\ \hfill & \le \sum _{{\mathbf{q}}_{SD}\in {\{-1,1\}}^{2N_D}}\sum _{{\mathbf{q}}_{RD}\in {\{-1,1\}}^{2N_D}}\hfill & \sqrt{\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{H}}_{SD},{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{H}}_{RD},{\mathbf{x}}_R\}}\hfill \\ \hfill & \hfill \times & \sqrt{\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{H}}_{SD},{\tilde{\mathbf{x}}}_S\}\mathbb{P}\{{\tilde{\mathbf{x}}}_S\to {\tilde{\mathbf{x}}}_R\}\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{H}}_{RD},{\tilde{\mathbf{x}}}_R\}}.\hfill \end{array}$$

(21)

Given that ${\sum }_{i=1}^n\sqrt{a_i{b}_i{c}_i}\le {\sum }_{i=1}^n\sqrt{a_i}{\sum }_{i=1}^n\sqrt{b_i}{\sum }_{i=1}^n\sqrt{c_i}$ when $0\le a_i,b_i,c_i\le 1,\forall i$, the inequality in (21) can be upper-bounded accordingly as

$$\begin{array}{cc}\hfill & \mathbb{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S|{\mathbf{H}}_{SRD}\}\le \sum _{{\mathbf{q}}_{SD}\in {\{-1,1\}}^{2N_D}}\sqrt{\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{H}}_{SD},{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{H}}_{SD},{\tilde{\mathbf{x}}}_S\}}\hfill \\ \hfill & \times \sum _{{\mathbf{q}}_{RD}\in {\{-1,1\}}^{2N_D}}\sqrt{\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{H}}_{RD},{\mathbf{x}}_R\}\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{H}}_{RD},{\tilde{\mathbf{x}}}_R\}}\times \sqrt{\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}\mathbb{P}\{{\tilde{\mathbf{x}}}_S\to {\tilde{\mathbf{x}}}_R\}}.\hfill \end{array}$$

(22)

Subsequently, the PEP is obtained by taking the expectation over channel realizations, which results in

$$\begin{array}{cc}\hfill \mathcal{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S\}& \le \underset{\triangleq A1}{\underbrace{\mathbb{E}\{\sum _{{\mathbf{q}}_{SD}\in {\{-1,1\}}^{2N_D}}\sqrt{\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{H}}_{SD},{\mathbf{x}}_S\}\mathbb{P}\{{\mathbf{q}}_{SD}|{\mathbf{H}}_{SD},{\tilde{\mathbf{x}}}_S\}}\}}}\hfill \\ \hfill & \times \underset{\triangleq A2}{\underbrace{\mathbb{E}\{\sum _{{\mathbf{q}}_{RD}\in {\{-1,1\}}^{2N_{RD}}}\sqrt{\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{H}}_{RD},{\mathbf{x}}_R\}\mathbb{P}\{{\mathbf{q}}_{RD}|{\mathbf{H}}_{RD},{\tilde{\mathbf{x}}}_R\}}\}}}\hfill \\ \hfill & \times \underset{B}{\underbrace{\mathbb{E}\{\sqrt{\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}\mathbb{P}\{{\tilde{\mathbf{x}}}_S\to {\tilde{\mathbf{x}}}_R\}}\}}},\hfill \end{array}$$

(23)

where the detections of $A1$ and $A2$ at the destination and relay are independent. Terms $A1$ and $A2$ in (23) can be derived by using the following lemma

Lemma 1.

$$\mathbb{E}\{\sum _{{\mathbf{q}}_{ab}\in {\{-1,1\}}^{2N_b}}\mathbb{P}\{{\mathbf{q}}_{ab}|{\mathbf{H}}_{ab},{\mathbf{x}}_a\}\mathbb{P}\{{\mathbf{q}}_{ab}|{\mathbf{H}}_b,{\tilde{\mathbf{x}}}_a\}\} \le f{({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)}^{2N_b}.$$

(24)

where $f({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)=\left(1-{\displaystyle \frac{2}{\pi }}arctan\left({\displaystyle \frac{\parallel {\overline{\mathbf{x}}}_a+{\overline{\tilde{\mathbf{x}}}}_a\parallel }{\parallel {\overline{\mathbf{x}}}_a-{\overline{\tilde{\mathbf{x}}}}_a\parallel }}\right)\right)\left(\sqrt{{\displaystyle \frac{2N_a}{2N_a+{\gamma }_{ab}{\parallel {\overline{\mathbf{x}}}_a\parallel }^2}}}+\sqrt{{\displaystyle \frac{2N_a}{2N_a+{\gamma }_{ab}{\parallel {\overline{\tilde{\mathbf{x}}}}_a\parallel }^2}}}\right)+{\displaystyle \frac{2}{\pi }}arctan\left({\displaystyle \frac{\parallel {\overline{\mathbf{x}}}_a+{\overline{\tilde{\mathbf{x}}}}_a\parallel }{\parallel {\overline{\mathbf{x}}}_a-{\overline{\tilde{\mathbf{x}}}}_a\parallel }}\right)$.

Proof. 

See Appendix A. □

In addition, term B in (23) can be obtained using the following lemma.

Lemma 2.

$$\begin{array}{cc}\hfill \mathbb{E}\left\{\sqrt{\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}\mathbb{P}\{{\tilde{\mathbf{x}}}_S\to {\tilde{\mathbf{x}}}_R\}}\right\}& \le f{({\gamma }_{SR},{\widehat{\mathbf{x}}}_s,{\widehat{\mathbf{x}}}_R)}^{2N_b},\hfill \end{array}$$

(25)

where ${\widehat{\mathbf{x}}}_S={[{\mathbf{x}}_S^T,{\tilde{\mathbf{x}}}_S^T]}^T$ and ${\widehat{\mathbf{x}}}_R={[{\mathbf{x}}_R^T,{\tilde{\mathbf{x}}}_R^T]}^T$.

Proof. 

Jensen’s inequality is applied to the square root terms in B. Then,

$$\begin{array}{cc}\hfill \mathbb{E}\left\{\sqrt{\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}\mathbb{P}\{{\tilde{\mathbf{x}}}_S\to {\tilde{\mathbf{x}}}_R\}}\right\}& \le \sqrt{\mathbb{E}\left\{\mathbb{P}\{{\mathbf{x}}_S\to {\mathbf{x}}_R\}\mathbb{P}\{{\tilde{\mathbf{x}}}_S\to {\tilde{\mathbf{x}}}_R\}\right\}}\hfill \\ & \stackrel{\left(a\right)}{=}\sqrt{f{({\gamma }_{SR},{\widehat{\mathbf{x}}}_s,{\widehat{\mathbf{x}}}_R)}^{4N_b}},\hfill \end{array}$$

(26)

where the equality in $\left(a\right)$ follows by applying the PEP bound derived in [30], whose bound $\mathcal{P}\{{\mathbf{x}}_a\to {\tilde{\mathbf{x}}}_a\}\le f{({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)}^{2N_b}$. ${\widehat{\mathbf{x}}}_S={[{\mathbf{x}}_S^T,{\tilde{\mathbf{x}}}_S^T]}^T$ and ${\widehat{\mathbf{x}}}_R={[{\mathbf{x}}_R^T,{\tilde{\mathbf{x}}}_R^T]}^T$ are extended symbol vectors that share the same source-to-relay channel ${\mathbf{H}}_{SR}$. □

By using Lemma 1, the PEP in (23) is obtained by the following theorem.

Theorem 1.

Let ${\mathbf{x}}_S$ and ${\tilde{\mathbf{x}}}_S$ represent two symbol vectors to detect ${\tilde{\mathbf{x}}}_S(\ne {\mathbf{x}}_S)$ when ${\mathbf{x}}_S$ is transmitted. The PEP of the symbol vectors for the AML with one-bit ADCs in Rayleigh fading channels is upper bounded as

$$\begin{array}{cc}\hfill \mathcal{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S\}& \le f{({\gamma }_{SD},{\mathbf{x}}_S,{\tilde{\mathbf{x}}}_S)}^{2N_D}f{({\gamma }_{RD},{\mathbf{x}}_R,{\tilde{\mathbf{x}}}_R)}^{2N_D}f{({\gamma }_{SR},\left[\begin{array}{c}{\mathbf{x}}_S\\ {\tilde{\mathbf{x}}}_S\end{array}\right],\left[\begin{array}{c}{\mathbf{x}}_R\\ {\tilde{\mathbf{x}}}_R\end{array}\right])}^{2N_R},\hfill \end{array}$$

(27)

where

$$\begin{array}{cc}\hfill & f({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)\hfill \\ \hfill & =\left(1-{\displaystyle \frac{2}{\pi }}arctan\left({\displaystyle \frac{\parallel {\overline{\mathbf{x}}}_a+{\overline{\tilde{\mathbf{x}}}}_a\parallel }{\parallel {\overline{\mathbf{x}}}_a-{\overline{\tilde{\mathbf{x}}}}_a\parallel }}\right)\right)\left(\sqrt{{\displaystyle \frac{2N_a}{2N_a+{\gamma }_{ab}{\parallel {\overline{\mathbf{x}}}_a\parallel }^2}}}+\sqrt{{\displaystyle \frac{2N_a}{2N_a+{\gamma }_{ab}{\parallel {\overline{\tilde{\mathbf{x}}}}_a\parallel }^2}}}\right)\hfill \\ \hfill & +{\displaystyle \frac{2}{\pi }}arctan\left({\displaystyle \frac{\parallel {\overline{\mathbf{x}}}_a+{\overline{\tilde{\mathbf{x}}}}_a\parallel }{\parallel {\overline{\mathbf{x}}}_a-{\overline{\tilde{\mathbf{x}}}}_a\parallel }}\right).\hfill \end{array}$$

(28)

Theorem 1 shows that the upper bound of the PEP decreases with an increase in the number of receive antennas. This result is in line with the intuition because the probability of successfully distinguishing two distinct symbol vectors increases with a number of independent observations at the receiver, even if these observations are binary. However, this shows that the upper bound of the PEP does not vanish with an increase in the SNR increases because of the use of one-bit ADCs if ${\mathbf{x}}_a\ne -{\tilde{\mathbf{x}}}_a$.

3.2. Asymptotic Behavior

The asymptotic upper bound of the PEP is analyzed to gain deeper insight into how PEP influences the diversity gain of AML detection with one-bit ADCs. Therefore, the asymptotic behavior of the array gain function $f({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)$ in Theorem 1 is examined. Subsequently, the diversity gain of PEP is investigated for various combinations of pairwise symbol vectors.

The array gain function $f({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)$, which governs the behavior of the PEP, exhibits distinct asymptotic characteristics depending on whether ${\mathbf{x}}_a=-{\tilde{\mathbf{x}}}_a$ or ${\mathbf{x}}_a\ne -{\tilde{\mathbf{x}}}_a$. The asymptotic behavior of this array gain function is summarized in the following corollary.

Corollary 1.

Let ${\mathbf{x}}_a$ and ${\tilde{\mathbf{x}}}_a$ represent two distinct symbol vectors. Subsequently, the asymptotic behavior of the array gain function $f({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)$ as ${\gamma }_{ab}\to \infty$ is characterized as

• If ${\mathbf{x}}_a=-{\tilde{\mathbf{x}}}_a$, then

  $$\begin{array}{c}\hfill f({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)\to {\displaystyle \frac{2\sqrt{2N_a}}{\parallel {\overline{\mathbf{x}}}_a\parallel }}{\gamma }_{ab}^{-{\displaystyle \frac{1}{2}}}.\end{array}$$

  (29)
• If ${\mathbf{x}}_a\ne -{\tilde{\mathbf{x}}}_a$, then

  $$\begin{array}{c}\hfill f({\gamma }_{ab},{\mathbf{x}}_a,{\tilde{\mathbf{x}}}_a)\to {\displaystyle \frac{2}{\pi }}arctan\left({\displaystyle \frac{\parallel {\overline{\mathbf{x}}}_a+{\overline{\tilde{\mathbf{x}}}}_a\parallel }{\parallel {\overline{\mathbf{x}}}_a-{\overline{\tilde{\mathbf{x}}}}_a\parallel }}\right).\end{array}$$

  (30)

The corollary states that the asymptotic behavior of the array gain function remains constant when ${\mathbf{x}}_a\ne -{\tilde{\mathbf{x}}}_a$. In contrast, the function decays with increasing ${\gamma }_{ab}$ at a rate proportional to the order of ${\displaystyle \frac{1}{2}}$ when ${\mathbf{x}}_a=-{\tilde{\mathbf{x}}}_a$. This behavior arises because the two symbol vectors (i.e., ${\mathbf{x}}_a$ and ${\tilde{\mathbf{x}}}_a$) in Rayleigh fading channels become indistinguishable at the receiver when they are not sign-reversed pairs.

Using Corollary 1 and Theorem 1, the asymptotic behavior of PEP for AML using one-bit ADCs can be analyzed as

Corollary 2.

Suppose that ${\mathbf{x}}_S=-{\tilde{\mathbf{x}}}_S$ and ${\mathbf{x}}_R={\tilde{\mathbf{x}}}_R$. In this case, full diversity gain can theoretically be achieved if ${\widehat{\mathbf{x}}}_S=-{\widehat{\mathbf{x}}}_R$. However, under these conditions, it follows that ${\mathbf{x}}_S={\mathbf{x}}_R$, which contradicts the assumption ${\mathbf{x}}_S=-{\mathbf{x}}_R$. Thus, the PEP behaves as

$$\begin{array}{cc}\hfill \mathcal{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S\}& \to A_1{\gamma }_{SD}^{-N_D},\hfill \end{array}$$

(31)

where $A_1={\left({\displaystyle \frac{2}{\pi }}arctan\left({\displaystyle \frac{\parallel {\overline{\mathbf{x}}}_R+{\overline{\tilde{\mathbf{x}}}}_R\parallel }{\parallel {\overline{\mathbf{x}}}_R-{\overline{\tilde{\mathbf{x}}}}_R\parallel }}\right)\right)}^{2N_D}\times {\left({\displaystyle \frac{2}{\pi }}arctan\left({\displaystyle \frac{\parallel {\overline{\widehat{\mathbf{x}}}}_{\mathrm{S}}+{\overline{\widehat{\mathbf{x}}}}_{\mathrm{R}}\parallel }{\parallel {\overline{\widehat{\mathbf{x}}}}_{\mathrm{S}}-{\overline{\widehat{\mathbf{x}}}}_{\mathrm{R}}\parallel }}\right)\right)}^{2N_R}\times {\displaystyle \frac{2\sqrt{2N_S}}{\parallel {\overline{\mathbf{x}}}_S\parallel }}$.

In this corollary, the full diversity gain of $N_D+N_R$ is not achieved in cooperative DF MIMO systems with one-bit ADCs. This limitation arises because the full diversity gain can only be obtained when the involved symbol vectors in PEP satisfy specific sign-inversion (i.e., “minus”) conditions. However, these conditions are not satisfied in the presence of source-to-relay detection errors, fundamentally limiting the achievable diversity in these scenarios.

Corollary 3.

Under the conditions ${\mathbf{x}}_S=-{\tilde{\mathbf{x}}}_S$, ${\mathbf{x}}_S={\mathbf{x}}_R$, and ${\tilde{\mathbf{x}}}_S={\tilde{\mathbf{x}}}_R$, full diversity gain can be achieved because ${\mathbf{x}}_R=-{\tilde{\mathbf{x}}}_R$. In this case, PEP behaves as

$$\begin{array}{cc}\hfill \mathcal{P}\{{\mathbf{x}}_S\to {\tilde{\mathbf{x}}}_S\}& \to A_2{\gamma }_{SD}^{-N_D}{\gamma }_{RD}^{-N_D},\hfill \end{array}$$

(32)

where $A_2={\left({\displaystyle \frac{2}{\pi }}arctan\left({\displaystyle \frac{\parallel {\overline{\widehat{\mathbf{x}}}}_{\mathrm{S}}+{\overline{\widehat{\mathbf{x}}}}_{\mathrm{R}}\parallel }{\parallel {\overline{\widehat{\mathbf{x}}}}_{\mathrm{S}}-{\overline{\widehat{\mathbf{x}}}}_{\mathrm{R}}\parallel }}\right)\right)}^{2N_R}\times {\displaystyle \frac{2\sqrt{2N_S}}{\parallel {\overline{\mathbf{x}}}_S\parallel }}\times {\displaystyle \frac{2\sqrt{2N_R}}{\parallel {\overline{\mathbf{x}}}_R\parallel }}$.

Based on Corollary 3, assuming error-free decoding at the relay enables the cooperative MIMO system to achieve full diversity gain contributed by both the source–destination and relay–destination links. However, Corollaries 2 and 3 collectively indicate that the diversity gain of the AML detector with one-bit ADCs is limited to $N_D+min\{0,N_D\}$ because the decoding success and error at the relay cannot occur simultaneously. Consequently, the achievable diversity gain in the DF MIMO systems is reduced to $N_D$. This result contrasts with the case of the infinite-resolution ADCs studied in [15], where the diversity gain is given by $N_D+min\{N_R,N_D\}$.

4. Simulations

The numerical results are obtained via Monte Carlo simulations, averaging over ${10}^6$ independent realizations of the channel matrices and additive noise. Perfect channel state information is assumed at each node to demonstrate the accuracy of the proposed analytical results. Binary phase-shift keying (BPSK) modulation is employed throughout the simulations to focus on the PEP behavior because it facilitates the generation of sign-inverted symbol pairs relevant to the analysis. Therefore, the number of transmit antennas at the source, relay, and destination nodes is fixed at one, whereas the number of receive antennas is varied across simulation scenarios $(N_S,N_R,N_D)$. Each transmission block consists of 256 data symbols, and a rate-${\displaystyle \frac{1}{2}}$ turbo code is adopted for channel coding. The turbo code is based on a parallel concatenated structure using feedforward and feedback polynomials $(15, 13)$ in the octal notation. In addition, it is assumed that a source-to-relay link is available at the destination node for detection. The block error rate (BLER) and PEP are evaluated to provide the practical system performance and validation of the analytical findings. Key abbreviations and system parameters used in the simulations are listed in

Table 1. Key abbreviations and system parameters in simulation.

Abbreviations and System Parameters	Descriptions
DC	Direct Communication
MRC	Maximum Ratio Combining
SDF	Selective Decode and Forward
AML	Approximate Maximum Likelihood
JML	Join Maximum Likelihood
$N_S$	Number of Transmit Antennas at Source Node
$N_R$	Number of Transmit Antennas at Relay Node
$N_D$	Number of Transmit Antennas at Destination Node

.

The BLER performance of various detection schemes is compared with that of the AML detection method in Figure 2. The BLER of direct communication (DC) is included to highlight the benefits of incorporating a relay. Conventional MRC cooperation exhibits poorer BLER performance than other cooperative methods such as AML and soft decode-and-forward (SDF). Moreover, the slope of the MRC curve significantly differs from those of AML, SDF, and JML, indicating its inability to achieve a diversity gain owing to a lack of robustness against relay detection errors. In contrast, the AML, SDF, and JML schemes exhibit similar BLER slopes, demonstrating their effectiveness in mitigating the error propagation inherent in DF relaying. As a summary, the performance and complexity are provided in

Table 2. Performance and complexity comparison for different detections.

	MRC	SDF	JML	AML
Performance	Poor	Good	Good	Good
Complexity	Low	Low	High	Low

, as shown below:

Figure 3 presents the PEP of AML detection with one-bit ADCs under various transmit/receive antenna configurations when ${\mathbf{x}}_S=-{\tilde{\mathbf{x}}}_S$ and ${\mathbf{x}}_R={\tilde{\mathbf{x}}}_R$. To validate Corollary 2, the symbol pairs are selected as ${\left[{\mathbf{x}}_S^T,{\tilde{\mathbf{x}}}_S^T\right]}^T={\left[1,-1\right]}^T$ and $\left[{\mathbf{x}}_R^T,{\tilde{\mathbf{x}}}_R^T\right]={\left[1,1\right]}^T$, thereby satisfying ${\mathbf{x}}_S=-{\tilde{\mathbf{x}}}_S$ and ${\mathbf{x}}_R={\tilde{\mathbf{x}}}_R$. This setup corresponds to a scenario in which decoding errors occur at the relay. Further, the asymptotic upper bound derived in Corollary 2 is plotted for comparison. The asymptotic behavior is analyzed in terms of the slope of the PEP curves, which reflects the diversity gain. The slope of the simulated PEP curves closely matches that of the analytical bound for all antenna configurations. Given that the slope of the analytical bound reflects the diversity gain of the source-to-destination link $N_D$, AML detection does not achieve any diversity gain from source-to-relay link $N_S$ under relay decoding errors. Given that the diversity gain depends solely on the source-to-destination link in this case, the PEP improves with an increasing number of receive antennas $N_D$, which is consistent with the theoretical analysis.

In contrast, Figure 4 depicts the PEP for various receive antenna configurations when ${\mathbf{x}}_S=-{\tilde{\mathbf{x}}}_S$, ${\mathbf{x}}_S={\mathbf{x}}_R$, and ${\tilde{\mathbf{x}}}_S={\tilde{\mathbf{x}}}_R$. The symbol pairs are set as ${\left[{\mathbf{x}}_S^T,{\tilde{\mathbf{x}}}_S^T\right]}^T={\left[1,-1\right]}^T$ and ${\left[{\mathbf{x}}_R^T,{\tilde{\mathbf{x}}}_R^T\right]}^T={\left[1,-1\right]}^T$, ensuring that above conditions are satisfied. This configuration corresponds to a scenario in which decoding at the relay is successful, enabling the cooperative MIMO system to achieve full diversity gain. In this case, the slope of the analytical bound reflects the combined diversity gain from the source-to-relay and source-to-destination links, i.e., $2N_D$. Compared to the diversity gain of $N_D$ observed in Figure 3, the slope in Figure 4 is significantly steeper. The simulated PEP curves are closely aligned with the analytical bound derived in Corollary 3, demonstrating that AML detection can effectively exploit the additional diversity gain provided by the source-to-relay link when the relay correctly decodes the source symbol.

5. Conclusions

This study investigated the pairwise error probability bounds of cooperative DF MIMO systems using one-bit ADCs over Rayleigh fading channels. The analysis focused on the approximate joint maximum likelihood detection, which was designed for mitigating the error propagation effects commonly observed in DF relaying. A tractable upper bound on the PEP was derived using Jensen’s inequality and Chernoff’s bound. The asymptotic characteristics of the PEP were analyzed to gain further insight into system behavior, revealing how one-bit quantization affects the achievable diversity gain. A key finding was that approximate joint maximum likelihood detection achieved full diversity only when the symbol pairs were sign-inverted and the relay correctly decoded the source symbol vector. The diversity gain was limited to that provided by the source-to-destination link when the relay failed to decode correctly. This represents the first analytical performance characterization of approximate joint maximum likelihood detection for cooperative DF MIMO systems with one-bit ADCs because the nonlinearity of the sign function in one-bit quantization makes the analysis particularly challenging. The validity of the proposed analytical bounds was confirmed through Monte Carlo simulations, which demonstrated excellent agreement with the theoretical predictions.

A potential future direction for this study is to extend the analytical framework to scenarios involving multiple sources, relays, and destinations. In such multi-node settings, the relay’s transmission protocol may differ significantly from the single-node case, depending on the underlying cooperation strategy. Consequently, the analytical approach must be adapted accordingly, making this a distinct line of research beyond the scope of this study.