Spectral Efficiency Maximization for Mixed-Structure Cognitive Radio Hybrid Wideband Millimeter-Wave Transceivers in Relay-Assisted Multi-User Multiple-Input Multiple-Output Systems

Abstract

This paper proposes a cognitive radio network (CRN)-based hybrid wideband precoding for maximizing spectral efficiency in millimeter-wave relay-assisted multi-user (MU) multiple-input multiple-output (MIMO) systems. The underlying problem is NP-hard and non-convex due to the joint optimization of hybrid processing components and the constant amplitude constraint imposed by the analog beamformer in the radio frequency (RF) domain. Furthermore, the analog beamforming solution common to all sub-carriers adds another layer of design complexity. Two hybrid beamforming architectures, i.e., mixed and fully connected ones, are taken into account to tackle this problem, considering the decode-and-forward (DF) relay node. To reduce the complexity of the original optimization problem, an attempt is made to decompose it into sub-problems. Leveraging this, each sub-problem is addressed by following a decoupled design methodology. The phase-only beamforming solution is derived to maximize the sum of spectral efficiency, while digital baseband processing components are designed to keep interference within a predefined limit. Computer simulations are conducted by changing system parameters under different accuracy levels of channel-state information (CSI), and the obtained results demonstrate the effectiveness of the proposed technique. Additionally, the mixed structure shows better energy efficiency performance compared to its counterparts and outperforms benchmarks.

1. Introduction

Future wideband mobile wireless networks need to accommodate the explosively growing demand for ultra-high-speed data transmission. This requirement leads to the exploration of the millimeter-wave (mm-wave) frequency band (30–300 GHz), owing to its advantage of containing huge spectral resources [1]. Furthermore, mm-wave technology with large-scale multi-input multi-output (MIMO) (also referred to as massive MIMO) systems are two key findings to address the spectrum crunch and enhance spectral and energy efficiencies [2]. Therefore, it is possible to realize several newly emerging applications, such as smart environments, autonomous vehicles, industrial and agriculture automation, remote healthcare, high-quality three-dimensional (3D) video, holographic imaging, virtual reality, and augmented reality [3,4]. These applications essentially demand low latency, higher data rates, and enhanced connectivity. Additionally, innovative signal processing techniques are required to make mm-wave communication feasible on commercial grounds [5]. There are several reasons behind that, such as new hardware constraints, operations at higher frequencies and large bandwidths, mm-wave channel impairments, advanced circuit implementation, and power consumption challenges to deploy large-scale antenna arrays to attain high beamforming gain [6].

In the forthcoming wireless systems (e.g., beyond the 5th generation (B5G) networks), the inevitable proliferation of connected devices may lead to spectral congestion. Therefore, efficient utilization of available frequency resources is required to avoid this situation [7,8]. It was observed in the measurement campaign that conventional fixed license-based static spectrum allocation policies decrease spectrum utilization efficiency to a great extent [9,10,11]. To resolve this issue, a spectrum-sharing technique is suggested to enhance the efficiency of spectrum utilization, and this sharing mechanism is based on dynamic spectrum allocation policies [12,13]. Cognitive radio (CR) is a feasible solution that makes it possible to support dynamic spectrum access, allowing secondary users (SUs) to utilize licensed spectrum while ensuring the quality of service (QoS) of primary users (PUs) [14]. Just as minimizing the impact of interference on PUs caused by the SUs is important, it is also desirable to minimize the effect of interference on the SU due to the PU in CR communication. This task can be achieved by regulating the power transmission at the respective unit [15,16,17]. To maximize spectrum utilization in B5G wireless networks, it seems crucial to combine the dynamic spectrum allocation strategy with mm-wave transmission. Specifically, novel applications from diverse fields, such as wireless backhaul, the Internet of Things (IoT), and intelligent transportation systems, can be realized by exploiting the adaptability of CR technology combined with the advantages of mm-wave frequencies [18].

It is worth noting that the characteristics of mm-wave channels are different compared to the lower part of the frequency spectrum due to the smaller wavelength of transmitted signals. For instance, huge path loss, limited propagation paths, lower diffraction, and highly directional wireless channels are associated with mm-wave frequencies [19]. To overcome the poor characteristics of mm-wave channels, the combination of large-scale MIMO and hybrid beamforming is considered the most attractive solution [20]. In particular, massive MIMO can achieve significant beamforming gain, while hybrid processing reduces hardware complexity, cost, and power consumption by significantly reducing the number of RF chains. However, hybrid precoding design in practical wideband systems is a challenging task, as it requires the analog RF precoder and combiner common to all sub-carriers [21]. In mm-wave transmission, there is a definite need to address the blockage sensitivity of signals. This impairment makes it quite difficult to establish a reliable non-line-of-sight (NLOS) communication link due to excessive path loss [22]. Fortunately, relay-assisted transmission can facilitate decreasing transmission power and enhance network coverage, link quality, and communication range [23,24]. In conclusion, relay-assisted MIMO communication networks improve the overall performance, especially at the edge of the cell.

It is worth highlighting that conventional fully digital beamforming results in prohibitive hardware complexity and cost for large-scale MIMO systems, as this precoding demands the same number of RF chains as the number of antennas in an array [25,26]. Among other alternatives, hybrid beamforming has been widely investigated to provide a good trade-off between complexity and performance. In a hybrid architecture, the beamforming process is divided into the digital baseband and the analog RF domains. This precoding strategy significantly decreases the number of RF chains due to the low-dimensional digital baseband processing components [27]. Moreover, multiplexing gain is achieved through the baseband processing part, while beamforming gain is obtained via the RF processing unit. Note that a network of phase shifters can be employed for the practical implementation of the analog RF beamformer [28]. In general, hybrid beamforming architectures can be categorized as (1) a fully connected structure where all the RF chains are connected to each antenna in an array—this structure enables each RF chain to achieve its full beamforming gain—and (2) a partially connected structure where each RF chain is connected to a non-overlapping subset of the antennas [29]. A sub-connected architecture uses less phase shifters for a given number of antennas and RF chains in comparison to a fully connected one. Therefore, the partially connected architecture reduces hardware implementation complexity and provides a cost-effective and energy-efficient hybrid precoding solution, but at the cost of reducing some beamforming gain [30]. Contrary to the fully connected structure, it is challenging to sustain fine control over the beam with the sub-connected structure, which can lead to less accurate beamforming. Furthermore, the partially connected architecture limits the flexibility of large antenna arrays, which may cause a significant reduction in performance [31,32]. It is worth noting that orthogonal frequency division multiplexing (OFDM) can support substantially higher data rates over a wide frequency range, making it a suitable choice for CRN. Hence, it is required to design a hybrid broadband mm-wave transceiver for the CRN-based MU-MIMO OFDM systems. Also, an energy-efficient solution can be obtained using a mixed hybrid structure rather than a fully connected one. To derive efficient hybrid processing components at different communicating nodes by taking interference at the PU into consideration is quite a challenging aspect of this problem.

1.1. Related Work

The works in [33,34,35,36,37,38,39] illustrate that the highly directional beamforming capability of mm-wave technology can improve user data rates and effectively mitigate interference simultaneously. From the perspective of spectrum sharing, these features of mm-wave transmission are quite useful. The authors of [33] analytically prove that license sharing among operators can enhance the performance of mm-wave systems. In comparison to the exclusive license model, the authors in [34] demonstrate that performance gains might be increased by up to 130% using spectrum-sharing techniques. An optimization framework is proposed in [36] for mm-wave CRN by incorporating joint beamforming to maximize the capacity and fairness of users, base-station (BS) association, and coordination. In [39], the authors examine the performance of spectrum-sharing systems in the 26 GHz and 70 GHz frequency bands, considering both ideal and non-ideal beamforming approaches in a real propagation environment. Based on their research, a spectrum-sharing system can achieve a capacity that is two to three times higher than an exclusive licensing network. Nevertheless, these advantages are unlikely to be achieved due to beamforming errors and poor interference mitigation techniques, particularly in situations with a low signal-to-interference plus noise ratio (SINR). The authors in [40] suggest a technique for interference management in a spectrum-sharing network, and its viability is also proved through computer simulations.

Hybrid precoding techniques have been extensively examined in both single-user (SU) and MU-MIMO communication networks. In [41,42,43,44,45,46,47,48,49,50,51], the authors propose hybrid beamforming algorithms for wideband mm-wave MIMO systems. In [41], the authors develop a principal component analysis (PCA)-based hybrid broadband beamforming algorithm, where a frequency-selective fully digital precoder is employed to design the frequency-independent analog RF beamformer. Additionally, this work is restricted to SU-MIMO-OFDM networks. The authors in [42] develop a hybrid precoding method for mm-wave SU-MIMO and MU multi-input single-output (MU-MISO) systems. This technique can achieve performance equal to the corresponding fully digital beamforming, provided the number of RF chains doubles the number of transmitted data streams. In [43], the authors propose the OFDM-based hybrid transceiver for massive MIMO systems, and this work is an extension of the previous algorithm in [42] to make it compatible with practical broadband systems. Based on statistical channel information, a practical, low-complexity hybrid precoding technique is suggested in [47], which employs an efficient searching algorithm to select the best possible beamforming vectors from codebooks. The constrained Tucker2 tensor decomposition technique is used in [48] to design the hybrid broadband transceiver for mm-wave MU-MIMO communication networks. Furthermore, the focus of this scheme is to enhance the capacity of the equivalent baseband channels by minimizing the impact of inter-user interference (IUI). In [50], the authors exploit an alternating minimization algorithm for phase extraction to approximate the full-complexity matrix from the corresponding analog processing component, considering mm-wave MU-MIMO communication networks. The authors of [51] propose alternating minimization (Alt-Min) algorithms to address hybrid beamforming for mm-wave MIMO systems using frequency-selective channels. The authors in [52] suggest the orthogonal matching pursuit (OMP) algorithm for hybrid precoding design by assuming frequency-flat channels. Moreover, this technique exploits the sparsity of mm-wave channels to derive the analog RF and digital baseband processing components.

In [53,54], the authors investigate relay-assisted hybrid precoding for maximizing spectral efficiency in single-user and MU mm-wave massive MIMO networks with a fully connected structure. Moreover, the amplify-and-forward (AF) relaying protocol is considered at the relay station in these designs. The works in [55,56] also focus on relay-based hybrid beamforming for maximizing energy efficiency in single-user mm-wave large-scale MIMO systems using partially connected structures at communicating nodes. Specifically, iterative successive approximation (ISA) and alternating direction method of multiplier (ADMM) algorithms are used to achieve the desired goals in [50,55], respectively. In [57], the authors examine the hybrid transceiver for multi-hop AF relay systems by taking channel errors into account. A mixed hybrid architecture, comprising both partially connected and fully connected structures, is proposed in [58] based on matrix factorization to achieve near-optimal performance. This hybrid precoding strategy shows lower hardware complexity and higher energy efficiency when compared with the corresponding fully connected structure. The authors of [59] present a codebook-based generalized sub-array connected (GSAC) structure for hybrid transceiver design to enhance energy efficiency. The authors in [60] design an analog beamformer that aims to minimize the channel estimation error by leveraging the channel’s long-term properties. Then, the baseband processing matrix is computed by employing simple digital signal processing such as maximum ratio combining/maximum ratio transmission (MRC/MRT). In [61], hybrid beamforming for mm-wave MIMO relay networks is suggested using imperfect CSI, where the AF protocol is adopted at relay nodes. Moreover, both algorithms [60,61] consider fully connected hybrid structures.

Previous works show that substantial research has been conducted on spectrum sharing in mm-wave MIMO systems and hybrid beamforming design in mm-wave MIMO OFDM networks. Nevertheless, hybrid transceivers that integrate the concept of CRN have not been studied extensively. The authors of [62] attempt to leverage spectrum sharing for designing hybrid transceivers by taking backhaul networks into consideration. The hybrid beamforming techniques in [63,64] primarily focus on maximizing the minimum secrecy rate of all SUs while accounting for practical limitations. In [65], the authors propose MU underlay cognitive hybrid transceiver designs while keeping interference to incumbent users within a predefined threshold. This technique is also applicable to both the downlink and uplink, provided that complete CSI is available.

The authors of [54,66,67,68,69,70] primarily focus their attention on designing mm-wave relay-based hybrid precoding for MU-MIMO systems. In [66], the authors investigate the hybrid beamforming technique for sum rate maximization with mixed and fully connected structures using the DF relaying protocol. This design does not support frequency-selective channels, as the RF and baseband processing matrices are derived under narrowband assumptions. The authors in [67] also develop the fully connected hybrid beamforming algorithm, where the coordinated beam alignment procedure is adopted for designing the RF processing matrices. Moreover, a digital baseband solution is obtained using a non-linear precoding method. In [68], the authors also suggest the hybrid precoding method for mm-wave massive MU-MIMO DF relay systems, taking mixed structure into account. This scheme is not suitable for wideband systems, as the analog and digital beamforming components are derived considering frequency-flat fading channels. Moreover, this algorithm shows relatively better performance when compared with the presented scheme in [66]. The authors in [69] propose the hybrid transceiver for mm-wave massive MU-MIMO relay-assisted networks with partially connected structures, considering the AF relaying principle. Just like [66,68], this technique suffers from the same limitations. In particular, the hybrid precoding techniques presented in [66,67,68,69] are not applicable when cognitive radio comes into play. The algorithm proposed in [70] is compatible with CR-based fully connected hybrid transceivers, but there is a definite need to reduce cost, energy, and hardware complexity without significant performance degradation. This work is an attempt to address this gap, considering the mixed hybrid structure to achieve the desired goal. The distinguishing feature of this work compared to similar existing techniques [54,66,67,68,69,70] is illustrated in

Table 1. Contrasting the proposed scheme with the existing relay-based hybrid beamforming for mm-wave MU-MIMO networks.

	[54]	[66]	[67]	[68]	[69]	[70]	Proposed
Relay-assisted mm-wave MU-MIMO system	✓	✓	✓	✓	✓	✓	✓
Frequency-selective channel	✕	✕	✕	✕	✕	✓	✓
Multiple data streams per user	✓	✓	✓	✓	✓	✓	✓
Minimum number of RF chains	✕	✕	✕	✕	✕	✓	✓
Support of CR technology	✕	✕	✕	✕	✕	✓	✓
Mixed hybrid structure	✕	✕	✕	✓	✕	✕	✓
Numerical results with imperfect CSI	✕	✕	✕	✕	✕	✓	✓

.

1.2. Motivation and Contribution

Contrary to the hybrid transceiver designs reported in the literature for relay-assisted MU-MIMO systems [54,66,67,68,69,70], there does not exist any CRN-based hybrid wideband mm-wave beamforming scheme that attempts to maximize the sum spectral efficiency, exploiting a low-complexity and energy-efficient structure. This provides motivation for the proposed hybrid transceiver design. To achieve the desired goal, the presented technique considers a relay-assisted MU-MIMO with a mixed structure, which comprises a combination of partially connected and fully connected architectures. This structure leads to a more practical solution that enhances energy efficiency when compared with the corresponding fully connected architecture. Additionally, the suggested mixed structure achieves sum spectral efficiency close to fully connected hybrid precoding with relatively low hardware complexity. Finally, the underlying hybrid beamforming design also leads to the efficient utilization of spectral resources, as the concept of CRN is also included in the proposed algorithm. The main contribution of this paper is summarized as follows:

• We investigate the hybrid broadband transceiver for sum spectral efficiency maximization in mm-wave relay-assisted MU-MIMO systems by incorporating the idea of CR technology to avoid inefficient spectrum allocation. The proposed algorithm primarily considers the mixed hybrid structure at the relay node and the partially connected architecture at each SU. This hybrid beamforming problem does not exist in the present literature to the best of the authors’ knowledge. Furthermore, the scenario investigated in this paper is consistent with practical considerations, as the underlying network architecture attempts to utilize the spectral resources efficiently while maximizing the sum rate using a low-complexity and energy-efficient hybrid structure, taking the transmitted power and interference constraints into account.
• In contrast to the existing hybrid precoding techniques for relay-assisted MU-MIMO systems [54,66,67,68,69,70], the proposed design considers frequency-selective channels, the CR paradigm, and mixed structures simultaneously, which offers great potential for unexplored applications. It is worth mentioning that the formulated optimization problem is non-convex due to the element-wise constant amplitude constraints imposed by the analog processing components. Also, the joint optimization of several complex matrix variables makes the original optimization problem NP-hard. Furthermore, it is required to derive the RF beamforming solution common to all sub-carriers. Consequently, design complexity and the associated computational complexity become prohibitively high. Therefore, the solution to the problem at hand is more challenging.
• To reduce the complexity of the optimization problem and make it tractable, the proposed algorithm first attempts to decompose the challenging task into two single-hop sum rate maximization sub-problems by exploiting the notion of information theory and the structural characteristics of DF relays. One sub-problem aims to maximize the sum rate from the source to relay decoding, while the other focuses on maximizing the sum spectral efficiency from relay encoding to multiple SUs. Then, a decoupled design approach is followed to drive the analog RF and digital baseband processing components. In particular, the derivation of RF processing components in each sub-problem attempts to maximize the sum rate, while digital baseband processing matrices focus their attention on minimizing interference. Furthermore, the mixed hybrid architecture at the relay node and the partially connected structure at each SU lead to an energy-efficient and low-complexity solution.
• The spatial wideband effect on the performance of the proposed scheme is also described through computer simulations. Furthermore, the impact of a uniform circular array under high-frequency transmission is also demonstrated using simulation results. Additionally, the proposed scheme also shows less computational complexity when compared with several other existing techniques.
• Numerical results are generated by changing system parameters over a wide range for both mixed and fully connected structures. Also, the performance curves are obtained under imperfect channel state information (CSI), considering the mixed hybrid architecture. It is evident from the simulation results that the suggested fully connected structure achieves performance close to full-complexity digital precoding, while the mixed hybrid structure shows relatively lower performance compared to the fully connected one. Furthermore, minor degradation in performance occurs when channel estimation error increases gradually. The obtained results also show that the proposed mixed architecture achieves significantly higher energy efficiency when compared to its fully connected counterpart. Finally, the achieved spectral efficiency and energy efficiency demonstrate the effectiveness of the proposed approach.

The rest of the paper is structured as follows: Section 2 describes the system model, the frequency domain mm-wave channel model under consideration, and problem formulation. Section 3 delineates the proposed hybrid transceiver design. The complexity analysis of the suggested technique is briefly discussed in Section 4, where a comparison is also made with other hybrid precoding algorithms. Section 5 presents simulation results for the performance evaluation, and concluding remarks are given in Section 6.

Notation: Upper-case and lower-case boldface letters denote matrices and vectors, respectively. ${\mathit{A}}^H$, ${\mathit{A}}^T$, $\measuredangle \mathit{A}$, ${\Vert \mathit{A}\Vert }_{\mathit{F}}$, $\left|\mathit{A}\left(i,j\right)\right|$, $\mathit{A}\left(:,i\right)$, and $\mathit{A}\left(:,1:j\right)$ represent the conjugate transpose, transpose, element-wise phase, Frobenius norm, element-wise modulus, $i$-th column, and first $j$ columns of a matrix $\mathit{A}$, respectively. $ℂ$ depicts the field of complex numbers, $ℝ$ specifies the field of real numbers, ${\mathit{I}}_m$ shows the identity matrix of order $m\times m$, and $\mathcal{C}\mathcal{N}\left(\mathbf{0}, {\sigma }^2{\mathit{I}}_n\right)$ describes the complex Gaussian distribution with mean $\mathbf{0}$ and covariance matrix ${\sigma }^2{\mathit{I}}_n$. The determinant and trace of a matrix $\mathit{A}$ are given as $\mathit{det}\left(\mathit{A}\right)$ and $Tr\left(\mathit{A}\right)$, respectively. $\mathbb{E}[.]$ shows the expectation operator, and $bd\left({\mathit{A}}_1,\dots ,{\mathit{A}}_K\right)$ stands for a block diagonal matrix with sub-matrices ${\mathit{A}}_1,\dots ,{\mathit{A}}_K$.

Table 2. Symbolic representation.

Symbol	Definition
$N_t/N_r/N_{d_k}$	Number of antennas at the source/relay node/$k$-th SU
$N_t^{RF}/N_r^{RF}/N_{d_k}^{RF}$	Number of RF chains at the source/relay node/$k$-th SU
${\mathit{s}}_k\left[n\right]/\mathit{s}\left[n\right]$	Number of data streams transmitted to the $k$-th SU/K SUs
$\mathit{x}\left[n\right]$	Source-transmitted signal after hybrid precoding
${\mathit{V}}_{RF}/{\mathit{V}}_{BB}\left[n\right]$	Common analog RF beamformer/frequency-selective digital baseband precoder at the source
$P_s$	Maximum allowable power at the source
${\mathit{y}}_r\left[n\right]$	Signal received at the relay node
$d_1\left[n\right]$	Interference experienced by the PU due to the source-transmitted signal
${\mathit{y}}_{r_1}\left[n\right]$	Baseband signal received at the output of relay hybrid combiner
${\mathit{F}}_{RF,1}/{\mathit{F}}_{RF,2}$	Common RF combiner/precoder at the relay station
${\mathit{F}}_{BB,1}\left[n\right]/{\mathit{F}}_{BB,2}\left[n\right]$	Frequency-dependent baseband combiner/precoder at the relay
${\mathit{F}}_1\left[n\right]/{\mathit{F}}_2\left[n\right]$	Relay hybrid combiner/precoder
$\mathit{F}\left[n\right]/{\mathit{F}}_{BB}\left[n\right]$	Relay hybrid filter/combined baseband processing component at the relay
${\mathit{y}}_{r_2}\left[n\right]$	Relay transmitted signal after hybrid beamforming
$d_2\left[n\right]$	Interference experienced by the PU due to the relay-transmitted signal
$\mathit{H}\left[n\right]/{\mathit{H}}_{PU}\left[n\right]/{\mathit{G}}_{d_k}\left[n\right]$	Channel matrix from the source to the relay in frequency-domain/source to the PU/relay to the $k$-th SU
${\mathit{n}}_r\left[n\right]/{\mathit{z}}_k\left[n\right]$	Noise vector at the relay/$k$-th SU
${\mathit{y}}_{d_k}^{+}\left[n\right]/{\mathit{y}}_{d_k}\left[n\right]$	Signal received at the $k$-th SU without hybrid processing/with hybrid processing
${\mathit{W}}_{RF,k}/{\mathit{W}}_{BB,k}\left[n\right]$	Frequency-independent RF combiner/frequency-dependent baseband combiner at the $k$-th SU
${\mathit{H}}_{eq}\left[n\right]={\mathit{F}}_{RF,1}^H\mathit{H}\left[n\right]{\mathit{V}}_{RF}$	Baseband equivalent channel from the source to the relay
${\mathit{G}}_{eq,k}\left[n\right]={\mathit{W}}_{RF,k}^H{\mathit{G}}_{d_k}\left[n\right]{\mathit{F}}_{RF,2}$	Baseband equivalent channel from the relay to the $k$-th SU
$C_k\left[n\right]/R_{avg}$	Capacity of the k-th SU at the n-th sub-carrier/average over $N_{sub}$ carriers
${\mathit{F}}_2^k\left[n\right]={\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}^k\left[n\right]$	Relay hybrid precoder corresponding to the $k$-th SU

shows the summary of the main symbols.

2. System Model and Problem Formulation

This section describes a system model of the underlying hybrid transceiver with two different structures: (1) a fully connected structure and (2) a mixed architecture. Furthermore, the frequency-domain mm-wave channel model and problem formulation are also given in this section.

2.1. System Model

Consider a downlink of the CRN-based relay-assisted MU-MIMO system with hybrid precoding as illustrated in Figure 1, where the source node, relay station, and the $k$-th SU are equipped with $N_t$, $N_r$, and $N_{d_k}$ antennas, respectively. The number of RF chains installed at the above-mentioned communicating nodes is represented as $N_t^{RF}$, $N_r^{RF}$ and $N_{d_k}^{RF}$, respectively. The source node transmits data streams to $K$ SUs through the relay node, while a direct communication link is employed for the PU. It is assumed that the direct transmission link between the source and $K$ SUs is not favorable owing to excessive path loss and deep fading. To transmit $N_s$ data streams from the cognitive radio base station (CRBS) to each SU, it is necessary for the source and relay to handle $K{N}_s$ data streams. This requirement leads to the essential conditions $K{N}_s\le \min \left(N_t^{RF}, N_r^{RF}\right)\ll \min \left(N_t, N_r\right)$ and $N_s\le \min \left(N_{d_k}^{RF}\right)\ll \min \left(N_{d_k}\right)$. These conditions enable efficient multi-stream transmission using a significantly small number of RF chains. To avoid relay-induced signal-space collisions, another condition $N_t\ge N_r\ge \sum _{k=1}^K{N}_{d_k}$ also needs to be satisfied.

In relay-based MIMO networks, two time slots are required to complete the transmission from the source to the destination. In the first time slot, the data streams of all intended users are transmitted from the source to the relay node. In the second time slot, the received signal at the relay station is processed and then forwarded to the end users. From this perspective, the communication link from the CRBS to the SUs can be decomposed into a single-user MIMO system (from the source to the relay station) and a MU-MIMO system (from the relay node to multiple users).

Let ${\mathit{s}}_k\left[n\right]\in {ℂ}^{N_s\times 1}$ be a vector of complex data streams intended to the $k$-th SU such that $\mathbb{E}\left[{\mathit{s}}_k\left[n\right]{\mathit{s}}_k^H\left[n\right]\right]={\mathit{I}}_{N_s},\forall n\in \{1,\dots ,N_{\mathit{sub}}\}$, and let $\mathit{s}\left[n\right]\in {ℂ}^{K{N}_s\times 1}$ be a vector of complex information symbols to $K$ SUs, which can be written as $\mathit{s}\left[n\right]={\left[{\mathit{s}}_1^T\left[n\right],\dots ,{\mathit{s}}_K^T\left[n\right]\right]}^T$, where $n$ denotes the sub-carrier index, and $N_{sub}$ specifies the total number of sub-carriers. At the CRBS, the frequency-selective digital baseband beamformer ${\mathit{V}}_{BB}\left[n\right]=\left[{\mathit{V}}_{BB,1}\left[n\right],\dots ,{\mathit{V}}_{BB,K}\left[n\right]\right]\in {ℂ}^{N_t^{RF}\times K{N}_s}$ is first employed to precode $K{N}_s$ data streams, specified by vector $\mathit{s}\left[n\right]$. Then, the frequency-flat analog RF beamformer ${\mathit{V}}_{RF}\in {ℂ}^{N_t\times N_t^{RF}}$ is deployed to enhance the beamforming gain. The transmitted signal $\mathit{x}\left[n\right]\in {ℂ}^{N_t\times 1}$ from the source at the $n$-th sub-carrier is given as $\mathit{x}\left[n\right]={\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\mathit{s}\left[n\right]={\mathit{V}}_{RF}\sum _{k=1}^K{\mathit{V}}_{BB,k}\left[n\right]{\mathit{s}}_k\left[n\right]$. Considering the transmit power constraint at the source, we have $\mathbb{E}\left[\mathit{x}\left[n\right]{\mathit{x}}^H\left[n\right]\right]=Tr\left\{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB }\left[n\right]\right){\left({\mathit{V}}_{RF}{\mathit{V}}_{BB }\left[n\right]\right)}^H\right\}\le P_s, \forall n\in \left\{1,\dots ,N_{sub}\right\}$, where $P_s$ indicates the maximum transmit power.

Note that ${\mathit{V}}_{RF}$ is common to all $N_{sub}$ sub-carriers. Therefore, the frequency-independent analog processing component needs to be derived for hybrid wideband transceivers. The signal received ${\mathit{y}}_r\left[n\right]\in {ℂ}^{N_r\times 1}$ at the relay station can be modeled as

$${\mathit{y}}_r\left[n\right]=\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\displaystyle \sum }_{k=1}^K{\mathit{V}}_{BB,k}\left[n\right]{\mathit{s}}_k\left[n\right]+{\mathit{n}}_r\left[n\right],$$

(1)

where $\mathit{H}\left[n\right]\in {ℂ}^{N_r\times N_t}$ represents the channel matrix from the source to relay station in frequency-domain, and ${\mathit{n}}_r\left[n\right]\in {ℂ}^{N_r\times 1}$ is the zero mean circularly symmetric complex Gaussian (ZMCSCG) noise with variance ${\sigma }_r^2$, i.e., ${\mathit{n}}_r\left[n\right]~\mathcal{C}\mathcal{N}\left(0, {\sigma }_r^2{\mathit{I}}_{N_r}\right)$. Due to the source-transmitted signal $\mathit{x}\left[n\right]\in {ℂ}^{N_t\times 1}$, the interference experienced by the PU can be characterized as

$$d_1\left[n\right]=\Vert {\mathit{H}}_{PU}\left[n\right]{\mathit{V}}_{RF}{\displaystyle \sum }_{k=1}^K{\mathit{V}}_{BB,k}\left[n\right]{\Vert }_F^2,$$

(2)

where ${\mathit{H}}_{PU}\left[n\right]$ denotes the channel matrix from the source to the PU at the $n$-th sub-carrier. The equivalent baseband signal ${\mathit{y}}_{r_1}\left[n\right]\in {ℂ}^{K{N}_s\times 1}$ received at the output of relay hybrid combiner ${\mathit{F}}_1\left[n\right]={\mathit{F}}_{RF,1}{\mathit{F}}_{BB,1}\left[n\right]\in {ℂ}^{N_r\times K{N}_s}$ can be expressed as

$${\mathit{y}}_{r_1}\left[n\right]={\left({\mathit{F}}_{RF,1}{\mathit{F}}_{BB,1}\left[n\right]\right)}^H\left(\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\displaystyle \sum }_{k=1}^K{\mathit{V}}_{BB,k}\left[n\right]{\mathit{s}}_k\left[n\right]+{\mathit{n}}_r\left[n\right]\right),$$

(3)

where ${\mathit{F}}_{RF,1}\in {ℂ}^{N_r\times N_r^{RF}}$ and ${\mathit{F}}_{BB,1}\left[n\right]\in {ℂ}^{N_r^{RF}\times K{N}_s}$ are the frequency-flat analog combiner and the frequency-dependent digital baseband combiner at the relay node, respectively. Applying hybrid beamforming, the transmitted signal ${\mathit{y}}_{r_2}\left[n\right]\in {ℂ}^{N_r\times 1}$ from the relay node is given as

$$\begin{gathered} {\mathit{y}}_{r_2}\left[n\right]={\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}\left[n\right]{\mathit{y}}_{r_1}\left[n\right] \\ =\underset{relay hybrid filter }{\underbrace{\left({\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}\left[n\right]\right){\left({\mathit{F}}_{RF,1}{\mathit{F}}_{BB,1}\left[n\right]\right)}^H}}\left(\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\displaystyle \sum }_{k=1}^K{\mathit{V}}_{BB,k}\left[n\right]{\mathit{s}}_k\left[n\right]+{\mathit{n}}_r\left[n\right]\right), \end{gathered}$$

(4)

where the relay node is supposed to apply frequency-dependent digital baseband precoder ${\mathit{F}}_{BB,2}[n]\in {ℂ}^{N_r^{RF}\times {\mathit{KN}}_s}$ followed by the frequency-independent analog RF beamformer ${\mathit{F}}_{RF,2}\in {ℂ}^{N_r\times N_r^{RF}}$. It is worth highlighting that the partially connected RF processing component in hybrid precoding and combining can further reduce the cost, energy, and hardware implementation complexity when compared with its fully connected counterpart. Therefore, we mainly focus on the partially connected analog precoder at the relay node and the RF combiner at each SU. At the relay transmitter, it is assumed that $N_r/N_r^{RF}$ antennas are connected to each of $N_r^{RF}$ chains. Under this condition, ${\mathit{F}}_{RF,2}$ becomes a block-diagonal matrix that contains a set of vectors ${\mathit{p}}_1, {\mathit{p}}_2,\dots ,{\mathit{p}}_{N_r^{RF}}$, each with length $N_r/N_r^{RF}$. Hence, the structure of ${\mathit{F}}_{RF,2}$, under the partially connected architecture, is given as

$${\mathit{F}}_{RF,2}=\left(\frac{1}{\sqrt{N_r}}\right)\left[\begin{array}{ccc}{\mathit{p}}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{p}}_{N_r^{RF}}\end{array}\right],$$

(5)

where ${\mathit{p}}_i\in {ℂ}^{N_r/N_r^{RF}\times 1}$ is the analog beamforming vector associated with the $i$-th sub-array. Owing to the relay-transmitted signal ${\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}\left[n\right]{\mathit{y}}_{r_1}\left[n\right]\in {ℂ}^{N_r\times 1}$ to the SUs, the interference experienced by the PU can be represented as $d_2\left[n\right]=\Vert {\mathit{H}}_{PU}\left[n\right]{\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}\left[n\right]{\mathit{y}}_{r_1}{\left[n\right]\Vert }_F^2$. Using (4), the relay hybrid filter $\mathit{F}\left[n\right]\in {ℂ}^{N_r\times N_r}$ is defined as $\mathit{F}\left[n\right]=\left({\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}\left[n\right]\right){\left({\mathit{F}}_{RF,1}{\mathit{F}}_{BB,1}\left[n\right]\right)}^H$, which can be compactly described as ${\mathit{F}}_{RF,2}{\mathit{F}}_{BB}\left[n\right]{\mathit{F}}_{RF,1}^H$, where ${\mathit{F}}_{BB}\left[n\right]{\mathit{F}}_{BB,2}\left[n\right]{\mathit{F}}_{BB,1}^H\left[n\right]\in {ℂ}^{N_r^{\mathit{RF}}\times N_r^{\mathit{RF}}}$ is the combined baseband processing matrix at the relay node. The received signal ${\mathit{y}}_{d_k}^{+}\left[n\right]\in {ℂ}^{N_{d_k}\times 1}$ at the $k$-th SU under the assumption of block-fading channel can be characterized as

$${\mathit{y}}_{d_k}^{+}\left[n\right]={\mathit{G}}_{d_k}\left[n\right]\mathit{F}\left[n\right]\left(\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\displaystyle \sum }_{k=1}^K{\mathit{V}}_{BB,k}\left[n\right]{\mathit{s}}_k\left[n\right]+{\mathit{n}}_r\left[n\right]\right)+{\mathit{z}}_k\left[n\right],$$

(6)

where ${\mathit{G}}_{d_k}\left[n\right]\in {ℂ}^{N_{d_k}\times N_r}$ denotes the frequency domain channel between the relay node and the $k$-th SU, and ${\mathit{z}}_k\left[n\right]\in {ℂ}^{N_{d_k}\times 1}$ specifies the ZMCSCG noise with variance ${\sigma }_k^2$, i.e., ${\mathit{z}}_k\left[n\right]~\mathcal{C}\mathcal{N}\left(0, {\sigma }_k^2{\mathit{I}}_{N_{d_k}}\right)$. The baseband equivalent signal ${\mathit{y}}_{d_k}\left[n\right]\in {ℂ}^{N_s\times 1}$ after passing ${\mathit{y}}_{d_k}^{+}\left[n\right]$ in (6) through the hybrid combiner ${\mathit{W}}_{d_k}\left[n\right]={\mathit{W}}_{RF,k}{\mathit{W}}_{BB,k}\left[n\right]\in {ℂ}^{N_{d_k}\times N_s}$ at the $k$-th SU is given as ${\mathit{y}}_{d_k}\left[n\right]={\left({\mathit{W}}_{RF,k}{\mathit{W}}_{BB,k}\left[n\right]\right)}^H{\mathit{y}}_{d_k}^{+}\left[n\right]$, where ${\mathit{W}}_{RF,k}\in {ℂ}^{N_{d_k}\times N_{d_k}^{RF}}$, ${\mathit{W}}_{BB,k}\left[n\right]\in {ℂ}^{N_{d_k}^{RF}\times N_s}$ are the common analog RF and the frequency-selective digital baseband combiners at the $k$-th SU. Under the assumption of the partially connected structure, the RF combiner ${\mathit{W}}_{RF,k}$ is constrained to be a block-diagonal matrix as follows:

$${\mathit{W}}_{RF,k}=\left(\frac{1}{\sqrt{N_{d_k}}}\right)\left[\begin{array}{ccc}{\mathit{q}}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{q}}_{N_{d_k}^{RF}}\end{array}\right],$$

(7)

where ${\mathit{q}}_j\in {ℂ}^{N_{d_k}/N_{d_k}^{RF}\times 1}$ is the analog beamforming vector associated with the $j$-th sub-array. Here, $N_{d_k}/N_{d_k}^{RF}$ antennas are connected to each of $N_{d_k}^{RF}$ chains at the $k$-th SU, and the block-diagonal matrix in (7) is composed of a set of vectors ${\mathit{q}}_1, {\mathit{q}}_2,\dots ,{\mathit{q}}_{N_{d_k}^{RF}}$, each with length $N_{d_k}/N_{d_k}^{RF}$. Another useful representation of ${\mathit{y}}_{d_k}\left[n\right]$ that facilitates writing the capacity expression is given as

$${\mathit{y}}_{c_k}\left[n\right]=\{\begin{array}{c}{\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{G}}_{eq,k}\left[n\right]{\mathit{F}}_{BB}\left[n\right]{\mathit{H}}_{eq}\left[n\right]{\displaystyle {\displaystyle \sum }_{k=1}^K}{\mathit{V}}_{BB,k}\left[n\right]{\mathit{s}}_k\left[n\right]+\\ {\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{G}}_{eq,k}\left[n\right]{\mathit{F}}_{BB}\left[n\right]{\mathit{n}}_r^{+}\left[n\right]+{\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{z}}_k^{+}\left[n\right]\end{array},$$

(8)

where ${\mathit{H}}_{eq}\left[n\right]={\mathit{F}}_{RF,1}^H\mathit{H}\left[n\right]{\mathit{V}}_{RF}\in {ℂ}^{N_r^{RF}\times N_t^{RF}}$ denotes the equivalent baseband channel from the CRBS to relay station, ${\mathit{G}}_{eq,k}\left[n\right]={\mathit{W}}_{RF,k}^H{\mathit{G}}_{d_k}\left[n\right]{\mathit{F}}_{RF,2}\in {ℂ}^{N_{d_k}^{RF}\times N_r^{RF}}$ represents the equivalent baseband channel from the relay node to the $k$-th SU, ${\mathit{n}}_r^{+}\left[n\right]={\mathit{F}}_{RF,1}^H{\mathit{n}}_r\left[n\right]\in {ℂ}^{N_r^{RF}\times 1}$ depicts the noise vector at the output of the relay RF combiner, and ${\mathit{z}}_k^{+}\left[n\right]={\mathit{W}}_{RF,k}^H{\mathit{z}}_k\left[n\right]\in {ℂ}^{N_{d_k}^{RF}\times 1}$ specifies the noise vector at the $k$-th SU after passing through the analog RF processing matrix. It is worthwhile to mention that the noise distribution does not change after multiplication with the RF precoder/combiner, as given in [71]. This result leads to the conclusion that ${\mathit{z}}_k^{+}\left[n\right]$ and ${\mathit{n}}_r^{+}\left[n\right]$ follow the same distribution as that of ${\mathit{z}}_k\left[n\right]$ and ${\mathit{n}}_r\left[n\right]$, respectively. Using (8), the capacity expression of the $k$-th SU at the $n$-th sub-carrier can be represented as [70]

$$C_k\left[n\right]=\left(\frac{1}{2}\right){\log }_2\det \left({\mathit{I}}_{N_s}+\frac{{\mathit{X}}_k\left[n\right]{\mathit{X}}_k^H\left[n\right]}{\underset{equivalent noise covariance matrix \left(={\mathit{R}}_k\left[n\right]\right)}{\underbrace{\left({\sigma }_r^2{\mathit{Y}}_k\left[n\right]{\mathit{Y}}_k^{\mathit{H}}\left[n\right]+{\sigma }_k^2{\mathit{W}}_{BB,k}^{\mathit{H}}\left[n\right]{\mathit{W}}_{BB,k}\left[n\right]\right)}}+{\Xi }_{\mathit{I}U\mathit{I}}\left[n\right]}\right),$$

(9)

where $\{\begin{array}{c}{\mathit{X}}_k[n]={\mathit{W}}_{BB,k}^H[n]{\mathit{G}}_{eq,k}[n]{\mathit{F}}_{BB}[n]{\mathit{H}}_{eq}[n]{\mathit{V}}_{BB,k}[n]\\ {\mathit{Y}}_k[n]={\mathit{W}}_{BB,k}^H[n]{\mathit{G}}_{eq,k}[n]{\mathit{F}}_{BB}[n]{\mathit{n}}_r^{+}[n]\\ {\mathit{Z}}_k[n]=\underset{\mathit{interuser\; interference\; (IUI)}}{\underbrace{{\mathit{W}}_{BB,k}^H[n]{\mathit{G}}_{\mathit{eq,k}}[n]{\mathit{F}}_{BB}[n]{\mathit{H}}_{eq}[n]\sum _{j=1,j\ne k}^K{\mathit{V}}_{BB,j}[n]}},{\mathit{\Xi }}_{\mathit{I}U\mathit{I}}[n]={\mathit{Z}}_k[n]{\mathit{Z}}_k^H[n]\end{array}$.

The pre-log factor (1/2) in (9) indicates that two time slots are required for signal transmission from the CRBS to $K$ SUs. The sum spectral efficiency averaged over $N_{sub}$ sub-carriers can be described as

$$R_{sum}=\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}\left({\displaystyle \sum }_{k=1}^K{C}_k\left[n\right]\right)=\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}\left({\displaystyle \sum }_{k=1}^K{\displaystyle \sum }_{i=1}^{N_s}{\mathit{log}}_2\left\{{\left(1+{\left(SNIR\right)}_{k_i}\left[n\right]\right)}_{SD}\right\}\right),$$

(10)

where ${\left(SNIR\right)}_{k_i}\left[n\right]$ is the signal-to-noise plus interference ratio from the CRBS to the $k$-SU corresponding to the i-th data stream under the assumption of Gaussian signaling.

2.2. Channel Model

In this paper, the clustered channel model is considered to characterize the mm-wave channels [72]. The mm-wave channels exhibit sparse scattering or spatial selectivity due to high propagation losses. Additionally, deploying a large number of antennas in massive MIMO systems within a small physical region leads to high antenna correlation. Therefore, low-rank matrices can be used to characterize the sparse scattering nature of mm-wave channels. In sum, the traditional rich scattering Rayleigh fading channel is no longer applicable for modeling mm-wave channels. To capture the mathematical structure of the mm-wave propagation environment, the geometric channel based on the extended Saleh–Valenzuela model is adopted. Specifically, the mathematical formulation of the frequency domain channel matrix, considering the uniform planar array (UPA), for the $n$-th sub-carrier can be expressed as [51]

$$\mathit{H}\left[n\right]=\beta {\displaystyle \sum }_{i=0}^{N_{cl}-1}{\displaystyle \sum }_{l=1}^{N_{ray}}{\alpha }_{il}{\mathit{a}}_r\left({\phi }_{il}^r,{\theta }_{il}^r\right) {\mathit{a}}_t{\left({\phi }_{il}^t,{\theta }_{il}^t\right)}^H e^{-j2\pi \frac{i.n}{N_{sub}}} ,$$

(11)

where $\beta =\sqrt{\frac{N_t{N}_r}{N_{cl}{N}_{ray}}}$ describes the normalization factor, $N_{cl}$ and $N_{ray}$ illustrate the number of clusters and the number of rays in each cluster, N_sub shows the total number of sub-carriers, ${\alpha }_{il}$~$\mathcal{C}\mathcal{N}\left(0,{\sigma }_{\alpha ,i}^2\right)$ specifies the complex gain, ${\theta }_{il}^t$ and ${\theta }_{il}^r$ denote the elevation angles of departure and arrival, and ${\phi }_{il}^t$ and ${\phi }_{il}^r$ represent the azimuth angles of departure and arrival of the $l$-th transmission path in the $i$-th propagation cluster. Moreover, ${\mathit{a}}_t\left({\phi }_{il}^t,{\theta }_{il}^t\right)$ and ${\mathit{a}}_r\left({\phi }_{il}^r,{\theta }_{il}^r\right)$ in (11) refer to the planar array response vectors at the transmitter and receiver, respectively. These array response vectors depend on the antenna array architecture, where each element needs to follow a constant modulus constraint. The uniform square planar array (USPA) with $\sqrt{N}\times \sqrt{N}$ antenna elements is considered in this work. Hence, the array response vector corresponding to the $l$-th ray in the $i$-th cluster is given as [51]

$$\mathit{a}\left({\phi }_{il},{\theta }_{il}\right)=\left(\frac{1}{\sqrt{N}}\right){\left[1,\dots ,e^{j\frac{2\pi }{\lambda }d\left(u\sin {\phi }_{il}\sin {\theta }_{il}+v\cos {\theta }_{il}\right)},\dots , e^{j\frac{2\pi }{\lambda }d\left(\left(\sqrt{N}-1\right)\sin {\phi }_{il}\sin {\theta }_{il}+\left(\sqrt{N}-1\right)\cos {\theta }_{il}\right)}\right]}^T,$$

(12)

where $\lambda$ and $d=\frac{\lambda }{2}$ are the signal wavelength and antenna spacing, and $0\le u<\sqrt{N}$ and $0\le v<\sqrt{N}$ are the antenna indices in the 2D plane. Note that the proposed algorithm and the obtained results can be extended to a uniform rectangular array (URA), and the corresponding array response vectors are given in [73].

2.3. Problem Formulation

The orientation of the proposed algorithm is to design a hybrid beamforming solution for mm-wave wideband relay-assisted MU-MIMO CRN. Furthermore, mixed and fully connected hybrid architectures are taken into consideration. It is important to mention that spectral efficiency and mean-squared error (MSE) are two optimization targets that are normally considered when the underlying problems involve the DF protocol at the relay station [55]. Note that the former is a crucial performance metric as far as hybrid transceiver designs are concerned. Hence, the prime objective of this work is to derive a set of hybrid processing components that maximize the sum spectral efficiency given in (10). It is also required to satisfy the transmit power constraint at the source while keeping interference experienced by the PU within a predefined threshold. Additionally, the analog RF beamforming components need to follow the element-wise constant amplitude constraints.

Therefore, the optimization problem can be formulated as follows:

$$\begin{gathered} \underset{{\left\{\begin{array}{c}{\mathit{V}}_{RF},{\mathit{V}}_{BB}\left[n\right], {\mathit{F}}_{RF,1},{\mathit{F}}_{BB,1}\left[n\right],\\ {\mathit{F}}_{BB,2}\left[n\right],{\mathit{F}}_{RF,2}, {\mathit{W}}_{RF,k},{\mathit{W}}_{BB,k}\left[n\right]\end{array}\right\}}_{k=1, n=1}^{K, N_{sub}}}{\max }{R}_{sum} \\ s. t. \\ \{\begin{array}{c}|{\mathit{V}}_{RF}(x,y)|=\frac{1}{\sqrt{N_t}},|{\mathit{W}}_{RF,k}(x,y)|=\frac{1}{\sqrt{N_{d_k}}},\\ |{\mathit{F}}_{RF,1}(x,y)|=|{\mathit{F}}_{RF,2}(x,y)|=\frac{1}{\sqrt{N_r}},\mathrm{\forall }x,y,k,\\ {\mathit{F}}_{RF,2}=\left[\begin{array}{ccc}{\mathit{p}}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{p}}_{N_r^{RF}}\end{array}\right],\\ {\mathit{W}}_{RF}=\left[\begin{array}{ccc}{\mathit{W}}_{RF,1}& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{W}}_{RF,K}\end{array}\right],\\ {\mathit{F}}_{BB,2}[n]=\left[\begin{array}{ccc}{\mathit{F}}_{BB,2}^{[1]}[n]& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{F}}_{BB,2}^{[K]}[n]\end{array}\right],\\ {\mathit{W}}_{BB}[n]=\left[\begin{array}{ccc}{\mathit{W}}_{BB,1}[n]& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{W}}_{BB,K}[n]\end{array}\right],\\ {\Vert {\mathit{V}}_{RF}{\mathit{V}}_{BB}[n]\Vert }_F^2\le P_s,\\ d_1[n]\le J_1,\\ d_2[n]\le J_2,\mathrm{\forall }n\end{array}, \end{gathered}$$

(13)

where $J_1$, $J_2$, and $P_s$ represent the interference threshold experienced by the PU due to the source-transmitted signal, the relay-transmitted signal, and the upper bound for the source-transmitted power, respectively. The solution of the optimization problem in (13) seeks a joint optimization over several complex matrix variables, namely ${\mathit{V}}_{RF}$, ${\mathit{V}}_{BB}\left[n\right]$, ${\mathit{F}}_{RF,1}$, ${\mathit{F}}_{BB,1}\left[n\right]$, ${\mathit{F}}_{BB,2}\left[n\right]$, ${\mathit{F}}_{RF,2}$, ${\mathit{W}}_{RF,k}$, and ${\mathit{W}}_{BB,k}\left[n\right]$ for each SU $k$ and each sub-carrier $n$. It is worth mentioning that joint optimization, in this case, is usually NP-hard. Moreover, the element-wise constant modulus constraints imposed by the analog RF processing matrices, i.e., precoders and combiners, make the problem non-convex. Therefore, the global optimal solution is intractable. To address these challenges, an endeavor is made to divide the complicated optimization problem (13) into sub-problems. One sub-problem corresponds to the first transmission phase, i.e., from source to relay station, and deals with the optimization of complex variables ${\mathit{V}}_{RF}$, ${\mathit{V}}_{BB}\left[n\right]$, ${\mathit{F}}_{RF,1}$, and ${\mathit{F}}_{BB,1}\left[n\right]$. The other sub-problem focuses on deriving the decision variables ${\mathit{F}}_{BB,2}\left[n\right]$, ${\mathit{F}}_{RF,2}$, ${\mathit{W}}_{RF,k}$, and ${\mathit{W}}_{BB,k}\left[n\right]$ that correspond to the second transmission phase, i.e., from the relay to $K$ SUs.

3. Hybrid Transceiver Design Based on SNR Maximization

The cooperative communication network, where the DF relay station is deployed between the source and destination, acts like a cascade of two sub-networks. Therefore, the part of the system from the source to the relay receiver corresponds to one sub-network, while the part of the system from the relay transmitter to the destination indicates the other sub-network. In the underlying hybrid transceiver, the decoded signal at the output of the relay hybrid combiner is applied to the input of the relay hybrid precoder. This process is referred to as cascading. Exploiting this property of the DF relay node, the section of the network from the CRBS to the SUs can be decomposed into two relatively independent sub-systems, which is the first step towards reducing the complexity of the original optimization problem (13). Let $R_1$ be the spectral efficiency of the point-to-point MIMO system from the source to the relay node and $R_2$ be the spectral efficiency of the point-to-multipoint MIMO system from the relay station to the SUs. It is possible to determine the overall sum rate of the two sub-systems by maximizing the minimum between $R_1$ and $R_2$. Taking advantage of this important feature, the sum rate of the entire SN can be expressed as [68]

$$R_{sum}=\left(\frac{1}{2}\right)\min \left(R_1,R_2\right).$$

(14)

Leveraging the separation of the sum rate into two parts, the optimization problem formulated in (13) can be divided into two sub-problems as

$$\begin{gathered} \underset{{\left\{{\mathit{V}}_{RF},{\mathit{V}}_{BB}\left[n\right],{\mathit{F}}_{RF,1}{\mathit{F}}_{BB,1}\left[n\right]\right\}}_{n=1}^{N_{sub}}}{\max }{R}_1 \\ s. t. \\ \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{V}}_{RF}\left(x,y\right)\right|=\frac{1}{\sqrt{N_t}} ,\left|{\mathit{F}}_{RF,1}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}}, \forall x,y\end{array}, \end{gathered}$$

(15)

$$\begin{gathered} \underset{{\left\{{\mathit{F}}_{BB,2}\left[n\right],{\mathit{F}}_{RF,2}, {\mathit{W}}_{RF,k},{\mathit{W}}_{BB,k}\left[n\right]\right\}}_{k=1, n=1}^{K, N_{sub}}}{\mathit{max}}{R}_2 \\ s. t. \\ \{\begin{array}{c}{\mathit{F}}_{RF,2}=\left[\begin{array}{ccc}{\mathit{p}}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{p}}_{N_r^{RF}}\end{array}\right],\\ {\mathit{W}}_{RF}=\left[\begin{array}{ccc}{\mathit{W}}_{RF,1}& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{W}}_{RF,K}\end{array}\right],\\ {\mathit{F}}_{BB,2}[n]=\left[\begin{array}{ccc}{F}_{BB,2}^{[1]}[n]& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{F}}_{BB,2}^{[K]}[n]\end{array}\right],\\ {\mathit{W}}_{BB}[n]=\left[\begin{array}{ccc}{\mathit{W}}_{BB,1}[n]& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{W}}_{BB,K}[n]\end{array}\right],\\ d_2[n]\le J_2,\\ d_2[n]\le J_2,\\ |{\mathit{F}}_{RF,2}(x,y)|=\frac{1}{\sqrt{N_r}},|{\mathit{W}}_{RF,k}(x,y)|=\frac{1}{\sqrt{N_{d_k}}},\mathrm{\forall }x,y.\end{array} \end{gathered}$$

(16)

3.1. Source Analog Precoder and Relay RF Combiner

The goal is to derive the RF precoder and combiner that maximize the sum spectral efficiency. Since the achievable rate $R_1$ is a function of complex matrix variables $\left({\mathit{V}}_{RF}, {\left\{{\mathit{V}}_{BB}\left[n\right]\right\}}_{n=1 }^{ N_{sub} }, {\mathit{F}}_{RF,1}, {\left\{{\mathit{F}}_{BB,1}\left[n\right]\right\}}_{n=1 }^{ N_{sub} }\right)$, joint optimization of the sum rate is computationally inefficient. Therefore, a decoupled approach is exploited to design the required analog beamforming components, which allows converting the sub-optimization problem (15) into the following form.

$$\begin{gathered} \underset{{\left\{{\mathit{V}}_{RF},{\mathit{V}}_{BB}\left[n\right]\right\}}_{ n=1}^{ N_{sub}}}{\mathit{max}}\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}{\log }_2\mathit{det}\left({\mathit{I}}_{K{N}_s}+\frac{1}{{\sigma }_r^2}\left(\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right){\left(\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\right) \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{V}}_{RF}\left(x,y\right)\right|=\frac{1}{\sqrt{N_t}} ,\left|{\mathit{F}}_{RF,1}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}}, \forall x,y\end{array}. \end{gathered}$$

(17)

According to Lemma 1 [41], the optimization problem in (17) can be approximated as

$$\begin{gathered} \underset{{\left\{{\mathit{V}}_{RF},{\mathit{V}}_{BB}\left[n\right]\right\}}_{ n=1}^{ N_{sub}}}{\max }\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}{\Vert \left({\mathit{V}}_{FD}^{opt}\left[n\right]\right)}^H{\mathit{V}}_{RF}{\mathit{V}}_{BB}{\left[n\right]\Vert }_F^2 \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{V}}_{RF}\left(x,y\right)\right|=\frac{1}{\sqrt{N_t}} ,\left|{\mathit{F}}_{RF,1}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}}, \forall x,y\end{array}. \end{gathered}$$

(18)

where ${\mathit{V}}_{FD}^{opt}\left[n\right]\in {ℂ}^{N_t\times K{N}_s}$ is the fully digital precoder corresponding to the $n$-th sub-carrier. If the hybrid beamformer ${\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]$ is sufficiently close to the full complexity precoder with considerably high SNR, then the optimization problem (18) becomes equivalent to (17). Furthermore, assuming ${\mathit{V}}_{BB}\left[n\right]$ as a unitary matrix allows us to transform the problem (18) into another equivalent form as

$$\begin{gathered} \underset{{\mathit{V}}_{RF}}{\max }\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}{\Vert {\left({\mathit{V}}_{FD}^{opt}\left[n\right]\right)}^H{\mathit{V}}_{RF}\Vert }_F^2 \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{V}}_{RF}\left(x,y\right)\right|=\frac{1}{\sqrt{N_t}} ,\forall x,y\end{array}. \end{gathered}$$

(19)

Since the frequency-domain MIMO channels ${\left\{\mathit{H}\left[n\right]\right\}}_{ n=1}^{ N_{sub}}$ have the same row/column space [49], ${\mathit{V}}_{RF}$ is common to all sub-carriers. From this perspective, ${\mathit{V}}_{RF}$ can be considered as a representation of the mentioned column space, and this observation inspires us to develop the analog beamformer using the principal component analysis framework. Under this framework, the data set matrix ${\mathit{V}}_{data}=\left[{\mathit{V}}_{FD}^{opt}\left[1\right], {\mathit{V}}_{FD}^{opt}\left[2\right], \dots ,{\mathit{V}}_{FD}^{opt}\left[N_{sub}\right] \right]$ can be employed to determine the principal components that constitute ${\mathit{V}}_{RF}$. Applying singular value decomposition (SVD) on ${\mathit{V}}_{data}$ leads to a stable solution with low complexity. Therefore, ${\mathit{V}}_{data}={\mathit{U}}_{{\mathit{V}}_{data}}{\mathit{\Sigma }}_{{\mathit{V}}_{data}}{\mathit{V}}_{{\mathit{V}}_{data}}^H$ and the sub-optimal solution can be given as follows:

$${\mathit{V}}_{RF}=\left(\frac{1}{\sqrt{N_t}}\right)\exp \left\{j arg\left({\mathit{U}}_{{\mathit{V}}_{data}}\left(:,1:N_t^{RF}\right)\right)\right\}.$$

(20)

It is worth highlighting that the obtained RF beamforming vectors have the largest projections on the respective eigenmodes, i.e., they cast maximum energy along those eigenmode directions [74]. Additionally, they help achieve maximum beamforming gain and keep the interference that PU experiences within a predetermined threshold. We can also find a sub-optimal solution to the problem (19) by deriving the lower bound of the objective function. An alternative solution to design the ${\mathit{V}}_{RF}$ is as follows:

$$\begin{gathered} {\displaystyle \sum }_{n=1}^{N_{sub}}{\Vert {\left({\mathit{V}}_{FD}^{opt}\left[n\right]\right)}^H{\mathit{V}}_{RF}\Vert }_F^2={\Vert {\left({\mathit{V}}_{FD}^{opt}\left[1\right]\right)}^H{\mathit{V}}_{RF}\Vert }_F^2+\cdots +\Vert {\left({\mathit{V}}_{FD}^{opt}\left[N_{sub}\right]\right)}^H{\mathit{V}}_{RF}{\Vert }_F^2 \\ \ge \Vert \left(\frac{{\left({\mathit{V}}_{FD}^{opt}\left[1\right]\right)}^H+\dots +{\left({\mathit{V}}_{FD}^{opt}\left[N_{sub}\right]\right)}^H}{N_{sub}}\right){\mathit{V}}_{RF}{\Vert }_F^2=\Vert {\mathit{V}}_{opt}{\mathit{V}}_{RF}{\Vert }_F^2, \end{gathered}$$

(21)

where ${\mathit{V}}_{opt}=\left(\frac{{\left({\mathit{V}}_{FD}^{opt}\left[1\right]\right)}^H+\dots +{\left({\mathit{V}}_{FD}^{opt}\left[N_{sub}\right]\right)}^H}{N_{sub}}\right)$. The sub-optimal solution to the problem (19) can also be expressed as ${\mathit{V}}_{RF}=\left(\frac{1}{\sqrt{N_t}}\right)\exp \left\{jarg{\left({\mathit{V}}_{opt}\right)}^H\right\}$. Based on the derived ${\mathit{V}}_{RF}$ and ${\left\{{\mathit{V}}_{BB}\left[n\right]\right\}}_{n=1}^{N_{sub}}$, the relay RF combiner ${\mathit{F}}_{RF,1}$ can be obtained by maximizing the received SNR. Using (3), the achievable rate can be expressed as

$$R_1=\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}\left({\log }_2\det \left({\mathit{I}}_{K{N}_s}+{R_{N_1}}^{-1}\left({\mathit{F}}_{BB,1}^H\left[n\right]{\mathit{F}}_{RF,1}^H\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right){\left({\mathit{F}}_{BB,1}^H\left[n\right]{\mathit{F}}_{RF,1}^H\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\right)\right),$$

(22)

where ${\mathit{R}}_{N_1}=\left({\left({\mathit{F}}_{RF,1}{\mathit{F}}_{BB,1}\left[n\right]\right)}^H\left({\mathit{F}}_{RF,1}{\mathit{F}}_{BB,1}\left[n\right]\right){\sigma }_r^2\right)$ is the noise covariance matrix. It is worth noting that the analog beamforming matrix does not change the noise distribution [71]. Additionally, assuming ${\mathit{F}}_{BB,1}\left[n\right], \forall n$ as a unitary matrix keeps the noise power the same, i.e., ${\sigma }_r^2$ after applying the hybrid combiner $\left({\mathit{F}}_{RF,1}{\mathit{F}}_{BB,1}\left[n\right]\right)$ at the relay node. Hence, the total post-processing SNR, which is determined after hybrid combining, is given as

$$SNR=\frac{\Vert {\mathit{F}}_{BB,1}^H\left[n\right]{\mathit{F}}_{RF,1}^H\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\mathit{V}}_{BB}{\left[n\right]\Vert }_F^2}{{\sigma }_r^2}.$$

(23)

It is known that capacity expression in MIMO systems is directly related to SNR. Therefore, the sub-optimization problem (15) for deriving ${\mathit{F}}_{RF,1}$ can be approximately written as

$$\begin{gathered} \underset{{\left\{{\mathit{F}}_{RF,1},{\mathit{F}}_{BB,1}\left[n\right],\right\}}_{n=1}^{N_{sub}}}{\max }\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}\left(\frac{\Vert {\mathit{F}}_{BB,1}^H\left[n\right]{\mathit{F}}_{RF,1}^H\mathit{H}\left[n\right]{\mathit{V}}_{RF}{\mathit{V}}_{BB}{\left[n\right]\Vert }_F^2}{{\sigma }_r^2}\right) \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{F}}_{RF,1}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}}, \forall x,y\end{array}. \end{gathered}$$

(24)

Furthermore, the sub-problem in (24) for maximizing the post-processing SNR can be equivalently transformed into the received power maximization problem as follows:

$$\begin{gathered} \underset{{\left\{{\mathit{F}}_{RF,1},{\mathit{F}}_{BB,1}\left[n\right],\right\}}_{n=1}^{N_{sub}}}{\max }\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}Tr\left\{\left({\mathit{F}}_{BB,1}^H\left[n\right]{\mathit{H}}_{eq}\left[n\right]{\mathit{V}}_{BB}\left[n\right]\right){\left({\mathit{F}}_{BB,1}^H\left[n\right]{\mathit{H}}_{eq}\left[n\right]{\mathit{V}}_{BB}\left[n\right]\right)}^H\right\} \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{F}}_{RF,1}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}}, \forall x,y\end{array}. \end{gathered}$$

(25)

Again, joint optimization of ${\mathit{F}}_{RF,1}$ and ${\left\{{\mathit{F}}_{BB,1}\left[n\right]\right\}}_{n=1}^{N_{sub}}$ in (24) is challenging due to the coupling between these processing matrices. This gives motivation for designing these beamforming components separately. Therefore, the digital baseband processing components ${\left\{{\mathit{V}}_{BB}\left[n\right], {\mathit{F}}_{BB,1}\left[n\right]\right\}}_{n=1}^{N_{sub}}$ in (24) can be ignored following a decoupled approach to reduce the complexity associated with the solution. Once the frequency-independent relay analog combiner is derived, the standard method can be used to find these baseband processing matrices to minimize interference among transmitted data streams. Therefore, the optimization problem (25) can be suppressed as

$$\begin{gathered} \underset{{\mathit{F}}_{RF,1}}{\max }\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}Tr\left\{{\mathit{H}}_{eq}\left[n\right]{\mathit{H}}_{eq}^H\left[n\right]\right\}=Tr\left\{\left({\mathit{F}}_{RF,1}^H\mathit{H}\left[n\right]{\mathit{V}}_{RF}\right){\left({\mathit{F}}_{RF,1}^H\mathit{H}\left[n\right]{\mathit{V}}_{RF}\right)}^H\right\} \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{F}}_{RF,1}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}}, \forall x,y\end{array}. \end{gathered}$$

(26)

For designing the ${\mathit{F}}_{RF,1}$, the objective function in (26) can be further simplified as

$$\begin{gathered} \underset{{\mathit{F}}_{RF,1}}{\max }\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}Tr\left\{{\mathit{F}}_{RF,1}^H{\mathit{H}}_{tmp}\left[n\right]{\mathit{H}}_{tmp}^H\left[n\right]{\mathit{F}}_{RF,1}\right\} \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{F}}_{RF,1}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}}, \forall x,y\end{array}, \end{gathered}$$

(27)

where ${\mathit{H}}_{tmp}\left[n\right]=\mathit{H}\left[n\right]{\mathit{V}}_{RF}$. The product ${\mathit{H}}_{tmp}\left[n\right]{\mathit{H}}_{tmp}^H\left[n\right]$ in the objective function of (27) can be replaced with the matrix given as

$$\mathit{A}={\displaystyle \sum }_{n=1}^{N_{sub}}\left(\frac{{\mathit{H}}_{tmp}\left[n\right]{\mathit{H}}_{tmp}^H\left[n\right]}{N_{sub}}\right),$$

(28)

where $\mathit{A}\in {ℂ}^{N_r\times N_r}$ is the mean of the covariance of frequency domain channels ${\mathit{H}}_{tmp}\left[n\right]$ taken over $N_{sub}$ sub-carriers. Using (28), the optimization problem in (27) can be written as

$$\begin{gathered} \underset{{\mathit{F}}_{RF,1}}{\max }Tr\left\{{\mathit{F}}_{RF,1}^H\mathit{A} {\mathit{F}}_{RF,1}\right\} \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\mathit{F}}_{RF,1}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}}, \forall x,y\end{array}. \end{gathered}$$

(29)

The function $Tr\left\{{\mathit{F}}_{RF,1}^H\mathit{A} {\mathit{F}}_{RF,1}\right\}$ in (29) can also be expressed as

$$Tr\left\{{\mathit{F}}_{RF,1}^H\mathit{A} {\mathit{F}}_{RF,1}\right\}={\displaystyle \sum }_{m=1}^{N_r^{RF}}{\left({\mathit{f}}_{RF,1}^H\right)}^{\left[m\right]}\mathit{A}{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]},$$

(30)

where ${\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}\in {ℂ}^{N_r\times 1}$ is the $m$-th analog beamforming vector of ${\mathit{F}}_{RF,1}$, and ${\left({\mathit{f}}_{RF,1}^H\right)}^{\left[m\right]}\mathit{A}{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}$ is the $m$-th diagonal element of ${\mathit{F}}_{RF,1}^H\mathit{A} {\mathit{F}}_{RF,1}\in {ℂ}^{N_r^{RF}\times N_r^{RF}}$. Using (30), the problem in (29) can be converted into the equivalent vector form as

$$\begin{gathered} \underset{{\left\{{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}\right\}}_{m=1}^{N_r^{RF}}}{\max }{\displaystyle \sum }_{m=1}^{N_r^{RF}}{\left({\mathit{f}}_{RF,1}^H\right)}^{\left[m\right]}\mathit{A}{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]} \\ s. t. \{\begin{array}{c}Tr\left\{{\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)}^H\left({\mathit{V}}_{RF}{\mathit{V}}_{BB}\left[n\right]\right)\right\}\le P_s, \forall n,\\ d_1\left[n\right]\le J_1,\\ \left|{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}\left(l\right)\right|=\frac{1}{\sqrt{N_r}}, \forall l\in \left\{1,\dots ,N_r\right\}\end{array}, \end{gathered}$$

(31)

where ${\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}\left(l\right)$ depicts the $l$-th element of the $m$-th column in ${\mathit{F}}_{RF,1}$. The primary objective while designing ${\mathit{F}}_{RF,1}$, is to decompose the channels into parallel sub-channels to ensure effective multi-steam transmission. The constant modulus constraints on the RF processing component pose a main challenge. Relaxing the constant amplitude constraints can lead to the optimal solution corresponding to ${\mathit{F}}_{RF,1}$ by performing eigenvalue decomposition (EVD) on $\mathit{A}$ (28). The optimal combiner can be determined by selecting the $N_r^{RF}$ eigenvectors that correspond to the $N_r^{RF}$ maximum eigenvalues. This matrix makes it possible to receive signals along the eigenmodes of the channel. Note that the eigenvectors cannot be directly used to rotate the signals when the element-wise constant amplitude constraints are taken into consideration. In this situation, the projection on each eigenmode can be considered a criterion for designing the RF beamforming vectors. The best beamforming matrix should be the one whose vectors have the largest projections on the respective eigenmodes. For instance, the EVD of $\mathit{A}$ in (28) is characterized as

$$\mathit{A}=\mathit{T}\mathit{\Sigma }{\mathit{T}}^H={\displaystyle \sum }_{r=1}^L{\lambda }_r{\mathit{t}}_r{\mathit{t}}_r^H,$$

(32)

where $\mathit{T}\in {ℂ}^{N_r\times N_r}$ is the unitary matrix, and $\mathit{\Sigma }\in {ℂ}^{N_r\times N_r}$ is a diagonal matrix that comprises eigenvalues. Furthermore, ${\mathit{t}}_r$ is the $r$-th vector in $\mathit{T}$, which is associated with the eigenvalue ${\lambda }_r$. From the perspective of mm-wave transmission, $L$ is the rank of the channel and describes the number of propagation paths. To determine the RF beamforming vector ${\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}$ in (31), there is a need to maximize the projection of the $m$-th data stream on the $m$-th eigenmode, i.e., $\left|{\mathit{t}}_m^H{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}\right|$. The maximal projection can be obtained by formulating the optimization problem that minimizes the mean squared error (MSE) between the unknown vector ${\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}$ and the corresponding unconstrained optimal vector. Therefore,

$$\begin{gathered} \underset{{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}}{\min }Tr\left\{{\left({\mathit{t}}_m^{opt}-{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}\right)}^H\left({\mathit{t}}_m^{opt}-{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}\right)\right\}=\Vert {\mathit{t}}_m^{opt}-{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}{\Vert }_2^2 \\ s. t. \left|{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}\left(l\right)\right|=\frac{1}{\sqrt{N_r}}, \forall l\in \left\{1,\dots ,N_r\right\}, m\in \left\{1,\dots ,N_r^{RF}\right\}, \end{gathered}$$

(33)

where ${\mathit{t}}_m^{opt}=\mathit{T}\left(:,m\right)\in {ℂ}^{N_r\times 1}$ is the optimal combining vector corresponding to the maximum eigenvalue in $\mathit{\Sigma }\in {ℂ}^{N_r\times N_r}$ (32), and ${\Vert .\Vert }_2$ stands for the $L_2$-norm of a vector. The objective function in (33), using the property of $L_2$-norm and trace operator, can be written as

$$\Vert {\mathit{t}}_m^{opt}-{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}{\Vert }_2^2=2-2 Tr\left(ℝ\left[{\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}{\left({\mathit{t}}_m^{opt}\right)}^H\right]\right).$$

(34)

It is obvious from (34) that the minimum value can be obtained when ${\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}$ has the same phase-values as ${\mathit{t}}_m^{opt}$. Under this condition, the projection of ${\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}$ on ${\mathit{t}}_m^{opt}$ becomes maximum, and this information can be exploited for designing the required phase-only combining vector as [75]

$${\left({\mathit{f}}_{RF,1}\right)}^{\left[m\right]}=\left(\frac{1}{\sqrt{N_r}}\right)\exp \left\{j arg\left({\mathit{p}}_m^{opt}\right)\right\},$$

(35)

where $arg(.)$ denotes the argument operator. Hence, the ${\mathit{F}}_{RF,1}$ is formulated as

$${\mathit{F}}_{RF,1}=\left[{\left({\mathit{f}}_{RF,1}\right)}^{\left[1\right]},\dots ,{\left({\mathit{f}}_{RF,1}\right)}^{\left[N_r^{RF}\right]}\right].$$

(36)

3.2. Baseband Beamformer Optimization

In the previous sub-section, the source analog precoder and the relay analog combiner were designed to maximize RF beamforming gain by maximizing projections along the respective eigenmodes. After deriving the above-mentioned phase-only processing components, it is possible to define the equivalent channel observed from the baseband processing units in the CRBS and the relay node. By diagonalizing this equivalent channel ${\mathit{H}}_{eq}\left[n\right]$ (8), the frequency-selective digital baseband precoders and combiners at the source and the relay node can be obtained. Hence, ${\mathit{H}}_{eq}\left[n\right]={\mathit{U}}_R\left[n\right]{\mathit{\Sigma }}_{SR}\left[n\right]{\mathit{V}}_S^H\left[n\right]$, ${\mathit{V}}_{BB}\left[n\right]={\mathit{V}}_S\left[n\right]\left(:,1:K{N}_s\right)$, ${\mathit{F}}_{BB,1}\left[n\right]={\mathit{U}}_R\left[n\right]\left(:,1:K{N}_s\right)$. These baseband processing components minimize interference among transmitted data streams from the source to the relay station indirectly.

3.3. Relay RF Precoder and RF Combiners at SUs

In the second time slot, the received signal ${\mathit{y}}_{d_k}\left[n\right]$ at the $k$-th SU in terms of the transmitted signal ${\mathit{y}}_{r_1}\left[n\right]$ (3) from the relay can be written as

$${\mathit{y}}_{d_k}\left[n\right]={\left({\mathit{W}}_{RF,k}{\mathit{W}}_{BB,k}\left[n\right]\right)}^H\left\{{\mathit{G}}_{d_k}\left[n\right]{\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}\left[n\right]{\mathit{y}}_{r_1}\left[n\right]+{\mathit{z}}_k\left[n\right]\right\}.$$

(37)

The compact representation of (37) is given as

$${\mathit{y}}_{d_k}^c\left[n\right]={\mathit{W}}_{d_k}^H\left[n\right]{\mathit{G}}_{d_k}\left[n\right]{\mathit{F}}_2\left[n\right]{\mathit{s}}^R\left[n\right]+{\mathit{W}}_{d_k}^H\left[n\right]{\mathit{z}}_k\left[n\right].$$

(38)

The received signal at the $k$-th SU user can also be represented as

$${\overline{\mathit{y}}}_{d_k}\left[n\right]={\mathit{W}}_{d_k}^H\left[n\right]{\mathit{G}}_{d_k}\left[n\right]{\mathit{F}}_2^k\left[n\right]{\mathit{s}}_k^R\left[n\right]+{\displaystyle \sum }_{j=1,j\ne k}^K{\mathit{W}}_{d_j}^H\left[n\right]{\mathit{G}}_{d_j}\left[n\right]{\mathit{F}}_2^j\left[n\right]{\mathit{s}}_j^R\left[n\right]+{\mathit{W}}_{d_k}^H\left[n\right]{\mathit{z}}_k\left[n\right],$$

(39)

where ${\mathit{W}}_{d_k}\left[n\right]={\mathit{W}}_{RF,k}{\mathit{W}}_{BB,k}\left[n\right]\in {ℂ}^{N_{d_k}\times N_s}$, ${\mathit{F}}_2\left[n\right]={\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}\left[n\right]\in {ℂ}^{N_r\times K{N}_s}$, ${\mathit{F}}_2^k\left[n\right]={\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}^k\left[n\right]$, and $s^R\left[n\right]={\mathit{y}}_{r_1}\left[n\right]\in {ℂ}^{K{N}_s\times 1}$. Assuming Gaussian signaling, the expression of $R_2$, using (39), is obtained as

$$R_2={\log }_2\det \left({\mathit{I}}_{N_s}+{\left({\mathit{R}}_{N_2}^k\right)}^{-1}\left({\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{G}}_{eq,k}\left[n\right]{\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]\right){\left({\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{G}}_{eq,k}\left[n\right]{\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]\right)}^H\right),$$

(40)

where ${\mathit{R}}_{N_2}^k={\mathit{W}}_{d_k}^H\left[n\right]\left({\displaystyle \sum }_{j\ne k}\left({\mathit{G}}_j\left[n\right]{\mathit{F}}_2^j\left[n\right]\right){\left({\mathit{G}}_j\left[n\right]{\mathit{F}}_2^j\left[n\right]\right)}^H+{\sigma }_k^2{\mathit{I}}_{N_{d_k}}\right){\mathit{W}}_{d_k}\left[n\right]$ shows the residual IUI plus noise. Using (40), the SNIR at the $k$-th SU after hybrid combining is expressed as

$${\left(SN\mathit{I}R\right)}_k=\frac{Tr\left(\left({\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{G}}_{eq,k}\left[n\right]{\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]\right){\left({\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{G}}_{eq,k}\left[n\right]{\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]\right)}^H\right)}{Tr\left({\mathit{W}}_{d_k}^H\left[n\right]\left({{\displaystyle \sum }}_{j\ne k}\left({\mathit{G}}_j\left[n\right]{\mathit{F}}_2^j\left[n\right]\right){\left({\mathit{G}}_j\left[n\right]{\mathit{F}}_2^j\left[n\right]\right)}^H+{\sigma }_k^2{\mathit{I}}_{N_{d_k}}\right){\mathit{W}}_{d_k}\left[n\right]\right)}.$$

(41)

It is evident from (41) that direct optimization is a difficult task due to the dependence on complex matrix variables. Additionally, the RF beamforming solution at the relay node and each SU essentially need to follow the element-wise constant amplitude constraints. Therefore, an endeavor is made to approximate the expression in (41) to reduce the complexity of the optimization problem. After determining the RF processing components in (41), digital baseband precoding matrices ${\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right], k\in \left\{1,\dots ,K\right\},$ can be obtained through the equivalent baseband channels ${\mathit{W}}_{RF,k}^H{\mathit{G}}_{d_k}\left[n\right]{\mathit{F}}_{RF,2}$, $k\in \left\{1,\dots ,K\right\}$, using the block diagonalization (BD) technique. This implies that $\left({\mathit{G}}_k\left[n\right]{\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}^j\left[n\right]\right){\left({\mathit{G}}_k\left[n\right]{\mathit{F}}_{RF,2}{\mathit{F}}_{BB,2}^j\left[n\right]\right)}^H=\mathbf{0}$, $\forall j\ne k$. Therefore, the SNIR in (41) can be reduced as

$${\left(\overline{SNIR}\right)}_k=\frac{Tr\left(\left({\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{G}}_{eq,k}\left[n\right]{\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]\right){\left({\mathit{W}}_{BB,k}^H\left[n\right]{\mathit{G}}_{eq,k}\left[n\right]{\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]\right)}^H\right)}{Tr\left({\sigma }_k^2{\mathit{W}}_{d_k}^H\left[n\right]{\mathit{W}}_{d_k}\left[n\right]\right)}.$$

(42)

It has been shown in [71] that RF processing components do not have any impact on the noise distribution, i.e., the noise vector shows the same variance after multiplication with the analog beam formers, as already mentioned. Moreover, the BD technique gives unitary matrices as the baseband processing components at the SUs. In conclusion, the noise power in (41) remains constant, and the optimization problem defined in (16) can be transformed into a received power maximization problem. Following this discussion and decoupled design methodology, the sub-problem in (16) can be approximately written as

$$\begin{gathered} \underset{{\mathit{F}}_{RF,2}, {\mathit{W}}_{RF,k}}{\max }\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}\left(Tr\left\{\left({\mathit{W}}_{RF,k}^H{\mathit{G}}_{d_k}\left[n\right]{\mathit{F}}_{RF,2}\right){\left({\mathit{W}}_{RF,k}^H{\mathit{G}}_{d_k}\left[n\right]{\mathit{F}}_{RF,2}\right)}^H\right\}\right) \\ s. t. \{\begin{array}{c}{\mathit{F}}_{RF,2}=\left[\begin{array}{ccc}{\mathsf{p}}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathsf{p}}_{N_r^{R\mathit{F}}}\end{array}\right],{\mathit{W}}_{RF}=\left[\begin{array}{ccc}{\mathit{W}}_{RF,1}& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{W}}_{RF,K}\end{array}\right],\\ d_2\left[n\right]\le J_2,\\ \left|{\mathit{F}}_{RF,2}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}} ,\left|{\mathit{W}}_{RF,k}\left(x,y\right)\right|=\frac{1}{\sqrt{N_{d_k}}}, \forall x,y\end{array}. \end{gathered}$$

(43)

When a large number of antennas are installed at the relay node, it will become highly probable that ${\mathit{F}}_{RF,2}{\mathit{F}}_{RF,2}^H\approx N_r{\mathit{I}}_r$ [43]. Furthermore,

$${\mathit{C}}_k=\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}{\mathit{G}}_{d_k}\left[n\right]{\mathit{G}}_{d_k}^H\left[n\right]\in {ℂ}^{N_{d_k}\times N_{d_k}}.$$

(44)

Under the assumption of the large antenna array and the mean of the covariance of frequency-domain channels (44), the problem in (43) can be expressed as

$$\begin{gathered} \underset{ {\mathit{W}}_{RF,k}}{\max }Tr\left({\mathit{W}}_{RF,k}^H{\mathit{C}}_k {\mathit{W}}_{RF,k}\right) \\ s. t. \{\begin{array}{c}{\mathit{W}}_{RF}=\left[\begin{array}{ccc}{\mathit{W}}_{RF,1}& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{W}}_{RF,K}\end{array}\right],\\ d_2\left[n\right]\le J_2,\\ \left|{\mathit{W}}_{RF,k}\left(x,y\right)\right|=\frac{1}{\sqrt{N_{d_k}}}, \forall x,y\end{array}. \end{gathered}$$

(45)

Except for the diagonal structure of ${\mathit{W}}_{RF,k}$, the problem in (45) is the same as that of (29), and hence, the same solution strategy can be applied to find the partially connected analog RF combiner at the $k$-th SU. To avoid repetition, the main design steps are shown here. The EVD of ${\mathit{C}}_k$ in (44) is given as

$${\mathit{C}}_k=\mathit{L}{\mathit{\Sigma }}_2{\mathit{L}}^H,$$

(46)

where $\mathit{L}\in {ℂ}^{N_{d_k}\times N_{d_k}}$ is the unitary matrix, and ${\mathit{\Sigma }}_2\in {ℂ}^{N_{d_k}\times N_{d_k}}$ is a diagonal matrix that contains eigenvalues. Let ${\mathit{L}}_k^{opt}\in {ℂ}^{N_{d_k}\times N_s}$ be the optimal combiner at the $k$-th SU that corresponds to the maximum eigenvalue in ${\mathit{\Sigma }}_2$, which is given as

$${\mathit{L}}_k^{opt}=\left[{\mathit{l}}_1,{\mathit{l}}_2,\dots ,{\mathit{l}}_{N_s}\right].$$

(47)

The required RF combiner ${\mathit{W}}_{RF, k}, k\in \left\{1,\dots ,K\right\}$ at the $k$-th SU can be determined by finding its projection as close as possible to the optimal combiner in (47). This target can be achieved by formulating the MMSE problem as

$$\begin{gathered} \underset{{\mathit{W}}_{RF, k}, k\in \left\{1,\dots ,K\right\}}{\min }{\Vert \left[\begin{array}{ccc}{\mathit{L}}_1^{opt}-{\mathit{W}}_{RF,1}& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{L}}_K^{opt}-{\mathit{W}}_{RF,K}\end{array}\right]\Vert }_F^2=\underset{{\mathit{W}}_{RF, k}}{\min }{\displaystyle \sum }_{k=1}^K\Vert {\mathit{L}}_k^{opt}-{\mathit{W}}_{RF,k}{\Vert }_F^2 \\ =\underset{{\mathit{W}}_{RF, k}}{\min }{\displaystyle \sum }_{k=1}^K\left(Tr\left\{{\left({\mathit{L}}_k^{opt}-{\mathit{W}}_{RF,k}\right)}^H\left({\mathit{L}}_k^{opt}-{\mathit{W}}_{RF,k}\right)\right\}\right) \\ s. t. \left|{\mathit{W}}_{RF,k}\left(x,y\right)\right|=\frac{1}{\sqrt{N_{d_k}}}. \end{gathered}$$

(48)

The minimum of the cost function in (48) occurs when ${\mathit{W}}_{RF,k}$ has the same phase values as ${\mathit{L}}_k^{opt}$. Therefore, ${\mathit{W}}_{RF,k}$ can be obtained using the following relation.

$${\mathit{W}}_{RF,k}=\left(\frac{1}{\sqrt{N_t}}\right)\exp \left\{j arg\left({\mathit{L}}_k^{opt}\right)\right\}.$$

(49)

To adjust ${\mathit{F}}_{RF,2}$ in accordance with ${\mathit{W}}_{RF,k}, \forall k$, the optimization problem based on received power maximization is approximated as

$$\begin{gathered} \underset{{\mathit{F}}_{RF,2}}{\max }\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}\left(Tr\left\{{\left({\mathit{W}}_{RF}^H\mathit{G}\left[n\right]{\mathit{F}}_{RF,2}\right)}^H\left({\mathit{W}}_{RF}^H\mathit{G}\left[n\right]{\mathit{F}}_{RF,2}\right)\right\}\right) \\ s. t. \{\begin{array}{c}{\mathit{F}}_{RF,2}=\left[\begin{array}{ccc}{\mathit{p}}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{p}}_{N_r^{R\mathit{F}}}\end{array}\right],\\ d_2\left[n\right]\le J_2,\\ \left|{\mathit{F}}_{RF,2}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}} ,\forall x,y\end{array} , \end{gathered}$$

(50)

where ${\mathit{W}}_{RF}=bd\left[{\mathit{W}}_{RF,1},\dots ,{\mathit{W}}_{RF,K}\right]\in {ℂ}^{K{N}_{d_k}\times K{N}_{d_k}^{RF}}$ is a block diagonal matrix and $\mathit{G}\left[n\right]={\left[{\mathit{G}}_{d_1}^T\left[n\right],\dots ,{\mathit{G}}_{d_K}^T\left[n\right]\right]}^T\in {ℂ}^{K{N}_{d_k}\times N_r}$. Just like (45), this problem can also be transformed into the following form:

$$\begin{gathered} \underset{ {\mathit{F}}_{RF,2}}{\max }Tr\left({\mathit{F}}_{RF,2}^H\mathit{D} {\mathit{F}}_{RF,2}\right) \\ s. t. \{\begin{array}{c}{\mathit{F}}_{RF,2}=\left[\begin{array}{ccc}{\mathit{p}}_1& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & {\mathit{p}}_{N_r^{R\mathit{F}}}\end{array}\right],\\ d_2\left[n\right]\le J_2,\\ \left|{\mathit{F}}_{RF,2}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}} ,\forall x,y\end{array}, \end{gathered}$$

(51)

where $\mathit{D}\in {ℂ}^{N_r\times N_r}$ is defined as

$$\mathit{D}=\left(\frac{1}{N_{sub}}\right){\displaystyle \sum }_{n=1}^{N_{sub}}{\mathit{G}}^H\left[n\right]{\mathit{W}}_{RF}{\mathit{W}}_{RF}^H\mathit{G}\left[n\right].$$

(52)

When EVD is performed on $\mathit{D}$, then

$$\mathit{D}=\mathit{E}{\mathit{\Sigma }}_3{\mathit{E}}^H,$$

(53)

where $\mathit{D}\in {ℂ}^{N_r\times N_r}$ is the unitary matrix and ${\mathit{\Sigma }}_3\in {ℂ}^{N_r\times N_r}$ is a diagonal matrix that contains eigenvalues. Using (53), the unconstrained optimal precoder at the relay ${\mathit{F}}_{opt}\in {ℂ}^{N_r\times N_r^{RF}}$ can be determined as ${\mathit{F}}_{opt}=E\left(:,1:N_r^{RF}\right)$. But this optimal solution does not fulfill the essential condition of constant modulus constraints associated with the desired RF beamforming solution, i.e., ${\mathit{F}}_{RF,2}$. However, it is possible to find the nearest point to the corresponding optimal precoder in high-dimensional space. This process helps in deriving the ${\mathit{F}}_{RF,2}$ from the unconstrained precoding matrix ${\mathit{F}}_{opt}$. To achieve this design objective, a problem is defined for deriving the required analog beamformer at the relay node as follows:

$$\begin{gathered} \underset{{\mathit{F}}_{RF,2}}{\min }\Vert {\mathit{F}}_{opt}-{\mathit{F}}_{RF,2}{\Vert }_F^2=Tr\left\{{\left({\mathit{F}}_{opt}-{\mathit{F}}_{RF,2}\right)}^H\left({\mathit{F}}_{opt}-{\mathit{F}}_{RF,2}\right)\right\} \\ s. t. \left|{\mathit{F}}_{RF,2}\left(x,y\right)\right|=\frac{1}{\sqrt{N_r}} ,\forall x,y. \end{gathered}$$

(54)

It is clear from (54) that the reconstruction loss cannot be fully avoided by achieving the exact lower bound, i.e., $\Vert {\mathit{F}}_{opt}-{\mathit{F}}_{RF,2}{\Vert }_F^2=0$, due to the element-wise non-convex constraints. Therefore, an endeavor is made to minimize the reconstruction error $\Vert {\mathit{F}}_{opt}-{\mathit{F}}_{RF,2}{\Vert }_F^2\approx 0$ in (54) to find ${\mathit{F}}_{RF,2}$. The objective function in (54) can be written as

$$\begin{gathered} Tr\left\{{\left({\mathit{F}}_{opt}-{\mathit{F}}_{RF,2}\right)}^H\left({\mathit{F}}_{opt}-{\mathit{F}}_{RF,2}\right)\right\}=\Vert {\mathit{F}}_{opt}{\Vert }_F^2+\Vert {\mathit{F}}_{RF,2}{\Vert }_F^2-Tr\left\{2ℝ\left({\mathit{F}}_{RF,2}{\mathit{F}}_{opt}^H\right)\right\} \\ =\Vert {\mathit{F}}_{opt}{\Vert }_F^2+\Vert {\mathit{F}}_{RF,2}{\Vert }_F^2-2{\displaystyle \sum }_{x=1}^{N_r}{\displaystyle \sum }_{y=1}^{N_r^{RF}}ℝ\left\{{\mathit{F}}_{RF,2}\left(x,y\right){\mathit{F}}_{opt}^{*}\left(x,y\right)\right\}. \end{gathered}$$

(55)

It is obvious from (55) that the minimum value of $\Vert {\mathit{F}}_{opt}-{\mathit{F}}_{RF,2}{\Vert }_F^2$ can only be obtained when each $ℝ\left\{{\mathit{F}}_{RF,2}\left(x,y\right){\mathit{F}}_{opt}^{*}\left(x,y\right)\right\}$ attains its maximum value. As ${\mathit{F}}_{opt}\left(x,y\right)=\left|{\mathit{F}}_{opt}\left(x,y\right)\right|\mathit{exp}\left\{j\mathit{arg}\left({\mathit{F}}_{opt}\left(x,y\right)\right)\right\}$ and ${\mathit{F}}_{RF,2}\left(x,y\right)=\left|{\mathit{F}}_{RF,2}\left(x,y\right)\right|\mathit{exp}\left\{j\mathit{arg}\left({\mathit{F}}_{RF,2}\left(x,y\right)\right)\right\}$, therefore,

$$ℝ\left\{{\mathit{F}}_{RF,2}\left(x,y\right){\mathit{F}}_{opt}^{*}\left(x,y\right)\right\}=\left|{\mathit{F}}_{opt}\left(x,y\right)\right|\left|{\mathit{F}}_{RF,2}\left(x,y\right)\right|\cos \psi \le \left|{\mathit{F}}_{opt}\left(x,y\right)\right|\left|{\mathit{F}}_{RF,2}\left(x,y\right)\right|,$$

(56)

where $\psi =\mathit{arg}\left({\mathit{F}}_{RF,2}\left(x,y\right)\right)-\mathit{arg}\left({\mathit{F}}_{opt}\left(x,y\right)\right)$. When $\mathit{arg}\left({\mathit{F}}_{RF,2}\left(x,y\right)\right)=\mathit{arg}\left({\mathit{F}}_{opt}\left(x,y\right)\right)$ then $\cos \psi =1$, and this condition leads (56) to its maximum value. In summary, the required RF beamformer ${\mathit{F}}_{RF,2}$ shares the element-wise phase of the corresponding optimal beamformer ${\mathit{F}}_{opt}$. Hence, ${\mathit{F}}_{RF,2}$ is given as

$${\mathit{F}}_{RF,2}=\left(\frac{1}{\sqrt{N_r}}\right)\exp \left\{j\mathit{arg}\left({\mathit{F}}_{opt}\right)\right\}.$$

(57)

3.4. Digital Baseband Precoding and Combining

The section of the secondary network (SN) from the relay node to the SUs can be transformed in terms of the baseband equivalent channels ${\mathit{G}}_{eq, k}\left[n\right]={\mathit{W}}_{RF,k}^H{\mathit{G}}_k\left[n\right]{\mathit{F}}_{RF,2}, \forall k,n$, as ${\mathit{F}}_{RF,2}$ and ${\mathit{W}}_{RF,k}$ have already been derived. Taking advantage of this transformation, the frequency-selective low-dimensional baseband processing matrices ${\mathit{F}}_{BB,2}\left[n\right]$ and ${\mathit{W}}_{BB,k}\left[n\right]$ can be designed using conventional techniques by exploiting ${\mathit{G}}_{eq, k}\left[n\right],\forall k, n$. Minimizing the impact of IUI is the primary objective behind the design of baseband processing components. This goal can be achieved by dividing the MU-MIMO channel into multiple SU-MIMO channels, and the block diagonalization (BD) technique leads to the required solution [76]. The overall channel ${\mathit{G}}_{eq}^{RD}\left[n\right]\in {ℂ}^{N_r^{RF}\times K{N}_{d_k}^{RF}}$ from the relay to K SUs can be defined as ${\mathit{G}}_{eq}^{RD}\left[n\right]={\left[{\mathit{G}}_{eq,1}^T\left[n\right],\dots ,{\mathit{G}}_{eq,K}^T\left[n\right]\right]}^T\in {ℂ}^{N_r^{RF}\times K{N}_{d_k}^{RF}}$. The constraint ${\mathit{G}}_{eq,j}^{RD}\left[n\right]{\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]=0, j\ne k$ needs to be satisfied to eliminate the IUI, where ${\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]$ denotes the baseband precoder at the relay node corresponding to the $k$-th SU. This constraint demands that the received signal by the $k$-th user must fall in the null space of other user channels.

Define ${\tilde{\mathit{G}}}_{eq,k}^{RD}\left[n\right]={\left[{\mathit{G}}_{eq,1}^T\left[n\right],\dots ,{\mathit{G}}_{eq,k-1}^T\left[n\right],{\mathit{G}}_{eq,k+1}^T\left[n\right],\dots ,{\mathit{G}}_{eq,K}^T\left[n\right]\right]}^T\in {ℂ}^{N_r^{RF}\times \left(K-1\right)N_{d_k}^{RF}}$ that contains the individual baseband equivalent channels except for the intended user. Performing SVD on ${\tilde{\mathit{G}}}_{eq,k}^{RD}\left[n\right]$ leads to the desired null space as follows

$${\tilde{\mathit{G}}}_{eq,k}^{RD}\left[n\right]=\left({\tilde{\mathit{U}}}_{eq,k}^{RD}\left[n\right]\right)\left[\begin{array}{cc}{\tilde{\mathit{\Sigma }}}_{eq,k}^{RD}\left[n\right]& 0\\ 0& 0\end{array}\right]{\left[{\left({\tilde{\mathit{V}}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(1\right)}{\left({\tilde{\mathit{V}}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(0\right)}\right]}^H,$$

(58)

where ${\left({\tilde{\mathit{V}}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(1\right)}$ represents the subspace orthogonal bases, and ${\left({\tilde{\mathit{V}}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(0\right)}$ denotes the null space orthogonal bases of ${\tilde{\mathit{G}}}_{eq,k}^{RD}\left[n\right]$. Therefore,

$${\tilde{\mathit{G}}}_{eq,k}^{RD}\left[n\right]{\left({\tilde{\mathit{V}}}_{eq,i}^{RD}\left[n\right]\right)}^{\left(0\right)}=\left\{\begin{array}{c}0, i\ne k\\ {\tilde{\mathit{G}}}_{eq,k}^{RD}\left[n\right]{\left({\tilde{\mathit{V}}}_{eq,i}^{RD}\left[n\right]\right)}^{\left(0\right)}, i=k\end{array}\right\}.$$

(59)

Applying SVD on ${\mathit{G}}_{eq,k}\left[n\right]{\left({\tilde{\mathit{V}}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(0\right)}$ gives

$${\mathit{G}}_{eq, k}\left[n\right]{\left({\tilde{\mathit{V}}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(0\right)}=\left({\mathit{U}}_{eq,k}^{RD}\left[n\right]\right)\left[\begin{array}{cc}{\Sigma }_{eq,k}^{RD}\left[n\right]& 0\\ 0& 0\end{array}\right]{\left[{\left({\mathit{V}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(1\right)}{\left({\mathit{V}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(0\right)}\right]}^H,\forall k.$$

(60)

To eliminate IUI, ${\left({\mathit{V}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(1\right)}$ corresponding to the non-zero singular values is selected for designing the precoding matrix. Therefore, the ${\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]$ is given as

$${\mathit{F}}_{BB,2}^{\left[k\right]}\left[n\right]={\left({\tilde{\mathit{V}}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(0\right)}{\left({\mathit{V}}_{eq,k}^{RD}\left[n\right]\right)}^{\left(1\right)}, \forall k.$$

(61)

4. Complexity Analysis

Complexity falls into two main categories: (1) hardware implementation complexity and (2) computational complexity. From the perspective of hardware complexity, a fully digital array requires one dedicated RF chain per antenna. For instance, $N_t^{RF}=N_t$ RF chains are required for the digital implementation of the source node. It is evident from the above-mentioned condition that a hybrid architecture demands significantly less hardware complexity, as it considerably reduces the number of RF chains, i.e., $N_t^{RF}\ll N_t$. Among different hybrid structures, the fully connected one employs $N_r^{RF}{N}_r$ phase shifters, which indicates that each RF chain is connected to all antennas in an array. On the other hand, the partially connected structure uses $N_r$ phase shifters. This reduction in the number of phase shifters indicates the lower hardware implementation complexity of partially connected architectures. The proposed scheme employs a combination of fully connected and partially connected structures at the relay node, which is referred to as a mixed architecture. Additionally, the hybrid combiner of each SU also uses a partially connected structure. Therefore, the suggested method shows less hardware complexity when compared with its fully connected counterpart.

In the previous section, a hybrid broadband mm-wave beamforming design was proposed for mixed-structure and fully connected relay-assisted MU-MIMO networks. Also, the CR communication framework was taken into consideration for efficient spectrum utilization. The primary objective behind this technique was to avoid spectral congestion that might occur in the presence of a huge number of connected devices. In addition, the suggested architecture leads to a cost-effective, energy-efficient, and low-complexity solution.

A brief analysis of the computational complexity of the proposed scheme and a comparison with other existing hybrid beamforming techniques are provided in this section. The derivation of the common analog beamforming solution and frequency-dependent digital baseband processing component in wideband systems determines the overall computational complexity. The frequency-independent phase-only beamformer can be obtained by taking the average of the covariance matrices of frequency domain channels, and this operation requires matrix addition and multiplication. Considering matrix multiplication as a major contributing factor, the computational complexity for designing the RF processing components at different communicating nodes is in the order of matrix multiplication. Despite knowing that complex multiplication operations are computationally more expensive compared to complex addition operations, they are both considered one floating-point operation in evaluating the computational cost. For instance, the resultant matrix in (28) contains the product ${\mathit{H}}_{tmp}\left[n\right]{\mathit{H}}_{tmp}^{\mathit{H}}\left[n\right]$, which requires approximately $N_r^2{N}_t$ floating-point operations for its computation. Moreover, eigendecomposition needs to be performed to extract the optimal phase values to design the RF beamforming matrices at various communicating nodes. Hence, the derivation of the analog beamformer requires computational complexity in the order of $O\left\{\max \left({N_r}^3,{N_r}^2{N}_t{N}_{sub}\right)\right\}$. Furthermore, the computational cost associated with the evaluation of the baseband processing component is in the order of $O\left({\left(N_r^{RF}\right)}^3\right)$. Therefore, the total complexity of the proposed approach can be approximated as follows:

$$C^T\approx N_r^2{N}_t+{N_r}^2{N}_t{N}_{sub}+N_{d_k}^2{N}_r+N_{d_k}^2{N}_r{N}_{sub}+{\left(N_r^{RF}\right)}^3+{\left(N_{d_k}^{RF}\right)}^3.$$

(62)

The computational cost of hybrid transceiver design for OFDM-based large-scale MIMO systems [43] can be expressed as

$$N_{sub}\left(N_s^3+N_t^2{N}_r+N_t\left({N_s}^2+{N_r}^2\right)\right)+N_s\left({N_t}^2+{N_r}^2\right).$$

(63)

Finally, the hybrid beamforming technique proposed in [41] demands the following computational complexity:

$$N_s^4+N_s^3\left(N_t+N_r\right)+{\left(N_t{N}_r{N}_s\right)}^2+N_{sub}\left(N_s^3+N_t^2{N}_r+N_r^2{N}_s\right).$$

(64)

Table 3. Complexity of the proposed design and the other hybrid beamforming algorithms.

Algorithms	Complexity
Proposed, [70]	$N_t^2{N}_r+{N_t}^2{N}_r{N}_{sub}+N_{d_k}^2{N}_r+N_{d_k}^2{N}_r{N}_{sub}+{\left(N_r^{RF}\right)}^3+{\left(N_{d_k}^{RF}\right)}^3$
Hybrid Precoding [43]	$N_s^4+N_s^3\left(N_t+N_r\right)+{\left(N_t{N}_r{N}_s\right)}^2+N_{sub}\left(N_s^3+N_t^2{N}_r+N_r^2{N}_s\right)$
Hybrid Beamforming [47]	$N_{sub}\left(N_s^3+N_t^2{N}_r+N_t\left({N_s}^2+{N_r}^2\right)\right)+N_s\left({N_t}^2+{N_r}^2\right)$
Hybrid Transceiver [41]	$O\left(K{N}_{sub}{N}_r{N}_t^2+K{N}_{sub}{N}_s{N}_t^2\right)$
Algorithm [50]	$O\left(K{N}_{sub}{N}_r{N}_t^2+K{N}_{sub}{N}_s{N}_t{N}_t^{RF}\right)$

shows the complexities of different hybrid precoding methods. On the other hand, Figure 2 illustrates the computational complexity of the proposed hybrid transceiver and the hybrid processing techniques developed in [41,43,47,50] as a function of the number of antennas. To obtain these curves, computer simulations are conducted by changing the number of antennas over a wide range while keeping these parameters ($N_{sub}=64$, $N_t^{RF}=N_r^{RF}=K{N}_s$, $N_d^{RF}=N_s=5$, $N_{d_k}=9$, and $K=5$) constant. It is clear from the obtained results that the proposed method shows less computational cost compared to the algorithms presented in [41,43,47,50]. There is a large performance gap between the proposed approach and the algorithm developed in [43]. Also, this gap increases with the number of antennas. For instance, the computational complexity of the proposed scheme is approximately 40 times less than the hybrid transceiver suggested in [43] when 100 antennas are deployed at the source and relay station. Similarly, the computational cost of the proposed technique is approximately 87 times less than the hybrid precoding design in [43] with 225 antennas. In addition, a small performance gap is observed when compared with the algorithm given in [47].

5. Computer Simulations

This section provides numerical results to illustrate the effectiveness of the proposed hybrid precoding algorithm for mixed structures and the corresponding fully connected architecture. Specifically, the sum spectral efficiency and energy efficiency performance of the presented technique are compared with full-complexity digital precoding and other well-known hybrid beamforming designs. It is worth mentioning that computer simulations are conducted by changing system configuration parameters over a wide range, considering perfect CSI and imperfect CSI with different accuracy factors to show robustness. To obtain simulation results, the mm-wave MIMO channel is generated according to the model (12) given in Section 2. The number of propagation paths in each cluster and the number of clusters are set to $N_{ray}=10$ and $N_c=5$, respectively. The angles of departure and the angles of arrival in (12) are generated by following the Laplacian distribution with a uniformly distributed mean over [0, 2π). Moreover, the complex gain of each propagation path is assumed to satisfy the distribution ${\alpha }_{il}$~$\mathcal{C}\mathcal{N}\left(0,{\sigma }_{\alpha ,i}^2\right)$, where ${\sigma }_{\alpha ,i}^2=1 \forall i$ is the average power of each cluster. In addition, the operating central frequency $f_c$ and the total number of sub-carriers $N_{sub}$ are set as follows: 28 GHz and 64, respectively. Note that the proposed algorithm is also applicable to any arbitrary number of sub-carriers (e.g., 128, 256, etc.). The upper bound for the transmit power of CRBS, transmission range, and normalized beamforming gain are given as $N_s$, 200–450 m, 1, respectively. All simulation results are obtained by averaging over 500 random channel realizations. It is worth highlighting that the estimated channel matrix $\overline{\mathit{H}}\left[n\right]$ can be characterized as [77]

$$\overline{\mathit{H}}\left[n\right]=\delta \mathit{H}\left[n\right]+\sqrt{1-{\delta }^2}\mathit{E},$$

(65)

where $\mathit{H}\left[n\right]$ denotes the actual channel matrix, $\delta \in \left[0, 1\right]$ shows the accuracy factor of $\overline{\mathit{H}}\left[n\right]$, and $\mathit{E}$ indicates the error matrix whose elements are independent and identically distributed $\left(i.i.d\right)$ Gaussian random variables, i.e., $\mathcal{C}\mathcal{N}\left(0, 1\right)$. The main simulation parameters are shown in

Table 4. System parameters to generate numerical results.

Parameters	Values
Number of data streams	$N_s=1~16$
Number of RF chains	$N_t^{RF}=N_r^{RF}=K{N}_s,N_{d_k}^{RF}=N_s$
Number of antennas	$N_t=N_r=49~256,N_{d_k}=4~64$
Number of data transmission paths	$N_{cl}=5,N_{ray}=10$
Number of frequency sub-carriers	$N_{sub}=64$
Carrier frequency	$f_c=28\mathrm{GHz}$
Number of secondary users	$K=2~16$

.

5.1. Spectral Efficiency Evaluation

Figure 3 illustrates the sum spectral efficiency performance of the proposed hybrid beamforming as a function of SNR, considering mixed structure and fully connected architecture. The number of antennas deployed at the source and relay node is set as $N_t=N_r=256$, while 64 antennas are installed at each SU, i.e., $N_{d_k}=64$. It is assumed that the CRBS is serving $K=4$ SUs. Moreover, the number of transmitted data streams is set to $N_s=\left\{2, 4, 8\right\}$ under the assumption that $N_t^{RF}=N_r^{RF}=K{N}_s$ and $N_{d_k}^{RF}=N_s$, i.e., the number of RF chains is equal to the number of transmitted data streams at the respective communicating nodes. The obtained results demonstrate that the proposed technique achieves performance close to its fully digital counterpart with the fully connected architecture. On the other hand, the proposed scheme shows relatively lower sum spectral efficiency with the mixed structure when compared with the corresponding fully connected and full-complexity digital ones. It can also be seen from the obtained curves that performance degradation occurs in a gradual manner as channel estimation error increases, i.e., $\delta =0.9 \left(10\%\right), 0.8 \left(20\%\right), 0.7 \left(30\%\right)$. Just like the mixed structure, the performance of the fully connected architecture follows the same pattern when channel estimation error increases in discrete steps. It is evident from the obtained curves that the performance gap increases slightly by increasing the number of transmitted data streams. However, the nearly consistent performance of the proposed algorithm, even with a large number of data streams per user, indicates that IUI is effectively suppressed.

Figure 4 plots the sum spectral efficiency of the proposed approach when 144 antennas are deployed at the source and relay nodes and 36 antennas are installed at each SU, i.e., $N_t=N_r=144$ and $N_{d_k}=36$. To generate simulation results, the least number of RF chains is considered at the respective communicating units, i.e., $N_t^{RF}=N_r^{RF}=K{N}_s$ and $N_{d_k}^{RF}=N_s$. The system is assumed to serve $K=3$ SUs, and the number of transmitted data streams to each user is set as N_s = {2, 3, 4}.

It is clear from the obtained curves that the proposed hybrid transceiver achieves performance close to full-complexity beamforming with the fully connected architecture. However, a small degradation in performance occurs when the mixed structure is taken into consideration. It is also obvious from the obtained results that the proposed algorithm shows consistent performance regardless of changing system parameters. To describe the impact of channel estimation error on the performance of mixed hybrid structures, numerical results are obtained under imperfect CSI at different accuracy levels, i.e., $\delta =0.85, 0.75$. Again, the decreasing accuracy factor leads to gradual degradation in performance, as shown in Figure 4.

Figure 5 describes the spectral efficiency performance of the proposed scheme by further increasing the number of users compared to the previous two cases. To obtain numerical results, the number of antennas at different communicating nodes and the number of transmitted data streams to each SU are set as follows: $N_t=N_r=144, N_{d_k}=16$ and $N_s=\left\{2, 4\right\}$, respectively. It is assumed that the CRBS is serving six users, i.e., $K=6$. The proposed algorithm with the fully connected structure achieves performance close to the upper bound defined by fully digital precoding, while a small degradation in performance occurs with the mixed architecture. Furthermore, a gradual decrease in performance is observed when the accuracy of the estimated channel decreases by considering $\delta =0.9, 0.8, 0.7$. It is obvious from Figure 5 that the obtained results follow a similar pattern as observed in the previous cases. This consistency in performance under changing system parameters indicates the usefulness of the suggested method.

Figure 6 depicts the achievable rate of the proposed approach, considering URA at communicating nodes. To generate numerical results, the number of antennas, transmitted data streams, and SUs are given as $N_t=N_r=200$, $N_{d_k}=50$, $N_s=5$, $K=4$, respectively. The obtained results demonstrate that the fully connected hybrid transceiver achieves performance close to the unconstrained fully digital precoding. Moreover, there is a minor performance gap between the mixed structure and the fully connected one, assuming perfect CSI. However, the performance of the proposed scheme decreases gradually by reducing the accuracy of the estimated channel matrix.

5.2. Impact of Number of Users on Spectral Efficiency

Figure 7 plots the sum spectral efficiency of the proposed hybrid transceiver as a function of the number of users in the secondary network (SN). The SNR value is set to 5 [dB], and the number of users increases from 2 to 16. Furthermore, the number of antennas at communicating nodes and the number of transmitted data streams to each SU are given as $N_t=N_r=256$, $N_{d_k}=16$, and $N_s=2$.

Since the effect of IUI increases with the number of users, the rate of change of sum spectral efficiency decreases accordingly. In the case of perfect CSI, the proposed hybrid precoding with the fully connected structure achieves performance close to fully digital beamforming. However, performance degradation occurs when the mixed architecture is taken into consideration. Additionally, the performance of the proposed method also decreases gradually when channel estimation error increases. Finally, the proposed algorithm outperforms hybrid precoding designs presented in [51,52].

5.3. Impact of Number of Antennas on Spectral Efficiency

Figure 8 shows the spectral efficiency performance of the proposed method by varying the number of antennas at the source, relay node, and SUs simultaneously. To visualize the impact of this change, the other system parameters, such as the number of users in the SN and transmitted data streams, are kept constant. It is known that high beamforming gain can be obtained by deploying a large number of transmit and receive antennas, as it results in a narrow beam formation. Numerical results are obtained by following an ascending order for the deployment of antennas at communicating units. The number of antennas at the source, relay station, and each SU is set as 64 (16) and 256 (64) to evaluate the sum spectral efficiency performance. While increasing the number of antennas at the source and relay, it is further assumed, without loss of generality, that $N_t=N_r$. Moreover, $N_s$ and $K$ are set as 2 and 4, respectively. The spectral efficiency increases in a gradual fashion by increasing the number of antennas, irrespective of hybrid beamforming structures. It is obvious from the obtained results that the suggested fully connected hybrid precoding achieves performance close to full-complexity digital beamforming. On the other hand, the presented mixed hybrid processing structure achieves relatively lower performance when compared with its fully connected hybrid and conventional fully digital counterparts. Simulation results demonstrate that minor performance degradation occurs when the estimated channel matrix (82) with $\delta =0.9, 0.8, 0.7$ is taken into consideration.

Figure 9 illustrates the sum spectral efficiency of the proposed approach as a function of the number of antennas at the source and relay node, while the number of antennas at the SUs is kept constant. The number of antennas installed at the source and the relay node is set to $N_t=N_r=\left\{64, 144, 256\right\}$. The other parameters like $N_{d_k}=16$, $K=4$, and $N_s=4$ are also kept constant while conducting computer simulations. Just like in the previous cases, the least number of RF chains is employed at the respective communicating nodes, i.e., $N_t^{RF}=N_r^{RF}=K{N}_s$ and $N_{d_k}^{RF}=N_s$, to obtain simulation results.

Under this condition, the system utilizes minimum power for signal transmission and reduces the cost factor as well. From the obtained curves in Figure 9, it is obvious that sum spectral efficiency increases with an increase in the number of antennas, but the rate of change decreases in a gradual manner. This behavior shows that saturation will occur at a significantly large number of antennas, where the rate of change of sum spectral efficiency becomes negligibly small. Simulation results indicate that the suggested fully connected transceiver achieves near-optimal performance, while relatively lower performance is obtained using its mixed-structure counterpart. The performance gap of the proposed method increases gradually by increasing the channel estimation error, irrespective of the hybrid precoding architecture, as shown in Figure 9.

The sum spectral efficiency of the proposed technique is plotted in Figure 10 as a function of the number of antennas at the SUs, while the number of data streams per SU, the number of SUs, the number of antennas at the relay station, and the number of antennas at the source are kept constant. To generate simulation results for visualizing the effect of this change, the fixed parameters are set as $N_t=N_r=256$, $K=4$, and $N_s=4$. Furthermore, each SU is equipped with the number of antennas $N_{d_k}=\left\{4, 16, 36, 64\right\}$. The obtained performance curves show that spectral efficiency increases and the rate of change decreases by increasing the number of antennas at SUs. It is also evident from the obtained results that large-scale antenna arrays make it possible to enhance beamforming gain without deploying additional expensive RF chains. Like the previous results, the proposed algorithm with the fully connected structure achieves performance close to that of the full-complexity solution. Moreover, the proposed approach with the mixed structure obtains relatively lower performance at a small number of antennas in comparison to both fully digital and fully connected hybrid solutions. But it approaches the performance of the fully connected architecture with a relatively large number of antennas. In addition, the performance degradation occurs in a gradual fashion by increasing the channel estimation error, i.e., $\delta =0.9, 0.8, 0.7$.

5.4. Impact of Number of Data Streams on Spectral Efficiency

Figure 11 depicts the sum spectral efficiency performance of the proposed approach by changing the number of data streams per user. In this case, the number of antennas at communicating units and the number of SUs are kept constant, and these parameters are set as $N_t=N_r=256$, $N_{d_k}=64$ and $K=4$. The number of data streams varies from 2 to 16, and performance curves are obtained at SNR = −10, 0, and 15 [dB]. Simulation results demonstrate that sum spectral efficiency increases and the rate of change decreases when the number of data streams per SU increases. In addition, there is a slight increase in the performance gap between the proposed hybrid precoding and the upper bound defined by full-complexity digital beamforming with an increase in the number of data streams. It is evident from the obtained results that the performance of the fully connected hybrid transceiver is comparatively better than the performance of the mixed hybrid structure. Also, the sum spectral efficiency of the suggested fully connected structure is approximately consistent at different SNR values. There is a minor decrease in the performance of the mixed hybrid structure at $\delta =0.8, 0.7$ when compared with the corresponding curve using perfect CSI, which is obtained by taking the fully connected architecture into account.

5.5. Performance Evaluation with Other Hybrid Beamforming Algorithms

The sum spectral efficiency performance of the suggested method is compared with conventional fully digital beamforming and hybrid precoding techniques given in [51,52], as illustrated in Figure 12 and Figure 13. To show the usefulness of the presented technique, numerical results are obtained by changing the number of antennas at communicating nodes, the number of users in the secondary network, and the number of transmitted data streams to each user. Using the system parameters $N_t=N_r=144$, $N_{d_k}=36$, $K=3$, $N_s=2$, and $N_{sub}=64$, the sum spectral efficiency of various algorithms is plotted in Figure 12.

It is obvious from the performance evaluation curves that the proposed hybrid precoding with the fully connected structure can achieve spectral efficiency close to full-complexity digital beamforming. In addition, the proposed scheme shows relatively lower performance when the mixed hybrid structure is taken into account. It is obvious from Figure 12 that minor degradation in the performance of mixed hybrid precoding occurs, considering imperfect CSI at $\delta =0.85, 0.75$. Furthermore, the proposed algorithm shows significantly better performance when compared with hybrid beamforming techniques suggested in [51,52].

Similarly, the sum spectral efficiency of different algorithms is plotted in Figure 13, where system parameters are set to $N_t=N_r=256$, $N_{d_k}=64$, $K=4$, $N_s=4$, and $N_{sub}=64$. It is evident from the obtained curves that the suggested hybrid precoding solution gives performance close to the upper bound when the fully connected structure is taken into consideration. On the other hand, degradation in performance occurs in the case of mixed hybrid architecture, as the underlying structure limits the flexibility of large-scale antenna systems. Additionally, a slight decrease in performance is observed owing to channel estimation error. Also, the proposed technique outperforms well-known hybrid beamforming designs presented in [51,52].

5.6. Energy Efficiency Evaluation

Energy consumption is of great practical importance for hybrid transceiver design in mm-wave relay-assisted MU-MIMO systems. This design aspect leads to sub-connected or mixed-structure hybrid beamforming, which enhances the energy efficiency (EE) in the domain of hybrid precoding when compared to the corresponding fully connected structure and full-complexity digital design. The number of phase shifters for a given number of RF chains, antennas, and data streams distinguishes both hybrid processing architectures. It is worthwhile to mention that the fully connected structure does better than the partially or mixed connected architecture in terms of spectral efficiency because it has more degrees of freedom (DoF) in the RF domain, as illustrated in the previous results. The number of phase shifters $N_{PS}$ for the relay node with fully connected and mixed structures can be expressed as

$$N_{PS}=\{\begin{array}{c}{N}_r{N}_r^{RF}, fully-connected\\ N_r, partially-connected \end{array}.$$

(66)

The relay hybrid precoder and combiner use partially connected and fully connected structures, respectively. Similarly, the source hybrid beamformer and the hybrid combiner at each destination utilize the fully connected and sub-connected architectures, respectively. Hence, the overall network and relay fall into the category of mixed hybrid structures. It is obvious from (66) that the mixed-structure hybrid relay filter requires fewer phase shifters in comparison to its fully connected counterpart. This is an indication that the relay with the mixed architecture consumes less energy than the corresponding fully connected one. The EE is defined as [51]

$$EE=\frac{R_{sum}}{P_{total}} \left(\mathrm{bps}/\mathrm{Hz}/\mathrm{W}\right).$$

(67)

It is worth mentioning that the power amplifier (PA) and the low-noise amplifier (LNA) connected to each antenna on the transmitter and receiver, respectively, the RF chains and phase shifters on both transmitter and receiver, analog-to-digital converters (ADC) and digital-to-analog converters (DAC), and the digital baseband processor are the components in the relay hybrid filter that consume power. Therefore, the total consumed power of the DF relay station is given as

$$P_{total}=2N_r^{RF}{P}_{RF}+N_r^{RF}\left(P_{\mathit{A}DC}+P_{D\mathit{A}C}\right)+N_r\left(P_{P\mathit{A}}+P_{LN\mathit{A}}\right)+2N_{PS}{P}_{PS}+2P_{BB},$$

(68)

where $P_{RF}$, $P_{\mathit{A}DC}$, $P_{D\mathit{A}C}$, $P_{P\mathit{A}}$, $P_{LN\mathit{A}}$, $P_{PS}$, and $P_{BB}$ show the power consumed by the RF chain, analog-to-digital converter, digital-to-analog converter, power amplifier, low-noise amplifier, phase shifter, and digital baseband processor, respectively. Specifically, the energy consumption of fully digital precoding $P_{FD}$, the proposed fully connected hybrid architecture $P_{\mathit{F}C}$, and the proposed mixed-structure hybrid transceiver $P_{PC}$ are shown in

Table 5. Energy consumption of different precoding techniques.

Architecture	Energy Consumption at the Relay Node
$P_{\mathit{F}D}$	$2P_{BB}+N_r\left(2P_{RF}+P_{P\mathit{A}}+P_{LN\mathit{A}}+P_{\mathit{A}DC}+P_{D\mathit{A}C}\right)$
$P_{\mathit{F}C}$	$2P_{BB}+2N_r^{RF}{P}_{RF}+N_r\left(P_{P\mathit{A}}+P_{LN\mathit{A}}\right)+2N_r{N}_r^{RF}{P}_{PS}+N_r^{RF}\left(P_{\mathit{A}DC}+P_{D\mathit{A}C}\right)$
$P_{PC}$	$2P_{BB}+2N_r^{RF}{P}_{RF}+N_r\left(P_{P\mathit{A}}+P_{LN\mathit{A}}\right)+2N_r{P}_{PS}+N_r^{RF}\left(P_{\mathit{A}DC}+P_{D\mathit{A}C}\right)$

.

Using (67), the EE performance of different precoding architectures is compared numerically in this sub-section. This task can be accomplished by plotting the EE of the proposed hybrid beamforming structures and full-complexity precoding as a function of the number of RF chains and the number of antennas at the relay station. To evaluate sum spectral efficiency, the system parameters are set as $N_t=N_r=100$, $N_{d_k}=25$, $K=4$, $N_s=1$, $SNR=10 \left[\mathrm{dB}\right]$, and $N_{sub}=64$. Furthermore, the power simulation parameters are given as follows: $P_{BB}=200 \mathrm{mW}$, $P_{RF}=100 \mathrm{mW}$, $P_{LN\mathit{A}}=P_{P\mathit{A}}=100 \mathrm{mW}$, $P_{PS}=10 \mathrm{mW}$, $P_{D\mathit{A}C}=110 \mathrm{mW}$, and $P_{\mathit{A}DC}=200 \mathrm{mW}$. The obtained curves in Figure 14 indicate that the EE decreases as the number of RF chains increases. Owing to the high energy consumption, the EE of the DF-based fully connected architecture decreases more rapidly in comparison to the mixed relay hybrid structure. Also, the EE of the mixed structure decreases slowly as the number of phase shifters in the partially connected hybrid beamformer at the relay transmitter is equal to the number of antennas (66). Finally, the energy consumption is independent of the number of relay RF chains in fully digital beamforming, which always keeps its EE stable. Note that these results are consistent with the theoretical analysis.

Figure 15 demonstrates the EE comparison of different algorithms by changing the number of antennas at the relay station. To evaluate the sum rate, the system parameters are set as follows: $N_t=N_r=\left\{100, 144, 196, 256\right\}$, $N_{d_k}=16$, $K=4$, $N_s=1$, $SNR=10 \left[\mathrm{dB}\right]$, and $N_{sub}=64$. It is clear from the obtained curves that the EE of the mixed DF-based scheme degrades more slowly than that of the corresponding fully connected hybrid and fully digital counterparts. Hence, the proposed mixed-structure technique achieves significantly higher EE by increasing the number of antennas.

Figure 16 and Figure 17 describe the relation between sum spectral efficiency and EE by varying the number of transmit antennas at the relay node. Furthermore, the number of data streams and the number of SUs are also changed to gain more insight into the above-mentioned relationship. Figure 16 plots the sum spectral efficiency and the corresponding EE when the number of antennas at the source and relay node are set as $N_t=N_r=\left\{64,100, 144, 196, 256\right\}$, while $N_{d_k}=16$, $K=4$, $N_s=1$, $SNR=10 \left[\mathrm{dB}\right]$, and $N_{sub}=64$ are kept constant, whereas the number of antennas at the CRBS and the relay station is set as $N_t=N_r=\left\{64, 144, 256\right\}$, and the other constant parameters are given as $N_{d_k}=16$, $K=8$, $N_s=2$, $SNR=10 \left[\mathrm{dB}\right]$, and $N_{sub}=64$ to evaluate the sum spectral efficiency and the corresponding EE, as shown in Figure 17. It is evident from the obtained curves that an inverse relationship exists between spectral efficiency and EE. Again, these results are compatible with the theoretical analysis.

In the case of unconstrained fully digital precoding, the spectral efficiency is maximum, but this technique achieves minimum EE. The suggested mixed-structure hybrid transceiver, on the other hand, has the highest EE, but its spectral efficiency is lower than that of other beamforming methods.

Figure 18 illustrates the EE performance of different beamforming methods as a function of SNR. The simulation results are obtained by setting $N_t=N_r=144$, $N_{d_k}=36$, $K=4$, $N_s=2$, and $N_{sub}=64$. As expected, the presented mixed-structure hybrid beamforming technique shows the highest EE performance in comparison to the other two precoding methods.

5.7. Performance Evaluation at Higher Frequencies (Beam Splitting Effect)

The common (frequency-independent) analog RF processing components are required under wideband assumptions, as already mentioned. However, this design does not hold true when large antenna arrays are deployed, and signal transmission uses significantly higher frequencies. Since the array response vectors become frequency-dependent, the assumption of frequency-flat array response vectors does not remain valid at considerably higher frequencies. These conditions cause a beam-squint effect that splits beams into different physical directions at different frequencies. This process is also termed a spatial wideband effect in the time domain. Serious performance degradation may occur if this problem is not addressed properly while designing hybrid wideband transceivers. To describe the spatial wideband effect on the performance of the proposed algorithm, the following mm-wave channel matrix in the frequency domain is considered [1].

$${\mathit{H}}_{BS}\left[n\right]=\gamma {\displaystyle \sum }_{i=1}^{N_{cl}}{\displaystyle \sum }_{l=1}^{N_{ray}}{\alpha }_{il}{\mathit{a}}_r\left({\phi }_{i,l,n}\right) {\mathit{a}}_t{\left({\psi }_{i,l,n}\right)}^H e^{-j2\pi {\tau }_{i,l}{f}_n} ,$$

(69)

where ${\mathit{a}}_t(.)$ and ${\mathit{a}}_r(.)$ are the beam steering vectors at the source and destination, respectively. Notation ψ_i,l,n and ${\phi }_{i,l,n}$ specify the angle of departure and arrival, respectively, corresponding to the $l$-th relay in the $i$-th cluster at sub-carrier frequency $n$. ${\alpha }_{il}$ and ${\tau }_{i,l}$ represent the path gain and path delay of the $l$-th propagation path in the $i$-th cluster, respectively, and $\gamma =\sqrt{\frac{N_t{N}_r}{N_{cl}{N}_{ray}}}$ stands for the normalization factor. The beam steering vectors ${\mathit{a}}_t\left({\psi }_{i,l,n}\right)$ and ${\mathit{a}}_r\left({\phi }_{i,l,n}\right)$ can be characterized as [1]

$${\mathit{a}}_t\left({\psi }_{i,l,n}\right)=\frac{1}{\sqrt{N_t}}{\left[1, e^{j2\pi {\psi }_{i,l,n}},\dots ,e^{j\left(N_t-1\right)2\pi {\psi }_{i,l,n}}\right]}^T,$$

(70)

$${\mathit{a}}_r\left({\phi }_{i,l,n}\right)=\frac{1}{\sqrt{N_r}}{\left[1, e^{j2\pi {\phi }_{i,l,n}},\dots ,e^{j\left(N_r-1\right)2\pi {\phi }_{i,l,n}}\right]}^T,$$

(71)

where ${\psi }_{i,l,n}$ and ${\phi }_{i,l,n}$ can be determined using the closed-form expressions as follows:

$${\psi }_{i,l,n}=\frac{f_n}{c}d\sin \left({\omega }_{i, l}\right),$$

(72)

$${\phi }_{i,l,n}=\frac{f_n}{c}d\sin \left({\mu }_{i, l}\right),$$

(73)

where $d=\frac{c}{2f_c}=\frac{\lambda }{2}$ is the antenna spacing; $\lambda$ is the wavelength at the central frequency $f_c$, which cannot be changed after the antenna fabrication; and $c$ represents the speed of light. ${\omega }_{i, l}\in \left[0, 2\pi \right)$ and ${\mu }_{i, l}\in \left[0, 2\pi \right)$ denote the angle of departure and the angle of arrival at the source and destination, respectively, and $f_n=f_c+\frac{f_s}{N_{sub}}\left(n-1-\frac{N_{sub}-1}{2}\right)$, where $f_s$ shows the bandwidth. The simulation parameters for describing the spatial wideband effect on the performance of the proposed technique are given in

Table 6. Simulation parameters for spatial wideband effect.

Parameters	Values
Number of data streams	$N_s=3, 4$
Number of SUs	$K=4$
Number of sub-carriers	$N_{sub}=64$
Number of antennas at the source, relay node, and each SU	$N_t=N_r=36, 64$$,N_{d_k}=9, 16$
Number of clusters; number of rays/cluster	$N_{cl}=4, N_{ray}=5$
Maximum time delay	${\tau }_{max}=20 \mathrm{ns}$
Central frequency	$f_c=30 \mathrm{GHz}$
Bandwidth	$f_s=3 \mathrm{GHz}$

.

Figure 19 demonstrates the impact of the spatial wideband effect on the sum spectral efficiency of the proposed technique, where the number of antennas $N_t=N_r=36$, $N_{d_k}=9$ and the number of transmitted data streams $N_s=3$ are taken into consideration. It is assumed that the system is serving four SUs, i.e., $K=4$. The optimal unconstrained fully digital beamforming defines the upper bound for comparing the performance of the presented approach. Numerical results illustrate that the proposed algorithm can achieve more than 91% of the optimal sum rate with a fully connected structure. Also, the suggested mixed hybrid structure can obtain approximately 86% of the optimal spectral efficiency.

Figure 20 compares the performance of the proposed algorithm with the corresponding full-complexity precoding by increasing the number of antennas at communicating nodes. The number of antennas, the number of SUs, and the number of data streams are given as $N_t=N_r=64$, $N_{d_k}=16$, $K=4$, and $N_s=4$, respectively. In this case, the fully connected structure shows more than 85% of the optimal sum rate, while the mixed structure achieves nearly 79% of the optimal spectral efficiency. Computer simulations demonstrate that performance decreases by increasing the number of antennas compared to the previous results, which is consistent with the theoretical analysis. In conclusion, the proposed scheme shows relatively lower performance under spatial wideband effects.

5.8. Impact of Uniform Circular Array (UCA)

Before describing the impact of the uniform circular array (UCA) on the performance of the proposed method, it is worth mentioning that wideband communication with UCA leads to a beam defocus effect, in contrast to the spatial wideband effect [78]. This results in a considerable performance loss if measures are not taken to tackle this challenge. To generate simulation results, the frequency domain wireless channel based on the Saleh–Valenzuela model [78] is given as follows:

$${\mathit{H}}_{UC\mathit{A}}\left[m\right]=\eta {\displaystyle \sum }_{l=1}^L{g}_l{\mathit{a}}_t^m\left({\psi }_l\right) {\mathit{a}}_r^m{\left({\phi }_l\right)}^H e^{-j2\pi {\tau }_l{f}_m},$$

(74)

where ${\mathit{a}}_t^m(.)$ and ${\mathit{a}}_t^m(.)$ are the array response vectors at the source and destination, respectively. The complex gain and delay of the $l$-th propagation path are denoted by $g_l$ and ${\tau }_l$, respectively, while $\eta =\sqrt{N/L}$ stands for the normalization factor, where $N$ shows the number of antenna elements in UCA. The angle of departure and the angle of arrival corresponding to the $l$-th path is represented as ${\psi }_l$ and ${\phi }_l$, respectively. The beam steering vector considering the physical direction $\psi$ can be expressed as follows:

$${\mathit{a}}_t^m\left(\psi \right)=\frac{1}{\sqrt{N}}{\left[e^{j{\rho }_m\cos \left(\psi -{\Phi }_0\right)},\dots ,e^{j{\rho }_m\cos \left(\psi -{\Phi }_{N-1}\right)}\right]}^T,$$

(75)

where ${\rho }_m=\frac{2\pi R{f}_m}{c}$ for $m=1,\dots ,M$ and ${\Phi }_n=\frac{2\pi n}{N}$ for $n=1,\dots ,N-1$. To satisfy the strict requirements of the array deployment, a uniform linear array (ULA) is used for the destination. Hence, the array response vector at the secondary user is modeled as follows:

$${\mathit{a}}_r^m\left(\phi \right)=\frac{1}{\sqrt{N}}{\left[1,e^{j\frac{2\pi d{f}_m}{c}\sin \left(\phi \right)},\dots ,e^{j\frac{2\pi \left(N-1\right)d{f}_m}{c}\sin \left(\phi \right)}\right]}^T.$$

(76)

The simulation parameters for visualizing the impact of UCA on the performance of the proposed technique are given in

Table 7. Simulation parameters for the impact of UCA.

Parameters	Values
Number of data streams	$N_s=8, 16$
Number of SUs	$K=4$
Number of sub-carriers	$N_{sub}=64$
Number of antennas at the source, relay node, and each SU	$N_t=N_r=64, 256$, $N_{d_k}=16, 64$
Number of propagation paths	$L=20$
Maximum time delay	${\tau }_{max}=25 \mathrm{ns}$
Central frequency	$f_c=30 \mathrm{GHz}$
Bandwidth	$f_s=3 \mathrm{GHz}$

(shown on the next page).

Figure 21 illustrates the spectral efficiency performance of the proposed approach, considering UCA at the source and relay nodes, while a uniform linear array (ULA) is deployed at each SU. The optimal fully digital beamforming is considered as the baseline for comparison. The number of antennas and the number of data streams are set as follows: $N_t=N_r=64$, $N_{d_k}=16$, $K=4$, and $N_s=8$. It is evident from the obtained results that the fully connected structure obtains approximately 83% of the optimal value, while the mixed architecture achieves nearly 76% of the baseline.

Figure 22 visualizes the impact on the performance of the proposed algorithm by increasing the number of antennas and the number of transmitted data streams. To generate simulation results, the various system parameters are given as $N_t=N_r=256$, $N_{d_k}=64$, $K=4$, and $N_s=16$. Numerical results show that the proposed method achieves approximately the same performance as obtained with a small number of antennas and data streams.

Note: It is worth noting that the proposed technique is different from the previous work [70], where a fully connected hybrid structure was considered at communicating nodes. The primary focus of this work is to derive an energy-efficient hybrid beamforming solution for CR-based relay-assisted MU-MIMO systems under frequency-selective channels. To achieve this goal, a combination of a fully connected architecture and a partially connected structure is employed at the relay node, while a partially connected structure is deployed at each SU. This structure is of great practical importance to achieve an energy-efficient hybrid beamforming solution while maximizing the achievable sum rate. All simulation results focus on comparing the performance of the proposed mixed hybrid architecture with the corresponding fully connected structure. Additionally, a sub-section is included in the simulation results that describes the energy consumption of the fully connected architecture, mixed structure, and fully digital precoding. Numerical results are also provided to show the energy efficiency of mixed structures in comparison to fully connected hybrid precoding and full-complexity digital beamforming. Finally, the spatial wideband effect and the impact of UCA on the performance of the presented technique are also included to show the effectiveness of the proposed approach through simulation results.

6. Conclusions

This paper investigates a hybrid wideband mm-wave transceiver for maximizing the sum rate in relay-assisted MU-MIMO CRNs. The underlying network architecture is of great practical importance in enhancing the efficiency of spectrum utilization. In addition, the overall performance can also be improved due to the cooperative communication paradigm. To achieve an energy-efficient solution for hybrid precoding, a combination of partially connected and fully connected structures is taken into consideration. A comparative study is also conducted between fully connected and mixed structures through computer simulations, which describes the better energy efficiency performance of the latter architecture compared to the former one. To reduce the complexity of the original optimization problem, an endeavor is made to decompose it into sub-problems. The focus of the proposed algorithm is to maximize the sum rate of SUs by keeping interference experienced by the PU within an acceptable limit. This task can be achieved by following a decoupled approach to address each sub-problem, where the RF and baseband processing components are derived separately. Moreover, the sum rate is maximized through the analog beamformers at different communicating nodes, and interference is minimized via frequency-dependent digital precoders and combiners. It is evident from the obtained results that the proposed method achieves performance close to that of its fully connected counterpart when different system parameters are changed over a wide range. A small performance degradation occurs in a gradual manner by increasing the channel estimation error in discrete steps. Simulation results also demonstrate that the mixed hybrid structure shows better energy efficiency when compared with its fully connected and full-complexity counterparts. This aspect of the suggested design is of prime significance. For promising future research directions: (1) The proposed design approach can be extended to multi-relay MIMO networks with other practical constraints due to the CR technology. These constraints mainly include interference from primary networks and power allocation to achieve different goals, such as minimizing the asymptotic outage probability, maximizing end-to-end throughput, and improving transmission performance. (2) It is also of great interest to investigate the performance of the proposed scheme by replacing the relay node with reconfigurable intelligent surfaces (RISs). They play an essential role in finding energy-efficient solutions with low complexity. Additionally, the deployment of a large number of RISs can achieve the same performance as relay networks.