Sensing and Deep CNN-Assisted Semi-Blind Detection for Multi-User Massive MIMO Communications

Abstract

Attaining precise target detection and channel measurements are critical for guiding beamforming optimization and data demodulation in massive multiple-input multiple-output (MIMO) communication systems with hybrid structures, which requires large pilot overhead as well as substantial computational complexity. With benefits from the powerful detection characteristics of MIMO radar, we aim for designing a novel sensing-assisted semi-blind detection scheme in this paper, where both the inherent low-rankness of signal matrix and the essential knowledge about geometric environments are fully exploited under a designated cooperative manner. Specifically, to efficiently recover the channel factorizations via the formulated low-rank matrix completion problem, a low-complexity iterative algorithm stemming from the alternating steepest descent (ASD) method is adopted to obtain the solutions in case of unknown noise statistics. Moreover, we take one step forward by employing the denoising convolutional neural network (DnCNN) to preprocess the received signals due to its favorable performance of handling Gaussian denoising. The overall paradigm of our proposed scheme consists of three stages, namely (1) target parameter sensing, (2) communication signal denoising and (3) semi-blind detection refinement. Simulation results show that significant estimation gains can be achieved by the proposed scheme with reduced training overhead in a variety of system settings.

1. Introduction

With the vigorous development of the wireless communication industry and the deep integration with intelligent information processing technology, various remote sensing applications, e.g., vehicle-to-everything, smart manufacturing and environmental monitoring, come to the force, which puts forward increasing demand for network capacity and spectrum utilization [1,2,3]. As an important solution to support ultra-high transmission rate and intelligence requirements of the future mobile communications, massive multiple-input multiple-output (MIMO) inherently allows for simultaneous serving of multiple users by deploying large-scale antennas at a base station (BS) and facilitates the generation of directed beams towards target users, thereby expanding connected density and reducing the mutual interference in scenarios involving multiple users [4,5]. To obtain larger antenna gains and alleviate the pressure of spectrum scarcity, exploring novel transceiver design and frame structure operating at higher frequencies, such as the millimeter wave (mmWave) band [6,7], paves the way to guarantee reliable communications while enabling satisfactory user experiences.

1.1. Background

Considering that major obstacles preventing the deployment of massive MIMO are related to huge hardware costs and power consumptions imposed on the excessive number of radio frequency (RF) chains, hybrid analog/digital structures have been utilized to canonically route the RF chains via a well-designed phase-shifter network [8,9,10]. Furthermore, to compensate for prominent path loss encountered in mmWave channels and mitigate severe interference among multiple users, optimization on hybrid beamforming is expected to strike a favorable tradeoff between cost and rate [11,12,13]. One of the critical premises for obtaining the optimal beamforming is to acquire channel state information. Nevertheless, accurate channel estimation is challenging due to increased dimensionality of signal matrix and the application of hybrid architecture that limits the available observations. In this way, developing an efficient receiver scheme to enhance the transmission performance is essential in hybrid massive MIMO systems.

Recently, based on the fact that radar and communication systems share similar underline signal processing and gradually work in approximate frequency bands, there emerges a technological trend to integrate both functionalities into a single system [14,15,16], which offers an exciting opportunity to implement radar sensing by inherent wireless infrastructures. Given the potential ability of rapid and wide-range detection, it would be beneficial to leverage the radar sensing to fully extract geometric information from the surrounding environment [17,18] and thus assist in obtaining more precise parameter estimations. Toward this end, the coexistence of radar and communication components will constitute the primary architecture of this paper for enhanced performance gains.

1.2. Problem Being Addressed

Against the above background, as the dimensionality of antennas and carrier frequencies proliferate, the task of effectively balancing detection performance with computational cost becomes increasingly arduous. Consequently, the critical problem to be addressed is to propose novel solutions for comprehensive extraction of valuable information from the received signals in hybrid massive MIMO communication systems, thereby contributing to more reliable and cost-effective channel estimators.

1.3. Existing Solutions

In the literature, various channel estimators have been conducted for guiding the follow-up decoding and beamforming operations in hybrid massive MIMO systems. Among them, many techniques refer to the non-blind detections, which perform explicit channel estimation basically by transmitting and dealing with pilot symbols [19,20,21]. However, as the number of antennas increases, the pilot overhead could be enlarged accordingly and makes a negative impact on spectral efficiency. To address such issue, several research works began to focus on leveraging the sparse representation exhibited in delay-domain or angle-domain of mmWave channels [22,23,24], based on which a number of compressed-sensing (CS) tools have been adopted to figure out the formulated sparse recovery problem. Considering the grid-based CS estimator may lead to effects of basis mismatch, another gridless method appeared to construct continuous parameter domain by employing the atomic norm minimization [25]. Moreover, in order to avoid substantial computational complexity caused by high-dimensional optimizations and exploit more information from the payload data, our prior work [26] has proposed a semi-blind detection scheme for a hybrid massive MIMO system over frequency-selective fading channels, which utilized the uplink data as virtual pilots to improve the accuracy of channel estimation and data detection. However, the existing research has not sufficiently capitalized on the intrinsic features of massive MIMO systems and realistic channel characteristics, which constitute a significant challenge for achieving better tradeoff between training overhead and detection performance.

In addition to searching for rich scattering assumptions at higher frequencies, the low-rankness of massive MIMO systems has also been widely considered. In [27], according to the fact that both the number of BS antennas and the number of symbols in one coherence block are typically larger than the number of active users, the joint channel and data estimation was modeled as a low-rank matrix completion problem, for which two iterative algorithms were put forward to effectively recover the original channel and data. Moreover, schemes exploiting both the low-rank property and the knowledge of array responses have been proposed in [28,29,30], where the initial channel estimates provided by solutions to an inductive matrix completion would be further refined through multiple stages. Without loss of generality, most of the existing methods need to recover the matrix factorizations following an alternating minimization procedure [31,32,33]. Adhering to the existing methodologies, the intricate matrix multiplication or matrix inversion would induce unacceptable computational costs, especially when generalized to large-scale problems. As an effort to relieve the computational burden, an alternating steepest descent (ASD) algorithm was introduced in [34] to replace the least squares (LS) subproblem solutions with exact line-search updates. In view of the simplicity and moderate accuracy in the presence of non-negligible noise, we are inspired to tailor the ASD algorithm for estimating channels in hybrid massive MIMO systems.

Instead of simply obeying stochastic distributions, more realistic channels consisting of multiple propagation paths usually display a certain geometric structure. Since the process of communication channel estimation is similar to the radar target detection to some extent, research on making full use of sensing parameters to support communication functionality has attracted growing attention [35,36,37]. For example, by combining both functionalities of phased-MIMO radar and hybrid communication in the mmWave band, a novel strategy for joint target detection and channel estimation was proposed in [35]. To reduce the spectral resources as well as hardware cost, the employment of radar sensing for assisting the prediction of motion parameters in vehicular communication networks was investigated in [36]. Moreover, a robust MIMO-radar-aided channel estimation scheme deployed in multi-user (MU) MIMO communication systems has proven to be advantageous in improving the estimation accuracy with fewer training overhead [37]. Given that channel estimation may suffer from severe performance degradation in low signal-to-noise ratio (SNR) regions due to conceivable noise enhancement along with the LS-based estimation process, a supervised deep-learning (DL)-based signal denoiser was also involved in [37] to eliminate the noise on received signals before estimating the channel gains. However, training a neural network with high-dimensional signals and large datasets seems not tolerable in terms of complexity and latency [38]. Therefore, developing a cost-effective signal denoising module to make the state-of-the-art intelligent algorithms practically implementable for massive MIMO channel estimation is worth exploring.

1.4. Motivations and Objectives

A summary of the existing literature is provided in

Table 1. Existing works comparison on receiver techniques.

Existing Works	System Models	Receiver Techniques	Limitations or Features
[19,20,21]	mmWave MIMO communications	non-blind detection based on pilots only	Pilot overhead can be enlarged greatly with massive antennas.
[22,23,24]	mmWave MIMO communications	CS leveraging the channel sparsity	Gird-based estimator may lead to effects of basis mismatch.
[25]	mmWave MIMO communications	atomic norm minimization	High-dimensional optimizations may cause substantial computational complexity.
[26]	mmWave MIMO communications	semi-blind detection aided with payload data	Neglect of geometric structure of channel distributions.
[27]	massive MIMO communications	LS leveraging the channel low-rankness	Underexploitation of realistic channel characteristic with multiple propagation paths.
[28,29,30]	mmWave MIMO communications	multiple stages exploiting both the low-rankness and sparsity	High computational complexity when generalized to large-scale problems.
[35]	joint radar and communications	multiple signal classification (MUSIC) for angle estimation	Joint signal processing strategy can simultaneously detect targets while estimating the communication channel.
[36]	joint radar and communications	maximum likelihood (ML) estimator	Sensing parameter estimation can promote the dynamic topology construction of surrounding environments.
[37]	joint radar and communications	LS detection vs. DL-based denoiser	Efficient estimation with fewer training resources by eliminating noise before recovering channels.

. The motivation for this paper stems from the notable limitations in current receiver techniques for hybrid massive MIMO communication systems. As one can see, challenges of large pilot overhead and substantial computational complexity will arise due to increased dimensionality and complicated signal nature in hybrid architectures, which restrict observation capabilities. Additionally, the growing demand for employing radar sensing in extracting environment information motivates us to seek advisable coordination between sensing and communication modules. In light of these challenges, the objective of this paper is to develop an efficient detection scheme tailored for hybrid massive MIMO systems with the aid of radar sensing, aimed at enhancing estimation performance and alleviating computational burdens.

1.5. Our Proposal and Contributions

In this paper, we propose a sensing and deep CNN-assisted semi-blind detection refinement scheme for a hybrid massive MU-MIMO system, where different functional modules, i.e., radar and communication, are integrated into the BS by partitioning multiple antennas. Specifically, both the low-rankness and geometric characteristic of high-frequency channels are fully exploited in a progressive manner. To reduce the large training overhead and substantial computational complexity, a novel transmission frame and an efficient receiver scheme are presented that can be split into three stages, namely (1) target parameter sensing, (2) communication signal denoising and (3) semi-blind detection refinement. Below, we crisply summarize the main contributions in this paper:

• A novel time-division duplex (TDD) transmission frame capable of coordinating the radar and communication operations is designed, based on which the root multiple signal classification (MUSIC) algorithm is firstly applied to environment sensing and then the extracted target angle information is utilized for refining the subsequent communication detection results.
• A generic representation for analog combining with phase shifters is considered, and the signal recovery problem is transformed into a low-rank matrix completion. To obtain the matrix factorizations with lower complexity, an iterative algorithm modified from ASD is proposed without any prior knowledge of noise statistics. In addition, different from the conventional pilot-only method, the semi-blind detection scheme is employed with reduced training overhead.
• A pre-trained denoising convolutional neural network (DnCNN) is adopted to preprocess the received signals before performing the semi-blind detection, which attempts to handle Gaussian noise removal with unknown noise level and shows powerful ability of improved accuracy especially in low SNR regions.

The remainder of this paper is organized as follows. In Section 2, the system model is presented and the conventional pilot-only estimation method is reviewed. By transforming the signal recovery problem into a low-rank matrix completion, our proposed sensing and deep CNN-assisted semi-blind detection scheme is detailed with three stages in Section 3. Simulation results are provided in Section 4. Finally, conclusions are given in Section 5.

Notations: The list of acronyms used in this paper is summarized in

Table 2. List of acronyms.

Abbreviations	Expansion
ALS	Alternating Least Squares
AoA	Angle-of-Arrival
AoD	Angle-of-Departure
BS	Base Station
CNN	Convolutional Neural Network
CS	Compressed Sensing
DnCNN	Denoising Convolutional Neural Network
DL	Deep Learning
LMMSE	Linear Minimum Mean Squared Error
LoS	Line-of-Sight
LS	Least Squares
mmWave	Millimeter Wave
MIMO	Multiple-Input Multiple-Output
MU-MIMO	Multi-User MIMO
MUSIC	Multiple Signal Classification
NMSE	Normalized Mean Squared Error
RF	Radio Frequency
RMSE	Root Mean Squared Error
SNR	Signal-to-Noise Ratio
SVD	Singular Value Decomposition
TDD	Time-Division Duplex
UE	User Equipment
ULA	Uniform Linear Array

. Unless otherwise specified, bold uppercase letters are used to represent matrices, bold lowercase letters are used to represent vectors and scalars are denoted by normal font. $\mathbf{A}(i,:)$ and $\mathbf{A}(:,j)$ denote the i-th row and j-th column of $\mathbf{A}$, respectively. $A_{i,j}$ denotes the $(i,j)$-th entry of $\mathbf{A}$. ${\mathbf{A}}^T$, ${\mathbf{A}}^H$ and ${\mathbf{A}}^{*}$ denote the operations of transpose, Hermitian transpose and element-wise conjugate, respectively. $\mathrm{diag}\left(\mathbf{a}\right)$ stands for a diagonal matrix with diagonal elements given by the vector $\mathbf{a}$. $\mathbb{E}$ and $\mathrm{vec}(·)$ represent the expectation operation and vectorization operation, respectively.

2. System Model and Background

In this paper, we consider a sensing-assisted hybrid massive MU-MIMO communication system, as shown in Figure 1, where K single-antenna users are served by a BS with two modules: one for the uplink communication and the other for the target sensing. Similarly as introduced in [37], the system operates in TDD mode and the multi-functional transmission frame structure is designed as shown in Figure 2, which consists of two parts. In the former part, the sensing module transmits probing signals and receives the echoes to detect the target directions. In the latter part, the communication module receives the pilots and payload data to complete the channel estimation and recover the uplink signals.

Suppose the uniform linear arrays (ULAs) are employed and are parallel with the users such that the angle-of-arrival (AoA) is identical to the angle-of-departure (AoD). Given the massive MIMO scenario and asymptotical orthogonality of steering vectors, we assume the sensing signals are emitted through line-of-sight (LoS) channels. Meanwhile, for the uplink communication, the transmitted signals from each single-antenna user will be scattered by the others around it when regarding them as point targets [36]. In addition, we consider the block-fading channel model, whose coefficients stay constant during the coherence block T.

Specifically, we first introduce the sensing model for assisting the AoA/AoDs estimation of target users. Then, the uplink communication model is presented, based on which the channel estimation problem for the hybrid massive MU-MIMO system is formulated. Finally, the conventional linear minimum mean squared error (LMMSE) estimator for solving the corresponding problem is reviewed.

2.1. Sensing Model

For the sensing module, we consider the mono-static radar case, where both the transmit and receiver arrays are equipped with $N_s$ antenna elements and we assume there is sufficient isolation between them. Let ${\mathbf{X}}_s\in {\mathbb{C}}^{N_s\times T_s}$ denote the probing signal sent by the BS, then the corresponding echoes reflected by target users can be expressed as [35]

$${\mathbf{Y}}_s=\sum _{k=1}^K{\beta }_k\mathbf{b}\left({\theta }_k\right){\mathbf{b}}^H\left({\theta }_k\right){\mathbf{X}}_s+{\mathbf{N}}_s,$$

(1)

where ${\mathbf{N}}_s\sim \mathcal{CN}(0,{\sigma }_s^2\mathbf{I})$ represents the additive white Gaussian noise plus interference, ${\beta }_k\sim \mathcal{CN}(0,{\sigma }_{{\beta }_k}^2)$ denotes the reflection coefficient of the k-th target user, ${\theta }_k$ denotes the k-th user’s azimuth angle relative to the BS and $\mathbf{b}\left({\theta }_k\right)\in {\mathbb{C}}^{N_s\times 1}$ represents the steering vector satisfying

$$\mathbf{b}\left({\theta }_k\right)=\frac{1}{\sqrt{N_s}}{[1,e^{j\frac{2\pi }{\lambda }dsin\left({\theta }_k\right)},\dots ,e^{j\frac{2\pi }{\lambda }d(N_s-1)sin\left({\theta }_k\right)}]}^T,$$

(2)

where $\lambda$ and d represent the signal wavelength and inter-element spacing, respectively. Generally, we set $d=\lambda /2$.

For notational brevity, we arrange all steering vectors into a uniform matrix $\mathbf{B}(\mathbf{\Theta })=[\mathbf{b}\left({\theta }_1\right),\dots ,\mathbf{b}\left({\theta }_K\right)]\in {\mathbb{C}}^{N_s\times K}$ with $\mathbf{\Theta }=\{{\theta }_1,\dots ,{\theta }_K\}$ and set ${\mathbf{\Lambda }}_s=\mathrm{diag}({\beta }_1,\dots ,{\beta }_K)$. Then, we rewrite the sensing signal model in (1) as a matrix form given by

$${\mathbf{Y}}_s=\mathbf{B}(\mathbf{\Theta }){\mathbf{\Lambda }}_s{\mathbf{B}}^H(\mathbf{\Theta }){\mathbf{X}}_s+{\mathbf{N}}_s.$$

(3)

2.2. Uplink Communication Model

For the uplink communication module, suppose the BS adopts a hybrid analog–digital architecture with $N_a$ antennas and $N_r (N_r<N_a)$ RF chains to combine the incoming signals as depicted in Figure 1. Let $x_k\left[t\right]$ be the t-th transmitted symbol from user k with unit average power. Then, the received signals across $N_a$ BS antennas is given by

$${\mathbf{r}}_c\left[t\right]=\sum _{k=1}^K{\mathbf{h}}_k{x}_k\left[t\right]+{\mathbf{n}}_c\left[t\right],1\le t\le T_c,$$

(4)

where ${\mathbf{n}}_c\left[t\right]\sim \mathcal{CN}(\mathbf{0},{\sigma }_c^2\mathbf{I})$ and ${\mathbf{h}}_k\in {\mathbb{C}}^{N_a\times 1}$ denotes the channel between user k and the BS.

Considering the hybrid arrays deployed, ${\mathbf{r}}_c\left[t\right]$ will further pass through the RF phase shifter network. In particular, let $\mathbf{W}\left[t\right]\in {\mathbb{C}}^{N_r\times N_a}$ denote the analog combining matrix, then the signal model combined in the RF band is consequently formulated as

$${\mathbf{y}}_c\left[t\right]=\mathbf{W}\left[t\right]\mathbf{H}{\mathbf{x}}_c\left[t\right]+\mathbf{W}\left[t\right]{\mathbf{n}}_c\left[t\right],$$

(5)

where ${\mathbf{y}}_c\left[t\right]\in {\mathbb{C}}^{N_r\times 1}$ indicates the received signal after passing through the analog structure, ${\mathbf{x}}_c\left[t\right]\in {\mathbb{C}}^{K\times 1}$ denotes the transmitted signal vector from K users at time slot t and $\mathbf{H}\triangleq [{\mathbf{h}}_1,\dots ,{\mathbf{h}}_K]\in {\mathbb{C}}^{N_a\times K}$ denotes the channel matrix between K users and the BS.

Next, we elaborate on the high-frequency channel model containing the intrinsic geometric structure. Instead of simply modeling the channels by stochastic distributions as [27], we adopt the scattering model here to describe the geometric environment over which the communication takes place. Suppose the users equipped with single-antenna are randomly distributed within communication distance, then the transmitted signal from each user will be scattered by its surrounding users. In this case, according to the widely used Saleh–Valenzuela channel model as presented in [7], the multi-path channel vector between user k and the BS can be provided as

$${\mathbf{h}}_k=\sqrt{\frac{N_a}{K}}\sum _{l=1}^K{\alpha }_{l,k}\mathbf{a}\left({\theta }_{l,k}\right),$$

(6)

where ${\alpha }_{l,k}$ and ${\theta }_{l,k}$ denote the complex scattering coefficient and the azimuth angle relative to the BS for the l-th path of user k, respectively. Specifically, the path $l=k$ contributes to the direct path from user k to the BS [36]. In addition, $\mathbf{a}\left({\theta }_{l,k}\right)\in {\mathbb{C}}^{N_a\times 1}$ denotes the steering vector expressed as

$$\mathbf{a}\left({\theta }_{l,k}\right)=\frac{1}{\sqrt{N_a}}{[1,e^{j\frac{2\pi }{\lambda }dsin\left({\theta }_{l,k}\right)},\dots ,e^{j\frac{2\pi }{\lambda }d(N_a-1)sin\left({\theta }_{l,k}\right)}]}^T.$$

(7)

By referring to (5)–(6), we arrive at the input–output relationship for uplink communication signals, which is given by

$${\mathbf{y}}_c\left[t\right]=\mathbf{W}\left[t\right]\mathbf{A}(\mathbf{\Theta })\mathbf{G}{\mathbf{x}}_c\left[t\right]+\mathbf{v}\left[t\right],$$

(8)

where $\mathbf{v}\left[t\right]\triangleq \mathbf{W}\left[t\right]{\mathbf{n}}_c\left[t\right]$ and $\mathbf{v}\left[t\right]\sim \mathcal{CN}(\mathbf{0},{\sigma }_c^2\mathbf{W}\left[t\right]\mathbf{W}{\left[t\right]}^H)$, $\mathbf{A}(\mathbf{\Theta })=[\mathbf{a}\left({\theta }_1\right),\dots ,\mathbf{a}\left({\theta }_K\right)]\in {\mathbb{C}}^{N_a\times K}$ corresponds to the steering vector matrix and $\mathbf{G}\in {\mathbb{C}}^{K\times K}$ merges the channel path gains and scattering coefficients that satisfies

$$g_{i,j}=\sqrt{\frac{N_a}{K}}{\alpha }_{l,k},\forall i=l,j=k,i,j\le K.$$

(9)

According to the communication signal model in (8), even with sufficient resolution for target sensing parameters, i.e., the geometric pattern of the channel matrix $\mathbf{A}(\mathbf{\Theta })$ is known to the BS, the stochastic channel gains of different paths are still unable to infer. Therefore, a certain amount of pilots is required for completing the channel estimation.

2.3. The Conventional LMMSE Estimator

The conventional pilot-only estimation methods may cause large training overhead, especially with massive MIMO scenarios. To be specific, let ${\mathbf{X}}_p\triangleq [{\mathbf{x}}_c\left[1\right],\dots ,{\mathbf{x}}_c\left[T_p\right]]\in {\mathbb{C}}^{K\times T_p}$ denote the transmitted pilots, where $T_p$ indicates the number of pilots. Then, the signal model of (8) can be transformed as [27]

$${\mathbf{y}}_c\left[t\right]=\left({\mathbf{x}}_p^T\left[t\right]\otimes \mathbf{W}\left[t\right]\right)\left({\mathbf{I}}_K\otimes \mathbf{A}(\mathbf{\Theta })\right)\mathbf{g}+\mathbf{v}\left[t\right],$$

(10)

where $\mathbf{g}\triangleq \mathrm{vec}\left(\mathbf{G}\right)\in {\mathbb{C}}^{KK\times 1}$ and ⊗ denotes the Kronecker product.

In terms of whether the azimuth angles information is known or unknown to the BS, we define the channel counterparts to be estimated as $\mathbf{g}$ and ${\mathbf{h}}_u\triangleq \left({\mathbf{I}}_K\otimes \mathbf{A}(\mathbf{\Theta })\right)\mathbf{g}$, respectively. By further defining ${\mathbf{y}}_p\triangleq {[{\mathbf{y}}_c^T\left[1\right],\dots ,{\mathbf{y}}_c^T\left[T_p\right]]}^T\in {\mathbb{C}}^{N_r{T}_p\times 1}$, ${\mathbf{v}}_p\triangleq {[{\mathbf{v}}^T\left[1\right],\dots ,{\mathbf{v}}^T\left[T_p\right]]}^T\in {\mathbb{C}}^{N_r{T}_p\times 1}$ and ${\mathbf{W}}_p\triangleq [{\mathbf{x}}_p^T\left[1\right]\otimes \mathbf{W}\left[1\right];\dots ;{\mathbf{x}}_p^T\left[T_p\right]\otimes \mathbf{W}\left[T_p\right]]\in {\mathbb{C}}^{N_r{T}_p\times N_{a}K}$, the LMMSE estimates of the uplink channels can be given by [39]

$${\widehat{\mathbf{h}}}_u={({\mathbf{W}}_p^H{\mathbf{W}}_p+{\sigma }_c^2\mathbf{I})}^{-1}{\mathbf{W}}_p^H{\mathbf{y}}_p,$$

(11)

and

$$\widehat{\mathbf{g}}={\left({\mathbf{A}}_{\mathbf{I}}^H(\mathbf{\Theta }){\mathbf{W}}_p^H{\mathbf{W}}_p{\mathbf{A}}_{\mathbf{I}}(\mathbf{\Theta })+{\sigma }_c^2\mathbf{I}\right)}^{-1}{\mathbf{A}}_{\mathbf{I}}^H(\mathbf{\Theta }){\mathbf{W}}_p^H{\mathbf{y}}_p,$$

(12)

for which ${\mathbf{A}}_{\mathbf{I}}(\mathbf{\Theta })=\mathbf{I}\otimes \mathbf{A}(\mathbf{\Theta })$ and we suppose $\mathbf{W}\left[t\right]\mathbf{W}{\left[t\right]}^H=\mathbf{I}$ according to popular choices of analog combining matrix.

Upon determining the estimated channels, let ${\mathbf{X}}_d\triangleq [{\mathbf{x}}_c[T_p+1],\dots ,{\mathbf{x}}_c\left[T_c\right]]\in {\mathbb{C}}^{K\times T_d}$ with $T_d=T_c-T_p$ indicating the length of payload data, then the LMMSE-based data detection can be performed by [39]

$${\widehat{\mathbf{x}}}_d\left[t\right]={\left({\left(\mathbf{W}\left[t\right]\widehat{\mathbf{H}}\right)}^H\mathbf{W}\left[t\right]\widehat{\mathbf{H}}+{\sigma }_c^2\mathbf{I}\right)}^{-1}{\left(\mathbf{W}\left[t\right]\widehat{\mathbf{H}}\right)}^H{\mathbf{y}}_c\left[t\right],$$

(13)

where $\widehat{\mathbf{H}}={\mathrm{vec}}^{-1}\left({\widehat{\mathbf{h}}}_u\right)$ or $\widehat{\mathbf{H}}={\mathrm{vec}}^{-1}\left({\mathbf{A}}_{\mathbf{I}}(\mathbf{\Theta })\widehat{\mathbf{g}}\right)$ depends on whether the azimuth angles of target users are already known to the BS.

From the above discussions, we can conclude that for the conventional pilot-only estimator, the number of transmitted pilots and the variance of noise have significant influences on the accuracy of channel estimation. To be specific, for the most common case without knowledge of geometric information [27], since there are involved $N_a\times K$ unknowns in $\widehat{\mathbf{H}}$, the number of required pilots needs to scale with $T_p\propto N_{a}K/N_r$, which may occupy a large portion of transmission frame given the massive receive antennas. For the case where the channel estimation is conducted after resolving the AoA/AoDs [37], although the number of required pilots can be reduced to $T_p\propto KK/N_r$, the accuracy of the LMMSE estimator will be greatly affected by non-negligible noise even with super-resolution of angle detections [40]. Therefore, to reduce the large training overhead while maintaining acceptable estimation accuracy, we will propose a sensing and deep denoising-assisted semi-blind detection scheme to improve the performance achieved by the existing methods.

3. Proposed Sensing and Deep CNN-Assisted Semi-Blind Detection Scheme

In this section, we introduce the proposed scheme to recover the signals formulated in Section 2. Based on the transmission frame structure designed in Figure 2, the proposed estimator will successively address the AoAs estimation, low-rank matrix completion and semi-blind detection refinement to exploit both the low-rank property and the geometric knowledge of channels in hybrid massive MIMO systems. In particular, by integrating the low-complexity spectrum estimation algorithm and high-efficiency matrix completion algorithm, MIMO radar’s rapid detection characteristics can be fully utilized to assist in obtaining more accurate channel estimation as well as taking up less training overhead.

3.1. Target Parameter Sensing

Considering the potential superiority of MIMO radar in achieving high-resolution angular estimations of target users, especially in identifying user positions, scattering paths and channel scenarios, the primary task of the sensing module is to detect and acquire the geometric directions of target users.

As shown in Figure 2, the angular estimation phase at the sensing module consists of two parts, during which the BS firstly sends probing signals and then receives the echo signals. Suppose the azimuth angles of K users are randomly distributed as $\mathbf{\Theta }\sim \mathrm{U}[-\frac{\pi }{2},\frac{\pi }{2}]$. To achieve the optimal estimation performance, we assume the probing signals sent by different antennas are mutually orthogonal such that the energy is evenly dispersed at each angle, i.e.,

$${\mathbf{R}}_s=\frac{1}{T_s}{\mathbf{X}}_s{\mathbf{X}}_s^H=\frac{P_s}{N_a}{\mathbf{I}}_{N_s},$$

(14)

where $P_s$ denotes the total energy of the BS. Moreover, under the assumption that the additive white Gaussian noise is unrelated to the probing signals and the averaged received power at each antenna being fixed as one, we can obtain the covariance matrix of the received echoes as  

$${\mathbf{R}}_{Y_s}=\frac{1}{T_s}{\mathbf{Y}}_s{\mathbf{Y}}_s^H=\mathbf{B}(\mathbf{\Theta }){\mathbf{R}}_{\tilde{s}}{\mathbf{B}}^H(\mathbf{\Theta })+{\sigma }_s^2{\mathbf{I}}_{N_a},$$

(15)

where

$${\mathbf{R}}_{\tilde{s}}=\frac{1}{T_s}{\mathbf{\Lambda }}_s{\mathbf{B}}^H(\mathbf{\Theta })\mathbf{B}(\mathbf{\Theta }){\mathbf{\Lambda }}_s^{*}.$$

(16)

To recover the AoAs of different target users, subspace decomposition-based methods can be invoked to separate the effective signals from background noise. Specifically, the signal subspace contains information from different directions, while the noise subspace comprises unrelated noise information. Then, following the standard subspace decomposition process [41], we present the signal and noise subspaces by taking eigenvalue decomposition of (15) as follows

$${\mathbf{R}}_{Y_s}=({\mathbf{U}}_s,{\mathbf{U}}_n){\mathbf{\Lambda }}_r{({\mathbf{U}}_s,{\mathbf{U}}_n)}^H,$$

(17)

where ${\mathbf{\Lambda }}_r\in {\mathbb{C}}^{N_s\times N_s}$ is a diagonal matrix with eigenvalues in descending order and ${\mathbf{U}}_s\in {\mathbb{C}}^{N_s\times K}$ and ${\mathbf{U}}_n\in {\mathbb{C}}^{N_s\times (N_s-K)}$ correspond to eigenvector matrices that constitute the signal and noise subspaces, respectively. It can be seen that the columns of $\mathbf{B}(\mathbf{\Theta })$ and ${\mathbf{U}}_s$ span the same subspace. Without loss of generality, given the orthogonal relationship between the signal and noise subspaces, the pseudo-spectrum used for locating the AoAs of K target users can be readily formulated as

$$P_a\left(\theta \right)=\frac{1}{|{\mathbf{b}}^H\left(\theta \right){\mathbf{U}}_n{\mathbf{U}}_n^H\mathbf{b}\left(\theta \right)|}.$$

(18)

Instead of applying the classical approach that finds the angles at peaks through tedious spectrum search, we here employ the root-MUSIC approach, which transforms the problem of spectral peak detection into polynomial root finding, thus eliminating the dependence on a predefined angular grid. As a result, the proposed approach can avoid the grid mismatch phenomenon induced by superficial quantization and identify distinct source signals even when their angular differences are minimal, which allows for super-resolution angular estimation with low complexity. Specifically, according to [30], we rewrite the denominator of (18) as

$${\mathbf{b}}^H\left(\theta \right){\mathbf{U}}_n{\mathbf{U}}_n^H\mathbf{b}\left(\theta \right)=\sum _{n=0}^{N_s-1}\sum _{m=0}^{N_s-1}{e}^{-j\xi m sin\theta }{C}_{m,n}{e}^{j\xi n sin\theta }=\sum _{l=-N_s+1}^{N_s-1}{C}_l{e}^{-j\xi l sin\theta },$$

(19)

where $\xi =\frac{2\pi d}{\lambda }$, $\mathbf{C}={\mathbf{U}}_n{\mathbf{U}}_n^H$ represents the correlation matrix with $C_l$ denoting the sum of elements along its l-th diagonal. Let $z\triangleq e^{-j\xi sin\theta }$, then we can further simplify (19) to a polynomial

$$f\left(z\right)=\sum _{l=-N_s+1}^{N_s-1}{C}_l{z}^l.$$

(20)

Then, the $2(N_s-1)$ roots of $f\left(z\right)$ exactly correspond to the poles of the MUSIC spectrum given by

$$z_i=\left|z_i\right|e^{j\arg \left(z_i\right)},i=1,\dots ,2(N_s-1),$$

(21)

K roots out of which lying closest to the unit circle are chosen to yield the AoAs as

$${\widehat{\theta }}_i={sin}^{-1}(-\frac{\arg \left(z_i\right)}{\xi }),i=1,\dots ,K,$$

(22)

where $\arg \left(z_i\right)$ denotes the phase angle of $z_i$.

3.2. Low-Rank Matrix Completion

When the sensing module obtains the knowledge of all the AoAs of target users, the angle information will be transferred to the communication module, which is devoted to identifying the coefficients of different scattering paths by using the uplink pilot and payload data. For the sake of saving the training overhead while making full use of available information at hand, we consider to explore the low-rankness of the massive MIMO system and then recover the uplink channels based on low-rank matrix completion.

Before constructing the signal recovery problem, we show a generic model of the analog combining matrix and discuss how the transmitted signals from K users are related to the received signals at the BS. Specifically, given the fully-connected hybrid structure as presented in Figure 1 such that each RF chain is connected to all antennas, various realizations of the phase shifter with constant modulus can be modeled by $\omega e^{j\varphi }$ and then make up the combining matrix $\mathbf{W}\left[t\right]$. To this end, we can regard the generation of analog combining matrix at each time slot as randomly selecting $N_r$ rows from a DFT matrix $\mathbf{F}\in {\mathbb{C}}^{N_a\times N_a}$ with elements given by [42]

$$F_{m,n}=\frac{1}{\sqrt{N_a}}{e}^{-\frac{j2mn\pi }{N_a}},m,n=0,\dots ,N_a-1.$$

(23)

According to the signal model formulated in (5), by further defining $\tilde{\mathbf{H}}\triangleq \mathbf{F}\mathbf{H}\in {\mathbb{C}}^{N_a\times K}$ and $\mathbf{z}\left[t\right]\triangleq \tilde{\mathbf{H}}{\mathbf{x}}_c\left[t\right]\in {\mathbb{C}}^{N_a\times 1}$, we can find that the received signal ${\mathbf{y}}_c\left[t\right]$ is exactly composed of $N_r$ elements from $\mathbf{z}\left[t\right]$. Then, let $\Omega \in {\mathbb{C}}^{N_r\times T_c}$ represent the index set indicating each independent selection of $\mathbf{W}\left[t\right]$ during the communication period, and denote ${\mathbf{Y}}_c^{\Omega }\in {\mathbb{C}}^{N_a\times T_c}$ as the observation matrix with entries in $\Omega$ given by ${\left\{{\mathbf{y}}_c\left[t\right]\right\}}_{t=1}^{T_c}$ and otherwise zero, then the received signals can be recast as a matrix form

$${\mathbf{Y}}_c^{\Omega }={\mathcal{P}}_{\Omega }(\mathbf{Z}+\mathbf{N}),$$

(24)

where $\mathbf{Z}\triangleq [\mathbf{z}\left[1\right],\dots ,\mathbf{z}\left[T_c\right]]\in {\mathbb{C}}^{N_a\times T_c}$, $\mathbf{N}\sim \mathcal{CN}(\mathbf{0},{\sigma }_c^2\mathbf{I})$ and ${\mathcal{P}}_{\Omega }(·)$ represents a linear operation that preserves the entries in $\Omega$ while filling them not in $\Omega$ with zero.

Based on (24), we plausibly view ${\mathbf{Y}}_c^{\Omega }$ as an incomplete observation of the matrix $\mathbf{Z}$ corrupted by the Gaussian noise $\mathbf{N}$, where $\mathbf{Z}=\mathbf{H}{\mathbf{X}}_c$ with ${\mathbf{X}}_c\triangleq [{\mathbf{x}}_c\left[1\right],\dots ,{\mathbf{x}}_c\left[T_c\right]]\in {\mathbb{C}}^{K\times T_c}$. Noting the fact that in massive MIMO scenarios, the number of users is normally much smaller than the number of BS antennas and the length of coherence block, i.e., $N_a,T_c\gg K$, the matrix $\mathbf{Z}$ displays an inherent low-rank property, i.e., $\mathrm{rank}\left[\mathbf{Z}\right]\le K\ll min\{N_a,T_c\}$. Moreover, since multiplying by unitary matrix does not change the distributions, $\tilde{\mathbf{H}}$ and $\mathbf{H}$ are interchangeable for the channel estimation. Consequently, as derived in [27], the channel and data estimation is formulated as the following low-rank matrix completion problem

$$\underset{\mathbf{U}\in {\mathbb{C}}^{N_a\times K},\mathbf{V}\in {\mathbb{C}}^{K\times T_c}}{min}\mathrm{rank}\left[\mathbf{U}\mathbf{V}\right],s.t.{\mathbf{Y}}_c^{\Omega }={\mathcal{P}}_{\Omega }(\mathbf{U}\mathbf{V}+\mathbf{N}),$$

(25)

where $\mathbf{U}$ and $\mathbf{V}$ are related to the solutions of channel and data estimates $\widehat{\mathbf{H}}$ and ${\widehat{\mathbf{X}}}_c$, respectively.

Different from the LMMSE estimator, which requires channel and noise second order statistics, we here propose to employ a low-complexity algorithm invoked in [34] to solve (25) without any prior information and exclusively based on matrix-wise updates. In this vein, considering the original rank-minimization problem is non-convex and generally NP-hard, an approximate relaxation scheme such as Frobenious norm minimization comes to the scene, which suggests to recover the matrix factorizations from

$$({\mathbf{U}}^{*},{\mathbf{V}}^{*})=\underset{\begin{array}{c}\mathbf{U}\in {\mathbb{C}}^{N_a\times K}\\ \mathbf{V}\in {\mathbb{C}}^{K\times T_c}\end{array}}{argmin}\frac{1}{2}{∥{\mathbf{Y}}_c^{\Omega }-{\mathcal{P}}_{\Omega }\left(\mathbf{U}\mathbf{V}\right)∥}_F^2.$$

(26)

To solve the above optimization problem, alternative minimization techniques, i.e., successively finding out $\mathbf{U}$ and $\mathbf{V}$ with the minimized residual, have been widely used due to their simplicity and flexibility. As an effort to circumvent the computational burden, a better choice called ASD algorithm incorporates a simple line-search to update solutions instead of dealing with the least squares subproblems. To be specific, let $\nabla f_{\mathbf{V}}\left(\mathbf{U}\right)$ and $\nabla f_{\mathbf{U}}\left(\mathbf{V}\right)$ define the gradient of the function $f(\mathbf{U},\mathbf{V})=\left|\right|{\mathbf{Y}}_c^{\Omega }-{\mathcal{P}}_{\Omega }\left(\mathbf{U}\mathbf{V}\right){\left|\right|}_F^2$ with respect to $\mathbf{U}$ and $\mathbf{V}$, respectively, the stepsizes for updating solutions along the steepest gradient descent direction can be explicitly computed as [34]

$$t_{\mathbf{U}}=\frac{\left|\right|\nabla f_{\mathbf{V}}\left(\mathbf{U}\right){\left|\right|}_F^2}{\left|\right|{\mathcal{P}}_{\Omega }(\nabla f_{\mathbf{V}}\left(\mathbf{U}\right)\mathbf{V}){\left|\right|}_F^2},$$

(27)

and

$$t_{\mathbf{V}}=\frac{\left|\right|\nabla f_{\mathbf{U}}\left(\mathbf{V}\right){\left|\right|}_F^2}{\left|\right|{\mathcal{P}}_{\Omega }(\mathbf{U}\nabla f_{\mathbf{U}}\left(\mathbf{V}\right)){\left|\right|}_F^2},$$

(28)

where

$$\nabla f_{\mathbf{V}}\left(\mathbf{U}\right)=-({\mathbf{Y}}_c^{\Omega }-{\mathcal{P}}_{\Omega }\left(\mathbf{U}\mathbf{V}\right)){\mathbf{V}}^H,$$

(29)

$$\nabla f_{\mathbf{U}}\left(\mathbf{V}\right)=-{\mathbf{U}}^H({\mathbf{Y}}_c^{\Omega }-{\mathcal{P}}_{\Omega }\left(\mathbf{U}\mathbf{V}\right)).$$

(30)

For initializations, the intuition behind (8) tells that the random realization of the channel mainly depends on the distribution of $\mathbf{G}$ assuming perfect estimates of angles. In view of this, given the angle information $\widehat{\mathbf{\Theta }}$ estimated by Section 3.1, we initialize ${\mathbf{U}}_0=\mathbf{A}\left(\widehat{\mathbf{\Theta }}\right){\mathbf{G}}_0\in {\mathbb{C}}^{N_a\times K}$ and ${\mathbf{V}}_0\in {\mathbb{C}}^{K\times T_c}$ with elements of ${\mathbf{G}}_0\in {\mathbb{C}}^{K\times K}$ and ${\mathbf{V}}_0$ drawing from i.i.d. Gaussian random variables with zero mean and unit variance. Then, the key idea of our proposed algorithm is to alternatively update $\mathbf{U}$ and $\mathbf{V}$ until the termination criteria are reached. Specifically, for the $(t+1)$-th iteration, after obtaining the steepest descent stepsizes and gradient descent directions following (27)–(30), we propose to update ${\mathbf{U}}_t$ to ${\mathbf{U}}_{t+1}$ in a two-step way, i.e.,

$${\mathbf{U}}_{t+1}^{\prime }={\mathbf{U}}_t-t_{{\mathbf{U}}_t}\nabla f_{{\mathbf{V}}_t}\left({\mathbf{U}}_t\right),{\mathbf{U}}_{t+1}=\mathbf{A}\left(\widehat{\mathbf{\Theta }}\right){\mathbf{G}}_{t+1},$$

(31)

where

$${\mathbf{G}}_{t+1}={\left({\mathbf{A}}^H\left(\widehat{\mathbf{\Theta }}\right)\mathbf{A}\left(\widehat{\mathbf{\Theta }}\right)\right)}^{-1}{\mathbf{A}}^H\left(\widehat{\mathbf{\Theta }}\right){\mathbf{U}}_{t+1}^{\prime }.$$

(32)

As a similar procedure, ${\mathbf{V}}_t$ will be updated to ${\mathbf{V}}_{t+1}$ by

$${\mathbf{V}}_{t+1}={\mathbf{V}}_t-t_{{\mathbf{V}}_t}\nabla f_{{\mathbf{U}}_{t+1}}\left({\mathbf{V}}_t\right),$$

(33)

where

$$t_{{\mathbf{V}}_t}=\frac{\left|\right|\nabla f_{{\mathbf{U}}_{t+1}}\left({\mathbf{V}}_t\right){\left|\right|}_F^2}{\left|\right|{\mathcal{P}}_{\Omega }({\mathbf{U}}_{t+1}\nabla f_{{\mathbf{U}}_{t+1}}\left({\mathbf{V}}_t\right)){\left|\right|}_F^2}.$$

(34)

Note that obtaining the gradient and stepsize will involve the product of a residual matrix and a $N_a\times K$ or $K\times T_c$ matrix. However, such operation only implements once at the beginning of each iteration and then can be efficiently used to update variables. Moreover, since the above approach replaces the least square solutions with an exact line-search step and thus avoids complex operations of matrix inversions, the computational cost for each iteration is much smaller than that of alternating least squares (ALS) and the conventional singular value decomposition (SVD) methods.

3.3. Refined Semi-Blind Detection

3.3.1. Ambiguity Removal

Although the above approach is able to provide solutions to (26) directly from the received observation ${\mathbf{Y}}_c^{\Omega }$, the non-uniqueness of matrix factorization may lead to great estimation bias. More clearly, suppose the initial channel and data estimates obtained from the ASD iterations are denoted by $\widehat{\mathbf{U}}$ and $\widehat{\mathbf{V}}$, respectively. If there exists an invertible matrix $\Sigma \in {\mathbb{C}}^{K\times K}$ satisfying $\mathbf{H}{\mathbf{X}}_c=\widehat{\mathbf{U}}\Sigma {\Sigma }^{-1}\widehat{\mathbf{V}}$, then solving (26) would not guarantee a unique recovery for $\mathbf{H}$ and $\mathbf{X}$. Therefore, a certain amount of pilots are required to resolve the ambiguity.

On this basis, considering the number of elements that need to be determined is $K\times K$, we develop a pilot and data placement as ${\mathbf{X}}_c\equiv [{\mathbf{X}}_p,{\mathbf{X}}_d]$, where ${\mathbf{X}}_p\in {\mathbb{C}}^{K\times T_p}$ with $T_p\propto K$. Then, we have

$$\widehat{\mathbf{V}}=\Sigma {\mathbf{X}}_c=\Sigma [{\mathbf{X}}_p,{\mathbf{X}}_d].$$

(35)

Due to the fact that pilot sequences are usually selected to be orthogonal, i.e., ${\mathbf{X}}_p{\mathbf{X}}_p^H=T_p·{\mathbf{I}}_K$, we can obtain $\Sigma$ as

$$\Sigma =\frac{\widehat{\mathbf{V}}(:,1\sim T_p)·{\mathbf{X}}_p^H}{T_p},$$

(36)

where $\widehat{\mathbf{V}}(:,i\sim j)$ refers to the sub-matrix consisting of i-th to j-th column vectors of $\widehat{\mathbf{V}}$. Upon resolving $\Sigma$, the data and channel estimation can be accordingly given by [26]

$${\widehat{\mathbf{X}}}_d={\Sigma }^{-1}\widehat{\mathbf{V}}(:,T_p+1\sim T_c),$$

(37)

and

$$\widehat{\mathbf{H}}=\widehat{\mathbf{U}}\widehat{\mathbf{V}}(:,1\sim T_p){\mathbf{X}}_p^H/T_p.$$

(38)

The process of our proposed sensing-assisted semi-blind detection scheme is detailed in Algorithm 1. By carefully inspecting the workflow of the proposed approach, we can observe that the resulting estimation is not directly obtained from matrix factorization solutions; indispensable pilots are needed to facilitate the overall estimation as shown in Line 14, which explains the origin of the name “semi-blind detection”. The advantage of employing semi-blind detection lies in the fact that while the conventional non-blind detection implements channel estimation based on pilots only as displayed in (11)–(12), semi-blind detection can reduce the number of required pilots from an order of $N_a$ to K, naturally leading to the consideration of its application in large-scale MIMO systems.

Since the core idea of semi-blind detection hinges on full exploitation of prior information carried by both known pilots and estimated payload data, the efficiency of semi-blind detection mainly depends on the efficiency of the preceding matrix completion processes as illustrated in Line 4–13. As the scale of antennas or users increases, some computational delays may arise. Fortunately, instead of adopting the conventional LMMSE/ALS, the proposed ASD also offers advantages in terms of cost-effectiveness, which will be validated by experimental results in Section 4.

Algorithm 1 The proposed sensing-assisted semi-blind detection with reduced pilot overhead
1:	Input $\mathbf{A}\left(\widehat{\mathbf{\Theta }}\right)\in {\mathbb{C}}^{N_a\times K}$, ${\mathbf{Y}}_c^{\Omega }\in {\mathbb{C}}^{N_a\times T_c}$, ${\mathbf{X}}_p\in {\mathbb{C}}^{K\times T_p}$
2:	Randomly initialize ${\mathbf{G}}_0\in {\mathbb{C}}^{K\times K}$ and ${\mathbf{V}}_0\in {\mathbb{C}}^{K\times T_c}$.
3:	Set ${\mathbf{U}}_0=\mathbf{A}\left(\widehat{\mathbf{\Theta }}\right){\mathbf{G}}_0$ and $t=0$.
4:	repeat
5:	Step (1) Update ${\mathbf{U}}_t$ to ${\mathbf{U}}_{t+1}$ as follows
6:	Compute $\nabla f_{\mathbf{V}}\left(\mathbf{U}\right)=-({\mathbf{Y}}_c^{\Omega }-{\mathcal{P}}_{\Omega }\left(\mathbf{U}\mathbf{V}\right)){\mathbf{V}}^H$
	and $t_{\mathbf{U}}=\left|\right|\nabla f_{\mathbf{V}}{\left(\mathbf{U}\right)\left|\right|}_F^2/\left|\right|{\mathcal{P}}_{\Omega }(\nabla f_{\mathbf{V}}\left(\mathbf{U}\right)\mathbf{V}){\left|\right|}_F^2$.
7:	${\mathbf{U}}_{t+1}^{\prime }\leftarrow {\mathbf{U}}_t-t_{{\mathbf{U}}_t}\nabla f_{{\mathbf{V}}_t}\left({\mathbf{U}}_t\right)$.
8:	${\mathbf{U}}_{t+1}\leftarrow \mathbf{A}\left(\widehat{\mathbf{\Theta }}\right){\left({\mathbf{A}}^H\left(\widehat{\mathbf{\Theta }}\right)\mathbf{A}\left(\widehat{\mathbf{\Theta }}\right)\right)}^{-1}{\mathbf{A}}^H\left(\widehat{\mathbf{\Theta }}\right){\mathbf{U}}_{t+1}^{\prime }$.
9:	Step (2) Update ${\mathbf{V}}_t$ to ${\mathbf{V}}_{t+1}$ as follows
10:	Compute $\nabla f_{\mathbf{U}}\left(\mathbf{V}\right)=-{\mathbf{U}}^H({\mathbf{Y}}_c^{\Omega }-{\mathcal{P}}_{\Omega }\left(\mathbf{U}\mathbf{V}\right))$
	and $t_{\mathbf{V}}=\left|\right|\nabla f_{\mathbf{U}}{\left(\mathbf{V}\right)\left|\right|}_F^2/\left|\right|{\mathcal{P}}_{\Omega }(\mathbf{U}\nabla f_{\mathbf{U}}\left(\mathbf{V}\right)){\left|\right|}_F^2$.
11:	${\mathbf{V}}_{t+1}\leftarrow {\mathbf{V}}_t-t_{{\mathbf{V}}_t}\nabla f_{{\mathbf{U}}_{t+1}}\left({\mathbf{V}}_t\right)$.
12:	Step (3) Update t to $t+1$.
13:	until Convergence
14:	Get $\Sigma \leftarrow \widehat{\mathbf{V}}(:,1\sim T_p)·{\mathbf{X}}_p^H/T_p$.
15:	Output $\widehat{\mathbf{H}}=\widehat{\mathbf{U}}\Sigma$

3.3.2. Signal Denoising

Given that the estimation accuracy of the semi-blind method partially relies on assumptions about the signal structure, it may suffer from pilot contamination and exhibit performance degradation, especially in the presence of high noise levels. In this way, it may be beneficial to firstly recover the latent clean signal from the noisy observations before performing the semi-blind detection for enhancing the reliability of payload data estimates.

With the recent development of deep learning, approaches based on CNNs have achieved remarkable success in the task of signal denoising, among which DnCNN formulated in [43] provides the state-of-the-art performance. Due to the integration of residual learning and batch normalization in the network architecture design, DnCNN has been validated to be highly effective in removing the Gaussian-type noise, which is exactly suitable for our considered signal models, especially at high noise levels. Specifically, the residual learning makes the inputs to each layer Gaussian-like distributed and less correlated with the image content, therefore avoid oversmoothing at high noise level. Furthermore, batch normalization alleviates the internal covariate shift by incorporating a normalization step and a re-scaling step before the nonlinear layer, which makes the model less sensitive to the noise level and generalizes well to inputs with strong noise. Moreover, DnCNN is shown to be able to extend to general denoising tasks, for example, the blind Gaussian denoising, which means the capacity of signal denoising without being informed of the noise level. Based on the above reasons, we propose to apply the DnCNN as a preprocessing procedure before performing the semi-blind detection, which aims to denoise the received signal ${\mathbf{Y}}_c\in {\mathbb{C}}^{N_r\times T_c}$ and expects better estimation accuracy in noisy environments.

Specifically, we choose to use the pre-trained model provided in [43] for Gaussian denoising, which takes the noisy signals as the input and generates the residual noise as the output. By subtracting the residual from the input, we then obtain the denoised signal ${\mathbf{Y}}_c^{\prime }$. It is worth noting that the model is without fine-tuning on our dataset because our signals exhibit similar distributions with the Gaussian noise, which can be seen by comparing Figure 3d,e, making it difficult for the DnCNN model to distinguish our signal from the noise. In this way, if the model is trained using our dataset, it tends to overestimate the noise level and over-smooth the input signal. On the contrary, the pre-trained model is trained with natural images with distinct distribution from the Gaussian noise, which can be seen by comparing Figure 3d,f, making it easier to implement proper denoising strength and avoid over-smoothing the input signal. Therefore, the pre-trained model is adopted, which not only saves the effort of the time-consuming training process but also enhances the denoising accuracy.

The overall framework of our proposed sensing and deep CNN-assisted semi-blind detection scheme for hybrid massive MU-MIMO systems is summarized in Figure 4, which can be divided into three stages, i.e., target parameter sensing, communication signal denoising and semi-blind detection refinement. In the sensing stage, the BS detects and acquires the azimuth angles of target user equipments (UEs) by sending probing signals and receiving echoes. Then, the uplink communication signals transmitted by UEs are processed in the RF band and denoised by using the pre-trained DnCNN model. Finally, based on the estimated angles and denoised signals, the refined semi-blind detection will be performed at the communication module.

4. Simulation Results

Simulation results are provided in this section to demonstrate the superior performance of the proposed sensing and deep CNN-assisted semi-blind detection scheme. We consider a hybrid massive MU-MIMO system as given in Figure 1, where $K=6$ single-antenna users tend to communicate with the BS-integrated radar and communication modules. For the sensing module, the transmit and receive arrays of radar are equipped with $N_s=64$, and the length of snapshots are set to be $T_s\ge N_s$. For the communication module, the receiver is equipped with $N_a=64$ antennas and $N_r=32$ chains. Here, we assume the pilot sequences are mutually orthogonal with the pilot length satisfying $T_p\ge K$. In addition, we choose the payload data independently from the QPSK constellation with unit average power. Moreover, all the scattering and reflection coefficients are supposed to obey the standard complex Gaussian distributions.

4.1. The Proposed Transmission Frame

We firstly compare the channel estimation performance of adopting the multi-functional transmission frame as shown in Figure 2 with some existing schemes, i.e., the conventional pilot-only estimator [39] and the original semi-blind estimator relying solely on the communication module without aid of radar sensing [27]. Specifically, the channel estimation performance is evaluated by the normalized mean squared error (NMSE) defined as $\mathrm{NMSE}=\mathbb{E}\left[\right||\widehat{\mathbf{H}}-{\mathbf{H}\left|\right|}_{\mathcal{F}}^2{]/\mathbb{E}[\left|\right|\mathbf{H}\left|\right|}_{\mathcal{F}}^2]$. For fair comparison, we assume the process of solving the low-rank matrix completion in Algorithm 1 is temporarily replaced by the same ALS iteration as derived in [27], and the noise variance is approximated according to $\mathrm{SNR}=10\lg (\overline{P}/{\sigma }^2)$ with $\overline{P}$ being the average received power. The other configurations for the transmission frame are set as $T_s=100$, $T_p=2K$ and $T_d=100$. Unless otherwise stated, each simulation result is obtained through Monte-Carlo with 1000 independent channel realizations. Figure 5 depicts the NMSE performance versus SNR by using different transmission schemes. We can observe that the proposed sensing-assisted scheme shows significant performance gains especially with low SNRs compared to the alternative estimators, which thanks to that more geometric information about the scattering environments can be gleaned from the target sensing stage, while the other two schemes neglect the structural characteristics of the multi-path channel models.

To further validate the superiority of utilizing radar sensing function to yield the target angles rather than extracting the path angles from the roughly estimated channels as conducted in [30], which designed a three-stage estimator (denoted by “TSTE”) and suggested to obtain the knowledge of array responses directly from an initial low-rank channel estimate, Figure 6 illustrates the root mean squared error (RMSE) defined as $\mathrm{RMSE}=10\lg \sqrt{\mathbb{E}\left[{\sum }_{k=1}^K\right||{\widehat{\theta }}_k-{\theta }_k|{|}_2^2/K]}$ to measure the performance of AoAs resolutions of target users by using different detection schemes. Since the estimation deviation of the subspace decomposition-based method mainly depends on the dimension of received covariance matrix and some preset parameters such as the array spacing, we consider two settings of $d/\lambda$. It can be seen that the proposed sensing-assisted detection scheme achieves better resolutions, especially when the inter-element spacing is larger. Moreover, it is noteworthy that the RMSE under our proposed scheme would improve as the SNR increases, while that obtained by the TSTE estimator only fluctuates slightly versus different SNR, which is reasonable because their path angles are learned from the low-rank matrix solutions instead of straight from the received echoes, and the noise components have been removed before constructing the signal subspaces.

4.2. Refined Semi-Blind Detection

Next, we illustrate the performance gain of our proposed Algorithm 1. To show the advantage of performing semi-blind detection at the receiver rather than the LS/LMMSE-based detection [37], suppose the comparison experiment is set as performing the LMMSE channel estimation after obtaining the knowledge of target angles from the sensing module. Considering two settings of pilot length, $T_p=K$ and $T_p=1.5K$, Figure 7 plots the NMSE performance of channel estimation with different receiver schemes. From Figure 7, we observe that the NMSE performance of Algorithm 1 outperforms the LMMSE method even if the sensing module is involved, which indicates the necessity of exploiting low-rank characteristics in massive MIMO systems to improve the estimation accuracy with relatively lower pilots. In addition, we can note that the NMSE performance of Algorithm 1 improves as the length of data sequences $T_d$ increases, while the performance of LMMSE remains almost the same in all settings. The reason is that by incorporating data into the decomposition process of received signals, Algorithm 1 can take full advantage of payload data as “virtual pilots” to glean more useful information over the conventional pilot-only method.

We then characterize the performances of Algorithm 1 in terms of the computational efficiency and the estimation accuracy without any prior knowledge of channel or noise statistics. By fixing $T_p=K$ and $T_d=100$, Figure 8 depicts the convergence rates by employing different types of iterations, where “R-ALS iteration” refers to the regularized alternating least squares approach proposed by [27] for solving the formulated low-rank matrix completion problem (26). As shown in Figure 8, both the ASD iterations presented in Algorithm 1 and the R-ALS iterations will converge within limited iterations, and their convergence behaviors are similar for different SNR levels. Although the convergence rate of R-ALS iterations seems faster that that of ASD iterations, the calculations of ${\mathbf{U}}_{t+1}$ and ${\mathbf{V}}_{t+1}$ in Algorithm 1 avoid two matrix inversion operations for each iteration, which would otherwise involve additional complexity with cubic order of the matrix dimension, i.e., $\mathcal{O}\left(N_a^3\right)$. Therefore, the total computational overhead incurred by Algorithm 1 can be relatively reduced especially with high-dimensional matrices.

To further validate the computational complexity of the proposed ASD approach, we conduct more experiments compared with the conventional LMMSE and R-ALS, focusing on their execution time and memory consumption. Following the principles of controlled variable experimental methodology, and with reference to the results illustrated in Figure 7 and Figure 8, we initially quantify the averaged computational cost for all three approaches, under the conditions of $\mathrm{SNR}=2 \mathrm{dB}$ and the number of iterations set to 6. Subsequently, given that the LMMSE approach would induce a single execution at the receiver adhered to Equations (12) and (13) without convergence issues, we only record the time and memory expenditures for the proposed ASD and R-ALS iterations by establishing the termination criterion as $ϵ=\left|\right|{\widehat{\mathbf{H}}}_t-{\widehat{\mathbf{H}}}_{t-1}{\left|\right|}_{\mathcal{F}}^2/\left|\right|{\widehat{\mathbf{H}}}_t{\left|\right|}_{\mathcal{F}}^2<0.01$. The outcomes are presented in

Table 3. Comparisons on computational cost by using different approaches.

Approaches	Averaged Computational Cost		Total Cost until Convergence
performance	memory	time	memory	time
proposed ASD	0.216 Kb	3.11 ms	10.22 Kb	22.80 ms
R-ALS	8.644 Kb	43.22 ms	46.61 Kb	86.96 ms
LMMSE	0.636 Kb	3.50 ms	/	/

, from which we can observe that the proposed ASD approach exhibits lower time and memory consumption compared to its existing alternatives, thereby demonstrating the advantages in reduced complexity and higher computational efficiency.

In order to visualize the influence with unknown prior knowledge on the estimation accuracy, Figure 9 compares the NMSE performance versus different SNRs by adopting Algorithm 1 and the R-ALS-based estimator, which mimics and modifies the process in [27] given the target angles. For the former scenarios without available noise, the maximum iteration number is set to be $t_{max}=20$; while for the latter, the regularization parameters used to incorporate the prior information are set to be consistent with the noise variance. It can be seen that benefiting from taking noise effect into account, the R-ALS estimator performs better with low SNR values, which is owing to the non-negligible noise that imposes great challenges on accurate estimations in such SNR ranges. Nevertheless, the performance differences between two algorithms become narrow as the SNR increases, and Algorithm 1 would even achieve better performance under the condition of higher SNRs. By combining the results in Figure 8 and Table 3, we can conclude that our proposed Algorithm 1 is a good choice for refining semi-blind detection due to its comparable performance and lower complexity.

4.3. Signal Denoising by DnCNN

Finally, we confirm the effectiveness of signal denoising by DnCNN before performing semi-blind detection in low SNR regions. To evaluate the denoising performance of adopting different DnCNN models for our received signal dataset, we select the pre-trained DnCNN models in [43] for Gaussian denoising with specific noise levels (which are referred to as DnCNN-15 and DnCNN-25) and for blind Gaussian denoising (which is referred to as DnCNN-B) as comparisons. During the denoising phase, we successively obtain the output ${\mathbf{Y}}_{denoise}$ as the denoised data of the input ${\mathbf{Y}}_{noisy}={\mathbf{Y}}_c$. Suppose the clean signal that refers to ${\mathbf{Y}}_{noiseless}$ is related to the combined results of $\mathbf{H}{\mathbf{X}}_c$, then the following performance metrics include NMSE and NMSE-Y defined as NMSE-Y = $\mathbb{E}\left[\right||{\mathbf{Y}}_{denoise}-{\mathbf{Y}}_{noiseless}{\left|\right|}_{\mathcal{F}}^2]/\mathbb{E}[\left|\right|{\mathbf{Y}}_{noiseless}{\left|\right|}_{\mathcal{F}}^2]$, which are used to measure the accuracy of channel estimation and signal denoising, respectively.

The averaged NMSE and NMSE-Y results over 1000 samples by adopting different signal denoising models are shown in

Table 4. Performance comparison by adopting different signal denoising models.

SNR Value	−5 dB		−4 dB		−3 dB
Selected Model	NMSE-Y	NMSE	NMSE-Y	NMSE	NMSE-Y	NMSE
Without DnCNN	3.1727	1.6561	2.5430	1.2967	1.9962	0.9956
DnCNN-15	1.5223	1.0362	1.1352	0.8159	0.8320	0.6679
DnCNN-25	0.8566	0.8875	0.8618	0.8951	0.8888	0.9199
DnCNN-B	0.9424	0.9635	0.9265	0.9488	0.8888	0.9042
SNR Value	−2 dB		−1 dB		0 dB
Selected Model	NMSE-Y	NMSE	NMSE-Y	NMSE	NMSE-Y	NMSE
Without DnCNN	1.6025	0.8733	1.2425	0.7477	0.9951	0.6051
DnCNN-15	0.6335	0.6242	0.4998	0.5947	0.4312	0.5720
DnCNN-25	0.9070	0.9363	0.9314	0.9552	0.9356	0.9610
DnCNN-B	0.9350	0.8631	0.8218	0.8875	0.7700	0.8672

, where the best results for each SNR value are highlighted in bold. As one can see, all the selected DnCNN models outperform the competing method without signal denoising when SNR is below $-2$ dB due to the consideration of noise influence in low SNR regions. In contrast, as the noise level decreases, both DnCNN-25 and DnCNN-B produce better results on NMSE-Y while fail to reach higher estimation accuracy, which is intuitively reasonable because the models with strong denoising power are likely to distort the original signal structures. It is pleasant to find that the model DnCNN-15, which is trained with lower noise level, can not only yield the best denoising performance on most of the discussed SNRs but also generate better channel estimates, which indicates the feasibility of deploying the signal denoising before performing the semi-blind detection.

5. Conclusions

In this paper, we have proposed a sensing and deep CNN-assisted semi-blind detection refinement scheme for uplink hybrid massive MU-MIMO systems. The main idea is to extract the geometric structure corresponding to the communication channels via radar sensing, based on which both the transmitted pilot and payload data are contributed to yield the refined estimates. We have treated the signal recovery problem as acquiring factorization solutions to a low-rank matrix completion, for which an iterative algorithm modified from ASD is presented without any prior knowledge of noise statistics. Specifically, a pre-trained DnCNN has been adopted for signal denoising to alleviate the noise enhancement along with the LS-based estimation process, especially in low SNR regions. The overall approach has been carried out in three progressive stages, i.e., target parameter sensing, communication signal denoising and semi-blind detection refinement. Consequently, simulation results have been provided to validate the superior performance of our proposed scheme. It is seen that the utilization of co-designed multi-functional transmission frame and the refined semi-blind receiver could achieve better estimation accuracy with reduced training overhead, which facilitates the availability of sensing-assisted communications.

Despite the advancements introduced by the proposed architecture, focusing on leveraging radar sensing capabilities to enhance communication gains, there still remain several limitations requiring further investigation. Instead of relying on the coexistence and information sharing between different modules, future work could develop more powerful signal processing techniques to integrate radar and communication modules into a unified hardware platform and decongest the RF environment by using the same signal for both functionalities. In such a case, devising sophisticated signal models that harmoniously incorporate the distinctions between LoS and multi-path channels might unveil new dimensions of system performance.

Another crucial aspect that needs to be considered in future work involves the time-varying property of propagation paths under user mobility scenarios, including time-delay parameters and Doppler shift effects. This aspect necessitates the BS to pre-compensate for these effects before demodulating the received signals for most robust detection performance. Toward this end, we plan to explore the time-varying channel tracking algorithms and compensation methods to mitigate the adverse effects of temporal fluctuations and phase deviations on estimation errors, which should be of great importance to cater to real-time applications.