Robust Hybrid Beamforming and Dynamic Subarray Design for Near-Field mmWave ISAC Systems Under Unknown Interference

Abstract

This paper investigates a near-field millimeter-wave (mmWave) integrated sensing and communication (ISAC) system under unknown interference. A base station equipped with a partially connected dynamic subarray hybrid architecture serves a legitimate user while performing target-oriented transmit beampattern shaping. Unlike existing works that assume perfect interference knowledge, we characterize the unknown interference channels via a robust spatial covariance uncertainty model. To exploit spatial degrees of freedom for interference suppression, the user employs a fully connected hybrid receiver. We formulate a robust transmit power minimization problem subject to worst-case communication signal-to-interference-plus-noise ratio (SINR) and sensing beampattern constraints, alongside constant-modulus and dynamic subarray hardware constraints. To solve this highly non-convex mixed discrete–continuous problem, we propose a two-layer alternating optimization framework. The inner layer optimizes the continuous and phase-quantized beamformers using successive convex approximation, while the outer layer refines the binary subarray connections via a penalty-augmented local discrete search. Extensive simulations demonstrate that explicitly modeling worst-case uncertainties ensures reliable ISAC performance in adversarial environments, and the dynamic subarray architecture systematically outperforms conventional fixed topologies in power efficiency. Additional robustness and sensitivity analyses show that these gains are most pronounced when sufficient spatial degrees of freedom remain, whereas excessive antenna failures, unmodeled strong multipath, or covariance drift outside the uncertainty envelope can erode the communication and sensing margins.

1. Introduction

In recent years, millimeter-wave (mmWave) communication equipped with massive multiple-input–multiple-output (MIMO) antenna arrays has emerged as a key enabling technology for high-rate wireless systems [1,2]. Since fully digital beamforming requires a dedicated radio frequency (RF) chain for each antenna element, its hardware cost and power consumption become prohibitive for large-scale mmWave arrays. Hybrid analog–digital beamforming therefore provides a practical architecture that balances implementation complexity and array gain: energy-efficient and low-resolution transceiver designs establish the hardware-motivated baseline [3,4], wideband and multi-user optimization methods address algorithmic scalability [5,6], and learning-aided or regularized hybrid designs further broaden the design toolbox [7,8]. Recent generative-model-based hybrid beamforming also illustrates the continuing development of data-driven large-array design [9]. Furthermore, as the number of antennas increases to extremely large scales to compensate for severe path loss, the Rayleigh distance expands significantly. This paradigm shift transitions the operating environment from the conventional far-field planar-wave model to the near-field spherical-wave model (i.e., the Fresnel region), fundamentally altering the spatial channel characteristics [10,11]; it also introduces spatial wideband and CSI-transfer issues that require dedicated near-field treatment [12].

Concurrently, integrated sensing and communication (ISAC) has garnered significant attention, as it leverages shared hardware platforms and spectral resources to simultaneously support communication and sensing functionalities. General joint radar–communication studies summarize the broader ISAC design space [13], while mmWave beamformer optimization and short-packet Pareto design show how sensing–communication tradeoffs appear in high-frequency arrays [14,15]. This dual-functional paradigm renders the beamforming design fundamentally multi-objective, necessitating a delicate balance between communication quality-of-service (QoS) and sensing accuracy. Recently, hybrid beamforming has been extensively investigated in ISAC systems to achieve this tradeoff cost-effectively. Narrowband and compressed-sampling mmWave ISAC formulations motivate hybrid transceiver design for shared sensing and communication [16,17], whereas wideband extensions address frequency-selective hybrid ISAC operation [18]. These developments enable capabilities such as multi-static cooperative localization and target tracking [19,20]. In parallel with beamforming-oriented ISAC studies, waveform and sequence design has also become an important layer of the ISAC literature. For example, the recently proposed DRIP family develops versatile space-time discrete-time sequences for ISAC signaling [21]. These works are complementary to the present study: sequence-level designs improve the temporal and spectral structure of shared signals, whereas our focus is the spatial hybrid-beamforming and subarray-reconfiguration layer for near-field robust power minimization.

Despite these advances, deploying ISAC systems in hostile electromagnetic environments introduces exceedingly challenging security and robustness issues. The broadcast nature of wireless signals makes ISAC systems vulnerable to malicious eavesdropping and intentional interference [22,23]. Recent communication-oriented studies have addressed anti-jamming or anti-interference hybrid beamforming for mmWave massive MIMO [24,25], secrecy-oriented hybrid beamforming for satellite-terrestrial and extended-reality links [26,27], and secure or adaptive hybrid precoding under signal-domain or platform-perturbation impairments [28,29]. Robust covariance-matrix reconstruction is also relevant to interference-statistics estimation in adaptive beamforming [30]. However, these studies predominantly focus on communication-only systems, secure links without explicit sensing constraints, or statistical interference processing without dynamic ISAC subarray design. In practical adversarial scenarios, however, the exact channel state information (CSI) of interferers is typically inaccessible, and the legitimate channels are subject to estimation errors. Furthermore, the sensing signals themselves can be exploited to generate artificial noise (AN) to counter eavesdroppers or suppress interference. Secure ISAC and covert mmWave hybrid MIMO designs directly motivate this use of auxiliary or sensing-related emissions [22,31], while high-frequency physical-layer security studies further show the value of spatial signal shaping against unauthorized reception [32]. However, such protection requires sophisticated beamforming designs that are robust against CSI uncertainties.

To further enhance the spatial degrees of freedom (DoFs) under limited RF chains, partially connected structures have been extensively studied as an even more cost-effective alternative to fully connected hybrid beamforming. Sub-connected relay and multi-user partially connected designs demonstrate the architectural feasibility of this approach [33,34], while energy-efficiency studies with low-resolution converters quantify its hardware-efficiency advantage [35]. Building upon this, dynamic subarray architectures have been proposed, which adaptively optimize the antenna-to-RF-chain connections. Hybridly connected and reconfigurable subarray structures provide the main architectural basis for such topology adaptation [36,37], and joint precoding in sub-connected systems further supports the need to co-design the connection topology with the digital/analog beamformers [38]. By dynamically reconfiguring the subarray topology, these architectures can significantly improve spectral efficiency and interference suppression capabilities [22]. However, existing studies typically focus on isolated aspects such as spectral-efficiency maximization, interference robustness [24], sensing beampattern shaping, or subarray architecture optimization [37,38]. There remains a distinct lack of a unified framework for near-field mmWave massive hybrid array systems that jointly accounts for ISAC, unknown adversarial interferers, imperfect CSI, and dynamic subarray design.

In this paper, motivated by this gap, we investigate a near-field partially connected massive hybrid array system serving a legitimate user and a sensing target in the presence of multiple unknown interferers. To effectively suppress unknown interference signals, the legitimate user is equipped with a multi-antenna fully connected hybrid receiver, which provides the necessary spatial DoFs for interference mitigation. Meanwhile, the sensing functionality is represented through transmit-side near-field beampattern shaping. The main contributions of this paper are summarized as follows:

(1)

We establish a comprehensive near-field ISAC system model with a dynamic subarray hybrid beamforming architecture at the base station (BS) and a fully connected hybrid receiver at the user. We formulate a narrowband robust transmit power minimization problem subject to a communication signal-to-interference-plus-noise ratio (SINR) requirement and transmit-side sensing beampattern constraints.

(2)

To address the unknown interference channels and imperfect CSI, we characterize the interferers via a robust spatial covariance uncertainty model and the legitimate channel via a bounded error model. The semi-infinite robust constraints are then rigorously transformed into deterministic, mathematically tractable surrogates.

(3)

We develop a two-layer alternating optimization (AO) framework to solve the highly non-convex mixed discrete–continuous design problem. The inner layer optimizes the continuous and phase-quantized beamforming variables using successive convex approximation (SCA), while the outer layer refines the binary dynamic subarray connections via a penalty-augmented local discrete search.

(4)

Extensive simulations validate the superiority and limitations of the proposed framework. The simulation study also evaluates initialization sensitivity, partial antenna failures, strong near-field specular reflections, sensing-gain fluctuation, covariance drift, and a small-scale exhaustive topology benchmark. The results demonstrate that explicitly modeling worst-case uncertainties saves significant transmit power in adversarial environments, and the dynamic subarray architecture systematically outperforms conventional fixed topologies across various hardware configurations and system requirements.

The remainder of this paper is organized as follows. Section 2 presents the near-field ISAC system model, including the hybrid transceiver architecture, the ISAC signal model and performance metrics, and the CSI and interference uncertainty models. Section 3 formulates the robust transmit power minimization problem and derives deterministic tractable reformulations. Section 4 develops the two-layer alternating-optimization framework, combining inner-layer successive convex approximation for continuous and quantized beamforming variables with outer-layer local search over the dynamic subarray connections. Section 5 reports numerical simulations and discusses beamforming behavior, robustness, and comparisons with baselines. Section 6 concludes the paper.

Unless stated otherwise, scalars are denoted by italic letters, vectors by bold lower-case letters, and matrices by bold upper-case letters. The superscripts ${(·)}^{\mathrm{T}}$ and ${(·)}^{\mathrm{H}}$ denote transpose and conjugate transpose (Hermitian), respectively. ${\parallel ·\parallel }_2$ and ${\parallel ·\parallel }_F$ denote the Euclidean norm of a vector and the Frobenius norm of a matrix, respectively; $\mathrm{Tr}(·)$ is the trace. $\Re \{·\}$ and $\angle [·]$ denote the real part and the phase of a complex scalar, respectively. ⊙ denotes the Hadamard (element-wise) product. For Hermitian matrices, $\mathbf{A}⪰\mathbf{B}$ means that $\mathbf{A}-\mathbf{B}$ is positive semidefinite (PSD). $\mathbb{E}[·]$ denotes statistical expectation; $\mathcal{CN}(·,·)$ denotes the circularly symmetric complex Gaussian distribution. ${\mathbf{I}}_n$ is the $n\times n$ identity matrix, and ${\lambda }_{\min }(·)$ denotes the minimum eigenvalue of a Hermitian matrix.

2. System Model

2.1. Near-Field Channel and Hybrid Transceiver Architecture

Consider a narrowband near-field mmWave ISAC system comprising a base station (BS), a legitimate user, a sensing target, and J unknown interferers. As shown in Figure 1, the BS is equipped with an $N_t$-element uniform linear array (ULA) driven by $N_t^{\mathrm{RF}}$ RF chains, where $N_t^{\mathrm{RF}}\ll N_t$. A partially connected hybrid transmit architecture is employed at the BS [36], where each antenna is connected to exactly one RF chain. Similarly, the legitimate user is equipped with an $N_r$-antenna ULA and $N_r^{\mathrm{RF}}$ RF chains ($N_r^{\mathrm{RF}}\ll N_r$), employing a fully connected hybrid receive combiner. The sensing target and the legitimate user are assumed to be located in the Fresnel (near-field) region of the BS transmit array [10]. The interferers may occupy arbitrary spatial locations relative to the BS and the user array; however, their exact CSI is inaccessible at the BS. Consequently, their aggregate effect is absorbed into a covariance-based robust interference model. Without loss of generality, the center of the BS array is chosen as the origin of the coordinate system. The coordinate of the n-th BS antenna element is given by $x_n^{\left(t\right)}=\left(n-{\displaystyle \frac{N_t+1}{2}}\right)d$ for $n=1,\dots ,N_t$, and the coordinate of the m-th receive antenna at the legitimate user is $x_m^{\left(r\right)}=\left(m-{\displaystyle \frac{N_r+1}{2}}\right)d$ for $m=1,\dots ,N_r$, where $d=\lambda /2$ is the inter-element spacing and $\lambda$ is the carrier wavelength.

For a generic point located at a distance r and angle $\theta$ relative to the array center, the distance between the point and the n-th BS transmit antenna is $r_n^{\left(t\right)}(r,\theta )=\sqrt{r^2+{\left(x_n^{\left(t\right)}\right)}^2-2r{x}_n^{\left(t\right)}\sin \theta }$. The corresponding near-field transmit steering vector, which captures the spherical wavefront [10], is defined as

$$\begin{array}{c}\hfill {\mathbf{a}}_t(r,\theta )=\sqrt{{\displaystyle \frac{1}{N_t}}}\left[\begin{array}{c}{e}^{-j{\displaystyle \frac{2\pi }{\lambda }}(r_1^{\left(t\right)}-r)}\\ e^{-j{\displaystyle \frac{2\pi }{\lambda }}(r_2^{\left(t\right)}-r)}\\ ⋮\\ e^{-j{\displaystyle \frac{2\pi }{\lambda }}(r_{N_t}^{\left(t\right)}-r)}\end{array}\right]\in {\mathbb{C}}^{N_t\times 1}.\end{array}$$

(1)

Similarly, the distance between the generic point and the m-th receive antenna at the user is $r_m^{\left(r\right)}(r,\theta )=\sqrt{r^2+{\left(x_m^{\left(r\right)}\right)}^2-2r{x}_m^{\left(r\right)}\sin \theta }$. The corresponding near-field receive steering vector is given by

$$\begin{array}{c}\hfill {\mathbf{a}}_r(r,\theta )=\sqrt{{\displaystyle \frac{1}{N_r}}}\left[\begin{array}{c}{e}^{-j{\displaystyle \frac{2\pi }{\lambda }}(r_1^{\left(r\right)}-r)}\\ e^{-j{\displaystyle \frac{2\pi }{\lambda }}(r_2^{\left(r\right)}-r)}\\ ⋮\\ e^{-j{\displaystyle \frac{2\pi }{\lambda }}(r_{N_r}^{\left(r\right)}-r)}\end{array}\right]\in {\mathbb{C}}^{N_r\times 1}.\end{array}$$

(2)

Note that when the propagation distance r is sufficiently large (i.e., exceeding the Rayleigh distance), the near-field steering vectors naturally degenerate to the standard far-field planar-wave steering vectors.

The legitimate user is located at $(r_b,{\theta }_b)$, and, under the standard separable spherical-wave channel approximation, the BS-to-user channel is modeled by a near-field geometric channel [1,10] with $L_b$ effective paths

$$\begin{array}{c}\hfill {\mathbf{H}}_b={\displaystyle \sum _{\ell =1}^{L_b}}{\alpha }_{b,\ell }{\mathbf{a}}_r(r_{b,\ell }^{\left(r\right)},{\varphi }_{b,\ell }){\mathbf{a}}_t^{\mathrm{H}}(r_{b,\ell }^{\left(t\right)},{\theta }_{b,\ell }),\end{array}$$

(3)

where ${\alpha }_{b,\ell }$ is the complex gain of the ℓ-th path, ${\theta }_{b,\ell }$ and ${\varphi }_{b,\ell }$ are the effective transmit and receive angles, and $r_{b,\ell }^{\left(t\right)}$ and $r_{b,\ell }^{\left(r\right)}$ are the effective near-field distances seen from the transmit and receive arrays, respectively. To facilitate tractable analysis while capturing the fundamental near-field propagation characteristics, we postulate a line-of-sight (LoS) dominant link, under which the channel is well-approximated by

$$\begin{array}{c}\hfill {\mathbf{H}}_b\approx {\alpha }_b{\mathbf{a}}_r(r_b,{\varphi }_b){\mathbf{a}}_t^{\mathrm{H}}(r_b,{\theta }_b),\end{array}$$

(4)

where ${\alpha }_b$ is the complex gain of the LoS path, ${\theta }_b$ and ${\varphi }_b$ are the effective transmit and receive angles, and $r_b$ denotes the center-to-center BS–user distance. This LoS-dominant model is adopted as the baseline channel used for tractable derivation and numerical comparison, but the proposed robust formulation is not restricted to a single physical path. If strong near-field specular reflections are resolvable or estimable, they can be included in the nominal multi-path channel ${\widehat{\mathbf{H}}}_b$ through the summation model above. If they are not explicitly modeled, their aggregate effect is treated as part of the residual CSI mismatch and is protected only when the induced perturbation remains within the adopted radius ${ϵ}_b$. Rapidly appearing strong reflections outside this envelope therefore constitute an out-of-set mismatch and are not covered by the formal worst-case guarantee.

The sensing target is located at $(r_t,{\theta }_t)$ and is represented by the near-field target steering vector ${\mathbf{a}}_t^{\mathrm{tar}}\triangleq {\mathbf{a}}_t(r_t,{\theta }_t)$. To enhance the spatial degrees of freedom (DoFs) of the partially connected architecture, we introduce a dynamic subarray mechanism at the BS. This mechanism is motivated by hybridly connected and reconfigurable subarray architectures [36,37], while the need to jointly optimize sub-connected precoding variables is supported by recent joint-precoding studies [38]. Let $\mathbf{S}\in {\{0,1\}}^{N_t\times N_t^{\mathrm{RF}}}$ denote the dynamic subarray connection matrix, where its elements are defined as

$$\begin{array}{c}\hfill S_{n,k}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if} \mathrm{transmit} \mathrm{antenna} n \mathrm{is} \mathrm{connected} \mathrm{to}\hfill \\ \hfill & \mathrm{RF} \mathrm{chain} k,\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

(5)

The partially connected structure requires that each transmit antenna is connected to exactly one RF chain, i.e., ${\sum }_{k=1}^{N_t^{\mathrm{RF}}}{S}_{n,k}=1$ for all $n=1,\dots ,N_t$. To avoid empty RF branches, we further impose that each RF chain serves at least one antenna, i.e., ${\sum }_{n=1}^{N_t}{S}_{n,k}\ge 1$ for all $k=1,\dots ,N_t^{\mathrm{RF}}$. The BS analog precoder ${\mathbf{F}}_{\mathrm{RF}}\in {\mathbb{C}}^{N_t\times N_t^{\mathrm{RF}}}$ is then defined as ${\mathbf{F}}_{\mathrm{RF}}=\mathbf{S}\odot {\mathbf{V}}_t$, where ${\mathbf{V}}_t\in {\mathbb{C}}^{N_t\times N_t^{\mathrm{RF}}}$ is the complex phase matrix. For each active connection where $S_{n,k}=1$, the constant-modulus constraint is given by $|{\left[{\mathbf{V}}_t\right]}_{n,k}|=\sqrt{1/M_k}$, where $M_k\triangleq {\sum }_{n=1}^{N_t}{S}_{n,k}$. Assuming finite-resolution phase shifters, the phase of the active elements must belong to a discrete set $\angle {\left[{\mathbf{V}}_t\right]}_{n,k}\in {\mathcal{Q}}_B\triangleq \{2\pi q/2^B:q=0,1,\dots ,2^B-1\}$, where B is the number of quantization bits for the phase shifters. For notational simplicity, the same B-bit quantization set ${\mathcal{Q}}_B$ is adopted at the receiver.

At the legitimate user, a fully connected hybrid receive combiner is employed to suppress the unknown interference. Given the massive number of antennas at the BS ($N_t\gg N_r$), a fully connected architecture would incur prohibitive power consumption from phase shifters; hence, the dynamic partially connected structure is adopted. Conversely, the user is equipped with fewer antennas but faces severe unknown interference, necessitating a fully connected architecture to maximize spatial degrees of freedom (DoFs) for precise interference nulling. This design choice explicitly avoids the need for an antenna activation matrix (i.e., no $\mathbf{S}$ matrix is required at the user side). Let the user-side analog combiner be ${\mathbf{W}}_{\mathrm{RF}}\in {\mathbb{C}}^{N_r\times N_r^{\mathrm{RF}}}$ and the digital combiner be ${\mathbf{w}}_{\mathrm{BB}}\in {\mathbb{C}}^{N_r^{\mathrm{RF}}\times 1}$. The overall receive combiner is $\mathbf{w}\triangleq {\mathbf{W}}_{\mathrm{RF}}{\mathbf{w}}_{\mathrm{BB}}\in {\mathbb{C}}^{N_r\times 1}$. The analog combiner is subject to constant-modulus constraints for all its entries, i.e., $|{\left[{\mathbf{W}}_{\mathrm{RF}}\right]}_{m,\ell }|={\displaystyle \frac{1}{\sqrt{N_r}}}$ for all $m=1,\dots ,N_r,\ell =1,\dots ,N_r^{\mathrm{RF}}$. The receive-side phase shifters are also quantized according to the same codebook ${\mathcal{Q}}_B$.

2.2. ISAC Signal Model and Performance Metrics

To support ISAC, the BS transmits a dual-functional waveform composed of a communication stream $s_c$ and a sensing-assisted stream $s_r$ [14,22]. The transmitted signal vector $\mathbf{x}\in {\mathbb{C}}^{N_t\times 1}$ is $\mathbf{x}={\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c{s}_c+{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r{s}_r$, where ${\mathbf{f}}_c,{\mathbf{f}}_r\in {\mathbb{C}}^{N_t^{\mathrm{RF}}\times 1}$ are the digital beamformers for communication and sensing, respectively. The data streams are assumed to be independent and normalized such that $\mathbb{E}\left[\right|s_c{|}^2]=1$ and $\mathbb{E}\left[\right|s_r{|}^2]=1$. For compactness, the transmit signal can be equivalently expressed as $\mathbf{x}={\mathbf{F}}_{\mathrm{RF}}{\mathbf{F}}_{\mathrm{BB}}\mathbf{s}$, where ${\mathbf{F}}_{\mathrm{BB}}=[{\mathbf{f}}_c,{\mathbf{f}}_r]$ and $\mathbf{s}={[s_c,s_r]}^{\mathrm{T}}$. The transmit covariance matrix ${\mathbf{R}}_x$ is given by

$$\begin{array}{c}\hfill {\mathbf{R}}_x=\mathbb{E}\left[\mathbf{x}{\mathbf{x}}^{\mathrm{H}}\right]={\mathbf{F}}_{\mathrm{RF}}\left({\mathbf{f}}_c{\mathbf{f}}_c^{\mathrm{H}}+{\mathbf{f}}_r{\mathbf{f}}_r^{\mathrm{H}}\right){\mathbf{F}}_{\mathrm{RF}}^{\mathrm{H}},\end{array}$$

(6)

and the total transmit power is $\mathrm{Tr}\left({\mathbf{R}}_x\right)$.

At the legitimate user, the received signal is corrupted by noise and the aggregate interference signal ${\mathbf{z}}_J={\sum }_{j=1}^J{\mathbf{g}}_j{q}_j$, where ${\mathbf{g}}_j\in {\mathbb{C}}^{N_r\times 1}$ is the channel from interferer j to the user and $q_j$ is the zero-mean interference signal emitted by interferer j [24]. The pre-combining receive vector is ${\mathbf{y}}_b={\mathbf{H}}_b\mathbf{x}+{\mathbf{z}}_J+{\mathbf{n}}_b$, where ${\mathbf{n}}_b\sim \mathcal{CN}(\mathbf{0},{\sigma }_b^2{\mathbf{I}}_{N_r})$ is the additive white Gaussian noise (AWGN). After applying the hybrid receive combiner $\mathbf{w}$, the scalar output is ${\widehat{y}}_b={\mathbf{w}}^{\mathrm{H}}{\mathbf{y}}_b$. Since the sensing-assisted stream $s_r$ does not carry desired communication information, it acts as self-induced interference. The post-combining communication SINR is defined as

$$\begin{array}{c}\hfill {\mathrm{SINR}}_b={\displaystyle \frac{{\left|{\mathbf{w}}^{\mathrm{H}}{\mathbf{H}}_b{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\right|}^2}{{\left|{\mathbf{w}}^{\mathrm{H}}{\mathbf{H}}_b{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\right|}^2+{\mathbf{w}}^{\mathrm{H}}{\mathbf{R}}_J\mathbf{w}+{\sigma }_b^2{\parallel \mathbf{w}\parallel }_2^2}},\end{array}$$

(7)

where ${\mathbf{R}}_J\triangleq \mathbb{E}\left[{\mathbf{z}}_J{\mathbf{z}}_J^{\mathrm{H}}\right]$ is the spatial covariance matrix of the aggregate interference, which can capture both independent and mutually correlated interferer emissions.

In this paper, the sensing functionality is evaluated via a transmit-side beampattern proxy [15]. Specifically, we adopt the transmit illumination gain as the primary sensing design metric, thereby circumventing the need to explicitly model echo reception, clutter dynamics, and downstream detection/estimation processing. The transmit gain at an arbitrary near-field point $(r,\theta )$ is defined by

$$\begin{array}{c}\hfill G(r,\theta )={\mathbf{a}}_t^{\mathrm{H}}(r,\theta ){\mathbf{R}}_x{\mathbf{a}}_t(r,\theta ).\end{array}$$

(8)

At the target location $(r_t,{\theta }_t)$, the desired sensing gain is

$$\begin{array}{c}\hfill G_t\triangleq G(r_t,{\theta }_t)={\mathbf{a}}_t^{\mathrm{H}}(r_t,{\theta }_t){\mathbf{R}}_x{\mathbf{a}}_t(r_t,{\theta }_t).\end{array}$$

(9)

Let ${\Omega }_{\mathrm{sl}}\subset {\mathbb{R}}_{+}\times [-\pi /2,\pi /2]$ denote a prescribed non-target angle-range region, excluding a guard region around the sensing target, over which sidelobe suppression is required.

2.3. CSI and Interference Uncertainty Models

In practical adversarial scenarios, the exact CSI of the legitimate user, the precise location of the sensing target, and the channels of the interferers are typically subject to estimation errors or are completely unavailable. We model the estimated legitimate MIMO channel as ${\mathbf{H}}_b={\widehat{\mathbf{H}}}_b+{\mathbf{E}}_b$, where ${\widehat{\mathbf{H}}}_b$ is the estimated channel and ${\mathbf{E}}_b$ is the bounded CSI error satisfying $\parallel {\mathbf{E}}_b{\parallel }_F \le {ϵ}_b$. Here, ${\widehat{\mathbf{H}}}_b$ is the nominal near-field CSI dictated by the LoS-dominant model in Section 2.1. The spherical-wave geometry is therefore retained in the nominal channel response; the Frobenius-norm uncertainty is not intended to replace this geometry. Instead, ${\mathbf{E}}_b$ is used as an engineering robustness envelope for the residual error after range, angle, synchronization, array-calibration, and model-mismatch effects have been absorbed into the estimated channel. A physically meaningful radius can be selected as an outer bound on the channel perturbations induced by the expected range-angle error region,

$$\begin{array}{c}\hfill {ϵ}_b\ge \underset{(\Delta r,\Delta \theta )\in {\mathcal{E}}_b}{\sup }{∥{\mathbf{H}}_b(r_b+\Delta r,{\theta }_b+\Delta \theta )-{\widehat{\mathbf{H}}}_b∥}_F+{ϵ}_{\mathrm{cal}}+{ϵ}_{\mathrm{model}},\end{array}$$

(10)

where ${\mathcal{E}}_b$ denotes the admissible physical localization-error set, and ${ϵ}_{\mathrm{cal}}$ and ${ϵ}_{\mathrm{model}}$ summarize residual calibration and model errors. This construction is conservative: it encloses the structured near-field manifold inside a tractable matrix-norm ball. The benefit is that the resulting robust SINR bound remains solvable within the mixed discrete–continuous hybrid-beamforming problem, whereas directly optimizing over range-angle perturbation manifolds would lead to highly non-convex semi-infinite constraints. If the actual physical mismatch exceeds the adopted radius, the formal worst-case guarantee should no longer be interpreted as strict.

The target location is subject to estimation errors bounded by maximum uncertainty radii ${ϵ}_r$ and ${ϵ}_{\theta }$ for range and angle, respectively. However, due to the severe spatial wideband effect and rapid phase variations in the near field, directly mapping physical location errors to the steering vector via Taylor expansion is mathematically invalid for practical error bounds. Instead, we construct a tractable ellipsoidal uncertainty set directly on the steering vector manifold:

$$\begin{array}{c}\hfill {\mathbf{a}}_t^{\mathrm{tar}}={\widehat{\mathbf{a}}}_t+\tilde{\mathbf{D}}{\delta }_t,{\parallel {\delta }_t\parallel }_2\le 1,\end{array}$$

(11)

where ${\widehat{\mathbf{a}}}_t\triangleq {\mathbf{a}}_t({\widehat{r}}_t,{\widehat{\theta }}_t)$ is the nominal steering vector, ${\delta }_t\in {\mathbb{C}}^{2\times 1}$ is the normalized perturbation vector, and $\tilde{\mathbf{D}}\in {\mathbb{C}}^{N_t\times 2}$ is a shape matrix designed to bound the worst-case phase variations within the physical $(r,\theta )$ error region. This ellipsoidal envelope provides a mathematically rigorous and tractable surrogate for near-field robustness.

Consequently, the target steering vector ${\mathbf{a}}_t^{\mathrm{tar}}$ is uncertain, i.e., ${\mathbf{a}}_t^{\mathrm{tar}}={\widehat{\mathbf{a}}}_t+\tilde{\mathbf{D}}{\delta }_t$. Since the interferers are uncooperative, their exact CSI is unobtainable at the BS [24]. Therefore, we model the aggregate interference through a robust spatial covariance uncertainty set ${\mathcal{U}}_J=\{\mathbf{R}⪰0:\parallel \mathbf{R}-{\widehat{\mathbf{R}}}_J{\parallel }_F\le {ϵ}_J\}$, where ${\widehat{\mathbf{R}}}_J⪰\mathbf{0}$ is a sample covariance matrix, or equivalently its PSD-projected estimate, and ${ϵ}_J$ is the uncertainty radius. In practice, ${\widehat{\mathbf{R}}}_J$ can be obtained from interference-dominant snapshots collected during guard/silent resources, sensing listen-only intervals, or data-aided residual observation after subtracting the reconstructed desired communication and sensing components. Given L snapshots ${\left\{{\mathbf{y}}_{\ell }\right\}}_{\ell =1}^L$, a typical implementation forms

$$\begin{array}{c}\hfill {\widehat{\mathbf{R}}}_J={\Pi }_{⪰0}\left({\displaystyle \frac{1}{L}}{\displaystyle \sum _{\ell =1}^L}{\mathbf{y}}_{\ell }{\mathbf{y}}_{\ell }^{\mathrm{H}}-{\widehat{\sigma }}_b^2{\mathbf{I}}_{N_r}\right),\end{array}$$

(12)

where ${\Pi }_{⪰0}(·)$ denotes projection onto the positive-semidefinite cone and ${\widehat{\sigma }}_b^2$ is the calibrated noise-floor estimate. In dynamic environments, this covariance may be refreshed using a sliding window or an exponential update ${\widehat{\mathbf{R}}}_{J,t}=(1-\alpha ){\widehat{\mathbf{R}}}_{J,t-1}+\alpha {\widehat{\mathbf{R}}}_{J,\mathrm{new}}$. The radius ${ϵ}_J$ should cover finite-sample covariance error, residual desired-signal subtraction error, and temporal drift over the update interval; for instance, it can be chosen from validation residuals or as a high-percentile bound on $\parallel {\widehat{\mathbf{R}}}_{J,t}-{\widehat{\mathbf{R}}}_{J,t-1}{\parallel }_F$. The robust guarantee applies when the true covariance remains inside ${\mathcal{U}}_J$; if the interference statistics drift outside this set, the design must update either ${\widehat{\mathbf{R}}}_J$, ${ϵ}_J$, or both.

3. Problem Formulation and Deterministic Reformulation

3.1. Original Robust Power-Minimization Problem

The design variables are the dynamic subarray matrix $\mathbf{S}$, the transmit phase matrix ${\mathbf{V}}_t$, the communication and sensing digital beamformers ${\mathbf{f}}_c$ and ${\mathbf{f}}_r$, the user analog combiner ${\mathbf{W}}_{\mathrm{RF}}$, and the user digital combiner ${\mathbf{w}}_{\mathrm{BB}}$. With $\mathbf{w}={\mathbf{W}}_{\mathrm{RF}}{\mathbf{w}}_{\mathrm{BB}}$ and ${\mathbf{F}}_{\mathrm{RF}}=\mathbf{S}\odot {\mathbf{V}}_t$, the robust transmit power minimization problem is formulated as

$$\begin{array}{cc}\hfill {\mathcal{P}}_0:& \underset{\begin{array}{c}\mathbf{S},{\mathbf{V}}_t,{\mathbf{f}}_c,{\mathbf{f}}_r,\\ {\mathbf{W}}_{\mathrm{RF}},{\mathbf{w}}_{\mathrm{BB}}\end{array}}{\min }\mathrm{Tr}\left({\mathbf{R}}_x\right)\hfill \end{array}$$

(13a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& {\displaystyle \underset{\begin{array}{c}\parallel {\mathbf{E}}_b{\parallel }_F\le {ϵ}_b\\ {\mathbf{R}}_J\in {\mathcal{U}}_J\end{array}}{\min }\frac{{\left|{\mathbf{w}}^{\mathrm{H}}({\widehat{\mathbf{H}}}_b+{\mathbf{E}}_b){\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\right|}^2}{{\left|{\mathbf{w}}^{\mathrm{H}}({\widehat{\mathbf{H}}}_b+{\mathbf{E}}_b){\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\right|}^2+{\mathbf{w}}^{\mathrm{H}}{\mathbf{R}}_J\mathbf{w}+{\sigma }_b^2{\parallel \mathbf{w}\parallel }_2^2}\ge {\gamma }_b}\hfill \end{array}$$

(13b)

$$\begin{array}{cc}\hfill & {\displaystyle \underset{\begin{array}{c}{\delta }_t={[{\delta }_{t,1},{\delta }_{t,2}]}^{\mathrm{T}}\\ \parallel {\delta }_t{\parallel }_2\le 1\end{array}}{\min }{\mathbf{a}}_t^{\mathrm{H}}({\widehat{r}}_t+{ϵ}_r{\delta }_{t,1},{\widehat{\theta }}_t+{ϵ}_{\theta }{\delta }_{t,2}){\mathbf{R}}_x{\mathbf{a}}_t({\widehat{r}}_t+{ϵ}_r{\delta }_{t,1},{\widehat{\theta }}_t+{ϵ}_{\theta }{\delta }_{t,2})\ge {\Gamma }_t}\hfill \end{array}$$

(13c)

$$\begin{array}{cc}\hfill & G(r,\theta )\le {\Gamma }_{\mathrm{sl}},\forall (r,\theta )\in {\Omega }_{\mathrm{sl}},\hfill \end{array}$$

(13d)

$$\begin{array}{cc}\hfill & {\mathbf{F}}_{\mathrm{RF}}=\mathbf{S}\odot {\mathbf{V}}_t,\hfill \end{array}$$

(13e)

$$\begin{array}{cc}\hfill & {\displaystyle \sum _{k=1}^{N_t^{\mathrm{RF}}}}{S}_{n,k}=1,\forall n=1,\dots ,N_t,\hfill \end{array}$$

(13f)

$$\begin{array}{cc}\hfill & {\displaystyle \sum _{n=1}^{N_t}}{S}_{n,k}\ge 1,\forall k=1,\dots ,N_t^{\mathrm{RF}},\hfill \end{array}$$

(13g)

$$\begin{array}{cc}\hfill & S_{n,k}\in \{0,1\},\forall n,k,\hfill \end{array}$$

(13h)

$$\begin{array}{cc}\hfill & |{\left[{\mathbf{V}}_t\right]}_{n,k}|=\sqrt{{\displaystyle \frac{1}{M_k}}},\forall (n,k):S_{n,k}=1,\hfill \end{array}$$

(13i)

$$\begin{array}{cc}\hfill & |{\left[{\mathbf{W}}_{\mathrm{RF}}\right]}_{m,\ell }|={\displaystyle \frac{1}{\sqrt{N_r}}},\forall m,\ell ,\hfill \end{array}$$

(13j)

$$\begin{array}{cc}\hfill & \angle {\left[{\mathbf{V}}_t\right]}_{n,k}\in {\mathcal{Q}}_B,\forall (n,k):S_{n,k}=1,\hfill \end{array}$$

(13k)

$$\begin{array}{cc}\hfill & \angle {\left[{\mathbf{W}}_{\mathrm{RF}}\right]}_{m,\ell }\in {\mathcal{Q}}_B,\forall m,\ell .\hfill \end{array}$$

(13l)

In problem ${\mathcal{P}}_0$, the objective function and the constraints possess the following physical interpretations:

• (13a) is the objective function, which aims to minimize the total transmit power at the base station, represented by $\mathrm{Tr}\left({\mathbf{R}}_x\right)$.
• (13b) ensures that the worst-case post-combining communication SINR meets the minimum requirement ${\gamma }_b$, guaranteeing reliable communication for the legitimate user despite bounded CSI errors ${\mathbf{E}}_b$ and unknown interference covariance ${\mathbf{R}}_J$.
• (13c) guarantees that the worst-case transmit illumination gain toward the sensing target exceeds the threshold ${\Gamma }_t$, ensuring robust sensing performance under target location uncertainties ${\delta }_t$.
• (13d) enforces a spatial sidelobe suppression bound ${\Gamma }_{\mathrm{sl}}$ over the predefined non-target region ${\Omega }_{\mathrm{sl}}$, which mitigates sensing interference to other directions and reduces the risk of being intercepted by interferers.
• (13e) defines the partially connected hybrid precoding structure via ${\mathbf{F}}_{\mathrm{RF}}=\mathbf{S}\odot {\mathbf{V}}_t$, combining the binary connection matrix $\mathbf{S}$ and the phase shift matrix ${\mathbf{V}}_t$.
• (13f)–(13h) specify the dynamic subarray topology. Specifically, (13f) ensures each transmit antenna is connected to exactly one RF chain; (13g) prevents empty RF chains by ensuring each chain serves at least one antenna; and (13h) restricts the connection indicators to binary values.
• (13i) and (13j) represent the constant-modulus hardware constraints of the phase shifters. For the transmitter (13i), the power is normalized by the dynamically assigned subarray size $M_k$; for the fully connected receiver (13j), it is normalized by the total number of receive antennas $N_r$.
• (13k) and (13l) impose the finite-resolution (quantized) phase constraints on the active transmit and receive phase shifters, restricting their phases to the B-bit codebook ${\mathcal{Q}}_B$.

Problem ${\mathcal{P}}_0$ is highly non-trivial to solve directly due to the semi-infinite robust constraints, the binary subarray variables, the constant-modulus and quantized phase constraints, and the strong coupling among the transmit and receive beamforming variables.

3.2. Deterministic Tractable Reformulation of Robust Constraints

To derive a tractable design formulation, we proceed to recast the semi-infinite constraints in ${\mathcal{P}}_0$ into deterministic robust counterparts or locally accurate surrogates. Regarding the user-channel uncertainty, the ensuing communication-side reformulation should be interpreted as providing worst-case protection around the nominal near-field channel, bounded by the adopted unstructured CSI envelope.

First, for any fixed receive combiner $\mathbf{w}$, the worst-case interference covariance admits a closed-form characterization.

Lemma 1.

For any given $\mathbf{w}$, the worst-case aggregate interference power over the uncertainty set ${\mathcal{U}}_J$ satisfies

$$\begin{array}{c}\hfill \underset{{\mathbf{R}}_J\in {\mathcal{U}}_J}{\max }{\mathbf{w}}^{\mathrm{H}}{\mathbf{R}}_J\mathbf{w}={\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_J\mathbf{w}+{ϵ}_J{\parallel \mathbf{w}\parallel }_2^2.\end{array}$$

(14)

Proof. 

By writing ${\mathbf{R}}_J={\widehat{\mathbf{R}}}_J+({\mathbf{R}}_J-{\widehat{\mathbf{R}}}_J)$ and using the Frobenius Cauchy–Schwarz inequality, we obtain

$$\begin{array}{c}{\mathbf{w}}^{\mathrm{H}}{\mathbf{R}}_J\mathbf{w}={\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_J\mathbf{w}+\mathrm{Tr}\left(({\mathbf{R}}_J-{\widehat{\mathbf{R}}}_J)\mathbf{w}{\mathbf{w}}^{\mathrm{H}}\right)\hfill \\ \hfill \le {\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_J\mathbf{w}+\parallel {\mathbf{R}}_J-{\widehat{\mathbf{R}}}_J{\parallel }_F{\parallel \mathbf{w}{\mathbf{w}}^{\mathrm{H}}\parallel }_F.\end{array}$$

(15)

Because $\parallel \mathbf{w}{\mathbf{w}}^{\mathrm{H}}{\parallel }_F={\parallel \mathbf{w}\parallel }_2^2$ and $\parallel {\mathbf{R}}_J-{\widehat{\mathbf{R}}}_J{\parallel }_F\le {ϵ}_J$, we obtain the upper bound

$$\begin{array}{c}\hfill {\mathbf{w}}^{\mathrm{H}}{\mathbf{R}}_J\mathbf{w}\le {\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{R}}}_J\mathbf{w}+{ϵ}_J{\parallel \mathbf{w}\parallel }_2^2.\end{array}$$

(16)

If $\mathbf{w}=\mathbf{0}$, the claim is immediate. Otherwise, equality is attained by choosing

$$\begin{array}{c}\hfill {\mathbf{R}}_J^{★}={\widehat{\mathbf{R}}}_J+{ϵ}_J{\displaystyle \frac{\mathbf{w}{\mathbf{w}}^{\mathrm{H}}}{{\parallel \mathbf{w}\parallel }_2^2}},\end{array}$$

(17)

which belongs to ${\mathcal{U}}_J$ because ${\widehat{\mathbf{R}}}_J⪰\mathbf{0}$, $\mathbf{w}{\mathbf{w}}^{\mathrm{H}}⪰\mathbf{0}$, and $\parallel {\mathbf{R}}_J^{★}-{\widehat{\mathbf{R}}}_J{\parallel }_F={ϵ}_J$. Therefore, the stated maximum is achieved. □

Accordingly, define the effective worst-case interference-plus-noise covariance ${\tilde{\mathbf{R}}}_J\triangleq {\widehat{\mathbf{R}}}_J+({ϵ}_J+{\sigma }_b^2){\mathbf{I}}_{N_r}$. Next, introduce an auxiliary scalar ${\tau }_b\ge 0$ to upper bound the post-combining interference-plus-noise term. Because the desired-signal term and the self-interference term depend on the same CSI error matrix ${\mathbf{E}}_b$, the following split is adopted as a conservative sufficient condition for the robust SINR constraint

$$\begin{array}{c}\hfill \underset{\parallel {\mathbf{E}}_b{\parallel }_F\le {ϵ}_b}{\min }{\left|{\mathbf{w}}^{\mathrm{H}}({\widehat{\mathbf{H}}}_b+{\mathbf{E}}_b){\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\right|}^2\ge {\gamma }_b{\tau }_b,\end{array}$$

(18)

$$\begin{array}{c}\hfill \underset{\parallel {\mathbf{E}}_b{\parallel }_F\le {ϵ}_b}{\max }{\left|{\mathbf{w}}^{\mathrm{H}}({\widehat{\mathbf{H}}}_b+{\mathbf{E}}_b){\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\right|}^2+{\mathbf{w}}^{\mathrm{H}}{\tilde{\mathbf{R}}}_J\mathbf{w}\le {\tau }_b.\end{array}$$

(19)

By the Frobenius Cauchy–Schwarz inequality, for any $\mathbf{f}\in \{{\mathbf{f}}_c,{\mathbf{f}}_r\}$,

$$\begin{array}{c}\hfill \left|{\mathbf{w}}^{\mathrm{H}}{\mathbf{E}}_b{\mathbf{F}}_{\mathrm{RF}}\mathbf{f}\right|\le \parallel {\mathbf{E}}_b{\parallel }_F\parallel \mathbf{w}{\left({\mathbf{F}}_{\mathrm{RF}}\mathbf{f}\right)}^{\mathrm{H}}{\parallel }_F\le {ϵ}_b{\parallel \mathbf{w}\parallel }_2{\parallel {\mathbf{F}}_{\mathrm{RF}}\mathbf{f}\parallel }_2.\end{array}$$

(20)

Therefore, the above sufficient condition can be tractably enforced through the following deterministic second-order-cone type constraints:

$$\begin{array}{c}\hfill \left|{\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\right|-{ϵ}_b{\parallel \mathbf{w}\parallel }_2{\parallel {\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\parallel }_2\ge \sqrt{{\gamma }_b{\tau }_b},\end{array}$$

(21)

$$\begin{array}{c}\hfill {\left(\left|{\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\right|+{ϵ}_b{\parallel \mathbf{w}\parallel }_2{\parallel {\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\parallel }_2\right)}^2+{\mathbf{w}}^{\mathrm{H}}{\tilde{\mathbf{R}}}_J\mathbf{w}\le {\tau }_b.\end{array}$$

(22)

It is worth noting that these inequalities constitute sufficient, albeit generally not necessary, conditions for the original robust SINR constraint. The inherent conservatism originates from two primary sources: first, the identical CSI perturbation ${\mathbf{E}}_b$ is decoupled across the desired-signal and self-interference terms via the auxiliary variable ${\tau }_b$; second, each constituent term is subsequently upper-bounded using norm inequalities. Consequently, this approximation may conservatively overestimate the transmit power required to guarantee robustness. While tighter bounds (e.g., via the S-procedure or Nemirovski Lemma) exist for quadratic forms, applying them to the fractional SINR term coupled with mixed-integer variables (dynamic subarrays) and quantized phases would render the subproblems computationally prohibitive. The adopted decoupling strategy strikes a necessary balance, providing a strictly tractable lower bound for the highly non-convex mixed-discrete optimization.

For the sensing constraint, we adopt the ellipsoidal steering vector uncertainty model from Section 2.3, where ${\mathbf{a}}_t^{\mathrm{tar}}={\widehat{\mathbf{a}}}_t+\tilde{\mathbf{D}}{\delta }_t$ with $\parallel {\delta }_t{\parallel }_2\le 1$. To construct the shape matrix $\tilde{\mathbf{D}}$, we utilize the spatial gradients:

$$\begin{array}{c}\hfill \tilde{\mathbf{D}}\triangleq \left[\begin{array}{cc}{\kappa }_r{\mathbf{d}}_r& {\kappa }_{\theta }{\mathbf{d}}_{\theta }\end{array}\right],\end{array}$$

(23)

where ${\mathbf{d}}_r={\displaystyle \frac{\partial {\mathbf{a}}_t(r,\theta )}{\partial r}}{|}_{({\widehat{r}}_t,{\widehat{\theta }}_t)}$ and ${\mathbf{d}}_{\theta }={\displaystyle \frac{\partial {\mathbf{a}}_t(r,\theta )}{\partial \theta }}{|}_{({\widehat{r}}_t,{\widehat{\theta }}_t)}$. The scaling factors ${\kappa }_r$ and ${\kappa }_{\theta }$ are chosen to empirically cover the target region. While this gradient-based shape matrix acts as a local surrogate, enforcing robustness over the entire ellipsoid $\parallel {\delta }_t{\parallel }_2\le 1$ provides a significantly wider protection margin than a nominal design.

The corresponding robust transmit-side illumination constraint is thus written as

$$\begin{array}{c}\hfill {({\widehat{\mathbf{a}}}_t+\tilde{\mathbf{D}}{\delta }_t)}^{\mathrm{H}}{\mathbf{R}}_x({\widehat{\mathbf{a}}}_t+\tilde{\mathbf{D}}{\delta }_t)\ge {\Gamma }_t,\forall {\parallel {\delta }_t\parallel }_2\le 1.\end{array}$$

(24)

For the above locally linearized semi-infinite constraint, a convenient sufficient LMI condition is that there exists a slack variable ${\mu }_t\ge 0$ such that

$$\begin{array}{c}\hfill \left[\begin{array}{cc}{\tilde{\mathbf{D}}}^{\mathrm{H}}{\mathbf{R}}_x\tilde{\mathbf{D}}+{\mu }_t{\mathbf{I}}_2& {\tilde{\mathbf{D}}}^{\mathrm{H}}{\mathbf{R}}_x{\widehat{\mathbf{a}}}_t\\ {\widehat{\mathbf{a}}}_t^{\mathrm{H}}{\mathbf{R}}_x\tilde{\mathbf{D}}& {\widehat{\mathbf{a}}}_t^{\mathrm{H}}{\mathbf{R}}_x{\widehat{\mathbf{a}}}_t-{\Gamma }_t-{\mu }_t\end{array}\right]⪰\mathbf{0}.\end{array}$$

(25)

This LMI should therefore be interpreted as a robust target-illumination guarantee for the adopted local beampattern proxy, rather than as a guarantee for a complete sensing receiver or detector.

Finally, the continuous sidelobe region ${\Omega }_{\mathrm{sl}}$ is numerically represented by a dense design grid ${\mathcal{G}}_{\mathrm{sl}}={\left\{(r_i^{\mathrm{sl}},{\theta }_i^{\mathrm{sl}})\right\}}_{i=1}^{N_{\mathrm{sl}}}$, and the continuous sidelobe requirement is approximated by the finite set of inequalities $G(r_i^{\mathrm{sl}},{\theta }_i^{\mathrm{sl}})\le {\Gamma }_{\mathrm{sl}}$ for $i=1,\dots ,N_{\mathrm{sl}}$. The approximation quality improves as the grid is refined; no exact between-grid guarantee is claimed in the present formulation.

3.3. Tractable Approximate Design Problem

Combining the above deterministic reformulations yields the following tractable approximate design problem used below:

$$\begin{array}{cc}\hfill {\mathcal{P}}_1:& \underset{\begin{array}{c}\mathbf{S},{\mathbf{V}}_t,{\mathbf{f}}_c,{\mathbf{f}}_r,\\ {\mathbf{W}}_{\mathrm{RF}},{\mathbf{w}}_{\mathrm{BB}},{\tau }_b,{\mu }_t\end{array}}{\min }\mathrm{Tr}\left({\mathbf{R}}_x\right)\hfill \end{array}$$

(26a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& {\displaystyle \left|{\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\right|-{ϵ}_b{\parallel \mathbf{w}\parallel }_2{\parallel {\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\parallel }_2\ge \sqrt{{\gamma }_b{\tau }_b}}\hfill \end{array}$$

(26b)

$$\begin{array}{cc}\hfill & {\displaystyle {\left(\left|{\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\right|+{ϵ}_b{\parallel \mathbf{w}\parallel }_2{\parallel {\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\parallel }_2\right)}^2+{\mathbf{w}}^{\mathrm{H}}{\tilde{\mathbf{R}}}_J\mathbf{w}\le {\tau }_b}\hfill \end{array}$$

(26c)

$$\begin{array}{cc}\hfill & {\displaystyle \left[\begin{array}{cc}{\tilde{\mathbf{D}}}^{\mathrm{H}}{\mathbf{R}}_x\tilde{\mathbf{D}}+{\mu }_t{\mathbf{I}}_2& {\tilde{\mathbf{D}}}^{\mathrm{H}}{\mathbf{R}}_x{\widehat{\mathbf{a}}}_t\\ {\widehat{\mathbf{a}}}_t^{\mathrm{H}}{\mathbf{R}}_x\tilde{\mathbf{D}}& {\widehat{\mathbf{a}}}_t^{\mathrm{H}}{\mathbf{R}}_x{\widehat{\mathbf{a}}}_t-{\Gamma }_t-{\mu }_t\end{array}\right]⪰\mathbf{0}}\hfill \end{array}$$

(26d)

$$\begin{array}{cc}\hfill & G(r_i^{\mathrm{sl}},{\theta }_i^{\mathrm{sl}})\le {\Gamma }_{\mathrm{sl}},i=1,\dots ,N_{\mathrm{sl}},\hfill \end{array}$$

(26e)

$$\begin{array}{cc}\hfill & {\mathbf{F}}_{\mathrm{RF}}=\mathbf{S}\odot {\mathbf{V}}_t,\hfill \end{array}$$

(26f)

$$\begin{array}{cc}\hfill & {\displaystyle \sum _{k=1}^{N_t^{\mathrm{RF}}}}{S}_{n,k}=1,\forall n=1,\dots ,N_t,\hfill \end{array}$$

(26g)

$$\begin{array}{cc}\hfill & {\displaystyle \sum _{n=1}^{N_t}}{S}_{n,k}\ge 1,\forall k=1,\dots ,N_t^{\mathrm{RF}},\hfill \end{array}$$

(26h)

$$\begin{array}{cc}\hfill & S_{n,k}\in \{0,1\},\forall n,k,\hfill \end{array}$$

(26i)

$$\begin{array}{cc}\hfill & |{\left[{\mathbf{V}}_t\right]}_{n,k}|=\sqrt{{\displaystyle \frac{1}{M_k}}},\forall (n,k):S_{n,k}=1,\hfill \end{array}$$

(26j)

$$\begin{array}{cc}\hfill & |{\left[{\mathbf{W}}_{\mathrm{RF}}\right]}_{m,\ell }|={\displaystyle \frac{1}{\sqrt{N_r}}},\forall m,\ell ,\hfill \end{array}$$

(26k)

$$\begin{array}{cc}\hfill & \angle {\left[{\mathbf{V}}_t\right]}_{n,k}\in {\mathcal{Q}}_B,\forall (n,k):S_{n,k}=1,\hfill \end{array}$$

(26l)

$$\begin{array}{cc}\hfill & \angle {\left[{\mathbf{W}}_{\mathrm{RF}}\right]}_{m,\ell }\in {\mathcal{Q}}_B,\forall m,\ell ,\hfill \end{array}$$

(26m)

$$\begin{array}{cc}\hfill & {\tau }_b\ge 0,{\mu }_t\ge 0.\hfill \end{array}$$

(26n)

Problem ${\mathcal{P}}_1$ affords a much cleaner robust interpretation than ${\mathcal{P}}_0$: constraints (26b) and (26c) jointly establish a conservative sufficient guarantee for the worst-case communication SINR, constraint (26d) enforces robust target illumination bounded by the adopted local near-field linearization model, and constraint (26e) imposes stringent sidelobe control over a dense spatial grid. Consequently, ${\mathcal{P}}_1$ serves as a mathematically tractable surrogate of ${\mathcal{P}}_0$ rather than a strictly equivalent reformulation. Nevertheless, ${\mathcal{P}}_1$ remains non-convex due to the coupled hybrid beamforming variables, the discrete phase constraints (26l) and (26m), and the binary dynamic subarray matrix (26i).

4. Alternating-Optimization Algorithm

4.1. Algorithmic Principle

Despite the deterministic reformulations, Problem ${\mathcal{P}}_1$ remains challenging due to three primary factors: (1) the transmit and receive beamformers remain strongly coupled; (2) even under the adopted local near-field linearization, the target-gain constraint is non-convex with respect to the beamforming variables, necessitating iterative convexification; and (3) the phase-quantized analog beamformers and the binary subarray matrix yield a highly non-convex mixed discrete–continuous design space. To circumvent these intractable issues, we develop a two-layer alternating optimization (AO) framework:

(1)

An inner-layer hybrid beamforming update for fixed subarray assignment $\mathbf{S}$;

(2)

An outer-layer dynamic subarray refinement based on local discrete search.

For the discrete updates, it is convenient to introduce a penalty-augmented merit function that measures both transmit power and current constraint violations. Define

$$\begin{array}{c}\hfill {\Psi }_t({\mathbf{R}}_x,{\mu }_t)\triangleq \left[\begin{array}{cc}{\tilde{\mathbf{D}}}^{\mathrm{H}}{\mathbf{R}}_x\tilde{\mathbf{D}}+{\mu }_t{\mathbf{I}}_2& {\tilde{\mathbf{D}}}^{\mathrm{H}}{\mathbf{R}}_x{\widehat{\mathbf{a}}}_t\\ {\widehat{\mathbf{a}}}_t^{\mathrm{H}}{\mathbf{R}}_x\tilde{\mathbf{D}}& {\widehat{\mathbf{a}}}_t^{\mathrm{H}}{\mathbf{R}}_x{\widehat{\mathbf{a}}}_t-{\Gamma }_t-{\mu }_t\end{array}\right],\end{array}$$

(27)

and the scalar violation functions

$$\begin{array}{c}\hfill c_b\triangleq \sqrt{{\gamma }_b{\tau }_b}-\left|{\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\right|+{ϵ}_b{\parallel \mathbf{w}\parallel }_2{\parallel {\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c\parallel }_2,\end{array}$$

(28)

$$\begin{array}{c}\hfill c_r\triangleq {\left(\left|{\mathbf{w}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\right|+{ϵ}_b{\parallel \mathbf{w}\parallel }_2{\parallel {\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r\parallel }_2\right)}^2+{\mathbf{w}}^{\mathrm{H}}{\tilde{\mathbf{R}}}_J\mathbf{w}-{\tau }_b,\end{array}$$

(29)

$$\begin{array}{c}\hfill c_{s,i}\triangleq G(r_i^{\mathrm{sl}},{\theta }_i^{\mathrm{sl}})-{\Gamma }_{\mathrm{sl}},i=1,\dots ,N_{\mathrm{sl}}.\end{array}$$

(30)

With ${\left[x\right]}_{+}\triangleq \max \{x,0\}$, the merit function is chosen as

$$\begin{array}{cc}\hfill {\mathcal{J}}_{\rho }\triangleq & \mathrm{Tr}\left({\mathbf{R}}_x\right)+{\rho }_b{\left[c_b\right]}_{+}^2+{\rho }_r{\left[c_r\right]}_{+}^2+{\rho }_t{\left[-{\lambda }_{\min }\left({\Psi }_t\right)\right]}_{+}^2\hfill \\ \hfill & +{\rho }_s{\displaystyle \sum _{i=1}^{N_{\mathrm{sl}}}}{\left[c_{s,i}\right]}_{+}^2,\hfill \end{array}$$

(31)

where ${\rho }_b,{\rho }_r,{\rho }_t,{\rho }_s>0$ are penalty parameters. When all constraints are satisfied, ${\mathcal{J}}_{\rho }$ reduces to the transmit power objective. This merit function is used to guide the discrete updates of ${\mathbf{W}}_{\mathrm{RF}}$, ${\mathbf{V}}_t$, and $\mathbf{S}$. Note that although the minimum eigenvalue function ${\lambda }_{\min }\left({\Psi }_t\right)$ is non-differentiable when multiple minimum eigenvalues exist, it is only evaluated as a scalar merit metric during the backtracking line search and discrete coordinate descent, thus avoiding any requirement for its gradient.

4.2. Inner-Layer AO for Fixed Subarray

For a fixed dynamic subarray matrix $\mathbf{S}$, the corresponding active set is determined, and only the continuous or phase-quantized beamforming variables need to be updated. The inner-layer AO contains four steps.

(1)

Digital receive combiner update.

Let ${\mathbf{u}}_c={\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_c$ and ${\mathbf{u}}_r={\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_r$, and define the interference-plus-noise covariance at the user as ${\mathbf{C}}_I\triangleq {\widehat{\mathbf{H}}}_b{\mathbf{u}}_r{\mathbf{u}}_r^{\mathrm{H}}{\widehat{\mathbf{H}}}_b^{\mathrm{H}}+{\tilde{\mathbf{R}}}_J$. For a fixed analog combiner ${\mathbf{W}}_{\mathrm{RF}}$, the digital combiner is updated by maximizing the following receive-SINR surrogate generalized Rayleigh quotient

$$\begin{array}{c}\hfill {\eta }_{\mathrm{BB}}\left({\mathbf{w}}_{\mathrm{BB}}\right)={\displaystyle \frac{{\left|{\mathbf{w}}_{\mathrm{BB}}^{\mathrm{H}}{\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{u}}_c\right|}^2}{{\mathbf{w}}_{\mathrm{BB}}^{\mathrm{H}}{\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\mathbf{C}}_I{\mathbf{W}}_{\mathrm{RF}}{\mathbf{w}}_{\mathrm{BB}}}}.\end{array}$$

(32)

When ${\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\mathbf{C}}_I{\mathbf{W}}_{\mathrm{RF}}$ is nonsingular, the exact maximizer is given, up to a nonzero scaling factor, by

$$\begin{array}{c}\hfill {\mathbf{w}}_{\mathrm{BB}}\propto {\left({\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\mathbf{C}}_I{\mathbf{W}}_{\mathrm{RF}}\right)}^{-1}{\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{u}}_c.\end{array}$$

(33)

In scenarios where the effective interference covariance ${\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\mathbf{C}}_I{\mathbf{W}}_{\mathrm{RF}}$ becomes singular or numerically ill-conditioned, we resort to a Tikhonov-regularized update direction to ensure numerical stability

$$\begin{array}{c}\hfill {\mathbf{w}}_{\mathrm{BB}}^{\mathrm{dir}}\propto {\left({\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\mathbf{C}}_I{\mathbf{W}}_{\mathrm{RF}}+{ϵ}_{\mathrm{reg}}{\mathbf{I}}_{N_r^{\mathrm{RF}}}\right)}^{-1}{\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b{\mathbf{u}}_c.\end{array}$$

(34)

where ${ϵ}_{\mathrm{reg}}>0$ is a small predetermined regularization parameter. In the implementation, it is selected as a numerical eigenvalue floor rather than as a new physical design variable. Specifically, we use

$$\begin{array}{c}\hfill {ϵ}_{\mathrm{reg}}=\max \left\{{ϵ}_0,{\kappa }_{\mathrm{reg}}{\displaystyle \frac{\mathrm{Tr}\left({\mathbf{W}}_{\mathrm{RF}}^{\mathrm{H}}{\mathbf{C}}_I{\mathbf{W}}_{\mathrm{RF}}\right)}{N_r^{\mathrm{RF}}}}\right\},\end{array}$$

(35)

with a small floor ${ϵ}_0$ and ${\kappa }_{\mathrm{reg}}\ll 1$. This trace-normalized choice scales the diagonal loading with the average effective interference-plus-noise power while preventing numerical singularity when the covariance rank is reduced by analog combining. In the simulations, ${ϵ}_0={10}^{-6}$ is used, and the same value is adopted whenever the trace-normalized term is smaller. The regularization only stabilizes the surrogate receive-combiner direction; accepted updates are still subject to the penalty-merit decrease test. Let ${\mathbf{w}}_{\mathrm{BB}}^{\mathrm{trial}}$ denote the normalized version of either the exact maximizer or the regularized direction above. To ensure monotonic decrease of the penalty merit function, the accepted update is taken as

$$\begin{array}{c}\hfill {\mathbf{w}}_{\mathrm{BB}}^{+}=(1-{\beta }_{\mathrm{BB}}){\mathbf{w}}_{\mathrm{BB}}+{\beta }_{\mathrm{BB}}{\mathbf{w}}_{\mathrm{BB}}^{\mathrm{trial}},{\beta }_{\mathrm{BB}}\in [0,1],\end{array}$$

(36)

where ${\beta }_{\mathrm{BB}}$ is chosen by backtracking so that ${\mathcal{J}}_{\rho }$ does not increase; in implementation, the search starts from ${\beta }_{\mathrm{BB}}=1$ and repeatedly shrinks it as ${\beta }_{\mathrm{BB}}\leftarrow {\varpi }_{\mathrm{bt}}{\beta }_{\mathrm{BB}}$ with a fixed ${\varpi }_{\mathrm{bt}}\in (0,1)$ until the first acceptable step is found. If no positive step size yields a decrease, we set ${\beta }_{\mathrm{BB}}=0$ and keep the current iterate. This receive-digital update is surrogate-based rather than the exact ${\mathbf{w}}_{\mathrm{BB}}$-block optimizer of ${\mathcal{P}}_1$; its role is to provide a low-cost nominal-SINR trial direction that is retained only when the full penalty merit function decreases.

(2)

Analog receive combiner update.

With ${\mathbf{w}}_{\mathrm{BB}}$ fixed, the analog combiner is updated by discrete coordinate descent over the quantized codebook. For each entry $(m,\ell )$, the feasible set is ${\mathcal{W}}_{m,\ell }=\{{\displaystyle \frac{1}{\sqrt{N_r}}}{e}^{j\phi }:\phi \in {\mathcal{Q}}_B\}$. Keeping all other entries unchanged, the new value of ${\left[{\mathbf{W}}_{\mathrm{RF}}\right]}_{m,\ell }$ is chosen as ${\left[{\mathbf{W}}_{\mathrm{RF}}\right]}_{m,\ell }^{★}=\arg {\min }_{w\in {\mathcal{W}}_{m,\ell }}{\mathcal{J}}_{\rho }({\mathbf{W}}_{\mathrm{RF}}\leftarrow w)$. All entries are cyclically updated until the decrease in ${\mathcal{J}}_{\rho }$ becomes smaller than a prescribed threshold. This update is monotonic because each coordinate is optimized over a finite set while the other coordinates are fixed.

(3)

Digital transmit beamformer update.

For fixed $(\mathbf{S},{\mathbf{V}}_t,{\mathbf{W}}_{\mathrm{RF}},{\mathbf{w}}_{\mathrm{BB}})$, the digital block is optimized by an inner successive convex approximation (SCA). Define

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \mathbf{Q}& \triangleq {\mathbf{F}}_{\mathrm{RF}}^{\mathrm{H}}{\mathbf{F}}_{\mathrm{RF}},{\mathbf{h}}_b\triangleq {\mathbf{F}}_{\mathrm{RF}}^{\mathrm{H}}{\widehat{\mathbf{H}}}_b^{\mathrm{H}}\mathbf{w},\hfill \\ \hfill {\mathbf{A}}_i& \triangleq {\mathbf{F}}_{\mathrm{RF}}^{\mathrm{H}}{\mathbf{a}}_t(r_i^{\mathrm{sl}},{\theta }_i^{\mathrm{sl}}){\mathbf{a}}_t^{\mathrm{H}}(r_i^{\mathrm{sl}},{\theta }_i^{\mathrm{sl}}){\mathbf{F}}_{\mathrm{RF}},\hfill \end{array}\end{array}$$

(37)

for $i=1,\dots ,N_{\mathrm{sl}}$. At the q-th SCA step, let ${\mathbf{f}}_c^{\left[q\right]}$, ${\mathbf{f}}_r^{\left[q\right]}$, and ${\tau }_b^{\left[q\right]}$ be the current digital iterate, with ${\tau }_b^{\left[q\right]}\ge {\tau }_{\min }$, where ${\tau }_{\min }>0$ is a small predetermined numerical safeguard. Since the global phase of ${\mathbf{f}}_c$ does not affect either the objective or the quadratic power terms, we use the phase-aligned lower bound

$$\begin{array}{c}\hfill \left|{\mathbf{h}}_b^{\mathrm{H}}{\mathbf{f}}_c\right|\ge \Re \left\{e^{-j{\varphi }_c^{\left[q\right]}}{\mathbf{h}}_b^{\mathrm{H}}{\mathbf{f}}_c\right\},{\varphi }_c^{\left[q\right]}\triangleq \angle \left({\mathbf{h}}_b^{\mathrm{H}}{\mathbf{f}}_c^{\left[q\right]}\right).\end{array}$$

(38)

If ${\mathbf{h}}_b^{\mathrm{H}}{\mathbf{f}}_c^{\left[q\right]}=0$, we simply set ${\varphi }_c^{\left[q\right]}=0$, since any phase choice yields the same real-part lower bound at that degenerate point.

In addition, because $\sqrt{{\tau }_b}$ is concave, its first-order Taylor expansion at ${\tau }_b^{\left[q\right]}$ gives a global upper bound $\sqrt{{\tau }_b}\le \sqrt{{\tau }_b^{\left[q\right]}}+{\displaystyle \frac{{\tau }_b-{\tau }_b^{\left[q\right]}}{2\sqrt{{\tau }_b^{\left[q\right]}}}}$. To obtain an explicit affine surrogate for the target-LMI terms, define for each $\nu \in \{c,r\}$

$$\begin{array}{c}\hfill {\mathbf{y}}_{\nu }\triangleq \left[\begin{array}{c}{\tilde{\mathbf{D}}}^{\mathrm{H}}{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_{\nu }\\ {\widehat{\mathbf{a}}}_t^{\mathrm{H}}{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_{\nu }\end{array}\right],{\mathbf{y}}_{\nu }^{\left[q\right]}\triangleq \left[\begin{array}{c}{\tilde{\mathbf{D}}}^{\mathrm{H}}{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_{\nu }^{\left[q\right]}\\ {\widehat{\mathbf{a}}}_t^{\mathrm{H}}{\mathbf{F}}_{\mathrm{RF}}{\mathbf{f}}_{\nu }^{\left[q\right]}\end{array}\right].\end{array}$$

(39)

Then, ${\Psi }_t({\mathbf{R}}_x,{\mu }_t)$ can be rewritten as

$$\begin{array}{c}\hfill {\Psi }_t({\mathbf{R}}_x,{\mu }_t)=\sum _{\nu \in \{c,r\}}{\mathbf{y}}_{\nu }{\mathbf{y}}_{\nu }^{\mathrm{H}}+\left[\begin{array}{cc}{\mu }_t{\mathbf{I}}_2& \mathbf{0}\\ {\mathbf{0}}^{\mathrm{H}}& -{\Gamma }_t-{\mu }_t\end{array}\right].\end{array}$$

(40)

Using the PSD identity $({\mathbf{y}}_{\nu }-{\mathbf{y}}_{\nu }^{\left[q\right]}){({\mathbf{y}}_{\nu }-{\mathbf{y}}_{\nu }^{\left[q\right]})}^{\mathrm{H}}⪰\mathbf{0}$, we obtain the Loewner lower bound

$$\begin{array}{c}\hfill {\mathbf{y}}_{\nu }{\mathbf{y}}_{\nu }^{\mathrm{H}}⪰{\mathbf{y}}_{\nu }{\left({\mathbf{y}}_{\nu }^{\left[q\right]}\right)}^{\mathrm{H}}+{\mathbf{y}}_{\nu }^{\left[q\right]}{\mathbf{y}}_{\nu }^{\mathrm{H}}-{\mathbf{y}}_{\nu }^{\left[q\right]}{\left({\mathbf{y}}_{\nu }^{\left[q\right]}\right)}^{\mathrm{H}}.\end{array}$$

(41)

Accordingly, the explicit affine surrogate is defined as

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\widehat{\Psi }}_t^{\left[q\right]}({\mathbf{f}}_c,{\mathbf{f}}_r,{\mu }_t)\triangleq & \sum _{\nu \in \{c,r\}}({\mathbf{y}}_{\nu }{\left({\mathbf{y}}_{\nu }^{\left[q\right]}\right)}^{\mathrm{H}}+{\mathbf{y}}_{\nu }^{\left[q\right]}{\mathbf{y}}_{\nu }^{\mathrm{H}}\hfill \\ \hfill & -{\mathbf{y}}_{\nu }^{\left[q\right]}{\left({\mathbf{y}}_{\nu }^{\left[q\right]}\right)}^{\mathrm{H}})+\left[\begin{array}{cc}{\mu }_t{\mathbf{I}}_2& \mathbf{0}\\ {\mathbf{0}}^{\mathrm{H}}& -{\Gamma }_t-{\mu }_t\end{array}\right],\hfill \end{array}\end{array}$$

(42)

which satisfies ${\widehat{\Psi }}_t^{\left[q\right]}({\mathbf{f}}_c,{\mathbf{f}}_r,{\mu }_t)⪯{\Psi }_t({\mathbf{R}}_x,{\mu }_t)$ for all $({\mathbf{f}}_c,{\mathbf{f}}_r,{\mu }_t)$.

Using these relations, the digital variables are updated by solving the following convex surrogate problem at the q-th iteration, where $t_c,t_r$, and $v_r$ are auxiliary variables introduced to reformulate the non-convex terms into convex constraints:

$$\begin{array}{cc}\hfill {\mathcal{P}}_2^{\left[q\right]}:& \underset{{\mathbf{f}}_c,{\mathbf{f}}_r,{\tau }_b,{\mu }_t,t_c,t_r,v_r}{\min }{\mathbf{f}}_c^{\mathrm{H}}\mathbf{Q}{\mathbf{f}}_c+{\mathbf{f}}_r^{\mathrm{H}}\mathbf{Q}{\mathbf{f}}_r\hfill \end{array}$$

(43a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& {\displaystyle \Re \left\{e^{-j{\varphi }_c^{\left[q\right]}}{\mathbf{h}}_b^{\mathrm{H}}{\mathbf{f}}_c\right\}-{ϵ}_b{\parallel \mathbf{w}\parallel }_2{t}_c\ge \sqrt{{\gamma }_b}\left(\sqrt{{\tau }_b^{\left[q\right]}}+\frac{{\tau }_b-{\tau }_b^{\left[q\right]}}{2\sqrt{{\tau }_b^{\left[q\right]}}}\right)}\hfill \end{array}$$

(43b)

$$\begin{array}{cc}\hfill & \parallel {\mathbf{Q}}^{1/2}{\mathbf{f}}_c{\parallel }_2\le t_c,\hfill \end{array}$$

(43c)

$$\begin{array}{cc}\hfill & |{\mathbf{h}}_b^{\mathrm{H}}{\mathbf{f}}_r|\le v_r,\hfill \end{array}$$

(43d)

$$\begin{array}{cc}\hfill & \parallel {\mathbf{Q}}^{1/2}{\mathbf{f}}_r{\parallel }_2\le t_r,\hfill \end{array}$$

(43e)

$$\begin{array}{cc}\hfill & {\left(v_r+{ϵ}_b{\parallel \mathbf{w}\parallel }_2{t}_r\right)}^2+{\mathbf{w}}^{\mathrm{H}}{\tilde{\mathbf{R}}}_J\mathbf{w}\le {\tau }_b,\hfill \end{array}$$

(43f)

$$\begin{array}{cc}\hfill & {\widehat{\Psi }}_t^{\left[q\right]}({\mathbf{f}}_c,{\mathbf{f}}_r,{\mu }_t)⪰\mathbf{0},\hfill \end{array}$$

(43g)

$$\begin{array}{cc}\hfill & {\mathbf{f}}_c^{\mathrm{H}}{\mathbf{A}}_i{\mathbf{f}}_c+{\mathbf{f}}_r^{\mathrm{H}}{\mathbf{A}}_i{\mathbf{f}}_r\le {\Gamma }_{\mathrm{sl}},i=1,\dots ,N_{\mathrm{sl}},\hfill \end{array}$$

(43h)

$$\begin{array}{cc}\hfill & {\tau }_b\ge {\tau }_{\min },{\mu }_t\ge 0.\hfill \end{array}$$

(43i)

Problem ${\mathcal{P}}_2^{\left[q\right]}$ is convex and can be efficiently solved by standard interior-point solvers. Note that the convex quadratic constraint (43f) can be equivalently recast as a standard second-order cone (SOC) constraint ${∥{\left[v_r+{ϵ}_b{\parallel \mathbf{w}\parallel }_2{t}_r,{\left({\tilde{\mathbf{R}}}_J^{1/2}\mathbf{w}\right)}^{\mathrm{T}},{\displaystyle \frac{{\tau }_b-1}{2}}\right]}^{\mathrm{T}}∥}_2\le {\displaystyle \frac{{\tau }_b+1}{2}}$, which facilitates efficient implementation in such solvers.

Before the first SCA solve ($q=0$), an initial digital point $({\mathbf{f}}_c^{\left[0\right]},{\mathbf{f}}_r^{\left[0\right]},{\tau }_b^{\left[0\right]})$ must be specified. In practice, ${\mathbf{f}}_c^{\left[0\right]}$ and ${\mathbf{f}}_r^{\left[0\right]}$ can be obtained by projecting the estimated user and target steering vectors, respectively, onto the current RF-precoder subspace, while ${\tau }_b^{\left[0\right]}$ is initialized no smaller than ${\tau }_{\min }$. If this initialization renders the first convex surrogate ${\mathcal{P}}_2^{\left[0\right]}$ infeasible, we solve a one-time slack-augmented feasibility-restoration problem at the current linearization point to obtain a workable starting point. Introduce nonnegative slacks ${\xi }_b,{\xi }_r,{\xi }_t$, and ${\left\{{\xi }_{s,i}\right\}}_{i=1}^{N_{\mathrm{sl}}}$, together with positive weights ${\omega }_b,{\omega }_r,{\omega }_t,{\omega }_s$, and solve

$$\begin{array}{cc}\hfill {\mathcal{P}}_{\mathrm{FR}}^{\left[0\right]}:& \underset{\begin{array}{c}{\mathbf{f}}_c,{\mathbf{f}}_r,{\tau }_b,{\mu }_t,t_c,t_r,v_r,\\ {\xi }_b,{\xi }_r,{\xi }_t,\left\{{\xi }_{s,i}\right\}\end{array}}{\min }{\omega }_b{\xi }_b+{\omega }_r{\xi }_r+{\omega }_t{\xi }_t+{\omega }_s{\displaystyle \sum _{i=1}^{N_{\mathrm{sl}}}}{\xi }_{s,i}\hfill \end{array}$$

(44a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& {\displaystyle \Re \left\{e^{-j{\varphi }_c^{\left[0\right]}}{\mathbf{h}}_b^{\mathrm{H}}{\mathbf{f}}_c\right\}-{ϵ}_b{\parallel \mathbf{w}\parallel }_2{t}_c+{\xi }_b\ge \sqrt{{\gamma }_b}\left(\sqrt{{\tau }_b^{\left[0\right]}}+\frac{{\tau }_b-{\tau }_b^{\left[0\right]}}{2\sqrt{{\tau }_b^{\left[0\right]}}}\right)}\hfill \end{array}$$

(44b)

$$\begin{array}{cc}\hfill & \parallel {\mathbf{Q}}^{1/2}{\mathbf{f}}_c{\parallel }_2\le t_c,\hfill \end{array}$$

(44c)

$$\begin{array}{cc}\hfill & |{\mathbf{h}}_b^{\mathrm{H}}{\mathbf{f}}_r|\le v_r,\hfill \end{array}$$

(44d)

$$\begin{array}{cc}\hfill & \parallel {\mathbf{Q}}^{1/2}{\mathbf{f}}_r{\parallel }_2\le t_r,\hfill \end{array}$$

(44e)

$$\begin{array}{cc}\hfill & {\left(v_r+{ϵ}_b{\parallel \mathbf{w}\parallel }_2{t}_r\right)}^2+{\mathbf{w}}^{\mathrm{H}}{\tilde{\mathbf{R}}}_J\mathbf{w}\le {\tau }_b+{\xi }_r,\hfill \end{array}$$

(44f)

$$\begin{array}{cc}\hfill & {\widehat{\Psi }}_t^{\left[0\right]}({\mathbf{f}}_c,{\mathbf{f}}_r,{\mu }_t)+{\xi }_t{\mathbf{I}}_3⪰\mathbf{0},\hfill \end{array}$$

(44g)

$$\begin{array}{cc}\hfill & {\mathbf{f}}_c^{\mathrm{H}}{\mathbf{A}}_i{\mathbf{f}}_c+{\mathbf{f}}_r^{\mathrm{H}}{\mathbf{A}}_i{\mathbf{f}}_r\le {\Gamma }_{\mathrm{sl}}+{\xi }_{s,i},i=1,\dots ,N_{\mathrm{sl}},\hfill \end{array}$$

(44h)

$$\begin{array}{cc}\hfill & {\tau }_b\ge {\tau }_{\min },{\mu }_t\ge 0,\hfill \end{array}$$

(44i)

$$\begin{array}{cc}\hfill & {\displaystyle {\xi }_b\ge 0,{\xi }_r\ge 0,{\xi }_t\ge 0,{\xi }_{s,i}\ge 0,\forall i}\hfill \end{array}$$

(44j)

Problem ${\mathcal{P}}_{\mathrm{FR}}^{\left[0\right]}$ is convex. In practice, it is used only when the first surrogate subproblem is infeasible; its solution is then taken as the reset current iterate before re-solving ${\mathcal{P}}_2^{\left[0\right]}$. A numerically large optimal aggregate slack indicates that the initialization is too aggressive for the adopted requirements or uncertainty radii.

Let $({\mathbf{f}}_c^{\mathrm{trial}},{\mathbf{f}}_r^{\mathrm{trial}},{\tau }_b^{\mathrm{trial}},{\mu }_t^{\mathrm{trial}})$ denote the solution of the current convex surrogate ${\mathcal{P}}_2^{\left[q\right]}$. The accepted update is obtained through damping, ${\mathbf{f}}_{\nu }^{+}=(1-{\beta }_{\mathrm{F}}){\mathbf{f}}_{\nu }+{\beta }_{\mathrm{F}}{\mathbf{f}}_{\nu }^{\mathrm{trial}}$ for $\nu \in \{c,r\}$, with the same rule applied to ${\tau }_b$ and ${\mu }_t$, where ${\beta }_{\mathrm{F}}\in [0,1]$ is selected so that ${\mathcal{J}}_{\rho }$ is non-increasing. As in the receive-digital step, the line search starts from ${\beta }_{\mathrm{F}}=1$ and shrinks geometrically by the same factor ${\varpi }_{\mathrm{bt}}$ until the first merit-decreasing step is found. If no positive step size yields a merit decrease, we set ${\beta }_{\mathrm{F}}=0$ and retain the current digital iterate. The SCA loop is repeated until both the relative decrease in the convex surrogate objective and the relative decrease in ${\mathcal{J}}_{\rho }$ become negligible.

(4)

Quantized analog transmit phase update.

For fixed $(\mathbf{S},{\mathbf{f}}_c,{\mathbf{f}}_r,{\mathbf{W}}_{\mathrm{RF}},{\mathbf{w}}_{\mathrm{BB}})$, the analog transmit phase matrix ${\mathbf{V}}_t$ is refined by discrete coordinate descent. For each active entry $(n,k)$ satisfying $S_{n,k}=1$, the feasible set is ${\mathcal{V}}_{n,k}=\{\sqrt{1/M_k}{e}^{j\phi }:\phi \in {\mathcal{Q}}_B\}$. Keeping all remaining entries fixed, the update is ${\left[{\mathbf{V}}_t\right]}_{n,k}^{★}=\arg {\min }_{v\in {\mathcal{V}}_{n,k}}{\mathcal{J}}_{\rho }({\mathbf{V}}_t\leftarrow v)$. The entries are successively visited over all active antenna–chain pairs until the decrease in ${\mathcal{J}}_{\rho }$ becomes sufficiently small. Since each coordinate is optimized over a finite codebook, the merit function is monotonically non-increasing during this step.

4.3. Outer-Layer Dynamic Subarray Update

After the inner-layer beamformer update converges for the current $\mathbf{S}$, the dynamic subarray is refined by one-antenna local search. Specifically, for each antenna n, let $k_{\mathrm{old}}\left(n\right)$ denote its currently assigned RF chain. For every candidate chain $k\ne k_{\mathrm{old}}\left(n\right)$, construct a tentative reassignment matrix ${\mathbf{S}}^{(n\to k)}$ satisfying $S_{n,k}^{(n\to k)}=1$ and $S_{n,k_{\mathrm{old}}\left(n\right)}^{(n\to k)}=0$ while keeping all other rows unchanged. Candidates that violate the non-empty-subarray condition are discarded. The unchanged assignment $k=k_{\mathrm{old}}\left(n\right)$ is also retained as a feasible baseline candidate. For each feasible tentative reassignment, the variables $({\mathbf{f}}_c,{\mathbf{f}}_r,{\mathbf{W}}_{\mathrm{RF}},{\mathbf{w}}_{\mathrm{BB}},{\tau }_b,{\mu }_t)$ are temporarily kept at their latest inner-layer values, and

(1)

The new subarray sizes $M_k$ are computed;

(2)

The magnitudes of the active entries of ${\mathbf{V}}_t$ are renormalized according to the updated $M_k$;

(3)

One fast transmit-phase refinement sweep is carried out using the rule in Step 4 above, yielding an associated refined phase matrix denoted by ${\mathbf{V}}_t^{(n\to k)}$;

(4)

The corresponding merit value ${\mathcal{J}}_{\rho }({\mathbf{S}}^{(n\to k)},{\mathbf{V}}_t^{(n\to k)})$ is evaluated.

The reassignment for antenna n is chosen as

$$\begin{array}{c}\hfill k_n^{★}=\arg \underset{k\in {\mathcal{K}}_n\cup \left\{k_{\mathrm{old}}\left(n\right)\right\}}{\min }{\mathcal{J}}_{\rho }\left({\mathbf{S}}^{(n\to k)},{\mathbf{V}}_t^{(n\to k)}\right),\end{array}$$

(45)

where ${\mathcal{K}}_n$ denotes the set of feasible destination chains for antenna n other than the current one, and by convention, ${\mathbf{S}}^{(n\to k_{\mathrm{old}}\left(n\right))}$ denotes the unchanged current assignment. Once $k_n^{★}$ is selected, both the accepted subarray row and the corresponding refined transmit phase matrix are updated according to $(\mathbf{S},{\mathbf{V}}_t)\leftarrow ({\mathbf{S}}^{(n\to k_n^{★})},{\mathbf{V}}_t^{(n\to k_n^{★})})$. After all antennas are visited once, the resulting matrix is denoted by ${\mathbf{S}}^{(t+1)}$. The outer-layer search terminates when one complete sweep over all antennas yields no further decrease in ${\mathcal{J}}_{\rho }$. While this greedy one-antenna search is heuristic and may converge to a local optimum, it avoids the prohibitive $\mathcal{O}\left({\left(N_t^{\mathrm{RF}}\right)}^{N_t}\right)$ complexity of exhaustive search. Because the search is local in the discrete topology space, its final solution can depend on the initial subarray partition. We therefore use deterministic balanced contiguous or interleaved partitions as reproducible warm starts, and for deployment scenarios where initialization sensitivity is critical, a low-cost multi-start variant can run the same local search from several random feasible partitions and retain the lowest-merit solution. The initialization dependence is quantified in Section 5.6. In practice, alternating this search with the continuous beamformer updates provides substantial performance gains over fixed topologies.

4.4. Complete AO Procedure

The overall procedure of the proposed AO algorithm is summarized below. Let $T_{\max }$, ${\tau }_{\min }$, ${ϵ}_{\mathrm{reg}}$, ${\varpi }_{\mathrm{bt}}$, ${\epsilon }_{\mathrm{in}}$, ${\epsilon }_{\mathrm{AO}}$, $N_{\mathrm{stag}}$, ${\epsilon }_{\mathrm{stag}}$, and ${\kappa }_{\rho }$ denote the prescribed algorithmic parameters.

(1)

Initialize a feasible subarray matrix ${\mathbf{S}}^{\left(0\right)}$, for example, by balanced contiguous or round-robin partition.

(2)

Initialize ${\mathbf{V}}_t^{\left(0\right)}$ using phase matching toward the estimated user direction, and initialize ${\mathbf{W}}_{\mathrm{RF}}^{\left(0\right)}$ using phase matching toward the dominant receive steering vector.

(3)

Construct an initial digital point $({\mathbf{f}}_c^{\left(0\right)},{\mathbf{f}}_r^{\left(0\right)})$, for example, by projecting the estimated user and target steering vectors onto the column space of ${\mathbf{F}}_{\mathrm{RF}}^{\left(0\right)}$. Initialize ${\tau }_b^{\left(0\right)}$ as any positive scalar satisfying ${\tau }_b^{\left(0\right)}\ge {\tau }_{\min }$, and set an initial nonnegative ${\mu }_t^{\left(0\right)}$. These quantities define the first SCA iterate, i.e., ${\mathbf{f}}_{\nu }^{\left[0\right]}={\mathbf{f}}_{\nu }^{\left(0\right)}$, for $\nu \in \{c,r\}$, together with ${\tau }_b^{\left[0\right]}={\tau }_b^{\left(0\right)}$ and ${\mu }_t^{\left[0\right]}={\mu }_t^{\left(0\right)}$. If ${\mathcal{P}}_2^{\left[0\right]}$ is infeasible, solve the slack-augmented restoration problem ${\mathcal{P}}_{\mathrm{FR}}^{\left[0\right]}$ once and use its solution to reset the initial digital iterate before entering the standard SCA loop.

(4)

For outer iteration $t=0,1,\dots ,T_{\max }$, execute the following substeps:

(5)

With ${\mathbf{S}}^{\left(t\right)}$ fixed, run the inner-layer AO cycle until the relative decrease in ${\mathcal{J}}_{\rho }$ becomes smaller than a prescribed threshold ${\epsilon }_{\mathrm{in}}$: update ${\mathbf{w}}_{\mathrm{BB}}$ and ${\mathbf{W}}_{\mathrm{RF}}$, solve the SCA-based digital subproblem to update $({\mathbf{f}}_c,{\mathbf{f}}_r,{\tau }_b,{\mu }_t)$, and then update ${\mathbf{V}}_t$ over the active transmit phases.

(6)

Evaluate the current merit value ${\mathcal{J}}_{\rho }^{\left(t\right)}$ after the inner-layer convergence for the current ${\mathbf{S}}^{\left(t\right)}$.

(7)

Update the dynamic subarray matrix $\mathbf{S}$ by one-antenna local search, simultaneously carrying forward the refined transmit phase matrix associated with each accepted reassignment, and obtain ${\mathbf{S}}^{(t+1)}$.

(8)

If the maximum constraint violation is below a prescribed tolerance and the relative reduction in ${\mathcal{J}}_{\rho }$ is smaller than ${\epsilon }_{\mathrm{AO}}$, terminate; otherwise, continue.

(9)

If the maximum constraint violation remains above the tolerance and its decrease over the latest $N_{\mathrm{stag}}$ outer sweeps is smaller than a prescribed threshold ${\epsilon }_{\mathrm{stag}}$, increase the penalty parameters according to ${\rho }_{\ell }\leftarrow {\kappa }_{\rho }{\rho }_{\ell }$ for $\ell \in \{b,r,t,s\}$ with ${\kappa }_{\rho }>1$ and repeat.

The final output is the tuple $(\mathbf{S},{\mathbf{V}}_t,{\mathbf{f}}_c,{\mathbf{f}}_r,{\mathbf{W}}_{\mathrm{RF}},{\mathbf{w}}_{\mathrm{BB}})$.

4.5. Algorithmic Interpretation and Descent Discussion

The proposed AO procedure is best interpreted as a pragmatic descent framework tailored for the penalized approximate problem associated with ${\mathcal{P}}_1$, rather than an exact block-coordinate solver for ${\mathcal{P}}_1$ itself.

(1)

The digital receive combiner step optimizes a nominal generalized Rayleigh-quotient surrogate and then uses the merit-decrease test to accept or reject the resulting trial direction; when the nominal interference matrix is ill-conditioned, a small Tikhonov regularization is used to maintain numerical stability rather than claiming exact generalized-Rayleigh optimality in the singular case.

(2)

The digital transmit block is handled by SCA on the conservative approximate model, where the communication-related non-convex terms are locally convexified and the target-LMI-related rank-one PSD terms are replaced by explicit affine Loewner lower bounds.

(3)

The analog transmit and receive phase matrices are updated by exact per-coordinate minimization over their finite quantized codebooks.

(4)

The dynamic subarray matrix is updated by feasible local search directly driven by the penalty-augmented objective, and each accepted reassignment carries along its associated refined transmit phase matrix so that the actual iterate is consistent with the evaluated merit value.

Proposition 1

(fixed-penalty descent). Consider one phase of the proposed AO algorithm in which the penalty weights ${\rho }_b,{\rho }_r,{\rho }_t,{\rho }_s$ and the uncertainty radii are fixed. Suppose that each continuous trial point is accepted only if it does not increase ${\mathcal{J}}_{\rho }$, and each analog-phase or subarray candidate is selected from its finite feasible set by explicit merit comparison. Then the generated penalized merit sequence is monotonically non-increasing and converges in objective value because ${\mathcal{J}}_{\rho }\ge 0$.

The proof follows directly from the update construction. The digital receive combiner uses a regularized generalized-Rayleigh direction followed by a backtracking merit test; the SCA-based digital transmit update is damped and accepted only when the current merit does not increase; the finite-resolution analog phase update performs per-coordinate discrete minimization; and the subarray update accepts only one-antenna reassignments that decrease the same merit function. Thus, every accepted block update is descent-preserving for the fixed-penalty surrogate. Since the merit is bounded below by zero, the scalar sequence converges.

Several qualifications apply to this fixed-penalty descent statement. First, it concerns the penalized approximate problem associated with ${\mathcal{P}}_1$, not the original semi-infinite mixed-integer robust problem ${\mathcal{P}}_0$. Second, when the penalty weights are increased, the objective being minimized changes, so the monotonicity statement is piecewise with respect to each fixed-penalty phase. Third, because the receive update is surrogate-based, the target LMI is locally linearized, and the subarray search is one-antenna local search, the algorithm is not guaranteed to reach a globally optimal solution or an exact stationary point of ${\mathcal{P}}_0$. The final output is therefore best interpreted as a feasible or near-feasible locally refined design for the adopted conservative surrogate, stable with respect to the employed phase-codebook sweeps and one-antenna topology moves.

4.6. Computational Complexity Analysis

Let $T_{\mathrm{out}}$ denote the number of outer subarray sweeps, $T_{\mathrm{in}}$ the number of inner AO iterations per fixed topology, $T_{\mathrm{SCA}}$ the number of SCA iterations or conic solves used by the digital transmit block, and $Q=|{\mathcal{Q}}_B|=2^B$ the phase-codebook size. The overall complexity can be summarized as

$$\begin{array}{c}\hfill \mathcal{O}\left(T_{\mathrm{out}}\left[T_{\mathrm{in}}\left(C_{\mathrm{BB}}+T_{\mathrm{SCA}}{C}_{\mathrm{SCA}}+C_{\mathrm{RF}}\right)+C_{\mathrm{sub}}\right]\right).\end{array}$$

(46)

The digital receive combiner update requires inversion of an $N_r^{\mathrm{RF}}\times N_r^{\mathrm{RF}}$ matrix, giving $C_{\mathrm{BB}}=\mathcal{O}\left({\left(N_r^{\mathrm{RF}}\right)}^3\right)$. The digital transmit update solves the convex surrogate ${\mathcal{P}}_2^{\left[q\right]}$. With $\mathcal{O}\left(N_t^{\mathrm{RF}}\right)$ complex digital variables, a $3\times 3$ target LMI, and several SOC/linear constraints, a compact interior-point estimate is $C_{\mathrm{SCA}}=\mathcal{O}\left({\left(N_t^{\mathrm{RF}}\right)}^3\right)$ for the reduced RF-domain surrogate, while first-order conic solvers scale mainly with the same reduced RF-chain dimension and the number of constraints. The analog combiner and precoder phase sweeps evaluate Q candidates for each active RF-domain coordinate, leading to $C_{\mathrm{RF}}=\mathcal{O}\left(Q(N_r{N}_r^{\mathrm{RF}}+N_t)\right)$, where the transmit term is proportional to the number of active antenna-to-chain connections. The outer local search considers at most $N_t(N_t^{\mathrm{RF}}-1)$ one-antenna reassignment candidates per sweep, so $C_{\mathrm{sub}}=\mathcal{O}\left(N_t(N_t^{\mathrm{RF}}-1)C_{\mathrm{eval}}\right)$, where $C_{\mathrm{eval}}$ is the cost of one merit evaluation plus a local phase refinement. Thus, the proposed method scales linearly with the number of candidate antenna reassignments and exponentially only with the phase-bit resolution through $Q=2^B$, while avoiding exhaustive topology search with $\mathcal{O}\left({\left(N_t^{\mathrm{RF}}\right)}^{N_t}\right)$ candidates.

5. Simulation Results and Analysis

In this section, we provide a comprehensive simulation framework to evaluate the performance of the proposed robust hybrid beamforming and dynamic subarray design for the near-field mmWave ISAC system under unknown interference. The simulations are designed to validate the effectiveness of the proposed alternating optimization (AO) algorithm, the benefits of the dynamic subarray architecture, and the robustness against imperfect CSI and unknown interferers.

All numerical experiments are conducted in Python 3.13.12 under Windows 10, using NumPy 2.3.2, SciPy 1.16.1, Matplotlib 3.10.5, and CVXPY 1.8.2. For the convex surrogate subproblems appearing in the digital transmit update, the conic solver SCS is invoked through CVXPY.

5.1. Simulation Setup

We consider a near-field mmWave ISAC system operating at $f_c=28$ GHz with antenna spacing $d=\lambda /2$. The BS is equipped with a ULA of $N_t\in \{64,128\}$ antennas and $N_t^{\mathrm{RF}}\in \{4,8\}$ RF chains, while the user has $N_r=16$ antennas and $N_r^{\mathrm{RF}}=4$ RF chains. Both employ B-bit phase shifters ($B\in \{4,6\}$). The BS is at the origin $(0,0)$. The nominal locations of the user and target are $({\widehat{r}}_b,{\widehat{\theta }}_b)=(10\mathrm{m},{30}^{\circ })$ and $({\widehat{r}}_t,{\widehat{\theta }}_t)=(8\mathrm{m},-{20}^{\circ })$, respectively. Interferers are randomly distributed in $[{40}^{\circ },{80}^{\circ }]$ at distances of 15–30 m. Assuming a system bandwidth of $B_w=100$ MHz and a typical noise figure, the noise power is set to ${\sigma }_b^2=-90$ dBm. The uncertainty bounds are set as ${ϵ}_b={\eta }_b{\parallel {\widehat{\mathbf{H}}}_b\parallel }_F$ (${\eta }_b\in [0,0.1]$), ${ϵ}_r=0.5$ m, ${ϵ}_{\theta }=2^{\circ }$, and ${ϵ}_J=0.05{\parallel {\widehat{\mathbf{R}}}_J\parallel }_F$. The default requirements are ${\gamma }_b=15$ dB, ${\Gamma }_t=20$ dBm, and ${\Gamma }_{\mathrm{sl}}=0$ dBm. The digital-combiner Tikhonov floor is set to ${ϵ}_0={10}^{-6}$. Parameter variations in specific experiments are explicitly stated. All convex subproblems are solved via CVXPY with SCS. The additional robustness and sensitivity analyses in Section 5.6 use reduced-scale instances to permit repeated random initializations, failure trials, and small-scale topology enumeration; each such setting is stated with the corresponding figure and is generated by the accompanying scripts in the simulation package.

5.2. Convergence and Complexity Analysis

This subsection validates the numerical stability and implementation efficiency of the proposed AO framework. To reveal the convergence mechanism, we adopt a reduced-scale instance with $N_t=16$, $N_t^{\mathrm{RF}}=4$, $N_r=8$, $N_r^{\mathrm{RF}}=2$, and $B=4$.

Figure 2 demonstrates that the inner-layer AO loop converges stably. The penalty merit function drops significantly in the first iteration due to the slack-augmented feasibility-restoration step, which moves the algorithm from an infeasible initialization to a feasible region. Consequently, the transmit power initially rises to satisfy the robust constraints and then decreases steadily alongside the merit function as the AO mechanism progressively removes redundant power expenditure.

Figure 3 illustrates the computational characteristics of the implementation. Consistent with the theoretical analysis in Section 4.6, the exact convex surrogate solutions dominate the total runtime, followed by the analog discrete searches. The feasibility restoration is invoked only once. Overall, the proposed AO framework exhibits stable merit-descent behavior and remains computationally manageable, providing a reliable numerical foundation for the subsequent large-scale simulations.

5.3. Transmit Beampattern and Sensing Performance

This subsection evaluates the spatial sensing capabilities of the optimized transmit covariance. We adopt a larger-aperture setting with $N_t=64$, $N_t^{\mathrm{RF}}=4$, $N_r=8$, $N_r^{\mathrm{RF}}=2$, and $r_t=4$ m. Since the target lies well inside the Rayleigh distance ($\approx 21.25$ m), the transmit response depends jointly on range and angle.

Figure 4 depicts the normalized 2D transmit illumination map. The optimized design concentrates the dominant sensing energy almost exactly at the desired target location $(4.0\mathrm{m},-{20}^{\circ })$. Furthermore, the beampattern explicitly reflects the near-field spherical-wave propagation, exhibiting a finite range-selective high-illumination region rather than remaining invariant with respect to distance.

Figure 5 compares the angular cuts at the target range $r=4$ m with and without the sidelobe constraint. The constrained design preserves the mainlobe while substantially reducing energy leakage outside the $\pm 8^{\circ }$ guard region. Both the maximum and integrated sidelobe levels are significantly improved, demonstrating that the sidelobe-aware robust design effectively suppresses aggregate out-of-sector leakage, which is crucial for reducing unintended illumination toward hostile directions.

5.4. Robustness Against Uncertainties and Interference

This subsection validates the effectiveness of the deterministic robust reformulations against imperfect CSI, unknown interference, and target location errors. We adopt a reduced-scale setting with $N_t=16$, $N_t^{\mathrm{RF}}=4$, $N_r=8$, and $N_r^{\mathrm{RF}}=2$. The non-robust baseline is obtained by setting the corresponding uncertainty parameter to zero during optimization.

Figure 6 and Figure 7 evaluate the impact of CSI uncertainty (${\eta }_b$) and interference covariance uncertainty (${ϵ}_J$), respectively. In both cases, the robust design requires progressively more transmit power as the uncertainty radius increases. However, this apparent power penalty is necessary: while the robust design successfully maintains the guaranteed SINR lower bound at the prescribed 7 dB level over the entire uncertainty range, the non-robust baseline suffers severe SINR degradation, dropping to $4.79$ dB at ${\eta }_b=0.05$ and drastically to $1.06$ dB at ${ϵ}_J=0.05$. This confirms that explicitly modeling worst-case uncertainties is essential for sustaining communication reliability.

Figure 8 evaluates the sensing robustness under target location uncertainty over 400 random realizations. The robust design shifts the entire gain distribution to the right, reducing the empirical outage ratio $\mathrm{Pr}\{G_t<{\Gamma }_t\}$ from $\mathrm{49\%}$ (non-robust) to $\mathrm{19\%}$. Although a residual outage remains—because the ellipsoidal relaxation $\tilde{\mathbf{D}}$ provides a tractable surrogate rather than a strict absolute envelope for all extreme near-field phase wraps—the robust design significantly improves the 10th percentile gain by over 6 dB. This validates the practical effectiveness of the adopted deterministic robust approximation for sensing.

5.5. Comparison Among Subarray Architectures

This subsection compares the proposed dynamic subarray against two standard fixed partially connected baselines: the fixed contiguous partition and the fixed interleaved partition. All schemes use the same hybrid transceiver model and AO updates; they differ only in whether the subarray topology is dynamically refined. The dynamic architecture is initialized from the interleaved partition.

Figure 9, Figure 10 and Figure 11 evaluate the required transmit power across varying communication SINR targets (${\gamma }_b$), sensing gain thresholds (${\Gamma }_t$), and phase-shifter resolutions (B), respectively. The results consistently demonstrate that the dynamic architecture requires the least transmit power across all configurations. For instance, at ${\gamma }_b=8$ dB (Figure 9), the dynamic design saves $\mathrm{18.6\%}$ and $\mathrm{50.8\%}$ power compared to the fixed interleaved and contiguous structures, respectively. Similar prominent advantages are observed under stringent sensing requirements (Figure 10) and across low-to-high phase-shifter resolutions (Figure 11). These comprehensive sweeps confirm that dynamic antenna-to-RF-chain adaptation systematically provides a robust and unambiguous power efficiency gain over rigid subarray structures.

5.6. Additional Robustness, Sensitivity, and Topology Benchmark Analyses

This subsection presents reduced-scale robustness and sensitivity analyses that characterize the practical behavior of the dynamic subarray architecture, its dependence on initialization, its degradation under hardware and channel mismatch, and its empirical distance to a small-scale topology benchmark. Unless otherwise stated, these reduced-scale analyses use $N_t=12$, $N_t^{\mathrm{RF}}=3$, $N_r=6$, $N_r^{\mathrm{RF}}=2$, and $B=3$, so that repeated random trials and exhaustive topology enumeration remain reproducible.

Figure 12 confirms that the one-antenna local search is initialization-sensitive. Across 20 random partitions, the final transmit power has a mean of $0.796$, a standard deviation of $0.292$, abest value of $0.336$, and a worst value of $1.377$. The balanced contiguous initialization converges to $0.983$, about $\mathrm{23.5\%}$ above the random mean, whereas the interleaved warm start converges to $0.147$, about $\mathrm{81.6\%}$ below the random mean. These observations support deterministic structured warm starts and, when runtime permits, multi-start random initialization. They also indicate that the dynamic subarray layer functions as a local topology refinement rather than a globally optimal topology solver.

Figure 13 evaluates the achievable target illumination gain under partial BS antenna failures using a $N_t=16$, $N_t^{\mathrm{RF}}=4$, $N_r=8$, $N_r^{\mathrm{RF}}=2$ instance. Without failures, the nominal target gain is $-7.91$ dB, above the sensing threshold ${\Gamma }_t=0.13$ ($-8.86$ dB), and the communication SINR lower bound is $5.01$ dB. When only one antenna is disabled ($\approx \mathrm{5\%}$), the mean target gain remains close to the nominal value, but the 10th-percentile gain drops to $-10.29$ dB and the sensing outage reaches $\mathrm{30\%}$. With five disabled antennas ($\approx \mathrm{30\%}$), the sensing outage increases to $\mathrm{65\%}$. Thus, the proposed design can absorb mild average degradation, but it cannot guarantee both communication and sensing performance once hardware failures remove too many effective spatial degrees of freedom or create unfavorable aperture holes.

Figure 14 studies two additional near-field specular paths with relative strength from 0 to $0.75$. When the reflections are not included in the nominal CSI, the worst-case SINR lower bound decreases by about $6.05$ dB at a relative strength of $0.75$, indicating that strong unmodeled multipath can drive the actual channel outside the adopted CSI uncertainty envelope. When the same reflections are included in the nominal CSI, the algorithm can redesign the hybrid beamformers and may even exploit moderate reflected paths as additional spatial diversity. However, at the strongest tested reflection level, the SINR lower bound again degrades, showing that very strong multipath still requires timely channel re-estimation, a larger uncertainty radius, or additional RF/spatial degrees of freedom.

Figure 15 reports target-gain fluctuation metrics under increasing range-angle uncertainty. At uncertainty levels 0, $0.5$, $1.0$, and $1.5$, the 10th-percentile target gains are $-8.48$, $-8.59$, $-8.67$, and $-8.85$ dB, respectively, while the worst sampled gain at the largest uncertainty level is $-9.04$ dB. The empirical outage probability remains zero up to level $1.0$ and increases to $\mathrm{8.3\%}$ at level $1.5$. These results complement the CDF in Figure 8 by showing variance, lower-tail degradation, and outage probability explicitly.

Table 1. Covariance-drift sensitivity analysis for a reduced-SINR demonstration case with ${\gamma }_b=-2$ dB.

${\mathit{ϵ}}_{\mathit{J},\mathbf{actual}}/{\mathit{ϵ}}_{\mathit{J},\mathbf{design}}$	Actual ${\mathit{ϵ}}_{\mathit{J}}$	SINR Lower Bound (dB)
0.0	0.00	1.46
0.5	0.01	0.84
1.0	0.02	0.30
1.5	0.03	−0.18
2.0	0.04	−0.62

further illustrates how rapidly varying interference statistics affect the robust design. In this reduced-SINR sensitivity analysis, the SINR lower bound remains above the $-2$ dB requirement even when the actual covariance perturbation is twice the design radius, but the margin shrinks monotonically from $1.46$ dB to $-0.62$ dB. This behavior is consistent with the role of ${ϵ}_J$: it provides a robustness margin only for covariance deviations covered by the chosen uncertainty set, and larger drift consumes the available SINR margin.

Figure 16 evaluates how close the local-search result can be to a small-scale exhaustive topology benchmark. Among 62 feasible non-empty partitions, the best enumerated power is $0.0877$. Starting from the interleaved partition and applying the local search yields the best enumerated value for the tested instance, whereas starting from the contiguous partition converges to $0.3273$, corresponding to a $\mathrm{273.3\%}$ gap. Although this reduced-scale benchmark does not establish global optimality for the original large-scale mixed-integer robust problem, it indicates that the proposed local search can reach the best enumerated topology from a favorable warm start and also explains why poor initial partitions should be avoided or supplemented by multi-start search.

Overall, the dynamic subarray architecture yields the largest power reductions when enough active antennas and RF-chain degrees of freedom remain, the interference covariance is updated on the same time scale as the environment, and the dominant propagation paths are captured by the nominal CSI or by the adopted uncertainty envelope. The formal guarantees do not apply, or can become overly conservative, when the uncertainty radius is smaller than the true mismatch, when covariance drift is faster than the update window, when too many antennas fail, when strong multipath components are unmodeled, or when the SINR and sensing thresholds exceed the spatial degrees of freedom available under finite-resolution phase shifters.

5.7. Comparison with Literature-Inspired Baselines

To demonstrate the practical significance of the proposed unified robust design, we compare it against three literature-inspired baselines adapted to our setting: (1) ISAC-HBF Baseline [16], a conventional non-robust ISAC scheme with a fixed contiguous subarray; (2) Robust-CSI ISAC Baseline [22], a dynamic subarray scheme robust to CSI errors but oblivious to interference and target uncertainties; and (3) Interference-Resilient Comm Baseline [24], a communication-centric scheme lacking sensing constraints.

Figure 17 compares the required transmit power across different SINR targets under the actual uncertain environment (${ϵ}_b=0.03,{ϵ}_J=0.02$). The transmit power of all schemes is scaled to meet the actual worst-case SINR lower bound. Remarkably, the proposed fully robust scheme requires drastically less scaled power. For instance, at ${\gamma }_b=8$ dB, it requires only $3.59$ units, whereas Baseline 2 and Baseline 1 require $10.25$ and $25.67$, respectively. This counter-intuitive power saving stems from the architectural superiority of the dynamic subarray combined with exact spatial nulling of the interference subspace, demonstrating that non-robust methods suffer severe power penalties to satisfy actual reliability constraints.

Figure 18 evaluates the multi-metric performance under an equal transmit power budget ($\approx 1.817$). The baselines exhibit severe vulnerabilities: Baseline 3 completely fails at sensing (target gain $-24.47$ dB, no sidelobe suppression), while Baseline 1 fails to provide reliable communication (guaranteed SINR drops to $-3.51$ dB) and suffers degraded sensing gain ($-10.24$ dB) due to channel mismatch. In sharp contrast, the proposed scheme successfully balances all objectives, maintaining the highest guaranteed SINR ($-0.55$ dB), strong target gain ($-5.84$ dB), and effective sidelobe suppression ($-0.93$ dBc). These results conclusively demonstrate the necessity of the proposed framework in hostile electromagnetic environments.

6. Conclusions

This paper investigated robust hybrid beamforming and dynamic subarray design for near-field mmWave ISAC systems with imperfect CSI and unknown aggregate interference. We formulated a worst-case transmit-power minimization problem that jointly enforces a communication SINR requirement, target-oriented near-field illumination, sidelobe suppression, finite-resolution phase constraints, and dynamic antenna-to-RF-chain connectivity. To make the problem tractable, the spherical-wave nominal channel was combined with conservative CSI and covariance uncertainty envelopes, and the resulting semi-infinite constraints were converted into deterministic surrogate constraints. A two-layer AO framework was then developed: the inner layer updates the digital and phase-quantized hybrid beamformers, while the outer layer performs penalty-driven one-antenna local search over the dynamic subarray topology.

The simulations show three main findings. First, robust uncertainty modeling prevents severe reliability loss: under CSI and covariance uncertainty, the robust design preserves the prescribed SINR lower bound, whereas non-robust baselines suffer large degradation. Second, dynamic subarray adaptation reduces the required transmit power relative to fixed contiguous and interleaved architectures, especially when communication and sensing constraints compete for limited RF-chain degrees of freedom. Third, the additional robustness and sensitivity analyses clarify the method’s practical limits. The subarray local search is sensitive to initialization, but structured warm starts and multi-start search can substantially reduce this risk; partial antenna failures and unmodeled strong specular reflections can erode the sensing and communication margins; and covariance drift consumes the robustness margin unless the nominal covariance and radius are updated at a suitable time scale.

These findings are subject to the adopted conservative surrogate model. The proposed algorithm guarantees monotone descent of the fixed-penalty merit function, but it does not guarantee global optimality for the original mixed-integer semi-infinite robust problem. Future work should therefore consider tighter near-field structured uncertainty sets, online covariance tracking, failure-aware reconfiguration with explicit hardware diagnostics, and global or learning-assisted initialization strategies for large-scale dynamic subarray search.