A Robust Complex α-Sigmoid Affine Projection Algorithm Under Non-Gaussian Noise

Abstract

To address the performance degradation of traditional adaptive filtering algorithms in environments with correlated input signals and non-Gaussian noise, this paper proposes a complex-valued affine projection algorithm based on the $\alpha$-Sigmoid cost function ($\alpha$-CSAP). The algorithm leverages the nonlinear characteristics of the $\alpha$-Sigmoid function and implicitly achieves variable step-size updates by introducing a normalization factor, which effectively suppresses impulsive noise interference and avoids matrix inversion, thereby reducing computational complexity. Theoretical analysis derives the steady-state mean square deviation (MSD) expression for the algorithm. Simulation results demonstrate that the proposed $\alpha$-CSAP algorithm exhibits superior performance compared to traditional complex adaptive filtering algorithms in both system identification and beamforming application scenarios.

1. Introduction

As a core technology in the field of digital signal processing, adaptive filtering, which does not require prior knowledge of the statistical characteristics of signals and noise and can dynamically adjust parameters to adapt to time-varying environments, has become a key means for radar systems to cope with complex signal and interference conditions [1,2,3,4]. Conventional fixed filters exhibit limited performance when dealing with non-stationary, nonlinear signals or unknown interference, whereas adaptive filtering, through iterative optimization algorithms such as Least Mean Square (LMS), Recursive Least Squares (RLS), and particle filtering, enables efficient denoising, target detection, and parameter estimation, and is widely applicable in engineering scenarios such as radar and communications [5,6,7].

In radar signal processing, adaptive filtering plays an indispensable and central role. Radar echoes are often affected by non-Gaussian noise such as spherically invariant clutter and pulse interference, making it difficult for traditional detection methods to achieve a satisfactory balance between detection probability and false alarm rate [8,9]. Adaptive filtering can effectively address such complex signal environments by dynamically adjusting filter weights or structures. For example, the adaptive matched filter can maintain a constant false alarm rate under unknown clutter covariance matrices, significantly improving radar target detection performance [10]; optimized particle filtering can achieve high-precision prediction of highly dynamic target trajectories, effectively handling tracking problems in nonlinear and non-Gaussian scenarios [11]. At the signal and data processing level, the RLS algorithm, with its fast convergence characteristics, is particularly suitable for processing non-stationary radar signals [12]. Furthermore, the integration of adaptive filtering with techniques such as digital beamforming and data fusion further enhances the anti-interference capability and target resolution accuracy of radar systems [13].

As a central direction in modern signal processing and machine learning, complex-valued adaptive filtering is of broad application value compared to real-valued filtering techniques, owing to its inherent suitability for complex signals prevalent in digital communications, power systems, radar, and other fields [14]. Traditional algorithms, often based on the Mean Square Error (MSE) criterion, are simple to implement but tend to become inaccurate or diverge in non-Gaussian impulsive noise environments. Moreover, they typically neglect the statistical properties of non-circular signals, leading to information loss [15]. The integration of information-theoretic criteria has led to algorithms based on complex correntropy and generalized complex correntropy, significantly enhancing robustness against non-Gaussian noise [16]. However, challenges remain, including the insufficient flexibility of single-kernel functions, incomplete utilization of error probability distributions, performance degradation under noisy input–output models, and the inherent trade-off between convergence speed and steady-state error [17]. In response, researchers have pursued optimization through various means, such as employing widely linear models to capture non-circular characteristics, introducing mixed-kernel functions to enhance adaptability, and designing fixed-point iterative or recursive forms to reduce complexity [18]. These efforts aim to better adapt these algorithms to complex engineering scenarios. Complementary to complex adaptive filtering, beamforming is a key technology in array signal processing. It enhances desired signals and suppresses interference by controlling the phase and amplitude of signals received by multiple sensors/antennas, demonstrating significant advantages in wireless communications, acoustic processing, and radar detection. Its core value lies in utilizing spatial diversity to improve signal quality and transmission efficiency, giving rise to specialized applications such as distributed collaborative beamforming [19], microphone array speech enhancement, and broadband beamforming with fractal antenna arrays [20]. Innovations in hybrid architectures and unsupervised methods further extend its applicability [21]. As important branches of adaptive signal processing, both complex adaptive filtering and beamforming face common challenges regarding adaptability in complex environments and balancing performance with complexity. Their technological integration and collaborative optimization have become important research directions driven by practical engineering needs.

In applications such as system identification, echo cancellation, and signal prediction, the performance of traditional adaptive filtering algorithms is significantly affected when the input signal is correlated. Their core operation lies in adjusting filter coefficients in real time to adapt to dynamic environments [22]. The classic LMS algorithm is widely adopted due to its simplicity, but its convergence speed degrades notably under correlated inputs. RLS algorithms converge faster but face high computational complexity, making them less suitable for real-time requirements [23]. To balance convergence speed and computational complexity, Ozeki and Umeda first proposed the Affine Projection (AP) algorithm in 1984 [24]. By reusing past input data, it achieves faster convergence than LMS while maintaining lower complexity than RLS, representing a major breakthrough in adaptive filtering [25]. However, practical challenges such as impulsive noise, highly correlated inputs, and sparse system characteristics impose limitations on traditional AP algorithms. For example, in acoustic echo cancellation and hands-free vehicular communications, impulsive interference can degrade algorithm performance [26]. In sparse system identification, conventional AP algorithms fail to exploit sparsity, leading to inefficient convergence [27]. In massive data processing scenarios, indiscriminate data updates result in wasted computational resources. Consequently, extensive research has been dedicated to improving the AP algorithm. Various strategies have been developed to enhance its performance. Data-selective updating mechanisms have been introduced to reduce computational load while maintaining estimation accuracy. Robust AP algorithms based on M-estimation have been designed to improve resilience against impulsive noise [28]. Variants incorporating sparsity-inducing penalties have been proposed for sparse system identification [29]. Furthermore, approaches that avoid direct or indirect matrix inversion have been adopted to lower the computational complexity of AP-type algorithms [30]. Additional techniques, such as subband processing and variable step-size strategies, have also been employed to further optimize convergence behavior under correlated inputs [31]. These improved algorithms demonstrate advantages in applications like system identification, echo cancellation, and beamforming. Nonetheless, achieving an optimal balance among robustness, real-time performance, and accuracy in complex environments remains an open challenge. Furthermore, while most existing AP algorithms are confined to the real domain, their application in the complex domain is increasingly important due to the prevalence of complex signals in radar signal processing.

Recently, researchers proposed an adaptive projection algorithm based on an $\alpha$-sigmoid cost function, demonstrating a superior performance when dealing with strongly correlated input signals and non-Gaussian noise environments [32]. This provides a new approach for radar echo cancellation and channel estimation. Building upon this, we propose a novel complex-valued adaptive filtering algorithm based on the $\alpha$-sigmoid cost function. The steady-state MSD of the proposed algorithm is derived analytically. The performance of the proposed algorithm is compared with classical complex adaptive filtering algorithms in simulations of system identification and beamforming. The results confirm the effectiveness and robustness of the proposed algorithm. Furthermore, the proposed algorithm offers a new potential solution for beamforming applications.

The remainder of this paper is organized as follows: Section 2 introduces the proposed algorithm. Section 3 provides a theoretical analysis of the algorithm, including the derivation of its steady-state mean MSD expression and an analysis of its computational complexity. Section 4 verifies the theoretical analysis and provides simulation results to demonstrate the superiority and robustness of the proposed algorithm. Section 5 concludes the paper.

2. The Proposed $\mathbf{\alpha }$-CSAP Algorithm

As illustrated in Figure 1, the system identification model considered in this work is given by

$$d\left(n\right)={\mathbf{x}}^H\left(n\right){\mathbf{w}}_0+z\left(n\right),$$

(1)

where ${\mathbf{w}}_0\in {\mathbb{C}}^{L_w\times 1}$ denotes the unknown system weight vector with $L_w$ being the filter length, $z\left(n\right)$ is the complex-valued additive noise, and the input vector $\mathbf{x}\left(n\right)\in {\mathbb{C}}^{L_w\times 1}$ is defined as $\mathbf{x}\left(n\right)={[x\left(n\right),x(n-1),\dots ,x(n-L_w+1)]}^T$. Here, ${(·)}^H$ denotes the Hermitian transpose. The output of the adaptive filter is expressed as $y\left(n\right)={\mathbf{x}}^H\left(n\right)\mathbf{w}(n-1)$; here, $\mathbf{w}(n-1)\in {\mathbb{C}}^{L_w\times 1}$ is the adaptive weight vector at time index $n-1$. Accordingly, the instantaneous error signal is given by

$$e\left(n\right)=d\left(n\right)-y\left(n\right)=d\left(n\right)-{\mathbf{x}}^H\left(n\right)\mathbf{w}(n-1).$$

(2)

In the AP algorithm framework, the projection order Q specifies the number of past input–error pairs used for coefficient adaptation. The stacked error vector is defined as $\mathbf{e}\left(n\right)={[e\left(n\right),e(n-1),\dots ,e(n-Q+1)]}^T\in {\mathbb{C}}^{Q\times 1}$ and the input data matrix is constructed as $\mathbf{X}\left(n\right)=[\mathbf{x}\left(n\right),\mathbf{x}(n-1),\dots ,\mathbf{x}(n-Q+1)]\in {\mathbb{C}}^{L_w\times Q}$. In order to enhance the convergence rate of the algorithm and effectively mitigate the adverse effects of non-Gaussian noise on the adaptation process, the $\alpha$-sigmoid function is employed as the cost function, as follows:

$$J\left(n\right)=\sum _{i=0}^{Q-1}{\displaystyle \frac{\alpha }{\alpha -1}}{\left(1+exp\left(-{\displaystyle \frac{{\left|e(n-i)\right|}^2}{2{\delta }^2}}\right)\right)}^{1-{\displaystyle \frac{1}{\alpha }}},$$

(3)

where the parameter $\alpha$ governs the steepness of the cost function, thereby determining the convergence speed of the algorithm: a larger $\alpha$ leads to a steeper function and faster convergence. The parameter $\delta$ adjusts the algorithm’s sensitivity to outliers. From a mathematical perspective, as the error $e\left(n\right)$ increases, the curvature of the cost function gradually flattens, and its derivative with respect to the weights approaches zero, thereby achieving robust suppression of outliers.

Taking the Wirtinger derivative of $J\left(n\right)$ with respect to the complex conjugate of the weight vector ${\mathbf{w}}^{\ast }(n-1)$ yields

$${\displaystyle \frac{\partial J\left(n\right)}{\partial {\mathbf{w}}^{\ast }(n-1)}}=-\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right),$$

(4)

where $\eta \left(n\right)\in {\mathbb{R}}^{Q\times Q}$ is a diagonal weighting matrix defined as

$$\eta \left(n\right)=\mathrm{diag}\{{\eta }_1\left(n\right),\dots ,{\eta }_Q\left(n\right)\},$$

(5)

with

$${\eta }_i\left(n\right)={\displaystyle \frac{exp\left(-\frac{{\left|e(n-i+1)\right|}^2}{2{\delta }^2}\right)}{{\delta }^2{\left(1+exp\left(-\frac{{\left|e(n-i+1)\right|}^2}{2{\delta }^2}\right)\right)}^{1/\alpha }}},i=1,2,\dots ,Q.$$

(6)

Under the gradient descent framework, a preliminary update expression for the algorithm is formulated as follows:

$$\mathbf{w}\left(n\right)=\mathbf{w}(n-1)+\mu \mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right).$$

(7)

where $\mu$ is the step size.

In the derivation of the $\alpha$ SAP algorithm, the introduction of the normalization factor ${\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2$ is a key step to realize the $L_2$-norm constraint. The squared Euclidean norm of the weight update vector is expressed as follows:

$${\parallel \mathbf{w}\left(n\right)-\mathbf{w}(n-1)\parallel }_2^2={\mu }^2{\parallel \mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)\parallel }_2^2.$$

(8)

As can be observed from Equation (8), when the noise contains large outliers, the error $e\left(n\right)$ will be significantly affected, which, in turn, substantially impacts the weight estimation and degrades the filter performance. The restriction of the filter update magnitude can effectively enhance the robustness of the filter. This can be expressed as follows:    

$${\parallel \mathbf{w}\left(n\right)-\mathbf{w}(n-1)\parallel }_2^2\le \epsilon ,$$

(9)

where $\epsilon$ is a positive scalar. Taking the partial derivative of Equation (6), and according to Equation (8), the diagonal elements of the weighting matrix $\eta \left(n\right)$ have an upper bound ${\eta }_{max}=1/({\delta }^2·2^{1/\alpha })$, which leads to the following:

$$\begin{array}{cc}\hfill {\mu }^2{\parallel \mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)\parallel }_2^2& \le {\mu }^2{\eta }_{max}^2{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2^2\hfill \\ \hfill {\displaystyle \frac{{\mu }^2{\parallel \mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)\parallel }_2^2}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2^2}}& \le {\mu }^2{\eta }_{max}^2\hfill \end{array}$$

(10)

At this point, by introducing a time-varying parameter ${\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2$ into the denominator of Equation (8) that shares the same order as the numerator, the upper bound for the squared norm of the squared Euclidean norm of the weight update vector is obtained, as follows:

$${\parallel \mathbf{w}\left(n\right)-\mathbf{w}(n-1)\parallel }_2^2\le {\mu }^2{\eta }_{max}^2.$$

(11)

The update formula with constraints is expressed as follows:

$$\mathbf{w}\left(n\right)=\mathbf{w}(n-1)+\mu {\displaystyle \frac{\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}}.$$

(12)

The proposed $\alpha$-CSAP algorithm is summarized in Algorithm 1. The introduction of the normalization factor can be understood from an implicit variable step-size perspective. In scenarios with large errors, the effective step size $\mu /{∥\mathbf{X}\left(n\right)\mathbf{e}\left(n\right)∥}_2$ automatically decreases, thereby suppressing impulse noise interference and preventing filter divergence. In scenarios with small errors, the effective step size relatively increases, accelerating the convergence process. Moreover, traditional affine projection algorithms require the inversion or pseudo-inversion of the input matrix, which is prone to numerical instability due to ill-conditioned matrices. The normalization factor avoids matrix inversion operations, significantly reducing computational complexity.

Algorithm 1: The $\alpha$-CSAP Algorithm

Require:   $x\left(n\right)$, $d\left(n\right)$, $L_w$, Q, $\mu$, $\alpha$, ${\delta }^2$, N Ensure:   $\mathbf{w}\left(n\right)$ 1: Initialize $\mathbf{w}\left(0\right)={\mathbf{0}}_{L_w\times 1}$ 2: for $n=1$ to N do 3:     Construct: $\mathbf{x}\left(n\right)={\left[x\left(n\right),x(n-1),\dots ,x(n-L_w+1)\right]}^T$ 4:     Construct: $\mathbf{X}\left(n\right)=\left[\mathbf{x}\left(n\right),\mathbf{x}(n-1),\dots ,\mathbf{x}(n-Q+1)\right]$ 5:     Compute: $y\left(n\right)={\mathbf{x}}^H\left(n\right)\mathbf{w}(n-1)$ 6:     Compute: $e\left(n\right)=d\left(n\right)-y\left(n\right)$ 7:     Form: $\mathbf{e}\left(n\right)={\left[e\left(n\right),e(n-1),\dots ,e(n-Q+1)\right]}^T$ 8:     for $i=1$ to Q do 9:         ${\displaystyle {\eta }_i\left(n\right)=\frac{exp\left(-{\displaystyle \frac{{\left|e(n-i+1)\right|}^2}{2{\delta }^2}}\right)}{{\delta }^2{\left[1+exp\left(-{\displaystyle \frac{{\left|e(n-i+1)\right|}^2}{2{\delta }^2}}\right)\right]}^{1/\alpha }}}$ 10:     end for 11:     Construct: $\eta \left(n\right)=diag\left\{{\eta }_1\left(n\right),\dots ,{\eta }_Q\left(n\right)\right\}$ 12:     ${\displaystyle \mathbf{w}\left(n\right)=\mathbf{w}(n-1)+\mu ·\frac{\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)}{{∥\mathbf{X}\left(n\right)\mathbf{e}\left(n\right)∥}_2}}$ 13: end for 14: return $\mathbf{w}\left(n\right)$

3. Theoretical Analysis

3.1. Unbiasedness in the Mean

This subsection employs a classical method from mean value analysis to conduct an approximate analysis of the mean unbiasedness of the proposed $\alpha$-CSAP algorithm under reasonable assumptions.

Define the weight error vector as

$$\tilde{\mathbf{w}}\left(n\right)=\mathbf{w}\left(n\right)-{\mathbf{w}}_0,$$

(13)

where ${\mathbf{w}}_0$ denotes the unknown system weight vector. Substituting (12) into the above equation yields the update formula with constraints, which is expressed as:

$$\tilde{\mathbf{w}}\left(n\right)=\tilde{\mathbf{w}}(n-1)+\mu {\displaystyle \frac{\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}}.$$

(14)

Under the affine projection framework with projection order Q, let the stacked desired signal vector be $\mathbf{d}\left(n\right)={[d\left(n\right),d(n-1),\dots ,d(n-Q+1)]}^T$ and the stacked noise vector be $\mathbf{z}\left(n\right)={[z\left(n\right),z(n-1),\dots ,z(n-Q+1)]}^T$. Thus, the stacked error vector can be expressed as

$$\mathbf{e}\left(n\right)=\mathbf{d}\left(n\right)-{\mathbf{X}}^H\left(n\right)\mathbf{w}(n-1)=-{\mathbf{X}}^H\left(n\right)\tilde{\mathbf{w}}(n-1)+\mathbf{z}\left(n\right).$$

(15)

Substituting the above expression into the normalized update rule of the proposed $\alpha$-CSAP algorithm yields the following weight error recursion:

$$\begin{array}{cc}\hfill \tilde{\mathbf{w}}\left(n\right)=& \tilde{\mathbf{w}}(n-1)-\mu {\displaystyle \frac{\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{X}}^T\left(n\right){\tilde{\mathbf{w}}}^{\ast }(n-1)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}}\hfill \\ \hfill & +\mu {\displaystyle \frac{\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{z}}^{\ast }\left(n\right)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}}.\hfill \end{array}$$

(16)

To facilitate the mean analysis, the following assumptions are adopted.

• (A1) The input matrix $\mathbf{X}\left(n\right)$ is statistically independent of the weight error vector $\tilde{\mathbf{w}}\left(n\right)$.
• (A2) In steady state, the diagonal weighting matrix $\eta \left(n\right)$ varies slowly and can be approximated by

  $$\eta \left(n\right)\approx \overline{g}{\mathbf{I}}_Q,$$

  (17)

  where $\overline{g}=\mathbb{E}\left\{{\eta }_i\left(n\right)\right\}$.
• (A3) ${\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2$ is weakly correlated with $\mathbf{X}\left(n\right)$ and $\tilde{\mathbf{w}}(n-1)$, while $\mathbf{z}\left(n\right)$ is zero-mean and independent of both.

Under this assumption, it follows that $\mathbb{E}\left\{{\displaystyle \frac{\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{z}}^{\ast }\left(n\right)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}}\right\}\approx \mathbf{0}$. Then, taking the expectation on both sides of (16) yields

$$\mathbb{E}\left\{\tilde{\mathbf{w}}\left(n\right)\right\}\approx \left[\mathbf{I}-\mu \overline{g}\mathbb{E}\left\{{\displaystyle \frac{\mathbf{X}\left(n\right){\mathbf{X}}^H\left(n\right)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}}\right\}\right]\mathbb{E}\left\{\tilde{\mathbf{w}}(n-1)\right\}.$$

(18)

When the input signal is persistently exciting, the matrix inside the expectation is positive semi-definite. Therefore, for a sufficiently small step size $\mu$, the only equilibrium point of (18) is

$$\underset{n\to \infty }{lim}\mathbb{E}\left\{\tilde{\mathbf{w}}\left(n\right)\right\}=\mathbf{0}.$$

(19)

This result indicates that the proposed $\alpha$-CSAP algorithm is unbiased in the mean.

3.2. Steady-State MSD Analysis

3.2.1. Theoretical Value Derivation

In this subsection, the steady-state MSD performance of the proposed $\alpha$-CSAP algorithm is analyzed. Due to the presence of nonlinear error weighting and normalization terms, the analysis is carried out using the energy conservation approach commonly adopted in the mean-square analysis of AP algorithms.

The MSD is defined as

$$\mathrm{MSD}\left(n\right)\triangleq \mathbb{E}\left\{\parallel \tilde{\mathbf{w}}{\left(n\right)\parallel }^2\right\}.$$

(20)

From the normalized update rule of the proposed algorithm, the weight error vector evolves as

$$\tilde{\mathbf{w}}\left(n\right)=\tilde{\mathbf{w}}(n-1)-\mu \mathbf{K}\left(n\right),$$

(21)

where

$$\mathbf{K}\left(n\right)={\displaystyle \frac{\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}}.$$

(22)

Taking the squared Euclidean norm on both sides of (21), the expectation yields

$$\begin{array}{cc}\hfill \mathbb{E}\left\{\parallel \tilde{\mathbf{w}}{\left(n\right)\parallel }^2\right\}=& \mathbb{E}\left\{\parallel \tilde{\mathbf{w}}{(n-1)\parallel }^2\right\}\hfill \\ \hfill & -2\mu \mathbb{E}\left\{Re\left[{\tilde{\mathbf{w}}}^H(n-1)\mathbf{K}\left(n\right)\right]\right\}\hfill \\ \hfill & +{\mu }^2\mathbb{E}\left\{{\parallel \mathbf{K}\left(n\right)\parallel }^2\right\}.\hfill \end{array}$$

(23)

Under steady-state conditions, $\mathbb{E}\left\{\parallel \tilde{\mathbf{w}}{\left(n\right)\parallel }^2\right\}=\mathbb{E}\left\{\parallel \tilde{\mathbf{w}}{(n-1)\parallel }^2\right\}$, the MSD satisfies the following energy balance relation:

$$2\mu \mathbb{E}\left\{Re\left[{\tilde{\mathbf{w}}}^H(n-1)\mathbf{K}\left(n\right)\right]\right\}={\mu }^2\mathbb{E}\left\{{\parallel \mathbf{K}\left(n\right)\parallel }^2\right\}.$$

(24)

Define the a priori error vector as

$${\mathbf{e}}_a\left(n\right)={\mathbf{X}}^H\left(n\right)\tilde{\mathbf{w}}(n-1).$$

(25)

Then,

$${\tilde{\mathbf{w}}}^H(n-1)\mathbf{K}\left(n\right)={\displaystyle \frac{{\mathbf{e}}_a^H\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}}.$$

(26)

Since $\eta \left(n\right)$ is a diagonal matrix with entries $\eta \left(\right|e\left(n\right)\left|\right),\dots ,\eta \left(\right|e(n-Q+1)\left|\right)$, the numerator in (26) can be expanded as

$${\mathbf{e}}_a^H\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)=\sum _{i=0}^{Q-1}\eta \left(\right|e(n-i)\left|\right)e_a^{\ast }(n-i)e^{\ast }(n-i).$$

(27)

Following the standard mean-square analysis of APs, it is assumed that under steady-state and small-step-size conditions, the correlation between the numerator and the normalization factor in (26) is weak. Thus, the following separation approximation is adopted:

$$\mathbb{E}\left\{{\displaystyle \frac{A}{B}}\right\}\approx {\displaystyle \frac{\mathbb{E}\left\{A\right\}}{\mathbb{E}\left\{B\right\}}}.$$

(28)

Applying this approximation to (26), and noting that all Q components are statistically equivalent in steady state, yields

$$\mathbb{E}\left\{Re\left[{\tilde{\mathbf{w}}}^H(n-1)\mathbf{K}\left(n\right)\right]\right\}\approx {\displaystyle \frac{Q\mathbb{E}\left\{Re\left[\eta \left(\right|e\left|\right)e_a^{\ast }{e}^{\ast }\right]\right\}}{\mathbb{E}\{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right){\parallel }_2\}}}.$$

(29)

Define

$${\phi }_c\triangleq {\mathbb{E}\{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2\}.$$

(30)

Under steady-state conditions, the diagonal weighting function varies slowly and can be approximated by its average value, i.e.,

$$\eta \left(\right|e\left|\right)\approx \overline{\eta },\overline{\eta }=\mathbb{E}\left\{\eta \right(\left|e\right|\left)\right\}.$$

(31)

Therefore, the above expression reduces to

$$\mathbb{E}\left\{Re\left[{\tilde{\mathbf{w}}}^H(n-1)\mathbf{K}\left(n\right)\right]\right\}\approx {\displaystyle \frac{Q\overline{\eta }}{{\phi }_c}}\mathbb{E}\left\{Re\left[e_a^{\ast }{e}^{\ast }\right]\right\}.$$

(32)

For the complex-valued error $e=e_a+z$, where $e_a$ is a proper complex Gaussian random variable and z denotes Bernoulli–Gaussian impulsive noise, the weighting function operates on the magnitude of e only. By invoking Price’s theorem under the complex-valued setting, the following approximation holds:

$$\mathbb{E}\left\{Re\left[e_a^{\ast }{e}^{\ast }\right]\right\}\approx {\displaystyle \frac{1}{\sqrt{\pi }}}{\displaystyle \frac{\mathbb{E}\left\{\right|e_a{|}^2\}(1-P_r)}{\sqrt{\mathbb{E}\left\{\right|e_a{|}^2\}+{\sigma }_{\alpha }^2}}}.$$

(33)

Moreover, under steady-state conditions,

$${\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2\approx \sqrt{Q\mathbb{E}\left\{\right|e\left(n\right){|}^2\}}.$$

(34)

Combining the above results yields

$$\mathbb{E}\left\{Re\left[{\tilde{\mathbf{w}}}^H(n-1)\mathbf{K}\left(n\right)\right]\right\}\approx {\displaystyle \frac{\overline{\eta }}{{\phi }_c\sqrt{Q\mathbb{E}\left\{\right|e\left(n\right){|}^2\}}}}·{\displaystyle \frac{1}{\sqrt{\pi }}}{\displaystyle \frac{\mathbb{E}\left\{\right|e_a\left(n\right){|}^2\}(1-P_r)}{\sqrt{\mathbb{E}\left\{\right|e_a\left(n\right){|}^2\}+{\sigma }_{\alpha }^2}}}.$$

(35)

The second-order term in (24) can be written as

$$\mathbb{E}\left\{{\parallel \mathbf{K}\left(n\right)\parallel }^2\right\}=\mathbb{E}\left[{\displaystyle \frac{\parallel \mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }{\left(n\right)\parallel }_2^2}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2^2}}\right].$$

(36)

Under steady-state and small-step-size conditions, this term can be approximated as

$$\mathbb{E}\left\{{\parallel \mathbf{K}\left(n\right)\parallel }^2\right\}\approx {\overline{\eta }}^2,$$

(37)

which is upper bounded by ${\eta }_{max}^2$ due to the boundedness of the weighting function.

Using the relations

$$\mathbb{E}\left\{\right|e_a{\left(n\right)|}^2\}={\sigma }_x^2{\mathrm{MSD},\mathbb{E}\left\{\right|e\left(n\right)|}^2\}\approx {\sigma }_x^2\mathrm{MSD}+{\sigma }_{\alpha }^2,$$

(38)

the steady-state MSD is finally obtained as

$$\mathrm{MSD}={\displaystyle \frac{\mu {\phi }_c{\eta }_{max}{\sigma }_{\alpha }^2\left(\mu {\phi }_c{\eta }_{max}+\frac{2Q}{\sqrt{\pi }}(1-P_r)\right)}{{\sigma }_x^2\left[\frac{4Q^2}{\pi }{(1-P_r)}^2-{\mu }^2{\phi }_c^2{\eta }_{max}^2\right]}}.$$

(39)

The above expression provides an accurate approximation of the steady-state MSD.

3.2.2. Theoretical Validation

To verify the accuracy of the theoretical steady-state MSD model of the proposed $\alpha$-CSAP algorithm, the steady-state MSD expression (39) derived from the theoretical analysis was validated through simulations under a projection order of $Q=2$, with the step-size parameter $\mu$ varying within the range $\mu \in [0.25,0.35]$. The experimental setup was as follows: the filter length $L_w$ was set to three values, 16, 24, and 32, each matching the order of the unknown system; the number of iterations was $N=1000$, which was sufficient for the algorithm to reach steady state; and the results were averaged over 100 Monte Carlo trials. The input signal was a normalized complex-valued white noise, and the noise environment was modeled as Bernoulli–Gaussian mixture noise with an impulse probability of $0.02$. Figure 2, Figure 3 and Figure 4 show the influence of the step-size parameter $\mu$ on the steady-state MSD for different filter lengths, with the projection order fixed at $Q=2$.

When $L_w=16$ (Figure 2), the theoretical MSD curve is slightly higher than the simulation data points, with an average error of 1.61 dB. As the filter length increases to $L_w=24$ (Figure 3), the theoretical and simulation results are in excellent agreement, verifying the accuracy of the derived expression under this configuration. When the filter length is further increased to $L_w=32$ (Figure 4), the predicted MSD values are slightly lower than the simulation results, with an average error of 1.35 dB. The slight deviations between the theoretical model and simulation results mainly originate from the necessary mathematical approximations adopted in the analysis, including the use of the separation assumption to handle the correlation of nonlinear terms, the steady-state averaging of the weighting function, and the approximate assumptions when applying Price’s theorem in the complex domain. Although these simplifications introduce systematic errors, they ensure the analytical tractability of the theoretical expressions. Experimental results demonstrate that the proposed theoretical model still accurately captures the overall trend of the steady-state MSD with respect to the step size, confirming the effectiveness of the analytical framework.

3.3. Computational Complexity Analysis

In this section, we conduct a comprehensive computational complexity analysis of the proposed $\alpha$-CSAP algorithm. The operation counting baseline follows complex arithmetic rules: complex addition requires two real additions, and complex multiplication requires four real multiplications and two real additions. For a length-n complex inner product, $4n-2$ real additions are required. The exponential function $exp(·)$, approximated using a second-order Taylor expansion, is estimated to need five multiplications and three additions.

Table 1. Computational complexity of $\alpha$-CSAP algorithm (per iteration).

Step	×	+	$exp(·)$	÷
Initialization: $\mathbf{w}\left(n\right)=\mathbf{0}$
1. $\mathbf{e}\left(n\right)=\mathbf{d}\left(n\right)-{\mathbf{X}}^H\left(n\right)\mathbf{w}(n-1)$	$4Q{L}_w$	$4Q{L}_w-2Q$	0	0
2. $|e_k{\left(n\right)|}^2$, $k=1,\dots ,Q$	$2Q$	Q	0	0
3. $exp(-|e_k\left(n\right){|}^2/\left(2{\delta }^2\right))$	$5Q$	$3Q$	Q	0
4. ${\eta }_i\left(n\right)=\frac{exp(·)}{{\delta }^2{(1+exp(·))}^{1/\alpha }}$	$4Q$	$2Q$	0	Q
5. $\mathbf{X}\left(n\right)\mathbf{e}\left(n\right)$	$4Q{L}_w$	$4Q{L}_w-2Q$	0	0
6. $\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)$	$4Q^2+4Q{L}_w$	$2Q^2+4Q{L}_w-2Q$	0	0
7. ${\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2$	$2Q+4$	$Q+1$	0	0
8. $\mathbf{w}\left(n\right)=\mathbf{w}(n-1)+\mu \frac{\mathbf{X}\left(n\right)\eta \left(n\right){\mathbf{e}}^{\ast }\left(n\right)}{{\parallel \mathbf{X}\left(n\right)\mathbf{e}\left(n\right)\parallel }_2}$	$4Q{L}_w+2$	$4Q{L}_w+1$	0	1
Total	$8Q{L}_w+4Q^2+13Q+6$	$8Q{L}_w+2Q^2+12Q-1$	Q	$Q+1$

details the computational complexity of the $\alpha$-CSAP algorithm per iteration, where $L_w$ is the filter length and Q is the projection order.

Table 2. Computational Comparison of Different Complex Adaptive Algorithms.

Algorithm	×	+	$exp(·)$	÷
CLMS	$8L_w+4$	$8L_w-2$	0	0
CNLMS	$12L_w+8$	$12L_w-5$	0	1
CMCC	$8L_w+26$	$8L_w+12$	1	0
CAP	$8Q{L}_w+4Q^2+4Q^3$	$8Q{L}_w+4Q^2+4Q^3-6Q$	0	0
$\alpha$-CSAP	$8Q{L}_w+4Q^2+13Q+6$	$8Q{L}_w+2Q^2+12Q-1$	Q	$Q+1$

compares the computational complexity of different complex adaptive filtering algorithms. The algorithms under comparison include the Complex Least Mean Square (CLMS), the Complex Norm Least Mean Squares (CNLMS), the Complex Maximum Correntropy Criterion (CMCC), and the Complex Affine Projection (CAP) algorithms.

In terms of optimization potential, by employing fast recursive filtering (FRF) and an approximation of $exp(·)$ requiring only three multiplications, the complexity of $\alpha$-CSAP can be reduced to $6Q{L}_w+3Q^2+8Q$ multiplications and $6Q{L}_w+Q^2+6Q$ additions.

The proposed $\alpha$-CSAP algorithm provides a good trade-off between computational complexity and performance. Although it requires more operations than LMS-type algorithms ($O\left(L_w\right)$), it avoids the $O\left(Q^3\right)$ matrix inversion of CAP and remains robust to impulsive noise and colored signals. In engineering applications, $L_w$ is often much larger than Q, so the complexity of the $\alpha$-CSAP algorithm is dominated by the $O\left(Q{L}_w\right)$ term, making it suitable for real-time implementation.

4. Simulation

4.1. System Identification Application

This section evaluates the system identification performance of the proposed $\alpha$-CSAP algorithm under various complex-valued non-Gaussian noise conditions, in comparison with several representative complex-valued adaptive filtering algorithms, including CLMS, CNLMS, CAP, and CMCC. The unknown system ${\mathbf{w}}_o$ to be identified is represented by a complex impulse response vector of length $L_w=16$, whose real and imaginary parts are independently drawn from a Gaussian distribution with zero mean and a variance of 0.5. The input signal $x\left(n\right)$ is a first-order complex autoregressive (AR(1)) process, generated by filtering complex-valued white Gaussian noise through a system function $H\left(z\right)=1/(1-0.9z^{-1})$, resulting in strong temporal correlation. The projection order is set to $Q=2$. For a fair comparison, all algorithmic parameters are adjusted to achieve the same initial convergence speed. Simulation results are averaged over 100 Monte Carlo trials, and the MSD learning curve is adopted as the performance metric. The MSD at the n-th iteration is calculated as follows:

$$\mathrm{MSD}\left(n\right)=10{log}_{10}\left(\parallel \mathbf{w}\left(n\right)-{\mathbf{w}}_0{\parallel }^2\right).$$

(40)

To thoroughly assess the algorithm’s robustness, the following three distinct and representative models for complex-domain non-Gaussian noise are employed:

1.

Impulsive Mixture Noise with Bernoulli–Gaussian Structure: The composite noise $z\left(n\right)$ is defined as $z\left(n\right)=A\left(n\right)z_1\left(n\right)+B\left(n\right)z_2\left(n\right)$. Here, $z_1\left(n\right)\sim \mathcal{CN}(0,{\sigma }_g^2)$ denotes the background complex Gaussian noise of variance ${\sigma }_g^2$. The impulsive element is represented by $z_2\left(n\right)\sim \mathcal{CN}(0,{\sigma }_i^2/Pr)$, having a larger variance ${\sigma }_i^2/Pr$. A Bernoulli random variable $A\left(n\right)$, distributed as $\mathrm{Bernoulli}(1-Pr)$ with $Pr=0.001$, governs the occurrence of impulses: $A\left(n\right)=1$ signifies no impulse, while its complement $B\left(n\right)=1-A\left(n\right)$ indicates an impulse event. All variables $A\left(n\right)$, $z_1\left(n\right)$, and $z_2\left(n\right)$ are mutually independent.

2.

Contaminated Complex Gaussian Noise (CG Noise): This noise model is synthesized by summing two independent zero-mean complex Gaussian components. The background component has a variance of 0.008 for the real part and 0.002 for the imaginary part. An intermittent impulsive component, with a variance of 8 for the real part and 2 for the imaginary part, is superimposed with an occurrence probability of 0.01. The combined signal exhibits a heavy-tailed distribution characteristic due to the occasional high-power impulses.

3.

Symmetric Complex $\alpha$-Stable Noise ($\alpha$-Stable Noise): This model adopts a complex-valued symmetric $\alpha$-stable distribution, formulated by $z\left(n\right)=z_R\left(n\right)+j{z}_I\left(n\right)$. The real component $z_R\left(n\right)$ and imaginary component $z_I\left(n\right)$ are generated as independent and identically distributed real $\alpha$-stable random variables, described by the distribution $z_{\alpha }({\alpha }_z,{\beta }_z,\nu ,\xi )$. The configuration employs the following: characteristic exponent ${\alpha }_z=1.9$, symmetry parameter ${\beta }_z=0$, scale parameter $\nu =1$, and location parameter $\xi =0$.

Figure 5, Figure 6 and Figure 7 illustrate the MSD convergence curves of various algorithms under three typical non-Gaussian noise environments. As can be seen, under the adverse conditions of strongly correlated input signals accompanied by non-Gaussian noise, the conventional CLMS algorithm exhibits a clear divergent trend. In contrast, the CNLMS algorithm effectively overcomes the divergence issue of LMS caused by step-size mismatch in non-stationary or impulsive signals through input signal power normalization of the step size, thereby maintaining system stability. However, the CAP algorithm suffers from a significantly degraded convergence performance under the influence of non-Gaussian noise. Furthermore, even when compared with robust algorithms specifically designed to suppress impulsive interference, the proposed $\alpha$-CSAP algorithm still achieves the lowest steady-state MSD across all three noise scenarios. This is primarily attributed to the synergistic design of the $\alpha$-sigmoid cost function and the ${\ell }_2$-norm constraint adopted in the $\alpha$-CSAP algorithm, which endows it with stronger anti-interference capability. In summary, the comparative results demonstrate that the proposed $\alpha$-CSAP algorithm exhibits superior robustness and a more accurate steady-state estimation performance when handling strongly correlated input signals and complex non-Gaussian noise.

4.2. Beamforming Application

This section evaluates the beamforming performance of the proposed $\alpha$-CSAP algorithm in impulsive noise environments. A hexagonal sensor array composed of 91 elements is employed, with an array radius set to three times the operating wavelength and an inter-element spacing of approximately half a wavelength. The operating frequency is 8 GHz, corresponding to a free-space wavelength of $\lambda =37.5$ mm. Five independent narrowband signals with Quadrature Phase Shift Keying (QPSK) modulation are simulated, including one desired signal and four strong interferences. The direction of arrival (DOA) of the desired signal is at an azimuth of ${90}^{\circ }$ and an elevation of ${45}^{\circ }$. The DOAs of the four interfering signals are $({20}^{\circ },{45}^{\circ })$, $({62}^{\circ },{45}^{\circ })$, $({120}^{\circ },{45}^{\circ })$, and $({145}^{\circ },{45}^{\circ })$, respectively. Herein, the azimuth angle $\theta$ is defined clockwise with respect to the array normal, and the elevation angle $\phi$ denotes the angle between the incident signal and the array plane.

To simulate a realistic strong interference environment, high-intensity impulsive noise is introduced, which is modeled by a Bernoulli–Gaussian mixture distribution. The impulse occurrence probability is set to $0.05$, and the variances of the corresponding Gaussian components are ${\sigma }_1^2=2$ and ${\sigma }_2^2=40$, respectively. The Signal-to-Noise Ratio (SNR) and Interference-to-Noise Ratio (INR) are set to 20 dB and 40 dB, respectively.

The core parameters of the $\alpha$-CSAP algorithm are systematically tuned, as follows: the adaptive filter length $L_w$ equals the number of array elements (i.e., 91), the projection order is $Q=4$, and the step size is set to $\mu =0.0001$. Furthermore, the characteristic exponent $\alpha$ and the scale parameter $\delta$ of the algorithm are configured as $1.5$ and 16, respectively. This configuration significantly enhances the stability and robustness of the algorithm under impulsive noise. For a fair comparison, the step sizes of the conventional algorithms are also adjusted accordingly, as follows: ${\mu }_{\mathrm{CLMS}}=0.001$ for the CLMS algorithm, ${\mu }_{\mathrm{CAP}}=0.0005$ for the CAP algorithm, and ${\mu }_{\mathrm{CMCC}}=0.0003$ for the CMCC algorithm. This ensures that all algorithms exhibit similar initial convergence speeds under identical simulation conditions.

As shown in the beam pattern results in Figure 8, while maintaining the same mainlobe gain, the $\alpha$-CSAP algorithm achieves deeper nulls at the four interference directions of ${20}^{\circ }$, ${62}^{\circ }$, ${120}^{\circ }$, and ${145}^{\circ }$, with null depths exceeding those of the other compared algorithms.

We consider the signal-to-interference-plus-noise ratio (SINR) as a metric to evaluate the performance of the beamforming system, which represents the ratio of the desired signal power to the interference-plus-noise power. As shown in

Table 3. Beampattern gain values and SINR performance comparison in impulsive noise environment.

Algorithm	Final SINR (dB)	SINR Improvement (dB)	Interference Direction (dB)
			20°	62°	120°	145°
$\alpha$-CSAP	34.72	+61.02	−80.95	−75.53	−75.76	−72.94
CLMS	18.72	+45.02	−43.52	−41.05	−43.10	−39.92
CAP	11.44	+37.74	−36.14	−35.33	−39.49	−38.04
CMCC	12.27	+38.57	−21.15	−21.73	−22.72	−20.28

Input SINR = −26.30 dB. The beampattern gain in the desired direction (90°) is normalized to 0.00 dB.

, under the harsh condition of an input SINR as low as $-26.30$ dB, the $\alpha$-CSAP algorithm elevates the output SINR to $34.72$ dB, achieving a remarkable improvement of $61.02$ dB. This performance enhancement significantly surpasses that of the other compared algorithms. These results fully demonstrate the superior capability of the $\alpha$-CSAP algorithm in interference suppression and overall beamforming performance.

5. Conclusions

This paper proposes a robust complex $\alpha$-CSAP suitable for non-Gaussian noise environments. By extending the affine projection algorithm based on the $\alpha$-Sigmoid cost function to the complex domain, the algorithm enhances robustness against non-Gaussian noise. The computational complexity is reduced by employing a normalization factor to avoid matrix inversion. In two typical application scenarios, system identification and beamforming, the proposed $\alpha$-CSAP algorithm demonstrates superior convergence characteristics and lower steady-state error compared to classical complex adaptive filtering algorithms such as CLMS, CAP, and CMCC. The results validate its effectiveness and engineering potential in complex-domain signal processing.