Complex-Valued FastICA Estimator with a Weighted Unitary Constraint: A Robust and Equivariant Estimator

Abstract

Independent component analysis (ICA), as a statistical and computational approach, has been successfully applied to digital signal processing. Performance analysis for the ICA approach is perceived as a challenging task to work on. This contribution concerns the complex-valued FastICA algorithm in the range of ICA over the complex number domain. The focus is on the robust and equivariant behavior analysis of the complex-valued FastICA estimator. Although the complex-valued FastICA algorithm as well as its derivatives have been widely used methods for approaching the complex blind signal separation problem, rigorous mathematical treatments of the robust measurement and equivariance for the complex-valued FastICA estimator are still missing. This paper strictly analyzes the robustness against outliers and separation performance depending on the global system. We begin with defining the influence function (IF) of complex-valued FastICA functional and followed by deriving its closed-form expression. Then, we prove that the complex-valued FastICA algorithm based on the optimizing cost function is linear-equivariant, depending only on the source signals.

1. Introduction

Independent component analysis (ICA) is a statistical and computational model, where the observed signals are considered as linear mixtures of underlying source signals. The purpose of ICA is to estimate mutually independent source signals from their linear mixtures without prior knowledge of sources and mixing coefficients [1,2]. Up to the present, ICA has been widely applied in diverse fields, such as signal processing [3,4], financial analysis [5,6], and so on. Hence, there is ever-increasing literature recognizing the essentiality of analyzing the properties of the employed ICA approaches from the aspect of theoretical support. The main purpose of this paper is to analyze the robustness and equivariance of the complex-valued fast fixed-point algorithm for ICA (complex-valued FastICA for short) [7,8], which is one of the most prominent algorithms in the complex number domain due to its faster convergence and easier implementation.

Among many publications, algorithmic property is one of the most fundamental properties for complex-valued ICA. Novey and Adali proposed complex ICA based on negentropy maximization (NM) [9] and provided the local stability condition of the algorithm. Furthermore, Qian and Wei gave a stability analysis from a unique perspective [10]. The authors pointed out that the NM-based ICA algorithm might iterate a poor separation vector even if the source signals meet the stability conditions. Reference [11] presented the Riemann and Lie structures of the complex unitary group, which generalized the topological properties of the complex-valued ICA. Koldovský and Tichavský [12] addressed the problem regarding the region of convergence of gradient-related ICA algorithms. The results showed that the size of the region of convergence was related to the employed algorithm and relied on the ratio of scales of source signals. E et al. [13] provided a performance analysis of the complex-valued FastICA algorithm with the M-estimator cost function, including stability and local convergence. They proved the existence of local optimal solutions and stability conditions.

Moreover, the statistical property is another fundamental property for the complex-valued ICA. Cramér-Rao bound (CRB) is essential for the ICA algorithms to describe the performance limit, and has been researched, e.g., in [14,15]; the authors derived a closed-form expression for the CRB of the separation parameter for complex-valued ICA. Fu et al. [16] established the theory for complex-valued ICA, providing the Cramér-Rao lower bound and identification conditions, which exploited diversities of non-Gaussianity, non-whiteness, and non-circularity. Furthermore, Koldovský et al. [17] analyzed the accuracy of fast dynamic independent vector analysis, showing asymptotic efficiency under given mixing and statistical models, which coincided with the Cramér-Rao lower bound derived in [15]. Reference [13] also analyzed the statistical properties of the complex-valued FastICA method, including uniformity and robustness (the sequences of the complex-valued FastICA estimator converge in probability to the true demixing vector). However, unfortunately, they addressed the problem concerning the robust property without a rigorous mathematical treatment. In this paper, we focus on deriving a closed-form expression of the robust measurement for the complex-valued FastICA functional.

The circular complex-valued FastICA (c-FastICA) algorithm was introduced by Bingham and Hyvärinen [7], as an extension of FastICA in the real domain [18,19]. It has received wide attention for solving complex-valued source signal separation due to its faster convergence and easier implementation. In order to improve the robustness against outliers, Chao and Douglas [20] selected the Huber M-estimator as nonlinearity in the cost function within the complex-circular FastICA algorithm. The local stability analysis showed that the improved approach was locally stable for Huber’s single-parameter M-estimator cost function with circular source signals. The aforementioned complex-valued FastICA algorithm, however, has poor performance when dealing with noncircular source signal separation. To address this obstacle, Novey and Adali extended it to the noncircular source signals separation scenario and analyzed the local convergence of the estimator [8]. Although there have been several attempts to study the statistical properties of the complex-noncircular FastICA algorithm (nc-FastICA for short), a rigorous analysis of the robust and equivariant behavior of nc-FastICA is still missing in the community. To fill this gap, this paper will analyze the robustness against outliers and the separation performance of the complex-valued FastICA estimator with a weighted unitary constraint.

The innovations of this paper are three-fold. First, we define the complex-valued FastICA functional and its influence function (IF) for the deflationary procedure. Then, a closed-form expression of the IF for the complex-valued FastICA functional is derived, which is utilized to measure robustness against outliers from a rigorous mathematical perspective. Third, we prove that the complex-valued FastICA algorithm is equivariant.

This paper is organized as follows: Section 2 provides the preliminaries of the complex-valued ICA model and the deflationary complex FastICA considered argument. Section 3 establishes the theories concerning the complex-valued FastICA functional and its influence function in the range of ICA over the complex number domain. Section 4 analyzes the equivariance of the complex FastICA estimator. Section 5 concludes the whole paper.

2. Preliminaries

2.1. Complex-Valued ICA Model

Assume that a complex-valued random vector $\mathbf{x}\in {\mathbb{C}}^n$ consists of two real-valued random vectors ${\mathbf{x}}^R,{\mathbf{x}}^I\in {\mathbb{R}}^n$ with imaginary unit j, $\mathbf{x}={\mathbf{x}}^R+j{\mathbf{x}}^I$, and the conjugate of $\mathbf{x}$ can be defined by ${\mathbf{x}}^{\ast }={\mathbf{x}}^R-j{\mathbf{x}}^I$. We denote by

$${\mathbf{m}}_{\mathbf{x}}=E\left\{\mathbf{x}\right\},$$

$$C_{\mathbf{x}}=E\left\{(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}}){(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}})}^H\right\},$$

$${\tilde{C}}_{\mathbf{x}}=E\left\{(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}}){(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}})}^T\right\},$$

the mean vector, covariance matrix, and complementary covariance matrix of $\mathbf{x}$, respectively, wherein $E\left\{\mathbf{x}\right\}=E\left\{{\mathbf{x}}^R\right\}+jE\left\{{\mathbf{x}}^I\right\}$, H denotes the conjugate transpose, and T represents the ordinary transpose. The complex-valued random vector $\mathbf{x}$ is circular if ${\tilde{C}}_{\mathbf{x}}=\mathbf{0}$, and otherwise noncircular.

Let $f(\mathbf{x},{\mathbf{x}}^{\ast })$ be an analytic function of a complex random vector $\mathbf{x}$ and its conjugate ${\mathbf{x}}^{\ast }$. In this paper, we utilize the Wirtinger calculus to obtain the partial derivatives with respect to $\mathbf{x}$ and ${\mathbf{x}}^{\ast }$, considering $\mathbf{x}$ and ${\mathbf{x}}^{\ast }$ as two variables independently [21].

The ICA model established in the complex number field can be considered an extension of a real-valued ICA model [2], which assumes that the observed signals are the linear combinations of the unknown source signals $\mathbf{s}={(s_1,s_2,\cdots ,s_n)}^T$, resulting in the generation of the observed signals vector $\mathbf{x}={(x_1,x_2,\cdots ,x_n)}^T$ according to

$$\mathbf{x}=\mathbf{As}=\sum _{i=1}^n{\mathbf{a}}_i{s}_i,$$

(1)

where $\mathbf{A}\in {\mathbb{C}}^{n\times n}$ is the unknown mixing matrix or coefficients matrix, which regulates the cumulative distribution function (cdf) of the observed signal $\mathbf{x}$ denoted by $F\left(\mathbf{x}\right)$, and ${\mathbf{a}}_i$ is the ith column of matrix $\mathbf{A}$. Generally, the following assumptions are needed in the ICA model:

Assumption 1. 

The source signals are of zero mean and unit variance, i.e., $C_{\mathbf{s}}=\mathbf{I}$ and the sources $s_1,s_2,\cdots ,s_n$ are statistically mutually independent, and at most, one source is Gaussian.

Assumption 2. 

The mixing matrix $\mathbf{A}$ is of full rank.

The primary aim of solving the ICA problem is to procure a matrix $\mathbf{B}$ named the demixing matrix, which can be utilized to estimate the source signals $\mathbf{s}$ by $\mathbf{y}={(y_1,y_2,\cdots ,y_n)}^T$ defined as follows:

$$\mathbf{y}=\mathbf{Bx}=\mathbf{BAs}=\mathbf{P}\mathrm{\Lambda }\mathbf{s},$$

(2)

where $\mathbf{P}$ and $\mathrm{\Lambda }$ are the permutation and diagonal matrices, respectively, representing the ambiguities of the ICA model (1). This can be obtained by the non-Gaussian-maximization criterion [2] up to the scales, phase, and order of source signals.

The data-centering procedure is needed when solving the ICA problem, as it simplifies the theory and algorithm. Considering the zero-mean source signals $\mathbf{s}$, which can be achieved by subtracting the mean vector ${\mathbf{m}}_{\mathbf{x}}$ from $\mathbf{x}$, we can similarly define their covariance matrix $C_{\mathbf{s}}$ and complementary covariance matrix ${\tilde{C}}_{\mathbf{s}}$. Especially, the source signal is circular if ${\tilde{C}}_{\mathbf{s}}=\mathbf{0}$, otherwise, it is noncircular. In this study, we analyze the robustness and equivariant characteristics of symmetric complex-valued FastICA under the more general case of noncircular sources.

2.2. Deflationary Complex FastICA

To date, various approaches have been developed for solving the complex ICA problem, with the complex-valued FastICA algorithm being one of the most prominent for finding the demixing matrix $\mathbf{B}$.

Let $\mathbf{b}$ be one of the columns of the demixing matrix $\mathbf{B}$, regarding a demixing vector. On the basis of Assumption 1, the estimated source signal $y={\mathbf{b}}^H\mathbf{x}$ should be of unit variance by searching for the solution on the weighted unitary constraint group $O\left(n\right)=\{\mathbf{b}\in {\mathbb{C}}^n:|\left|\mathbf{b}\right|{|}_{{\mathbf{C}}_{\mathbf{x}}}=1\}$ with a weighted vector norm, such that ${\left|\right|\mathbf{b}\left|\right|}_{{\mathbf{C}}_{\mathbf{x}}}^2={\mathbf{b}}^H{\mathbf{C}}_{\mathbf{x}}\mathbf{b}$. Hence, the cost function of the complex-valued FastICA algorithm with the weighted unitary constraint for the one-unit version has the following form:

$$J\left(\mathbf{b}\right)=E\left\{G\right(|{\mathbf{b}}^H{\mathbf{x}|}^2\left)\right\},\mathbf{b}\in O\left(n\right),$$

(3)

where $G:{\mathbb{R}}^{+}\bigcup \left\{0\right\}\to \mathbb{R}$ represents a smooth even function defined as the nonlinearity function (which determines the robustness of the complex-valued FastICA algorithm, which will be further involved in the forthcoming Section 3); $\mathbf{x}$ denotes the observed signals vector defined in (1).

We note that (i) the cost function performs on the absolute value $|{\mathbf{b}}^H{\mathbf{x}|}^2$ instead of a complex-valued vector, which results from seeking to maximize the expectation of the non-linear cost function; (ii) the optimization of the weighted unitary constraint group $O\left(n\right)$ guarantees that the estimated source signal $y={\mathbf{b}}^H\mathbf{x}$ has unit variance.

In order to solve the optimization problem (3), Bingham and Hyvärinen [7] presented a fast fixed-point type algorithm (called c-FastICA), which is capable of estimating the circular source signals. However, it has poor performance when dealing with noncircular source separation. To figure out this obstacle, Novey and Adali extended the c-FastICA algorithm to the noncircular source signal separation scenario (called the nc-FastICA algorithm), consisting of the following learning rule [8]:

• Step 1. Choose an arbitrary initial value of the unit norm for $\mathbf{b}\in O\left(n\right)$;
• Step 2. Run iterations:

  $$\begin{array}{c}\mathbf{b}\leftarrow -E\left\{g\right(|{\mathbf{b}}^H{\mathbf{x}|}^2){\left({\mathbf{b}}^H\mathbf{x}\right)}^{\ast }\mathbf{x}\}+E\{g^{\prime }\left(\right|{\mathbf{b}}^H{\mathbf{x}|}^2\left)\right|{\mathbf{b}}^H{\mathbf{x}|}^2\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +g\left(\right|{\mathbf{b}}^H{\mathbf{x}|}^2\left)\right\}\mathbf{b}+E\left\{{\mathbf{xx}}^T\right\}E\{g^{\prime }\left(\right|{\mathbf{b}}^H{\mathbf{x}|}^2\left)({\mathbf{b}}^H\mathbf{x}{)}^{\ast 2}\right\}{\mathbf{b}}^{\ast },\\ \hspace{1em}\hspace{1em} \mathbf{b}\leftarrow \frac{\mathbf{b}}{{\left|\right|\mathbf{b}\left|\right|}_{{\mathbf{C}}_{\mathbf{x}}}^2}\hfill \end{array}$$

  (4)

  until convergence, where the notations $g\left(z\right)$, $g^{\prime }\left(z\right)$ denote $\frac{dG\left(z\right)}{dz}$, and $\frac{dg\left(z\right)}{dz}$, respectively.

One needs to estimate several source signals; the one-unit complex FastICA algorithm (4) should be run several times with vectors ${\mathbf{b}}_1,{\mathbf{b}}_2,\cdots ,{\mathbf{b}}_n$. To prevent the vectors from converging to the same maxima, they need to be orthogonalized after each iteration using the Gram–Schmidt method [2], resulting in the following deflationary complex FastICA algorithm:

• Step 1. Choose an arbitrary initial value of unit norm for ${\mathbf{b}}_j\in O\left(n\right)$; set counter $j=1$;
• Step 2. Run iterations:

  $$\begin{array}{ccc}\hfill {\mathbf{b}}_j\leftarrow & -E\left\{g\right(|{\mathbf{b}}_j^H{\mathbf{x}|}^2){\left({\mathbf{b}}_j^H\mathbf{x}\right)}^{\ast }\mathbf{x}\}+E\{g^{\prime }\left(\right|{\mathbf{b}}_j^H{\mathbf{x}|}^2\left)\right|{\mathbf{b}}_j^H{\mathbf{x}|}^2\hfill & \\ & +g\left(\right|{\mathbf{b}}_j^H{\mathbf{x}|}^2\left)\right\}{\mathbf{b}}_j+E\left\{{\mathbf{xx}}^T\right\}E\{g^{\prime }\left(\right|{\mathbf{b}}_j^H{\mathbf{x}|}^2\left)({\mathbf{b}}_j^H\mathbf{x}{)}^{\ast 2}\right\}{\mathbf{b}}_j^{\ast },\hfill & \\ & {\mathbf{b}}_j\leftarrow {\mathbf{b}}_j-\sum _{i=1}^{j-1}\left({\mathbf{b}}_j^T{\mathbf{b}}_i\right){\mathbf{b}}_i\hfill & \\ & {\mathbf{b}}_j\leftarrow \frac{{\mathbf{b}}_j}{\left|\right|{\mathbf{b}}_j{\left|\right|}_{{\mathbf{C}}_{\mathbf{x}}}^2}\hfill \end{array}$$

  (5)

  until convergence;
• Step 3. Set $j\leftarrow j+1$ until $j=n$.

3. Robustness of the Complex-Valued FastICA Estimator

3.1. Nonlinearity

From a statistical viewpoint, the nonlinear function G provides information on the higher-order statistics in the expectation format $E\left\{G\right(|{\mathbf{b}}^H{\mathbf{x}|}^2\left)\right\}$, which determines the selection of g (the derivation of the nonlinear function G) in the algorithm (4). Hence, the statistical properties of the complex-valued FastICA estimator ${\mathbf{b}}_k$ lie essentially in the selection of nonlinear function G. In this paper, we analyze the robustness of the ${\mathbf{b}}_k$ via the IF.

Generally, robustness against outliers is a desirable property for an estimator, meaning that the estimator is insensitive to individual, highly erroneous observations. In this section, we mainly address the problem of how to measure the robustness of the complex-valued FastICA estimator ${\mathbf{b}}_k$? Heuristically, the value of the function $G\left(x\right)$ cannot grow fast with the increase in $\left|x\right|$ if one needs a robust estimator. Specifically, we list the classical nonlinearities $G\left(x\right)$ in

Table 1. The nonlinearities in the complex-valued FastICA algorithm.

Nonlinear Function $\mathit{G}\left(\mathit{x}\right)$	Derivation of $\mathit{G}\left(\mathit{x}\right)$
$G_{log}=log(1+x^2)$	$g_{log}=\frac{2x}{1+x^2}$
$G_{tanh}=logcosh\left(x\right)$	$g_{tanh}=tanh\left(y\right)$
$G_{kurt}=\frac{1}{2}{x}^4$	$g_{kurt}=2x^3$
$G_{sqrt}=\sqrt{1+x^2}$	$g_{sqrt}=\frac{x}{\sqrt{1+x^2}}$
$G_{Huber}=\left\{\begin{array}{cc}\frac{1}{2}{\left|x\right|}^2,\hfill & \left|x\right|\le \theta ,\hfill \\ \theta \left|x\right|-\frac{{\theta }^2}{2},\hfill & \left|x\right|>\theta .\hfill \end{array}\right.$	$g_{Huber}=\left\{\begin{array}{cc}x,\hfill & \left|x\right|\le \theta ,\hfill \\ \theta sgn\left(x\right),\hfill & \left|x\right|>\theta .\hfill \end{array}\right.$
$G_{Tukey}=\left\{\begin{array}{cc}\frac{{\theta }^2}{6}[1-{(1-\frac{x^2}{{\theta }^2})}^3],\hfill & \left|x\right|\le \theta ,\hfill \\ \frac{{\theta }^2}{6},\hfill & \left|x\right|>\theta .\hfill \end{array}\right.$	$g_{Tukey}=\left\{\begin{array}{cc}x{(1-\frac{x^2}{{\theta }^2})}^2,\hfill & \left|x\right|\le \theta ,\hfill \\ 0,\hfill & \left|x\right|>\theta .\hfill \end{array}\right.$

Note: The functions with subscripts denote the logarithm, hyperbolic tangent, kurtosis, mean square root, Huber M-estimator, and Tukey M-estimator functions, respectively

. The curves of the function $G\left(x\right)$ and its derivation with respect to x are plotted in Figure 1 and Figure 2.

From these Figures, one can yield that the Tukey M-estimator function implements a more robust estimator that is insensitive to outliers, and kurtosis gives a non-robust estimator that may be influenced by individual highly erroneous observations. In fact, the values of $G_{kurt}\left(x\right)$ (red line in Figure 1) are increased quickly from the beginning of 1 or −1 without a downward trajectory. With the increase in $\left|x\right|$, the outliers do not have much influence on the values of $G_{Tukey}\left(x\right)$ in the sense that the complex-valued FastICA estimator based on the Tukey M-estimator cost function is recommended with better robustness. This evidence can also be seen in the curves of the derivation of nonlinearities $G\left(x\right)$ in Figure 2. In this paper, we shall, hereafter, analyze the robustness of the complex-valued FastICA estimator via the IF.

3.2. Influence Function of Complex-Valued FastICA Functional

We analyze the robustness of the complex-valued FastICA estimator by concerning the deflationary version of the complex-valued FastICA algorithm. For convenience, we shall hereafter suppose that ${\mathbf{b}}_k$ is the kth column of the demixing matrix $\mathbf{B}$, which corresponds to the estimator for finding the kth source signal $s_k$ in the sense that the equation $y_k={\mathbf{b}}_k^H\mathbf{x}$ gives an estimation of the source signal $s_k$.

In the complex-valued ICA model $\left(1\right)$, the observed signal vector $\mathbf{x}$ comes from an unknown distribution with the cdf $F\left(\mathbf{x}\right)$. Hence, in order to give the measurement of the complex-valued FastICA estimator, we first define the complex-valued FastICA functional ${\mathbf{b}}_k\left(F\right)$ for the deflationary procedure and its influence function.

Definition 1. 

Assume that the observed vector $\mathbf{x}$ follows the complex-valued ICA model (1). The complex-valued FastICA functional ${\mathbf{b}}_k\left(F\right)$ for the deflationary procedure is defined as follows:

$${\mathbf{b}}_k\left(F\right)=arg\underset{\left|\right|{\mathbf{b}}_k{\left|\right|}_{{\mathbf{C}}_{\mathbf{x}}}}{max}E\left\{G\right(|{\mathbf{b}}_k^H{\mathbf{x}|}^2\left)\right\},$$

(6)

subject to the following weighted unitary constraint:

$${\mathbf{b}}_k{\left(F\right)}^H{C}_{\mathbf{x}}\left(F\right){\mathbf{b}}_j\left(F\right)=\left\{\begin{array}{cc}0,\hfill & j<k,\hfill \\ 1,\hfill & j=k.\hfill \end{array}\right.$$

(7)

where $G:{\mathbb{R}}^{+}\bigcup \left\{0\right\}\to \mathbb{R}$ represents a smooth even function and $C_{\mathbf{x}}\left(F\right)$ denotes the covariance matrix function of the observed vector $\mathbf{x}$ at the distribution $F\left(\mathbf{x}\right)$.

We note that the constraint condition (7) can be performed by the deflationary orthogonalization process, which separates the source signals one by one.

Definition 2. 

The influence function (IF) of the complex-valued FastICA functional ${\mathbf{b}}_k(·)$ at $F\left(\mathbf{x}\right)$ is given by the following:

$$IF(\mathbf{x},\mathbf{b},F\left(\mathbf{x}\right))=\underset{t\to 0}{lim}\frac{{\mathbf{b}}_k((1-t)F\left(\mathbf{x}\right)+t{▵}_{\mathbf{x}})-{\mathbf{b}}_k\left(F\left(\mathbf{x}\right)\right)}{t}$$

(8)

where ${▵}_{\mathbf{x}}$ denotes the probability measure, which puts mass 1 at point $\mathbf{x}\in {\mathbb{C}}^n$.

We note that the IF of the complex-valued FastICA functional ${\mathbf{b}}_k(·)$ is defined based on the fact that it is Gâteaux differentiable [22] at the distribution $F\left(\mathbf{x}\right)$ in ${\mathbb{C}}^n$. Thus, it can also be written as follows:

$$IF(\mathbf{x},\mathbf{b},F\left(\mathbf{x}\right))=\frac{\partial {\mathbf{b}}_k\left(F_t\left(\mathbf{x}\right)\right)}{\partial t}{|}_{t=0},$$

(9)

where $F_t\left(\mathbf{x}\right)=(1-t)F\left(\mathbf{x}\right)+t{▵}_{\mathbf{x}}$.

To simplify the notation, we shall hereafter replace the notations $IF(\mathbf{x},\mathbf{b},F(\mathbf{x}\left)\right)$ with $IF\left(\mathbf{b}\right)$ to denote the IF of the complex-valued FastICA functional ${\mathbf{b}}_k(·)$, ${\mathbf{C}}_{\mathbf{x}}\left(F\right)$ (Res. ${\mathbf{m}}_{\mathbf{x}}\left(F\right)$) by the ${\mathbf{C}}_{\mathbf{x}}$ (Res. ${\mathbf{m}}_{\mathbf{x}}$) to denote the covariance matrix function (Res. mean vector function) of the observed vector $\mathbf{x}$ at the distribution $F\left(\mathbf{x}\right)$, $F_t$ by $F_t\left(\mathbf{x}\right)$, respectively.

The significance of the IF lies in its heuristic explanation: it reports the influence of the contamination at point $\mathbf{x}$ on the estimate. From the robust statistics point of view, the IF quantifies the asymptotic bias resulting from contamination in the observation [22]. Hence, the forthcoming section provides a closed-form expression of the IF of the complex-valued FastICA functional ${\mathbf{b}}_k(·)$.

3.3. Robustness

In order to analyze the robustness, we use the Lagrangian multiplier method to deduce the specific expression of the influence function of the complex-valued FastICA functional. Plugging the weighted unitary constraint (7) into the cost function (3), the Lagrange function can be obtained as follows:

$$L({\mathbf{b}}_k,\lambda )=J\left({\mathbf{b}}_k\right)-{\lambda }_k[{\mathbf{b}}_k^H{\mathbf{C}}_{\mathbf{x}}{\mathbf{b}}_k-1]-\sum _{j=1}^{k-1}{\lambda }_j{\mathbf{b}}_j^H{\mathbf{C}}_{\mathbf{x}}{\mathbf{b}}_k,$$

(10)

where ${\lambda }_j\in \mathbb{R},j=1,\cdots ,k$ is the penalty factor; $J\left({\mathbf{b}}_k\right)$ is the cost function of the complex-valued FastICA. Note that the observed data, $\mathbf{x}$, are concerned without a preprocessing procedure, in the sense that $J\left({\mathbf{b}}_k\right)=E\left\{G\right(|y_{\mathbf{x},k}{|}^2\left)\right\}=E\left\{G\left(y_{\mathbf{x},k}{y}_{\mathbf{x},k}^{\ast }\right)\right\},$ wherein $y_{\mathbf{x},k}={\mathbf{b}}_k^H(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}})={\displaystyle \sum _{i=1}^n}{b}_i^{\ast }(x_i-m_i)$, and $b_i,x_i,m_i$ denote the ith elements for counterparts, respectively. After differentiating the $L({\mathbf{b}}_k,\lambda )$ with respect to ${\mathbf{b}}_k={(b_1,\cdots ,b_n)}^T$ and equating to zero, one can yield

$$\begin{array}{cc}\hfill {\nabla }_{{\mathbf{b}}_k}J\left({\mathbf{b}}_k\right)={\nabla }_{{\mathbf{b}}_k}{\lambda }_k& \left[{\mathbf{b}}_k^H{\mathbf{C}}_{\mathbf{x}}{\mathbf{b}}_k-1\right]+{\nabla }_{{\mathbf{b}}_k}\sum _{j=1}^{k-1}{\lambda }_j{\mathbf{b}}_j^H{\mathbf{C}}_{\mathbf{x}}{\mathbf{b}}_k,\hfill \end{array}$$

(11)

where ${\nabla }_{\mathbf{b}}$ denotes the gradient operator with respect to $\mathbf{b}$, yielding the following:

$$\frac{\partial E\left\{G\left(y_{\mathbf{x},k}{y}_{\mathbf{x},k}^{\ast }\right)\right\}}{\partial b_i}=E\left\{g\left(y_{\mathbf{x},k}{y}_{\mathbf{x},k}^{\ast }\right)y_{\mathbf{x},k}{[x_i-m_i]}^{\ast }\right\},$$

$$\frac{\partial {\lambda }_k{\mathbf{b}}_k^H{\mathbf{C}}_{\mathbf{x}}{\mathbf{b}}_k}{\partial b_i}={\lambda }_k{\mathbf{b}}_k^H{\mathbf{c}}_i,$$

$$\frac{\partial {\lambda }_j{\mathbf{b}}_j^H{\mathbf{C}}_{\mathbf{x}}{\mathbf{b}}_k}{\partial b_i}={\lambda }_j{\mathbf{b}}_j^H{\mathbf{c}}_i,$$

where ${\mathbf{c}}_i$ represents the ith column of the covariance matrix function ${\mathbf{C}}_{\mathbf{x}}\left(F\right)$. Furthermore,

$$E\left\{g\right(|y_{\mathbf{x},k}{|}^2\left)y_{\mathbf{x},k}{[\mathbf{x}-{\mathbf{m}}_{\mathbf{x}}]}^{\ast }\right\}={\lambda }_k{\mathbf{C}}_{\mathbf{x}}^T{\mathbf{b}}_k^{\ast }+\sum _{j=1}^{k-1}{\lambda }_j{\mathbf{C}}_{\mathbf{x}}^T{\mathbf{b}}_j^{\ast }.$$

(12)

For the sake of convenience in writing, we shall identify the following:

$$L_k\left(F\right)\triangleq E\left\{g\right(|y_{\mathbf{x},k}{|}^2\left)y_{\mathbf{x},k}{[\mathbf{x}-{\mathbf{m}}_{\mathbf{x}}]}^{\ast }\right\}$$

(13)

and

$$R_k\left(F\right)\triangleq {\lambda }_k{\mathbf{C}}_{\mathbf{x}}^T{\mathbf{b}}_k^{\ast }+\sum _{j=1}^{k-1}{\lambda }_j{\mathbf{C}}_{\mathbf{x}}^T{\mathbf{b}}_j^{\ast }$$

(14)

respectively, yielding the following:

$$L_k\left(F\right)=R_k\left(F\right).$$

(15)

Theorem 1. 

The influence function of the complex-valued FastICA functional ${\mathbf{b}}_k(·)$, $k\in \{1,2,\cdots ,n\}$, at the centered mixture distribution $F\left(\mathbf{x}\right)$ is as follows:

$$\begin{array}{c}\hfill IF\left({\mathbf{b}}_k\right)=-{\overline{s}}_k^{\ast }\sum _{i=1}^{k-1}[u_i{\left({\overline{s}}_i\right)}^{\ast }+{\overline{s}}_i]{\mathbf{b}}_i+\frac{1-|{\overline{s}}_k{|}^2}{2}{\mathbf{b}}_k+u_k\left({\overline{s}}_k\right)\sum _{i=k+1}^n{\overline{s}}_i{\mathbf{b}}_i\end{array}$$

(16)

where $u_k\left(x\right)=\frac{{\left(g\right(\left|x\right|}^2)-c_k)x^{\ast }-{\rho }_p^{\ast }}{c_k-{\mu }_k}$, $c_k=E\left\{g\right(|s_k{|}^2\left)\right|s_k{|}^2\}$, ${\mu }_k=E\left\{g\right(|s_k{|}^2\left)\right\}$, ${\rho }_k=E\left\{g\right(|s_k{|}^2\left)s_k^2\right\}$, $g(·)$ is the derivative operator of the nonlinearity $G(·)$ in the cost function (3), and ${\overline{s}}_i={\mathbf{b}}_i^H(\mathbf{z}-{\mathbf{m}}_{\mathbf{x}})$ denotes the projection of the centered contamination point $\mathbf{z}$ into the direction ${\mathbf{b}}_i$.

To prove Theorem 1, the following five Lemmas are needed, which will be proved in Appendix A and Appendix B.

Lemma 1. 

Assume that the observed data, $\mathbf{x}$, obey the distribution $F\left(\mathbf{x}\right)$ with the mean vector ${\mathbf{m}}_{\mathbf{x}}$, and the sources $\mathbf{s}$ follow Assumption 1. Let $y_{\mathbf{x},i}\left(F_t\right)={\mathbf{b}}_i^H\left(F_t\right)(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}})$, $L_k\left(F\right)=E\left\{g\right(|y_{\mathbf{x},k}{|}^2\left)y_{\mathbf{x},k}{[\mathbf{x}-{\mathbf{m}}_{\mathbf{x}}]}^{\ast }\right\}$ and $B_k\left(F\right)=E\left\{g\right(|y_{\mathbf{x},k}{|}^2\left)[\mathbf{x}-{\mathbf{m}}_{\mathbf{x}}]{[\mathbf{x}-{\mathbf{m}}_{\mathbf{x}}]}^H\right\}$, one obtains the following:

(a) 

$IF\left(y_{\mathbf{x},i}\right)=IF{\left({\mathbf{b}}_i\right)}^H(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}})-{\overline{s}}_i,$

(b) 

$L_k\left(F\right)=c_k{\mathbf{a}}_k^{\ast }$,

(c) 

$B_k\left(F\right)={\mu }_k{\mathbf{AA}}^H+(c_k-{\mu }_k){\mathbf{a}}_k{\mathbf{a}}_k^H$,

(d) 

${\mathbf{a}}_k^{H}IF\left({\mathbf{b}}_i\right)=-IF{\left({\mathbf{b}}_k\right)}^H{\mathbf{a}}_i-{\overline{s}}_k{\overline{s}}_i^{\ast }$, $i=1,\cdots ,k-1$,

(e) 

$2Re\left({\mathbf{a}}_k^{H}IF\left({\mathbf{b}}_k\right)\right)=1-{\left|{\overline{s}}_k\right|}^2$,

where ${\overline{s}}_i={\mathbf{b}}_i^H(\mathbf{z}-{\mathbf{m}}_{\mathbf{x}}),i=1,\cdots ,n$ denotes the projection of the centered contamination point $\mathbf{z}$ into the direction ${\mathbf{b}}_i$; $c_k=E\left\{g\right(|s_k{|}^2\left)\right|s_k{|}^2\}$, ${\mu }_k=E\left\{g\right(|s_k{|}^2\left)\right\}$. The notation of ${\mathbf{a}}_k$ represents the kth column of matrix $\mathbf{A}$.

Proof. 

See Appendix A. □

Lemma 2. 

Assume that the observed data, $\mathbf{x}$, obey the distribution $F\left(\mathbf{x}\right)$ with the mean vector ${\mathbf{m}}_{\mathbf{x}}$, and the sources $\mathbf{s}$ follow Assumption 1. For ${\lambda }_i,i=1,\cdots ,k$ in (10) at the distribution $F\left(\mathbf{x}\right)$, we have the following:

(a) 

${\lambda }_i\left(F\right)=0,i=1,\cdots ,k-1$,

(b) 

${\lambda }_k\left(F\right)=c_k$,

(c) 

$IF\left({\lambda }_i\right)=(c_k-{\mu }_k){\mathbf{a}}_k^{H}IF\left({\mathbf{b}}_i\right)+\left(g\left(\right|{\overline{s}}_k{|}^2)-{\mu }_k\right){\overline{s}}_k{\overline{s}}_i^{\ast }-{\rho }_k{s}_i^{\ast },i=1,\cdots ,k-1$,

(d) 

$IF\left({\lambda }_k\right)=\left(g\left(\right|{\overline{s}}_k{|}^2)-c_k\right)|{\overline{s}}_k{|}^2+E\{g^{\prime }(|{\overline{s}}_k{|}^2)v_k\left({\overline{s}}_k\right)|{\overline{s}}_k{|}^2\}-2Re\left(E\left\{g\right(|{\overline{s}}_k{|}^2\left){\overline{s}}_k{s}_k^{\ast }\right\}\right)$,

where $v_k\left({\overline{s}}_k\right)=|s_k{|}^2-|s_k{|}^2{\left|{\overline{s}}_k\right|}^2-2Re\left({\overline{s}}_k{s}_k^{\ast }\right)$, and $c_k$, ${\mu }_k$, ${\overline{s}}_i$, ${\rho }_k$ are defined as in Theorem 1.

Proof. 

See Appendix B. □

Based on Lemmas 1 and 2, we will now prove Theorem 1 from the following three steps:

Proof. 

Step 1. Differentiating $L_k\left(F_t\right)$ and $R_k\left(F_t\right)$ with respect to $t=0$ in (15), yields the following equation:

$$\frac{\partial L_k\left(F_t\right)}{\partial t}{|}_{t=0}=\frac{\partial R_k\left(F_t\right)}{\partial t}{|}_{t=0},$$

(17)

which aims at obtaining a closed-form expression of the influence function of the complex-valued FastICA functional $IF\left({\mathbf{b}}_k\right)$.

Step 2. Calculating the left-hand side of the Equation (17). Combing $F_t\left(\mathbf{x}\right)=(1-t)F\left(\mathbf{x}\right)+t{▵}_{\mathbf{x}}$ with Equation (13), we conclude the following:

$$\begin{array}{c}\hfill L_k\left(F_t\right)=(1-t)E\left\{g\right(|y_{\mathbf{x},k}{|}^2)y_{\mathbf{x},k}{[\mathbf{x}-{\mathbf{m}}_{\mathbf{x}}]}^{\ast }\}+tg(|y_{\mathbf{z},k}{|}^2)y_{\mathbf{z},k}{[\mathbf{z}-{\mathbf{m}}_{\mathbf{x}}]}^{\ast },\end{array}$$

(18)

Differentiating $L_k\left(F_t\right)$ with respect to $t=0$ and leveraging Lemma 1, one can obtain the following:

$$\begin{array}{ccc}\hfill \frac{\partial L_k\left(F_t\right)}{\partial t}{|}_{t=0}=& -L_k\left(F\right)+(1-t)\left[E\right\{g^{\prime }\left(\right|y_{\mathbf{x},k}{|}^2\left)\left(IF\left(y_{\mathbf{x},k}\right)y_{\mathbf{x},k}^{\ast }+y_{\mathbf{x},k}IF{\left(y_{\mathbf{x},k}\right)}^{\ast }\right)y_{\mathbf{x},k}{(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}})}^{\ast }\right\}\hfill & \\ & +E\left\{g\right(|y_{\mathbf{x},k}{|}^2)IF\left(y_{\mathbf{x},k}\right){(\mathbf{x}-{\mathbf{m}}_{\mathbf{x}})}^{\ast }\}-E\{g\left(\right|y_{\mathbf{x},k}{|}^2\left)y_{\mathbf{x},k}IF{\left({\mathbf{m}}_{\mathbf{x}}\right)}^{\ast }\right\}]{|}_{t=0}\hfill & \\ & +g\left(\right|y_{\mathbf{z},k}{|}^2)y_{\mathbf{z},k}{(\mathbf{z}-{\mathbf{m}}_{\mathbf{x}})}^{\ast }{|}_{t=0}\hfill & \\ & =-c_k{\mathbf{a}}_k^{\ast }+E\{g^{\prime }\left(\right|s_k{|}^2\left)\right|s_k{|}^2{v}_k\left({\overline{s}}_k\right)\}{\mathbf{a}}_k^{\ast }+{\left[{\mu }_k{\mathbf{AA}}^H+(c_k-{\mu }_k){\mathbf{a}}_k{\mathbf{a}}_k^H\right]}^{\ast }IF{\left({\mathbf{b}}_k\right)}^{\ast }\hfill & \\ & -2Re\left(E\left\{g\right(|s_k{|}^2\left){\overline{s}}_k{s}_k^{\ast }\right\}\right){\mathbf{a}}_k^{\ast }+{\rho }_k{\overline{s}}_k^{\ast }{\mathbf{a}}_k^{\ast }+\left[g\left(\right|{\overline{s}}_k{|}^2){\overline{s}}_k-{\rho }_k\right]{(\mathbf{z}-{\mathbf{m}}_{\mathbf{x}})}^{\ast }.\hfill \end{array}$$

Step 3. Calculating the right-hand side of (17). From (14), we have the following:

$$\begin{array}{c}\hfill R_k\left(F_t\right)={\lambda }_k\left(F_t\right){\mathbf{C}}_{\mathbf{x}}^T{\mathbf{b}}_k^{\ast }\left(F_t\right)+\sum _{j=1}^{k-1}{\lambda }_j\left(F_t\right){\mathbf{C}}_{\mathbf{x}}^T{\mathbf{b}}_j^{\ast }\left(F_t\right),\end{array}$$

(19)

Differentiating $R_k\left(F_t\right)$ with respect to $t=0$ and leveraging Lemma 2 one can obtain the following:

$$\begin{array}{ccc}\hfill \frac{\partial R_k\left(F_t\right)}{\partial t}{|}_{t=0}=& IF\left({\lambda }_k\right){\left({\mathbf{AA}}^H\right)}^T{\mathbf{b}}_k^{\ast }+{\lambda }_k\left(F\right)IF{\left({\mathbf{C}}_{\mathbf{x}}\right)}^T{\mathbf{b}}_k^{\ast }+{\lambda }_k\left(F\right){\left({\mathbf{AA}}^H\right)}^{T}IF{\left({\mathbf{b}}_k\right)}^{\ast }\hfill & \\ & +\sum _{j=1}^{k-1}\left[IF\left({\lambda }_j\right){\left({\mathbf{AA}}^H\right)}^T{\mathbf{b}}_j^{\ast }+{\lambda }_j\left(F\right)IF{\left({\mathbf{C}}_{\mathbf{x}}\right)}^T{\mathbf{b}}_j^{\ast }+{\lambda }_j\left(F\right){\left({\mathbf{AA}}^H\right)}^{T}IF{\left({\mathbf{b}}_j\right)}^{\ast }\right]\hfill & \\ & =IF\left({\lambda }_k\right){\mathbf{a}}_k^{\ast }+c_k\left[{(\mathbf{z}-{\mathbf{m}}_{\mathbf{x}})}^{\ast }{\overline{s}}_k-{\mathbf{a}}_k^{\ast }\right]+c_k{\left({\mathbf{AA}}^H\right)}^{T}IF{\left({\mathbf{b}}_k\right)}^{\ast }+\sum _{j=1}^{k-1}IF\left({\lambda }_j\right){\mathbf{a}}_j^{\ast }\hfill & \\ & =\left[g\left(\right|{\overline{s}}_k{|}^2\left)\right|{\overline{s}}_k{|}^2-c_k{\left|{\overline{s}}_k\right|}^2\right]{\mathbf{a}}_k^{\ast }-c_k{\mathbf{a}}_k^{\ast }+c_k{(\mathbf{z}-{\mathbf{m}}_{\mathbf{x}})}^{\ast }{\overline{s}}_k+c_k{\left({\mathbf{AA}}^H\right)}^{T}IF{\left({\mathbf{b}}_k\right)}^{\ast }\hfill & \\ & +E\{g^{\prime }\left(\right|s_k{|}^2\left)\right|s_k{|}^2{v}_k\left({\overline{s}}_k\right)\}{\mathbf{a}}_k^{\ast }-2Re\left(E\left\{g\right(|s_k{|}^2\left){\overline{s}}_k{s}_k^{\ast }\right\}\right){\mathbf{a}}_k^{\ast }-{\rho }_k\sum _{j=1}^{k-1}{\overline{s}}_j^{\ast }{\mathbf{a}}_j^{\ast }\hfill & \\ & +(c_k-{\mu }_k)\sum _{j=1}^{k-1}{\mathbf{a}}_k^{H}IF\left({\mathbf{b}}_j\right){\mathbf{a}}_j^{\ast }+\left[g\right(|{\overline{s}}_k{|}^2)-{\mu }_k]{\overline{s}}_k\sum _{j=1}^{k-1}{\overline{s}}_j^{\ast }{\mathbf{a}}_j^{\ast }.\hfill \end{array}$$

Step 4. Deriving the specific expression of the IF of the complex-valued FastICA function. Plugging the results derived in step 1 and step 2 into (17), after tedious manipulation, one can yield the following:

$$\begin{array}{ccc}\hfill IF\left({\mathbf{b}}_k\right)=& \frac{g\left(\right|{\overline{s}}_k{|}^2){\overline{s}}_k^{\ast }-c_k{\overline{s}}_k^{\ast }-{\rho }_k^{\ast }}{c_k-{\mu }_k}{\mathbf{B}}^H\mathbf{B}(\mathbf{z}-{\mathbf{m}}_{\mathbf{x}})-\frac{g\left(\right|{\overline{s}}_k{|}^2)-c_k}{c_k-{\mu }_k}{\left|{\overline{s}}_k\right|}^2{\mathbf{B}}^H\mathbf{B}{\mathbf{a}}_k+\frac{{\rho }_k^{\ast }{\overline{s}}_k}{c_k-{\mu }_k}{\mathbf{B}}^H\mathbf{B}{\mathbf{a}}_k\hfill & \\ & +\frac{{\rho }_k^{\ast }}{c_k-{\mu }_k}\sum _{j=1}^{k-1}{\overline{s}}_j{\mathbf{B}}^H\mathbf{B}{\mathbf{a}}_j-\sum _{j=1}^{k-1}{\mathbf{B}}^H\mathbf{B}{\mathbf{a}}_k^{T}IF{\left({\mathbf{b}}_j\right)}^{\ast }{\mathbf{a}}_j\hfill & \\ & -\frac{g\left(\right|{\overline{s}}_k{|}^2)-{\mu }_k}{c_k-{\mu }_k}{\overline{s}}_k^{\ast }\sum _{j=1}^{k-1}{\overline{s}}_j{\mathbf{B}}^H\mathbf{B}{\mathbf{a}}_j+\frac{1-|{\overline{s}}_k{|}^2}{2}{\mathbf{B}}^H\mathbf{B}{\mathbf{a}}_k\hfill & \\ & =\frac{g\left(\right|{\overline{s}}_k{|}^2){\overline{s}}_k^{\ast }-c_k{\overline{s}}_k^{\ast }-{\rho }_k^{\ast }}{c_k-{\mu }_k}\left(\sum _{j\ne k}^n{\mathbf{b}}_j{\overline{s}}_j+{\overline{s}}_k{\mathbf{b}}_k\right)-\frac{g\left(\right|{\overline{s}}_k{|}^2)-c_k}{c_k-{\mu }_k}{\left|{\overline{s}}_k\right|}^2{\mathbf{b}}_k+\frac{{\rho }_k^{\ast }{\overline{s}}_k}{c_k-{\mu }_k}{\mathbf{b}}_k\hfill & \\ & +\frac{{\rho }_k^{\ast }}{c_k-{\mu }_k}\sum _{j=1}^{k-1}{\overline{s}}_j{\mathbf{b}}_j-\sum _{j=1}^{k-1}{\mathbf{a}}_k^{T}IF{\left({\mathbf{b}}_j\right)}^{\ast }{\mathbf{b}}_j-\frac{g\left(\right|{\overline{s}}_k{|}^2)-{\mu }_k}{c_k-{\mu }_k}{\overline{s}}_k^{\ast }\sum _{j=1}^{k-1}{\overline{s}}_j{\mathbf{b}}_j+\frac{1-|{\overline{s}}_k{|}^2}{2}{\mathbf{b}}_k\hfill & \\ & =-s_k^{\ast }\sum _{j=1}^{k-1}{\overline{s}}_j{\mathbf{b}}_j+u_k\left({\overline{s}}_k\right)\sum _{j=k+1}^n{\overline{s}}_j{\mathbf{b}}_j-\sum _{j=1}^{k-1}{\mathbf{a}}_k^{T}IF{\left({\mathbf{b}}_j\right)}^{\ast }{\mathbf{b}}_j+\frac{1-|{\overline{s}}_k{|}^2}{2}{\mathbf{b}}_k.\hfill \end{array}$$

(20)

Combining the preceding equality concludes our proof ${\mathbf{a}}_k^{H}IF\left({\mathbf{b}}_j\right)=u_j\left({\overline{s}}_j\right){\overline{s}}_k$. □

Remark 1. 

In the proof of Theorem 1 and Lemmas 1–2, we need to consider the following facts:

$${\mathbf{A}}^H{\mathbf{b}}_k={\mathbf{e}}_k,$$

$$\mathbf{A}{\mathbf{e}}_k={\mathbf{a}}_k,$$

$${\mathbf{b}}_i^H{\mathbf{a}}_{\mathbf{j}}=\left\{\begin{array}{cc}0,\hfill & j<k\hfill \\ 1,\hfill & j=k\hfill \end{array}\right.$$

where $\mathbf{A}$ denotes the mixing matrix in ICA model (1); ${\mathbf{a}}_k$ is the kth column of matrix $\mathbf{A}$; ${\mathbf{e}}_k$ represents a vector with 1 in the kth element and 0 elsewhere; and ${\mathbf{b}}_i$ is the ith column of the demixing matrix $\mathbf{B}$.

Remark 2. 

Note that pre-multiplying both sides of Equation (20) by ${\mathbf{a}}_i^H$, we have the following:

$$\begin{array}{c}\hfill {\mathbf{a}}_i^{H}IF\left({\mathbf{b}}_k\right)=-s_k^{\ast }\sum _{j=1}^{k-1}{\overline{s}}_j{\mathbf{a}}_i^H{\mathbf{b}}_j+u_k\left({\overline{s}}_k\right)\sum _{j=k+1}^n{\overline{s}}_j{\mathbf{a}}_i^H{\mathbf{b}}_j-\sum _{j=1}^{k-1}{\mathbf{a}}_k^{T}IF{\left({\mathbf{b}}_j\right)}^{\ast }{\mathbf{a}}_i^H{\mathbf{b}}_j+\frac{1-|{\overline{s}}_k{|}^2}{2}{\mathbf{a}}_i^H{\mathbf{b}}_k,\end{array}$$

(21)

After analyzing (21), one can observe the following: ${\mathbf{a}}_i^{H}IF\left({\mathbf{b}}_k\right)=u_k\left({\overline{s}}_k\right){\overline{s}}_i$ for $i>k$, or ${\mathbf{a}}_k^{H}IF\left({\mathbf{b}}_j\right)=u_j\left({\overline{s}}_j\right){\overline{s}}_k$ for $k<j$.

Remark 3. 

The expression of complex-valued FastICA IF can also be written as follows:

$$\begin{array}{c}\hfill IF\left({\mathbf{b}}_k\right)=\left\{\begin{array}{cc}\frac{1-|{\overline{s}}_1{|}^2}{2}{\mathbf{b}}_1+u_1\left({\overline{s}}_1\right){\displaystyle \sum _{i=2}^n}{\overline{s}}_i{\mathbf{b}}_i,\hfill & k=1,\hfill \\ -{\overline{s}}_k^{\ast }{\displaystyle \sum _{i=1}^{k-1}}[u_i{\left({\overline{s}}_i\right)}^{\ast }+{\overline{s}}_i]{\mathbf{b}}_i+\frac{1-|{\overline{s}}_k{|}^2}{2}{\mathbf{b}}_k+u_k\left({\overline{s}}_k\right){\displaystyle \sum _{i=k+1}^n}{\overline{s}}_i{\mathbf{b}}_i,\hfill & 1<k<n,\hfill \\ -{\overline{s}}_n^{\ast }{\displaystyle \sum _{i=1}^{n-1}}[u_i{\left({\overline{s}}_i\right)}^{\ast }+{\overline{s}}_i]{\mathbf{b}}_i+\frac{1-|{\overline{s}}_n{|}^2}{2}{\mathbf{b}}_n,\hfill & k=n.\hfill \end{array}\right.\end{array}$$

(22)

By observing the closed-form expression of the IF for the complex-valued FastICA functional ${\mathbf{b}}_k$ in Theorem 1, we obtain that the IF of ${\mathbf{b}}_k$ is the weighted sum of the separation vector ${\mathbf{b}}_1,\cdots ,{\mathbf{b}}_n$ with the unbounded weight coefficient function, with respect to the projection of the contaminated point ${\overline{s}}_1,\cdots ,{\overline{s}}_n$. This finding confirms that the values of $IF\left({\mathbf{b}}_k\right)$ are large when the outliers are present in the source signals. As can also be seen in Theorem 1, the greater the values of the contaminated point ${\overline{s}}_1,\cdots ,{\overline{s}}_n$ presenting in the source signals, the higher the values of $IF\left({\mathbf{b}}_k\right)$.

4. Equivariance of the Complex-Valued FastICA Estimator

4.1. Equivariance of Complex-Valued ICA

Equivariance is a fundamental property in statistics [23], which makes the transformation of the parameter in the model equal to that of the sample data, which results in the following definition.

Definition 3. 

Assume that $\widehat{\mathbf{A}}=\mathcal{A}\left(\mathbf{X}\right)$ is an estimator of the mixing matrix $\mathbf{A}$ in the complex-valued ICA model (1), which relies on the sample matrix $\mathbf{X}=\left[\mathbf{x}\right(1),\cdots ,\mathbf{x}(T\left)\right]$ with T sample points. Then, we call the estimator $\widehat{\mathbf{A}}$ equivariant if

$$\mathcal{A}\left(\mathbf{TX}\right)=\mathbf{T}\mathcal{A}\left(\mathbf{X}\right)$$

(23)

for any invertible transformation $\mathbf{T}$.

In order to analyze the equivariance of the complex-valued FastICA estimator, the following Theorem is needed.

Theorem 2. 

Let $\mathbf{C}=\mathbf{BA}$ be the global mixing–demixing system of the complex-valued ICA model (1) and $\widehat{\mathbf{A}}$ be equivariant. Then the global mixing–demixing system $\mathbf{C}$ of ICA algorithms based on the optimizing cost function depends only on the source signal matrix $\mathbf{S}=\left[\mathbf{s}\right(1),\cdots ,\mathbf{s}(T\left)\right]$, instead of the mixing system $\mathbf{A}$ and demixing system $\mathbf{B}$.

Proof. 

Denote by $\mathbf{C}$ the mixing–demixing matrix $\mathbf{BA}$; the estimation of the source can be rewritten as follows:

$$\mathbf{y}=\mathbf{Cs}.$$

(24)

In fact, the main property of the equivariant estimator for the BSS problem is that it provides uniform performance in the sense that

$$\mathbf{y}=\mathbf{Bx}=\mathcal{A}{\left(\mathbf{X}\right)}^{-1}\mathbf{As}=\mathcal{A}{\left(\mathbf{AS}\right)}^{-1}\mathbf{As}=\mathcal{A}{\left(\mathbf{S}\right)}^{-1}\mathbf{s},$$

(25)

where $\mathbf{S}=\left[\mathbf{s}\right(1),\cdots ,\mathbf{s}(T\left)\right]$ is the source signal matrix with T sample points.

By comparing Equations (24) and (25), the result follows from the fact that $\mathbf{C}=\mathcal{A}{\left(\mathbf{S}\right)}^{-1}.$ □

Remark 4. 

From Theorem 2, the current estimation of source ${\mathbf{y}}_t$ estimated by the equivariant estimator relies only on the source signal matrix $\mathbf{S}$ and the source ${\mathbf{s}}_t$, rather than the mixing matrix $\mathbf{A}$.

Remark 5. 

Due to the fact that the estimator based on the optimizing cost function is equivariant [24], we only need to consider the equivariance of the complex-valued FastICA based on the global mixing–demixing system $\mathbf{C}=\mathbf{BA}$, i.e., the update learning rule of the estimated parameter for the ICA algorithm depends on the composition system.

4.2. Equivariance of Symmetric Complex-Valued FastICA

For convenience, we analyze the equivariance of the complex-valued FastICA algorithm by focusing on the symmetric version, in the sense that the source signals are estimated in parallel. In particular, the demixing vectors ${\mathbf{b}}_1,{\mathbf{b}}_2,\cdots ,{\mathbf{b}}_n$ are iterated simultaneously, as follows:

$$\begin{array}{cc}\hfill {\mathbf{b}}_1\leftarrow & -E\left\{g\right(|{\mathbf{b}}_1^H{\mathbf{x}|}^2){\left({\mathbf{b}}_1^H\mathbf{x}\right)}^{\ast }\mathbf{x}\}+E\{g^{\prime }\left(\right|{\mathbf{b}}_1^H{\mathbf{x}|}^2\left)\right|{\mathbf{b}}_1^H{\mathbf{x}|}^2+g\left(\right|{\mathbf{b}}_1^H{\mathbf{x}|}^2\left)\right\}{\mathbf{b}}_1+E\left\{{\mathbf{xx}}^T\right\}E\{g^{\prime }\left(\right|{\mathbf{b}}_1^H{\mathbf{x}|}^2\left)({\mathbf{b}}_1^H\mathbf{x}{)}^{\ast 2}\right\}{\mathbf{b}}_1^{\ast },\hfill \\ & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}⋮\hfill & \\ \hfill {\mathbf{b}}_n\leftarrow & -E\left\{g\right(|{\mathbf{b}}_n^H{\mathbf{x}|}^2){\left({\mathbf{b}}_n^H\mathbf{x}\right)}^{\ast }\mathbf{x}\}+E\{g^{\prime }\left(\right|{\mathbf{b}}_n^H{\mathbf{x}|}^2\left)\right|{\mathbf{b}}_n^H{\mathbf{x}|}^2+g\left(\right|{\mathbf{b}}_n^H{\mathbf{x}|}^2\left)\right\}{\mathbf{b}}_n+E\left\{{\mathbf{xx}}^T\right\}E\{g^{\prime }\left(\right|{\mathbf{b}}_n^H{\mathbf{x}|}^2\left)({\mathbf{b}}_n^H\mathbf{x}{)}^{\ast 2}\right\}{\mathbf{b}}_n^{\ast },\hfill \end{array}$$

where ${\mathbf{b}}_1,{\mathbf{b}}_2,\cdots ,{\mathbf{b}}_n$ is an orthonormal set. In order to prevent different parameters from converging to the same maxima, the estimate sources $y_1={\mathbf{b}}_1^H\mathbf{x},y_2={\mathbf{b}}_2^H\mathbf{x},\cdots ,y_n={\mathbf{b}}_n^H\mathbf{x}$ need to be decorrelated, which can be accomplished by the following classic symmetric orthogonalization approach involving matrix square roots

$$\mathbf{B}\leftarrow {\left(\mathbf{B}{\mathbf{B}}^H\right)}^{-\frac{1}{2}}\mathbf{B},$$

(26)

where $\mathbf{B}=({\mathbf{b}}_1,\cdots ,{\mathbf{b}}_n)$ is the matrix of the vectors. Note that the inverse square root ${\left(\mathbf{B}{\mathbf{B}}^H\right)}^{-\frac{1}{2}}$ can be acquired by the EVD of $\mathbf{B}{\mathbf{B}}^H=\mathbf{E}diag(d_1,d_2,\cdots ,d_n){\mathbf{E}}^H$. For any vector $\mathbf{h}={(h_1,\cdots ,h_n)}^T$, we denote the following:

$$diag\left(\mathbf{h}\right)=\left(\begin{array}{ccc}{h}_1& & 0\\ & \ddots & \\ 0& & h_n\end{array}\right).$$

Hence, we have the following:

$${\left(\mathbf{B}{\mathbf{B}}^H\right)}^{-\frac{1}{2}}=\mathbf{E}diag(d_1^{-\frac{1}{2}},d_2^{-\frac{1}{2}},\cdots ,d_n^{-\frac{1}{2}}){\mathbf{E}}^H.$$

(27)

Using the matrix representation, the symmetric version of the complex-valued FastICA algorithm can be rewritten by the following learning rule:

Step 1 

Choose arbitrary initial values for the ${\mathbf{b}}_1,\cdots ,{\mathbf{b}}_n\in O\left(n\right)$; orthogonalize matrix $\mathbf{B}=({\mathbf{b}}_1,\cdots ,{\mathbf{b}}_n)$ as in Step 2 below.

Step 2 

Let $\mathbf{y}=\mathbf{Bx}$, run iteration

$$\begin{array}{ccc}\hfill \mathbf{B}\leftarrow & -{E\left\{diag\right(g\left(\right|\mathbf{y}|}^2\left)\right){\mathbf{yx}}^H\}+E\{diag(g^{\prime }{\left(\right|\mathbf{y}|}^2{\left)\right)\left|\mathbf{y}\right|}^2\hfill & \\ & +{g\left(\right|\mathbf{y}|}^2\left)\right\}\mathbf{B}+E\left\{diag\right(g^{\prime }{\left(\right|\mathbf{y}|}^2\left)\right){\mathbf{y}}^2\}E\left\{{\mathbf{y}}^{\ast }{\mathbf{x}}^H\right\},\hfill & \\ & \mathbf{B}\leftarrow {\left(\mathbf{B}{\mathbf{B}}^H\right)}^{-\frac{1}{2}}\mathbf{B}\hfill \end{array}$$

(28)

until convergence.

Definition 4. 

Assume that $\mathbf{B}=\mathcal{B}\left(\mathbf{X}\right)$ is an estimator of the demixing matrix $\mathbf{B}$ in the complex-valued ICA model (1), which relies on the sample matrix $\mathbf{X}=\left[\mathbf{x}\right(1),\cdots ,\mathbf{x}(T\left)\right]$ with T sample points. Then, we call the estimator $\mathcal{B}\left(\mathbf{X}\right)$ linear-equivariant, if

$$\mathbf{C}=\mathcal{B}\left(\mathbf{S}\right),$$

(29)

where $\mathbf{x}\left(i\right)=\mathbf{As}\left(i\right),i=1,\cdots ,T$, $\mathbf{S}=\left[\mathbf{s}\right(1),\cdots ,\mathbf{s}(T\left)\right]$ and $\mathbf{C}=\mathbf{BA}$.

From Definition 4, we can obtain the equivariance of the symmetric complex-valued FastICA algorithm:

Theorem 3. 

The algorithm (28) for the complex-valued BSS is linear-equivariant and $\mathcal{B}\left(\mathbf{S}\right)=\mathcal{A}{\left(\mathbf{S}\right)}^{-1}$.

Proof. 

We denote by $\mathbf{C}$ the mixing–demixing matrix $\mathbf{BA}$, and the estimation of sources can be rewritten as follows:

$$\mathbf{y}=\mathbf{Cs}.$$

(30)

Plugging the symmetric orthogonalization into the algorithm (28), by post-multiplying $\mathbf{A}$ on both sides of the complex-valued FastICA algorithm, one can yield

$$\begin{array}{ccc}\hfill \mathbf{C}\leftarrow & -{E\left\{diag\right(g\left(\right|\mathbf{Cs}|}^2\left)\right){\mathbf{Css}}^H{\mathbf{C}}^H\mathbf{C}\}+E\{diag(g^{\prime }{\left(\right|\mathbf{Cs}|}^2{\left)\right)\left|\mathbf{Cs}\right|}^2\hfill & \\ & +{g\left(\right|\mathbf{Cs}|}^2\left)\right\}\mathbf{C}+E\left\{diag\right(g^{\prime }{\left(\right|\mathbf{Cs}|}^2{\left)\right)\left(\mathbf{Cs}\right)}^2\}E\left\{{\left(\mathbf{Cs}\right)}^{\ast }{\mathbf{s}}^H{\mathbf{C}}^H\mathbf{C}\right\}.\hfill \end{array}$$

(31)

From the above equation, we have $\mathbf{C}=\mathcal{B}\left(\mathbf{S}\right)$. Hence, the complex-valued BSS is linear-equivariant. The residual conclusion $\mathcal{B}\left(\mathbf{S}\right)=\mathcal{A}{\left(\mathbf{S}\right)}^{-1}$ can be obtained by combining Theorem 2, Definition 4 and Theorem 3. □

The results obtained demonstrate the following: (1) Theorems 2 and 3 suggest that the complex-valued FastICA estimator is of equivariance, namely, the performance of the complex-valued FastICA algorithm depends only on the source signal $\mathbf{s}$, rather than the mixing matrix $\mathbf{A}$ and demixing matrix $\mathbf{B}$; (2) There is an invertible relationship between the mixing matrix $\mathbf{A}$ and demixing matrix $\mathbf{B}$ under Assumption 2.

5. Conclusions

In order to provide a rigorous mathematical treatment of the robust measurement and equivariance for the complex-valued FastICA estimator, this paper analyzed the statistical properties of the complex-valued FastICA algorithm in the context of ICA over the complex number domain. Firstly, a closed-form expression of the complex-valued FastICA functional was derived and used to measure robustness against outliers. We found that the complex-valued FastICA algorithm based on Tukey’s single-parameter M-estimator cost function had the best separation performance in both circular and noncircular scenarios. Then, we proved that the complex-valued FastICA algorithm is equivariant in the sense that the global mixing-demixing system of the algorithm depended only on the source signals.