Steady-State Performance of an Adaptive Combined MISO Filter Using the Multichannel Affine Projection Algorithm

Abstract

The combination of adaptive filters is an effective approach to improve filtering performance. In this paper, we investigate the performance of an adaptive combined scheme between two adaptive multiple-input single-output (MISO) filters, which can be easily extended to the case of multiple outputs. In order to generalize the analysis, we consider the multichannel affine projection algorithm (APA) to update the coefficients of the MISO filters, which increases the possibility of exploiting the capabilities of the filtering scheme. Using energy conservation relations, we derive a theoretical behavior of the proposed adaptive combination scheme at steady state. Such analysis entails some further theoretical insights with respect to the single channel combination scheme. Simulation results prove both the validity of the theoretical steady-state analysis and the effectiveness of the proposed combined scheme.

1. Introduction

The convergence performance of an adaptive filtering is decisively affected by the choice of certain characteristic parameters of the filter. This is one of the main motivations behind the high number of adaptive algorithms and structures that are continually introduced in the literature to provide improvements of convergence and steady-state performance in certain scenarios. The design of an adaptive filter may be even very risky when no a priori information is provided on the model of the signal to be processed. One solution to this problem is represented by the adaptive combination of adaptive filters [1,2,3,4,5,6], which exploits the diversity of parallel adaptive branches. Combined structures of filtering have been applied to a wide variety of online adaptive filtering applications [7,8,9,10,11,12,13,14,15] due to their capabilities of detecting and selecting instant by instant the best-performing filter. To this end, the analysis of the mean squared performance aims at showing the effectiveness of adaptive combined schemes and how to exploit them the best [6,16,17,18,19,20].

In this paper, we focus on the combination of adaptive filters in the multichannel case. In particular, the combined structure that we take into account is composed of a convex combination of two adaptive multiple-input single-output (MISO) filters [8]. MISO filters are widely used in several applications, such as adaptive beamforming [21,22], OFDM systems [23], channel equalization [24], and audio and speech applications [25,26]. These have been chosen as a baseline scheme of the multichannel case, but the combined MISO structure, and the related analysis thereof, can also be easily extended to multiple-input multiple-output (MIMO) filters, as in [11], since an MIMO filter can be seen as a parallel of MISO filters. In order to adapt MISO filters, we adopt a multichannel affine projection algorithm (APA) [27,28], which, being characterized by a projection order, allows us to generalize the analysis with respect to both gradient-based and Hessian-based adaptive algorithms. Due to its robustness, the APA has been widely implemented with variable parameters to further improve performance [29,30,31,32,33].

A first novel insight in this paper concerns the combination of two multichannel APA filters differentiated according to the projection order. The combination of APAs using different projection orders has been addressed in [8,34,35,36,37,38], where, however, the combination takes into account only the current projection. Conversely, the proposed approach, which is also generalized with respect to the other filter parameters, considers a mixing parameter for each projection order. Moreover, in the choice of differentiating the projection order, and in particular setting a unitary order for one of the two MISO filters, the diversity between gradient-based and Hessian-based algorithms is exploited, thus yielding a performance improvement [1,16,39]. Some attempts in that sense were also provided in [40], but without a theoretical performance analysis.

In this paper, instead, a performance analysis is conducted in-depth, thus representing itself another novel contribution. Indeed, in order to show the capabilities of the proposed adaptive scheme, we derive a steady-state analysis of the combined algorithm, thus deriving a theoretical value of the excess mean squared error (EMSE) based on the energy conservation arguments [41]. In the literature, many efforts have been made to analyze the performance of single-channel APA filters [42,43] and of combined least mean squares (LMS)-like algorithms [1,6]. The use of the proposed scheme leads to further non-straightforward assumptions that increase the complexity of deriving a theoretical value for the EMSE. Experimental results prove, on the one hand, the validity of the steady-state EMSE analysis presented and, on the other hand, the effectiveness of the proposed combined filtering architecture.

The rest of the paper is organized as follows: The proposed adaptive combined MISO filter is introduced in Section 2, and the optimum theoretical EMSE is derived in Section 3. Then, the components of the theoretical EMSE are derived using energy conservation properties; specifically, the EMSE components for the individual MISO filters are derived in Section 4, and the EMSE component due to the combination is achieved in Section 5. Results are presented in Section 6, and finally, in Section 7 our conclusions are drawn.

Notation

In this paper, matrices are represented by boldface capital letters and vectors are denoted by boldface lowercase letters. Scalars are denoted by italic letters. Time-varying vectors and matrices show discrete-time index as a subscript index, while in time-varying scalar elements the time index is denoted in square brackets. A regression vector is represented as ${\mathbf{x}}_n\in {\mathbb{R}}^{M\times 1}={\left[\begin{array}{cccc}x\left[n\right]& x\left[n-1\right]& \dots & x\left[n-M+1\right]\end{array}\right]}^T$, where M is the overall vector length and $x\left[n-i\right]$ is individual scalar entry at the generic time instant $n-i$. A generic coefficient vector, in which all the elements depend on the same time instant, is denoted as ${\mathbf{w}}_n\in {\mathbb{R}}^{M\times 1}={\left[\begin{array}{cccc}{w}_0\left[n\right]& w_1\left[n\right]& \dots & w_{M-1}\left[n\right]\end{array}\right]}^T$, where $w_i\left[n\right]$ is the generic $i^{\mathrm{th}}$ individual entry at the $n^{\mathrm{th}}$ time instant. All vectors are represented as column vectors. The index related to a generic $j^{\mathrm{th}}$ filter is denoted as superscript, e.g., ${\mathbf{w}}_n^{\left(j\right)}$.

2. A Convex Combination Scheme for Adaptive MISO Filters

An adaptive MISO system is composed of a number P of adaptive filters in parallel, each one receiving a different input, and yields a single output. A combined system of MISO filters is characterized by the adaptive combination of the outputs of different MISO filters, which can be differentiated in several ways according to the updating rule or some parameter settings. In this paper, we consider the adaptive combination of a number of $J=2$ MISO filters adapted by using the same updating rule, defined by the regularized multichannel APA [25,27], but different projection orders.

The input signals are the same for each MISO system and are collected into data matrices. However, due to the different projection order, each MISO system uses its own sub-matrix. In particular, the $p^{\mathrm{th}}$ input sub-matrix of the $j^{\mathrm{th}}$ MISO system, for $p=1,\dots ,P$ and $j=1,2$, is represented as ${\mathbf{X}}_{p,n}^{\left(j\right)}\in {\mathbb{R}}^{K_j\times M}$:

$$\begin{array}{cc}\hfill {\mathbf{X}}_{p,n}^{\left(j\right)}& ={\left[\begin{array}{cccc}{\mathbf{x}}_{p,n}^{\left(j\right)}& {\mathbf{x}}_{p,n-1}^{\left(j\right)}& \dots & {\mathbf{x}}_{p,n-K_j+1}^{\left(j\right)}\end{array}\right]}^T\hfill \\ & =\left[\begin{array}{ccc}{x}_p\left[n\right]& \cdots & x_p\left[n-M+1\right]\\ x_p\left[n-1\right]& \cdots & x_p\left[n-M\right]\\ ⋮& \ddots & ⋮\\ x_p\left[n-K_j+1\right]& \cdots & x_p\left[n-M-K_j+2\right]\end{array}\right]\hfill \end{array}$$

(1)

where M is the length of the adaptive filters (same for all) and $K_j$ denotes the number of previous entries to keep in memory, i.e., the projection order, for the $j^{\mathrm{th}}$ MISO system. The concatenation of sub-matrices (1) provides the input data matrix for each MISO system, i.e., ${\mathbf{X}}_n^{\left(j\right)}\in {\mathbb{R}}^{K_j\times MP}=\left[\begin{array}{cccc}{\mathbf{X}}_{1,n}^{\left(j\right)}& {\mathbf{X}}_{2,n}^{\left(j\right)}& \dots & {\mathbf{X}}_{P,n}^{\left(j\right)}\end{array}\right]$. Each MISO filter can be represented by ${\mathbf{w}}_n^{\left(j\right)}\in {\mathbb{R}}^{MP\times 1}{\left[\begin{array}{cccc}{\mathbf{w}}_{1,n}^{\left(j\right),T}& {\mathbf{w}}_{2,n}^{\left(j\right),T}& \dots & {\mathbf{w}}_{P,n}^{\left(j\right),T}\end{array}\right]}^T$, with $j=1,2$. The output vector of each MISO filter ${\mathbf{y}}_n^{\left(j\right)}\in {\mathbb{R}}^{K_j\times 1}={\left[\begin{array}{cccc}{y}_0^{\left(j\right)}\left[n\right]& y_1^{\left(j\right)}\left[n\right]& \dots & y_{K_j-1}^{\left(j\right)}\left[n\right]\end{array}\right]}^T$ is then obtained as:

$${\mathbf{y}}_n^{\left(j\right)}={\mathbf{X}}_n^{\left(j\right)}{\mathbf{w}}_{n-1}^{\left(j\right)}.$$

(2)

Taking into account the vector ${\mathbf{d}}_n^{\left(j\right)}\in {\mathbb{R}}^{K_j\times 1}={\left[\begin{array}{cccc}d\left[n\right]& d\left[n-1\right]& \dots & d\left[n-K_j+1\right]\end{array}\right]}^T$ containing the last $K_j$ samples of the desired signal, it is possible to achieve the error signal vector ${\mathbf{e}}_n^{\left(j\right)}\in {\mathbb{R}}^{K_j\times 1}={\left[\begin{array}{cccc}{e}_0^{\left(j\right)}\left[n\right]& e_1\left(j\right)\left[n\right]& \dots & e_{K_j-1}\left(j\right)\left[n\right]\end{array}\right]}^T$ for each MISO filter:

$${\mathbf{e}}_n^{\left(j\right)}={\mathbf{d}}_n^{\left(j\right)}-{\mathbf{y}}_n^{\left(j\right)}.$$

(3)

It is worth noting that the difference of notation between the $k^{\mathrm{th}}$ entry $d\left[n-k\right]$ of ${\mathbf{d}}_n^{\left(j\right)}$ and the entry $y_k\left[n\right]$ of ${\mathbf{y}}_n^{\left(j\right)}$ (and the same for ${\mathbf{e}}_n^{\left(j\right)}$) is due to the fact that ${\mathbf{d}}_n^{\left(j\right)}$ is a regression vector collecting the last $K_j$ samples of the desired signal, while all the entries of ${\mathbf{y}}_n^{\left(j\right)}$ and ${\mathbf{e}}_n^{\left(j\right)}$ are computed at each time instant. This means, for example, that the entry of the desired signal for $k=0$ at the $n-1$ time instant has the same value of the entry for $k=1$ at the $n^{\mathrm{th}}$ time instant, and they both are denoted with $d\left[n-1\right]$. On the other hand, all the entries of the output and signal vectors may change at each time instant until convergence; hence, for example, $y_0^{\left(j\right)}\left[n-1\right]\ne y_1^{\left(j\right)}\left[n\right]$.

Therefore, the update rule of each MISO filter using the regularized multichannel affine projection algorithm [27,28]:

$${\mathbf{w}}_n^{\left(j\right)}={\mathbf{w}}_{n-1}^{\left(j\right)}+{\mu }_j{\mathbf{X}}_n^{\left(j\right),T}{\left({\delta }_j\mathbf{I}+{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{\mathbf{e}}_n^{\left(j\right)}$$

(4)

where ${\mu }_j$ is the step size and ${\delta }_j$ is the regularization factor for the $j^{\mathrm{th}}$ MISO filter, which is considered the same for both MISO filters.

The chosen method for combining MISO filters is the system-by-system combination scheme [8], depicted in Figure 1, since it involves the minimum necessary number of free parameters for single-stage combined architectures. According to this scheme, the overall output of the system is achieved by combining convexly the MISO filter outputs (2). This combination involves the sum of buffers with different lengths, which depends on the projection order $K_j$. This problem is addressed by using zero-padding, thus:

$${\mathbf{z}}_n\left\{\begin{array}{ccc}\in {\mathbb{R}}^{K_1\times 1}=& {\mathit{\lambda }}_n\odot {\mathbf{y}}_n^{\left(1\right)}+\left(\mathbf{1}-{\mathit{\lambda }}_n\right)\odot \left[{\mathbf{y}}_n^{\left(2\right)};{\mathbf{0}}^{\left(1\right)}\right],& \mathrm{if}{K}_1\ge K_2\\ \in {\mathbb{R}}^{K_2\times 1}=& {\mathit{\lambda }}_n\odot \left[{\mathbf{y}}_n^{\left(1\right)};{\mathbf{0}}^{\left(2\right)}\right]+\left(\mathbf{1}-{\mathit{\lambda }}_n\right)\odot {\mathbf{y}}_n^{\left(2\right)},& \mathrm{if}{K}_1<K_2\end{array}\right.$$

(5)

where ${\mathbf{0}}^{\left(1\right)}\in {\mathbb{R}}^{\left(K_1-K_2\right)\times 1}$ and ${\mathbf{0}}^{\left(2\right)}\in {\mathbb{R}}^{\left(K_2-K_1\right)\times 1}$ are vectors of zeros, $\mathbf{1}\in {\mathbb{R}}^{K_{\max }\times 1}$ are vectors of ones, where $K_{\max }=\max \left\{K_1,K_2\right\}$, and ${\mathit{\lambda }}_n\in {\mathbb{R}}^{K_{\max }\times 1}$ are mixing parameter vectors for $j=1,2$; finally, ⊙ denotes the Hadamard product.

The mixing parameters are adaptive weights that balance the combination between the two MISO filters, giving more importance to the best-performing one. Such awareness is obtained according to a mean squared error minimization. The adaptation of ${\mathit{\lambda }}_n$ is performed by using an auxiliary vector ${\mathbf{a}}_n$, whose elements are related to those of ${\mathit{\lambda }}_n$ by means of a sigmoidal function defined according to [8,44]:

$${\lambda }_k\left[n\right]=\beta \left(\frac{1}{1+e^{-a_k\left[n-1\right]}}-\alpha \right),k=0,\dots ,K_{\max }-1$$

(6)

where:

$$\alpha =\frac{1}{\left(1+e^4\right)},\beta =\frac{1}{1-2\alpha }.$$

(7)

The auxiliary vectors are updated by using a gradient descent rule; therefore, for $k=0,\dots ,K_{\max }-1$:

$$a_k\left[n\right]=a_k\left[n-1\right]+\frac{{\mu }_{\mathrm{c}}}{\beta r_k\left[n\right]}{e}_k\left[n\right]\left(y_k^{\left(1\right)}\left[n\right]-y_k^{\left(2\right)}\left[n\right]\right)·\left({\lambda }_k\left[n\right]+\alpha \beta \right)\left(\beta -\alpha \beta -{\lambda }_k\left[n\right]\right)$$

(8)

where ${\mu }_{\mathrm{c}}/r_k\left[n\right]$ represents a normalized step size [45],

$$r_k\left[n\right]=\gamma r_k\left[n-1\right]+\left(1-\gamma \right){\left(y_k^{\left(1\right)}\left[n\right]-y_k^{\left(2\right)}\left[n\right]\right)}^2$$

(9)

is the estimated power of $\left(y_k^{\left(1\right)}\left[n\right]-y_k^{\left(2\right)}\left[n\right]\right)$, and $\gamma$ is a smoothing factor close to one.

The entries $e_k\left[n\right]$ in (8), for $k=0,\dots ,K_{\max }-1$, of the overall error signal vector ${\mathbf{e}}_n$ are derived as:

$${\mathbf{e}}_n\in {\mathbb{R}}^{K_{\max }\times 1}=\left\{\begin{array}{cc}{\mathbf{d}}_n^{\left(1\right)}-{\mathbf{z}}_n,& \mathrm{if}{K}_{\max }=K_1\hfill \\ {\mathbf{d}}_n^{\left(2\right)}-{\mathbf{z}}_n,& \mathrm{if}{K}_{\max }=K_2\hfill \end{array}\right..$$

(10)

The overall error ${\mathbf{e}}_n$ represents the quantity that provides the awareness to the scheme about the best-performing filter, and it is used to adapt the mixing parameters at each iteration, as is possible to note in (8).

The combination described by (5) has a significant value due to its generalization properties, since it can be used:

• to find an optimal selection of filter parameters, e.g., by choosing different values for the step sizes ${\mu }_j$ or for the regularization factors ${\delta }_j$ in (4) [1,6];
• to improve the overall tracking abilities beyond the capabilities of the individual filters by selecting $K_1=1$ and $K_2>1$. Since the APA with unitary projection order is equivalent to the (normalized) least mean squares algorithm, this choice enables the scheme to show a combination between a gradient-based and a Hessian-based adaptive algorithm, which provides diversity to the scheme and leads to enhancing the tracking performance [1,16].

In those applications having limited computational resources, the complexity of the proposed method may represent a drawback. While the adaptation of the mixing parameters does not involve a significant increment of the computational load [1], the adaption of two MISO filters in parallel may require twice the resources. However, several implementation strategies have been proposed in the literature to reduce the overall complexity of a combined scheme significantly, such as using a low-cost filter as a companion to a high-cost one, rather than using parallelization, or taking advantage of redundancies in the two filter components (see [1] for further details). The implementation of such strategies would make the computational load of the combined scheme just slightly higher than the complexity of a single MISO filter.

In the next sections, we provide a theoretical derivation of the steady-state performance, which is useful to understand the capabilities of the proposed combined filtering scheme.

3. Optimum Mixing Parameters and EMSE

A first step to perform a mean squared performance analysis of the proposed combined scheme consists of deriving the expression of the optimal mixing parameter for each input projection that minimizes the mean squared error of the combined scheme. To this end, we first define the problem formulation, and then, we formalize the combination performance.

3.1. Stationary Data Model

Let us consider the following linear regression model:

$$d\left[n\right]={\mathbf{x}}_n^T{\mathbf{w}}^{\mathrm{o}}+v\left[n\right]$$

(11)

where ${\mathbf{x}}_n\in {\mathbb{R}}^{MP\times 1}={\left[\begin{array}{ccc}{\mathbf{x}}_{1,n}^T& \dots & {\mathbf{x}}_{P,n}^T\end{array}\right]}^T$ is the first projection (i.e., for $k=0$) of the data input matrix ${\mathbf{X}}_n^{\left(j\right)}$ (for the simplicity of notation, we omit the projection order subscript for $k=0$), and it is the same for both MISO filters. Moreover, in (11), ${\mathbf{w}}^{\mathrm{o}}\in {\mathbb{R}}^{MP\times 1}={\left[\begin{array}{ccc}{\mathbf{w}}_1^{\mathrm{o},T}& \dots & {\mathbf{w}}_P^{\mathrm{o},T}\end{array}\right]}^T$ is the optimal multichannel weight solution. With respect to the model of (11), we consider the following assumptions [6,41,42].

Assumption 1.

The signal $v\left[n\right]$ represents an independent and identically distributed (i.i.d.) noise added to the system output, whose variance is ${\sigma }_v^2$.

Assumption 2.

The input signals are assumed to be statistically independent of the noise signal.

Assumption 3.

We also assume that $\mathrm{E}\left\{{\mathbf{x}}_{p,n}\right\}=\mathbf{0}$, $\mathrm{E}\left\{d\left[n\right]\right\}=0$, $\mathrm{E}\left\{v\left[n\right]\right\}=0$, and define the covariance matrix $\mathrm{E}\left\{{\mathbf{x}}_{p,n}{\mathbf{x}}_{p,n}^T\right\}={\mathbf{R}}_{p,n}$.

We can define the weight error vector for the $j^{\mathrm{th}}$ MISO filter and for the multichannel combined scheme, respectively, as:

$$\begin{array}{cc}\hfill {\tilde{\mathbf{w}}}_n^{\left(j\right)}\in {\mathbb{R}}^{MP\times 1}& ={\mathbf{w}}^{\mathrm{o}}-{\mathbf{w}}_n^{\left(j\right)}\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\tilde{\mathbf{w}}}_n\in {\mathbb{R}}^{MP\times 1}& ={\mathbf{w}}^{\mathrm{o}}-{\mathbf{w}}_n\hfill \end{array}$$

(13)

We also introduce the a priori and the a posteriori estimation error signals for the $j^{\mathrm{th}}$ MISO system, respectively, as:

$$\begin{array}{cc}\hfill {{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\in {\mathbb{R}}^{K_j\times 1}& ={\mathbf{e}}_n^{\left(j\right)}-{\mathbf{v}}_n^{\left(j\right)}={\mathbf{X}}_n^{\left(j\right)}{\tilde{\mathbf{w}}}_{n-1}^{\left(j\right)}\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill {{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right)}\in {\mathbb{R}}^{K_j\times 1}& ={\mathbf{X}}_n^{\left(j\right)}{\tilde{\mathbf{w}}}_n^{\left(j\right)}\hfill \end{array}$$

(15)

and the a priori and the a posteriori estimation error signals for the combination scheme, respectively, as:

$$\begin{array}{cc}\hfill {{\mathbf{e}}_{\mathrm{a}}}_n\in {\mathbb{R}}^{K_{\max }\times 1}& ={\mathbf{e}}_n-{\mathbf{v}}_n={\mathbf{X}}_n{\tilde{\mathbf{w}}}_{n-1}\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {{\mathbf{e}}_{\mathrm{p}}}_n\in {\mathbb{R}}^{K_{\max }\times 1}& ={\mathbf{X}}_n{\tilde{\mathbf{w}}}_n\hfill \end{array}$$

(17)

where ${\mathbf{v}}_n^{\left(j\right)}\in {\mathbb{R}}^{K_j\times 1}={\left[\begin{array}{cccc}v\left[n\right]& v\left[n-1\right]& \dots & v\left[n-K_j+1\right]\end{array}\right]}^T$ contains the last $K_j$ samples of the irreducible noise, while ${\mathbf{v}}_n\in {\mathbb{R}}^{K_{\max }\times 1}$ contains the last $K_{\max }$ samples of $v\left[n\right]$, and ${\mathbf{x}}_n\in {\mathbb{R}}^{K_{\max }\times 1}$.

It is worth noting that while in the steady-state definition of the model (11), we consider the input at its current time instant, in (14) and (16), we take into account also the projections of the input signal.

Considering (10) and (5), the a priori error (16) can be also written as:

$${{\mathbf{e}}_{\mathrm{a}}}_n\in {\mathbb{R}}^{K_{\max }\times 1}=\left\{\begin{array}{cc}{\mathit{\lambda }}_n\odot {{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right)}+\left(\mathbf{1}-{\mathit{\lambda }}_n\right)\odot \left[{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right)};{\mathbf{0}}^{\left(1\right)}\right],& \mathrm{if}{K}_{\max }=K_1\hfill \\ {\mathit{\lambda }}_n\odot \left[{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right)};{\mathbf{0}}^{\left(2\right)}\right]+\left(\mathbf{1}-{\mathit{\lambda }}_n\right)\odot {{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right)},& \mathrm{if}{K}_{\max }=K_2\hfill \end{array}\right.$$

(18)

which can be denoted for simplicity of notation as:

$${{\mathbf{e}}_{\mathrm{a}}}_n={\mathit{\lambda }}_n\odot {{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right)}+\left(\mathbf{1}-{\mathit{\lambda }}_n\right)\odot {{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right)}$$

(19)

3.2. Formulation of the EMSE for the Combination

The mean squared performance of the combined MISO filter can be evaluated in terms of the excess mean squared error (EMSE) (see for example [1,22,41]), which can be defined according to the expression of the a priori error (19), for $k=0,\dots ,K_{\max }-1$:

$$\begin{array}{cc}\hfill {\xi }_k\left[n\right]& =\mathrm{E}\left\{{\left|{e_{\mathrm{a}}}_k\left[n\right]\right|}^2\right\}\hfill \\ & ={\lambda }_k^2\left[n\right]\mathrm{E}\left\{{\left|{e_{\mathrm{a}}}_k^{\left(1\right)}\left[n\right]\right|}^2\right\}+{\left(1-{\lambda }_k\left[n\right]\right)}^2\mathrm{E}\left\{{\left|{e_{\mathrm{a}}}_k^{\left(2\right)}\left[n\right]\right|}^2\right\}+2{\lambda }_k\left[n\right]\left(1-{\lambda }_k\left[n\right]\right)\mathrm{E}\left\{{e_{\mathrm{a}}}_k^{\left(1\right)}\left[n\right]{e_{\mathrm{a}}}_k^{\left(2\right)}\left[n\right]\right\}\hfill \\ & ={\lambda }_k^2\left[n\right]{\xi }_k^{\left(1\right)}\left[n\right]+{\left(1-{\lambda }_k\left[n\right]\right)}^2{\xi }_k^{\left(2\right)}\left[n\right]+2{\lambda }_k\left[n\right]\left(1-{\lambda }_k\left[n\right]\right){\xi }_k^{\left(12\right)}\left[n\right]\hfill \end{array}$$

(20)

where ${\xi }_k^{\left(j\right)}\left[n\right]$, $j=1,2$, are the EMSEs of the $k^{\mathrm{th}}$ projection for the two individual MISO filters and:

$${\xi }_k^{\left(12\right)}\left[n\right]=\mathrm{E}\left\{{e_{\mathrm{a}}}_k^{\left(1\right)}\left[n\right]{e_{\mathrm{a}}}_k^{\left(2\right)}\left[n\right]\right\}$$

(21)

represents the cross-EMSE, as defined in [6], which provides a measure of the ability of the combination to improve the performance with respect to both filter components [1]. The cross-EMSE satisfies the Cauchy–Schwartz inequality:

$${\left|{\xi }_k^{\left(12\right)}\left[n\right]\right|}^2\le {\xi }_k^{\left(1\right)}\left[n\right]{\xi }_k^{\left(2\right)}\left[n\right]$$

(22)

which implies that the magnitude of the cross-EMSE ${\xi }_k^{\left(12\right)}\left[n\right]$ can never be simultaneously larger than the individual EMSEs ${\xi }_k^{\left(1\right)}\left[n\right]$ and ${\xi }_k^{\left(2\right)}\left[n\right]$.

We want to minimize (20) to find the optimum mixing parameter for each projection. The minimization must take into account that ${\lambda }_k\left[n\right]$ is contained in the interval $\left[0,1\right]$ and that (20) is nonnegative and quadratic in ${\lambda }_k\left[n\right]$ [1]. Therefore, the optimum mixing parameter is given by:

$${\lambda }_k^{\mathrm{o}}\left[n\right]={\left.\frac{{\xi }_k^{\left(2\right)}\left[n\right]-{\xi }_k^{\left(1\right)}\left[n\right]}{{\xi }_k^{\left(1\right)}\left[n\right]+{\xi }_k^{\left(2\right)}\left[n\right]-2{\xi }_k^{\left(12\right)}\left[n\right]}\right|}_0^1={\left.\frac{\Delta {\xi }_k^{\left(2\right)}\left[n\right]}{\Delta {\xi }_k^{\left(1\right)}\left[n\right]+\Delta {\xi }_k^{\left(2\right)}\left[n\right]}\right|}_0^1$$

(23)

where $\Delta {\xi }_k^{\left(j\right)}\left[n\right]={\xi }_k^{\left(j\right)}\left[n\right]-{\xi }_k^{\left(12\right)}\left[n\right]$, $j=1,2$, and the vertical line denotes the limits of the interval for the optimum mixing parameters.

We now substitute (23) in (20) to find the definition of the optimum EMSE for all the input projections:

$$\begin{array}{cc}\hfill {\xi }_k^{\mathrm{o}}\left[n\right]& ={\xi }_k^{\left(1\right)}\left[n\right]-\left(1-{\lambda }_k^{\mathrm{o}}\left[n\right]\right)\Delta {\xi }_k^{\left(1\right)}\left[n\right]\hfill \\ & \equiv {\xi }_k^{\left(2\right)}\left[n\right]-{\lambda }_k^{\mathrm{o}}\left[n\right]\Delta {\xi }_k^{\left(2\right)}\left[n\right].\hfill \end{array}$$

(24)

The expressions of the optimum EMSE are valid for any time instant n and also at steady state, i.e., for $n\to \infty$. It is worth noting from (23) and (24) that the combination depends on the value of the cross-EMSE. In particular, we can identify four different cases [1,6]:

Case 1:

${\xi }_k^{\left(1\right)}\left[n\right]\le {\xi }_k^{\left(12\right)}\left[n\right]<{\xi }_k^{\left(2\right)}\left[n\right]$.

For this case, it is easy to verify that $\Delta {\xi }_k^{\left(1\right)}\left[n\right]\le 0$ and $\Delta {\xi }_k^{\left(2\right)}\left[n\right]>0$. In this situation, the optimum mixing parameter is ${\lambda }_k^{\mathrm{o}}\left[n\right]=1$, and the combined scheme turns out to perform like the best individual filter, i.e., the one with the lower EMSE, which is the first filter. Indeed, if we replace ${\lambda }_k^{\mathrm{o}}\left[n\right]$ in (24), we achieve:

$${\xi }_k^{\mathrm{o}}\left[n\right]\approx {\xi }_k^{\left(1\right)}\left[n\right].$$

Case 2:

${\xi }_k^{\left(1\right)}\left[n\right]>{\xi }_k^{\left(12\right)}\left[n\right]\ge {\xi }_k^{\left(2\right)}\left[n\right]$.

Here, we have that $\Delta {\xi }_k^{\left(1\right)}\left[n\right]>0$ and $\Delta {\xi }_k^{\left(2\right)}\left[n\right]\le 0$. Again, the combined filter turns out to perform like the best individual filter, which in this case is the second one. As a matter of fact, the optimum mixing parameter is ${\lambda }_k^{\mathrm{o}}\left[n\right]=0$; thus, the optimum EMSE is:

$${\xi }_k^{\mathrm{o}}\left[n\right]\approx {\xi }_k^{\left(2\right)}\left[n\right].$$

Case 3:

${\xi }_k^{\left(12\right)}\left[n\right]<min\left({\xi }_k^{\left(j\right)}\left[n\right]\right)$, $j=1,2$.

In this case, considering (24), it is easy to conclude that the cross-EMSE is lower than both individual EMSEs; therefore, $\Delta {\xi }_k^{\left(j\right)}\left[n\right]>0$, $j=1,2$, while ${\lambda }_k^{\mathrm{o}}\left[n\right]\in \left(0,1\right)$. This result can be justified by the fact that since the correlation between the a priori errors of both individual components is small, their weighted combination provides an estimation error of reduced variance [1]. In this case, the optimum EMSE is still represented by (24).

Case 4:

${\xi }_k^{\left(1\right)}\left[n\right]={\xi }_k^{\left(2\right)}\left[n\right]={\xi }_k^{\left(12\right)}\left[n\right]$.

For completeness, we consider also this particular case, whose condition is rather rare to find in practice. In this case, we have $\Delta {\xi }_k^{\left(j\right)}\left[n\right]=0$, $j=1,2$; thus, the optimum EMSE is:

$${\xi }_k^{\mathrm{o}}\left[n\right]={\xi }_k^{\left(j\right)}\left[n\right]={\xi }_k^{\left(12\right)}\left[n\right].$$

From these cases, it can be gathered that when the cross-EMSE is comprised between the two individual EMSEs (i.e., Cases 1 and 2), the cross-correlation between the a priori errors of the two filters is high enough so that their combination can perform at most as the best individual filter. However, when the cross-EMSE is lower than the individual EMSEs (i.e., Case 3), a lower excess error is achieved for the combination.

In order to achieve the expressions of the EMSEs resulting from the above cases, we need to derive the individual EMSEs of the two MISO filters and the cross-EMSE. To this end, we analyze the mean squared performance of the individual MISO filters in Section 4, from which we derive the expressions of ${\xi }_k^{\left(j\right)}\left[n\right]=0$, $j=1,2$, and the mean squared performance of the combination in Section 5, from which we derive the cross-EMSE ${\xi }_k^{\left(12\right)}\left[n\right]$.

4. Mean Squared Performance of Individual MISO APA Filters

Individual EMSEs ${\xi }_k^{\left(j\right)}\left[n\right]$, for $j=1,2$ and $k=0,\dots ,K_{\max }$, can be derived by a straightforward extension of the analysis in [42] to the multichannel case. To this end, we derive the energy conservation arguments.

4.1. Energy Conservation Relation for MISO Filters

We can reformulate the update equation of the $j^{\mathrm{th}}$ MISO filter (4), $j=1,2$, in terms of the weight error vector (12):

$${\tilde{\mathbf{w}}}_n^{\left(j\right)}={\tilde{\mathbf{w}}}_{n-1}^{\left(j\right)}-{\mu }_j{\mathbf{X}}_n^{\left(j\right),T}{\left({\delta }_j\mathbf{I}+{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{\mathbf{e}}_n^{\left(j\right)}.$$

(25)

We left-multiply both sides of (25) by ${\mathbf{X}}_n^{\left(j\right)}$:

$${\mathbf{X}}_n^{\left(j\right)}{\tilde{\mathbf{w}}}_n^{\left(j\right)}={\mathbf{X}}_n^{\left(j\right)}{\tilde{\mathbf{w}}}_{n-1}^{\left(j\right)}-{\mu }_j{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}{\left({\delta }_j\mathbf{I}+{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{\mathbf{e}}_n^{\left(j\right)}$$

(26)

in which we replace the expressions of the a priori and a posteriori estimation errors, respectively (14) and (15), thus yielding:

$${{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right)}={{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}-{\mu }_j{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}{\left({\delta }_j\mathbf{I}+{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{\mathbf{e}}_n^{\left(j\right)}.$$

(27)

Assuming that ${\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}$ is invertible, we can rewrite (27) as:

$${\left({\delta }_j\mathbf{I}+{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{\mathbf{e}}_n^{\left(j\right)}=\frac{1}{{\mu }_j}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}\left({{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}-{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right)}\right).$$

(28)

Replacing (28) in (25) and rearranging appropriately, we obtain:

$${\tilde{\mathbf{w}}}_n^{\left(j\right)}+{\mathbf{X}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}={\tilde{\mathbf{w}}}_{n-1}^{\left(j\right)}+{\mathbf{X}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right)}.$$

(29)

We evaluate the energy of (29), thus obtaining the energy conservation relation for the $j^{\mathrm{th}}$ MISO filter:

$${∥{\tilde{\mathbf{w}}}_n^{\left(j\right)}∥}^2+{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}={∥{\tilde{\mathbf{w}}}_{n-1}^{\left(j\right)}∥}^2+{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right)}$$

(30)

in which no approximations are used, but it is an exact relation showing how the energies of the weight-error vectors at two successive iterations are related to the weighted energies of the a priori and a posteriori estimation error vectors [42].

4.2. Variance Relation for MISO Filters

Now, we analyze the energy conservation relation (30) in the mean squared sense. To this end, we take the expectations of both sides of (30) and get:

$$\mathrm{E}\left\{{∥{\tilde{\mathbf{w}}}_n^{\left(j\right)}∥}^2\right\}+\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}=\mathrm{E}\left\{{∥{\tilde{\mathbf{w}}}_{n-1}^{\left(j\right)}∥}^2\right\}+\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right)}\right\}$$

(31)

Considering that at steady state (i.e., for $n\to \infty$), the weights no longer change, we have that $\mathrm{E}\left\{{∥{\tilde{\mathbf{w}}}_n^{\left(j\right)}∥}^2\right\}=\mathrm{E}\left\{{∥{\tilde{\mathbf{w}}}_{n-1}^{\left(j\right)}∥}^2\right\}$, so (31) reduces to:

$$\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}=\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right)}\right\}.$$

(32)

Using (27), the right-hand side of (32) becomes:

$$\begin{array}{cc}\hfill {\mu }_j\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(j\right)}\right\}& =\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\left({\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}-{\mu }_j\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\mathbf{B}}_n^{\left(j\right)}{\mathbf{e}}_n^{\left(j\right)}\right\}\hfill \\ & -{\mu }_j\mathrm{E}\left\{{\mathbf{e}}_n^{\left(j\right),T}{\mathbf{B}}_n^{\left(j\right)}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}+{\mu }_j^2\mathrm{E}\left\{{\mathbf{e}}_n^{\left(j\right),T}{\mathbf{A}}_n^{\left(j\right)}{\mathbf{e}}_n^{\left(j\right)}\right\}\hfill \end{array}$$

(33)

where:

$$\begin{array}{cc}\hfill {\mathbf{B}}_n^{\left(j\right)}& ={\left({\delta }_j\mathbf{I}+{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}\right)}^{-1}\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill {\mathbf{A}}_n^{\left(j\right)}& ={\mathbf{B}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right)}{\mathbf{X}}_n^{\left(j\right),T}{\mathbf{B}}_n^{\left(j\right)}.\hfill \end{array}$$

(35)

Replacing (33) in (32), we obtain the variance relation for the $j^{\mathrm{th}}$ MISO filter:

$${\mu }_j\mathrm{E}\left\{{\mathbf{e}}_n^{\left(j\right),T}{\mathbf{A}}_n^{\left(j\right)}{\mathbf{e}}_n^{\left(j\right)}\right\}=\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\mathbf{B}}_n^{\left(j\right)}{\mathbf{e}}_n^{\left(j\right)}\right\}+\mathrm{E}\left\{{\mathbf{e}}_n^{\left(j\right),T}{\mathbf{B}}_n^{\left(j\right)}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}.$$

(36)

4.3. Steady-State Performance for MISO Filters

We want to derive the expressions of individual EMSE at steady state so, taking into account the definition of the a priori estimation error signal for the $j^{\mathrm{th}}$ MISO filter (14) and Assumption 2, and neglecting the dependency of ${\tilde{\mathbf{w}}}_{n-1}^{\left(j\right)}$ on past noise samples, we can approximate the variance relation (36) as:

$${\mu }_j\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\mathbf{A}}_n^{\left(j\right)}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}+{\mu }_j\mathrm{E}\left\{{\mathbf{v}}_n^{\left(j\right),T}{\mathbf{A}}_n^{\left(j\right)}{\mathbf{v}}_n^{\left(j\right)}\right\}=2\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\mathbf{B}}_n^{\left(j\right)}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}.$$

(37)

We use this expression to evaluate the EMSE for $n\to \infty$. However, for this purpose, we need a further assumption [42].

Assumption 4.

At steady state, the input signal ${\mathbf{x}}_n$ is statistically independent of the a priori estimation error signals ${{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}$, $j=1,2$, and moreover,

$$\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}\right\}=\mathrm{E}\left\{{\left|e_{\mathrm{a}}^{\left(j\right)}\left[n\right]\right|}^2\right\}·{\mathbf{S}}^{\left(j\right)}$$

(38)

where $e_{\mathrm{a}}^{\left(j\right)}\left[n\right]$ is the a priori estimation error sample for the first projection (i.e., $k=0$; we omit the projection order subscript for the simplicity of notation), and the matrix ${\mathbf{S}}^{\left(j\right)}\in {\mathbb{R}}^{K_j\times K_j}$ assumes the following values:

$${\mathbf{S}}^{\left(j\right)}\approx \left\{\begin{array}{cc}\mathbf{I},& \mathit{for}\mathit{small}\mathit{values}\mathit{of}{\mu }_j\hfill \\ \mathbf{F},& \mathit{for}\mathit{large}\mathit{values}\mathit{of}{\mu }_j\hfill \end{array}\right.$$

(39)

being $\mathbf{F}\in {\mathbb{R}}^{K_j\times K_j}=\mathrm{diag}\left\{\begin{array}{cccc}1& 0& \dots & 0\end{array}\right\}$.

The derivation of the approximation in Assumption 4 can be found in Appendix A.

Using Assumption 4, we can rewrite (37). In particular, the first term on the left-hand side is equivalently written as:

$$\begin{array}{cc}\hfill \mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\mathbf{A}}_n^{\left(j\right)}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}& ={\mu }_j\mathrm{Tr}\left(\mathrm{E}\left\{{e_{\mathrm{a}}}_n^{\left(j\right)}{e_{\mathrm{a}}}_n^{\left(j\right),T}{\mathbf{A}}_n^{\left(j\right)}\right\}\right)\hfill \\ & ={\mu }_j\mathrm{E}\left\{{\left|e_{\mathrm{a}}^{\left(j\right)}\left[n\right]\right|}^2\right\}\mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right).\hfill \end{array}$$

(40)

Similarly, we can write the remaining terms of (37) respectively as follows:

$$\begin{array}{cc}\hfill {\mu }_j\mathrm{E}\left\{{\mathbf{v}}_n^{\left(j\right),T}{\mathbf{A}}_n^{\left(j\right)}{\mathbf{v}}_n^{\left(j\right)}\right\}& ={\mu }_j{\sigma }_v^2\mathrm{Tr}\left(\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill 2\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right),T}{\mathbf{B}}_n^{\left(j\right)}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}\right\}& =2\mathrm{E}\left\{{\left|e_{\mathrm{a}}^{\left(j\right)}\left[n\right]\right|}^2\right\}\mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{B}}_n^{\left(j\right)}\right\}\right).\hfill \end{array}$$

(42)

Therefore, (37) becomes:

$$\left(2\mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{B}}_n^{\left(j\right)}\right\}\right)-{\mu }_j\mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)\right)\mathrm{E}\left\{{\left|e_{\mathrm{a}}^{\left(j\right)}\left[n\right]\right|}^2\right\}={\mu }_j{\sigma }_v^2\mathrm{Tr}\left(\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)$$

(43)

from which we can derive the expression of the steady-state EMSE of the individual MISO filters as:

$${\xi }_{\infty }^{\left(j\right)}=\underset{n\to \infty }{lim}\mathrm{E}\left\{{\left|e_{\mathrm{a}}^{\left(j\right)}\left[n\right]\right|}^2\right\}=\frac{{\mu }_j{\sigma }_v^2\mathrm{Tr}\left(\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)}{\left(2\mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{B}}_n^{\left(j\right)}\right\}\right)-{\mu }_j\mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)\right)}.$$

(44)

It is worth noting that Equation (44) takes into account the information related to each projection and contained in the matrices ${\mathbf{A}}_n^{\left(j\right)}$, ${\mathbf{B}}_n^{\left(j\right)}$ and ${\mathbf{S}}^{\left(j\right)}$, but at the same time, it provides a unique steady-state EMSE value for all the projections; therefore, the dependence of ${\xi }_{\infty }^{\left(j\right)}$ on index k disappears.

The expression above can be simplified when the regularization factor ${\delta }_j$ is small enough, so that ${\mathbf{A}}_n^{\left(j\right)}\approx {\mathbf{B}}_n^{\left(j\right)}$. This implies $\mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)\approx \mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{B}}_n^{\left(j\right)}\right\}\right)$; hence, the EMSE (44) reduces to:

$${\xi }_{\infty }^{\left(j\right)}=\frac{{\mu }_j{\sigma }_v^2}{\left(2-{\mu }_j\right)}\frac{\mathrm{Tr}\left(\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)}{\mathrm{Tr}\left({\mathbf{S}}^{\left(j\right)}\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)}.$$

(45)

Now, we can distinguish two further approximations.

(i)

Small value of ${\mu }_j$.

If we assume a small value of the step size ${\mu }_j$, we have for Assumption 4 that ${\mathbf{S}}^{\left(j\right)}=\mathbf{I}$; hence, (45) becomes:

$${\xi }_{\infty }^{\left(j\right)}=\frac{{\mu }_j{\sigma }_v^2}{\left(2-{\mu }_j\right)}.$$

(46)

(ii)

Large value of ${\mu }_j$.

If we assume a large value of the step size ${\mu }_j$ (i.e., close to one), we have for Assumption 4 that ${\mathbf{S}}^{\left(j\right)}=\mathbf{F}$; hence, (45) becomes:

$${\xi }_{\infty }^{\left(j\right)}=\frac{{\mu }_j{\sigma }_v^2}{\left(2-{\mu }_j\right)}\frac{\mathrm{Tr}\left(\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)}{\mathrm{E}\left\{a_{11}^{\left(j\right)}\left[n\right]\right\}}$$

(47)

where $a_{11}^{\left(j\right)}\left[n\right]$ is the element on the first row and first column of ${\mathbf{A}}_n^{\left(j\right)}$. However, we can also assume the following approximations:

$$\begin{array}{cc}\hfill \mathrm{Tr}\left(\mathrm{E}\left\{{\mathbf{A}}_n^{\left(j\right)}\right\}\right)& \approx \mathrm{E}\left\{\frac{K_j}{{∥{\mathbf{x}}_n∥}^2}\right\}\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill \frac{1}{\mathrm{E}\left\{a_{11}^{\left(j\right)}\left[n\right]\right\}}& \approx \mathrm{Tr}\left({\mathbf{R}}_n^{\left(j\right)}\right)\hfill \end{array}$$

(49)

that yield the following expression for the steady-state EMSE of the $j^{\mathrm{th}}$ MISO filter:

$${\xi }_{\infty }^{\left(j\right)}=\frac{{\mu }_j{\sigma }_v^2}{\left(2-{\mu }_j\right)}\mathrm{Tr}\left({\mathbf{R}}_n^{\left(j\right)}\right)\mathrm{E}\left\{\frac{K_j}{{∥{\mathbf{x}}_n∥}^2}\right\}$$

(50)

which depends proportionally on the value of the projection order $K_j$.

5. Mean Squared Performance of the Combination of MISO Filters

Now, we want to derive the expression of the cross-EMSE ${\xi }_k^{\left(12\right)}\left[n\right]$ by carrying out similar procedural steps of the previous section, but considering the cross-information, as explained in detail in the following.

In order to derive the energy conservation relation for the combined scheme, similarly to [6], we consider (29) with ${\mathbf{X}}_n\in {\mathbb{R}}^{K_{\max }\times 1}$ and multiply the transpose of (29) for $j=1$ by (29) itself for $j=2$, and get:

$${\tilde{\mathbf{w}}}_n^{\left(1\right),T}{\tilde{\mathbf{w}}}_n^{\left(2\right)}+{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right),T}{\left({\mathbf{X}}_n{\mathbf{X}}_n^T\right)}^{-1}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right)}={\tilde{\mathbf{w}}}_{n-1}^{\left(1\right),T}{\tilde{\mathbf{w}}}_{n-1}^{\left(2\right)}+{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(1\right),T}{\left({\mathbf{X}}_n{\mathbf{X}}_n^T\right)}^{-1}{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(2\right)}.$$

(51)

Taking the expectations of both sides of (51) and considering that at steady state $\mathrm{E}\left\{{\tilde{\mathbf{w}}}_n^{\left(1\right),T}{\tilde{\mathbf{w}}}_n^{\left(2\right)}\right\}=\mathrm{E}\left\{{\tilde{\mathbf{w}}}_{n-1}^{\left(1\right),T}{\tilde{\mathbf{w}}}_{n-1}^{\left(2\right)}\right\}$, we obtain:

$$\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right),T}{\left({\mathbf{X}}_n{\mathbf{X}}_n^T\right)}^{-1}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right)}\right\}=\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(1\right),T}{\left({\mathbf{X}}_n{\mathbf{X}}_n^T\right)}^{-1}{{\mathbf{e}}_{\mathrm{p}}}_n^{\left(2\right)}\right\}.$$

(52)

Replacing (27) (with ${\mathbf{X}}_n\in {\mathbb{R}}^{K_{\max }\times 1}$) in (52) and after some simplifications, similarly to what was done for (36), it is possible to achieve the variance relation for the combined scheme at steady state:

$${\mu }_1{\mu }_2\mathrm{E}\left\{{\mathbf{e}}_n^{\left(1\right),T}{\mathbf{A}}_n{\mathbf{e}}_n^{\left(2\right)}\right\}={\mu }_1\mathrm{E}\left\{{\mathbf{e}}_n^{\left(1\right),T}{\mathbf{B}}_n{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right)}\right\}+{\mu }_2\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right),T}{\mathbf{B}}_n{\mathbf{e}}_n^{\left(2\right)}\right\}$$

(53)

where ${\mathbf{A}}_n$ and ${\mathbf{B}}_n$ are defined similarly to (34) and (35), respectively, but over $K_{\max }$. Considering (14), we can rewrite (53), similarly to (37), as:

$${\mu }_1{\mu }_2\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right),T}{\mathbf{A}}_n{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right)}\right\}+{\mu }_1{\mu }_2\mathrm{E}\left\{{\mathbf{v}}_n^T{\mathbf{A}}_n{\mathbf{v}}_n\right\}=\left({\mu }_1+{\mu }_2\right)\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right),T}{\mathbf{B}}_n{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right)}\right\}.$$

(54)

Now, similarly to Assumption 4, but with extension to the case of combined MISO filters, we can define the following assumption.

Assumption 5.

At steady state, the input signal ${\mathbf{x}}_n$ is statistically independent of the a priori estimation error signals ${{\mathbf{e}}_{\mathrm{a}}}_n^{\left(j\right)}$, $j=1,2$, and moreover,

$$\mathrm{E}\left\{{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(1\right)}{{\mathbf{e}}_{\mathrm{a}}}_n^{\left(2\right),T}\right\}=\mathrm{E}\left\{\left|e_{\mathrm{a}}^{\left(1\right)}\left[n\right]e_{\mathrm{a}}^{\left(2\right)}\left[n\right]\right|\right\}·{\mathbf{S}}^{\left(12\right)}$$

(55)

where the value of ${\mathbf{S}}^{\left(12\right)}\in {\mathbb{R}}^{K_{\max }\times K_{\max }}$ is defined according to the following cases.

• If we characterize the combination scheme according to the step-size values, generally ${\mu }_1$ small and ${\mu }_2$ large, e.g., to find an optimal selection of filter parameters, we have that:

  $${\mathbf{S}}^{\left(12\right)}\approx \mathbf{F}$$

  (56)

  where $\mathbf{F}$ is defined similarly to (39).
• On the other hand, if we want to provide diversity to the combined scheme and choose different projection orders, but the same step-size value, i.e., ${\mu }_1={\mu }_2=\mu$, we have that:

  $${\mathbf{S}}^{\left(12\right)}\approx \left\{\begin{array}{cc}\mathbf{I},& \mathit{for}\mathit{small}\mathit{values}\mathit{of}\mu \hfill \\ \mathbf{F},& \mathit{for}\mathit{large}\mathit{values}\mathit{of}\mu \hfill \end{array}\right..$$

  (57)

  Equation (57) holds as long as the SNR is high, as detailed in Appendix B.

The derivation of the approximation (55) in Assumption 5 can be found in Appendix B.

Using Assumption 5 and after some manipulations, similarly to what was done for (44), it is possible to derive the expression of the cross-EMSE from (54):

$${\xi }_{\infty }^{\left(12\right)}=\underset{n\to \infty }{lim}\mathrm{E}\left\{\left|e_{\mathrm{a}}^{\left(1\right)}\left[n\right]e_{\mathrm{a}}^{\left(2\right)}\left[n\right]\right|\right\}=\frac{{\mu }_1{\mu }_2{\sigma }_v^2\mathrm{Tr}\left(\mathrm{E}\left\{{\mathbf{A}}_n\right\}\right)}{\left({\mu }_1+{\mu }_2\right)\mathrm{Tr}\left({\mathbf{S}}^{\left(12\right)}\mathrm{E}\left\{{\mathbf{B}}_n\right\}\right)-{\mu }_1{\mu }_2\mathrm{Tr}\left({\mathbf{S}}^{\left(12\right)}\mathrm{E}\left\{{\mathbf{A}}_n\right\}\right)}.$$

(58)

As done in the previous section for ${\xi }_{\infty }^{\left(j\right)}$, we may further simplify also (58) when the regularization factor ${\delta }_j$ is small enough to be neglected, so that ${\mathbf{A}}_n\approx {\mathbf{B}}_n$, which implies:

$${\xi }_{\infty }^{\left(12\right)}=\frac{{\mu }_{12}{\sigma }_v^2\mathrm{Tr}\left(\mathrm{E}\left\{{\mathbf{A}}_n\right\}\right)}{\left(1-{\mu }_{12}\right)\mathrm{Tr}\left({\mathbf{S}}^{\left(12\right)}\mathrm{E}\left\{{\mathbf{A}}_n\right\}\right)}$$

(59)

where ${\mu }_{12}={\mu }_1{\mu }_2/\left({\mu }_1+{\mu }_2\right)$. Equation (59) can be further reduced according to the values of the step sizes. In particular, similarly to what was done for ${\xi }_{\infty }^{\left(j\right)}$ in Section 4.3, we can consider the following approximations.

(i)

Small values for both ${\mu }_j$.

This case is typical when we want to differentiate the combined scheme according to the projection order and we choose the same small value for both step sizes ${\mu }_1={\mu }_2=\mu$. Based on Assumption 5, we have that ${\mathbf{S}}^{\left(12\right)}=\mathbf{I}$; hence, (59) becomes:

$${\xi }_{\infty }^{\left(12\right)}=\frac{{\mu }_{12}{\sigma }_v^2}{\left(1-{\mu }_{12}\right)}.$$

(60)

(ii)

Large value for at least one ${\mu }_j$.

We can also consider the case for which ${\mathbf{S}}^{\left(j\right)}=\mathbf{F}$, according to Assumption 5. This may occur when we want to characterize the combined scheme according to the projection order and we choose the same large value for both step sizes ${\mu }_1={\mu }_2=\mu$, or also when we choose the same projection order, but one step-size value small and the other one large (close to one). In both of these situations, after some approximations similarly to (50), we have that (59) becomes:

$${\xi }_{\infty }^{\left(12\right)}=\frac{{\mu }_{12}{\sigma }_v^2}{\left(1-{\mu }_{12}\right)}\mathrm{Tr}\left({\mathbf{R}}_n\right)\mathrm{E}\left\{\frac{K_{\max }}{{∥{\mathbf{x}}_n∥}^2}\right\}$$

(61)

which depends proportionally on the value of the maximum projection order $K_{\max }$ between the two MISO filters.

We can note that, similarly to the case of the individual EMSEs in (44), even the steady-state cross-EMSE (58) provides a unique value for all the projections, although it considers the information related to them. This also implies a unique value at steady state for the optimum mixing parameter for all the projections. Therefore, considering (44) and (58), and related approximations, the expression of the theoretical EMSE of the combined MISO scheme (24) can be written at steady state as:

$$\begin{array}{cc}\hfill {\xi }_{\infty }& ={\xi }_{\infty }^{\left(1\right)}\left[n\right]-\left(1-{\lambda }_{\infty }\right)\Delta {\xi }_{\infty }^{\left(1\right)}\hfill \\ & \equiv {\xi }_{\infty }^{\left(2\right)}-{\lambda }_{\infty }\Delta {\xi }_{\infty }^{\left(2\right)}\hfill \end{array}$$

(62)

according to which the four different cases described in Section 3 can be easily derived.

6. Simulation Results

We evaluate the theoretical results derived in the previous sections by considering a system identification scenario. The unknown MISO system, described by the model of (11), was composed of $P=5$ impulse responses, each one formed with $M=7$ independent random values between $-1$ and one. The input signal was generated by means of a first-order autoregressive model, whose transfer function is $\sqrt{1-{\alpha }^2}/$$\left(1-\alpha z^{-1}\right)$, with $\alpha =0.8$, fed with an i.i.d. Gaussian random process. The length of the input signal was $L=$ 20,000 samples. An additive i.i.d. noise signal $v\left[n\right]$ with variance ${\sigma }_v^2=0.01$ (i.e., providing 20 dB of SNR) was added to form the desired signal. We took into account a combined MISO filter to identify the unknown MISO system. We might differentiate the two individual MISO filters of the combined scheme according to the step-size values or to the projection orders.

The theoretical EMSEs were achieved by considering (24), involving (45) and related approximations for individual EMSEs and (59) and related approximations for the cross-EMSE. The measured steady-state EMSEs were achieved according to:

$$\mathrm{EMSE}\left[n\right]=\mathrm{E}\left\{{\left(e\left[n\right]-v\left[n\right]\right)}^2\right\}$$

(63)

evaluated over 50 independent runs. We also report the measured EMSEs of the two individual MISO filters.

6.1. Performance Evaluation Using Different Step-Size Values

First, we analyzed the former case by choosing different step-size values for the two MISO filters. We chose the same projection order $K_1=K_2=4$ for both the MISO filters, and we varied the step-size values. In particular, we took into account the range of values $\left[0.001,1\right]$ for ${\mu }_2$, which represents the larger step size, and we set ${\mu }_1={\mu }_2/q$, where q is a proportionality factor. The parameter setting for the adaptive combination was characterized by: ${\mu }_{\mathrm{c}}=0.5$, $a_k\left[0\right]=0$ and $r_k\left[0\right]=1$ for $k=0,\dots ,K_{\max }-1$.

The results are shown in Figure 2 with different settings of the factor q, i.e., 4, 10, 25, and 100. These results prove the validity of the derived theoretical EMSE (59), as the measured EMSE values approached the theoretical ones. It is worth noting, in Figure 2, the effect of the combination, especially for small step-size values and high q.

We also evaluated the convergence performance of the combined MISO scheme. The scenario was similar to the previous experiment, but in this case, we considered $P=8$ unknown channels, each one formed by $M=10$ independent random values between $-1$ and one. The input signal was again generated by using a first-order autoregressive model, and its length was $L=$ 40,000. In order to study the ability of the combined filtering scheme to react to abrupt changes in the scenario, the coefficients of all the unknown channels were randomly reassigned at time instant $n=L/2$. Again, we considered additive noise with ${\sigma }_v^2=0.01$ at the output of the unknown MISO system. We chose a small step-size value ${\mu }_1=0.001$ for the first MISO filter and a larger one ${\mu }_2=0.1$ for the second MISO filter, while we considered the same projection order $K_1=K_2=2$ for both filters. Performance was evaluated in terms of the measured EMSE, averaged over 10,000 independent runs. We compared the adaptive combined MISO filter with the corresponding individual filters. Results are depicted in Figure 3, where it is worth noting that the MISO filter with ${\mu }_1$ shows a slow convergence rate, but a good precision at steady state, while the MISO filter with ${\mu }_2$ provides a faster convergence rate, but lower precision. The combined MISO filter was capable of exploiting the advantages of both the individual MISO filters, thus showing a fast convergence rate and good precision at steady state. Moreover, a performance gain due to the cross-EMSE was visible when the mixing parameter was changing its value from zero to one. In Figure 3, the evolution of the current projection of the mixing parameter, i.e., for $k=0$, is also depicted to highlight the changing behavior of the combined MISO filter.

6.2. Performance Evaluation Using Different Projection Orders

We analyzed also the second case of combination by differentiating the MISO filters according to the projection orders, which is very important since the combination of APA-based filters with different projection orders is able to provide diversity to the combined scheme [1,16]. In particular, we chose $K_1=1$ for the first MISO filter, which turns out to be a multichannel NLMSfilter, and $K_2=4$ for the second MISO filter. The parameter setting for the adaptive combination was the same as used for the first case. Different selections have been chosen for the step-size value, which was the same for both the individual MISO filters. We compared the theoretical values of the EMSE of (59) for the combined architecture with the measured steady-state EMSEs of (63) over 50 independent runs. We also compared the measured EMSEs of the two individual MISO filters. Results are shown in Figure 4 and prove the validity of (59), as again, the measured EMSE values approached the theoretical ones.

Even for this case, we evaluated the convergence performance of the combined MISO scheme. The scenario was the same as in the first case, and the input signal length was $L=$ 24,000. Again, we considered an abrupt change at time instant $n=L/2$. The combined MISO filter was composed of two individual MISO filters having different projection orders, $K_1=1$ and $K_2=4$, and same step sizes, ${\mu }_1={\mu }_2=0.01$. Performance was evaluated in terms of the measured EMSE (63), averaged over 10,000 independent runs. We compared the adaptive combined MISO filter with the corresponding individual filters. The results are depicted in Figure 5, where it is possible to notice that even in this case, the combined MISO filter was capable of exploiting the advantages of both the individual MISO filters, as also highlighted by the evolution of the mixing parameter on the current projection (i.e., ${\lambda }_0\left[n\right]$).

7. Conclusions

In this paper, we have studied an adaptive combination of MISO filters, based on the adaptation by the multichannel APA, which allows the filters to be differentiated according to the step size or to the projection order. We have analyzed the steady-state performance of such a combined scheme, thus deriving a theoretical value for the EMSE based on the energy conservation properties. Simulation results have proven the validity of such a theoretical study. We have also assessed the performance improvement involved by an adaptive combined MISO filter using different projection orders in a changing scenario. Future research lines include the analysis of tracking and the transient performance of a combined MISO filter, as well as the extension to multi-stage combined filtering architectures. Moreover, the scheme can be evaluated in practical system identification problems, like acoustics or communications applications, where its performance can be compared with that of alternative single-MISO adaptive filters, such as multichannel variable step-size APA filters.