Tensor-Based Recursive Least-Squares Adaptive Algorithms with Low-Complexity and High Robustness Features

Abstract

The recently proposed tensor-based recursive least-squares dichotomous coordinate descent algorithm, namely RLS-DCD-T, was designed for the identification of multilinear forms. In this context, a high-dimensional system identification problem can be efficiently addressed (gaining in terms of both performance and complexity), based on tensor decomposition and modeling. In this paper, following the framework of the RLS-DCD-T, we propose a regularized version of this algorithm, where the regularization terms are incorporated within the cost functions. Furthermore, the optimal regularization parameters are derived, aiming to attenuate the effects of the system noise. Simulation results support the performance features of the proposed algorithm, especially in terms of its robustness in noisy environments.

1. Introduction

Nowadays, tensor-based signal processing methods are employed in many important real-world applications [1]. Such techniques can provide efficient solutions for big data problems [2], machine learning field [3], and source separation applications [4]. Recently, the iterative version of the Wiener filter was proposed for the identification of multilinear forms [5], e.g., for high-dimensional system identification problems (with a large parameter space).

Among the most popular adaptive filters, the recursive least-squares (RLS) algorithm represents a notable choice [6,7]. This algorithm is well-known for its fast convergence rate, especially when processing correlated input signals, but also for its high computational complexity. Despite their prohibitive nature, in classical and tensor-based structures, the RLS methods outperform their popular counterparts, such as the affine projection algorithm (APA) [8] and the algorithms based on the least-mean-square (LMS) method [9,10,11,12,13,14,15], which have been preferred in real-world applications due to their low arithmetic requirements. However, due to the tensor-based approach, the RLS algorithms designed for the identification of bilinear and trilinear forms [16,17] have become computationally efficient, as compared to the conventional LMS solutions. Furthermore, the recently proposed tensor-based RLS algorithm (RLS-T) [5] is tailored for the identification of multilinear forms. Basically, the high-dimensional system identification problem can be solved by using a decomposition into lower-dimensional structures, tensorized together.

Another tensor-based efficient adaptive filter that employs the combination between the RLS algorithm and the dichotomous coordinate descent (DCD) iterations, namely RLS-DCD-T [18], has been recently released. This algorithm uses a tensorial decomposition for multilinear forms and it has proven to be a powerful solution for tensor-based least-squares adaptive systems with low-complexity [19,20]. The DCD portion of the algorithm exploits the nature of the corresponding correlation matrices and solves auxiliary systems of equations using only additions and bit-shifts, as an alternative to traditional matrix inversion methods. Motivated by the appealing performance of the RLS-DCD-T, we aim to improve its robustness in noisy environments and we develop a regularized version of this algorithm. We will propose a variable regularized version in order to mitigate effects associated with non-stationary environments.

The regularization parameters of the RLS-DCD-T algorithm are usually used in the initialization stage (in order to prevent numerical problems). However, their influence is practically annihilated in the steady-state due to the forgetting factors [6]. The idea of performing regularization for the entire run of the adaptive systems has been initially introduced with the classical RLS algorithm [21]. In order to attenuate the effects of the system noise, optimal expressions were derived for the regularization parameters, which are related to the signal-to-noise ratio (SNR). The methods have also been successfully applied on the RLS-NKP algorithm [22] and on the RLS algorithms designed for the identification of bilinear [16], respectively, trilinear [23] forms.

Following this introduction, Section 2 provides a brief review of the multilinear model, then, in Section 3, the RLS-DCD-T algorithm and the proposed regularized RLS-DCD-T algorithm are explained. Simulation results are provided in Section 4, while finally, Section 5 concludes the paper.

2. System Context

We consider N impulse response vectors ${\mathbf{h}}_i$ of lengths $L_i,i=1,2,\dots ,N$, which can be combined to form a larger multiple-input/single-output (MISO) global system. The N vectors describe the same number of corresponding individual channels:

$$\begin{array}{c}\hfill {\mathbf{h}}_i={\left[h_{i,1}{h}_{i,2}\cdots h_{i,L_i}\right]}^T,\end{array}$$

(1)

where the superscript ${}^T$ denotes the transpose operation and $h_{i,L_i}$ denotes the filter weights. For the rest of the paper, i will be employed for denoting elements associated with the i-th channel. The real-valued output signal corresponding to the global system at discrete-time index n can be expressed as:

$$\begin{array}{c}\hfill y\left(n\right)=\sum _{l_1=1}^{L_1}\sum _{l_2=1}^{L_2}\cdots \sum _{l_N=1}^{L_N}{x}_{l_1{l}_2\dots l_N}\left(n\right)h_{1,l_1}{h}_{2,l_2}\cdots h_{N,l_N},\end{array}$$

(2)

where the input signals from (2) can be aggregated to form the tensorial structure $\mathit{𝓧}\left(n\right)\in {\mathbb{R}}^{L_1\times L_2\times \cdots \times L_N}$ using the components ${\left(\mathit{𝓧}\right)}_{l_1{l}_2\dots l_N}\left(n\right)=x_{l_1{l}_2\dots l_N}\left(n\right)$. Consequently, the input-output structure in (2) can be expressed as a multilinear form using the mode-i product ${\times }_i$, explained in [24]:

$$\begin{array}{c}\hfill y\left(n\right)=\mathit{𝓧}\left(n\right){\times }_1{\mathbf{h}}_1^T{\times }_2{\mathbf{h}}_2^T{\times }_3\cdots {\times }_N{\mathbf{h}}_N^T.\end{array}$$

(3)

If $N-1$ of the vectors are considered fixed, then $y\left(n\right)$ represents a linear function of the remaining vector ${\mathbf{h}}_i$.

The expression in (3) will be further developed in order to obtain a more intuitive form. The vectors ${\mathbf{h}}_i$ will be combined using the outer product ∘, which can be expressed for two vectors as ${\mathbf{h}}_1\circ {\mathbf{h}}_2={\mathbf{h}}_1{\mathbf{h}}_2^T$ [15,18]. A new tensor $\mathbf{𝓗}\in {\mathbb{R}}^{L_1\times L_2\times \cdots \times L_N}$ can be defined as:

$$\begin{array}{c}\hfill \mathbf{𝓗}={\mathbf{h}}_1\circ {\mathbf{h}}_2\circ \cdots \circ {\mathbf{h}}_N,\end{array}$$

(4)

which is constituted with the elements ${\left(\mathbf{𝓗}\right)}_{l_1,l_2,\dots ,l_N}=h_{1,l_1}{h}_{2,l_2}\cdots h_{N,l_N}$. Furthermore, the vectorization operation applied to (4) leads to:

$$\begin{array}{c}\hfill \mathrm{vec}\left(\mathbf{𝓗}\right)={\mathbf{h}}_N\otimes {\mathbf{h}}_{N-1}\otimes \cdots \otimes {\mathbf{h}}_1,\end{array}$$

(5)

where, for any two vectors, $\mathrm{vec}\left({\mathbf{h}}_1\circ {\mathbf{h}}_2\right)={\mathbf{h}}_2\otimes {\mathbf{h}}_1$, with ⊗ denoting the Kronecker product. Consequently, the output signal in (2) can be written as:

$$\begin{array}{c}\hfill y\left(n\right)={\mathrm{vec}}^T\left(\mathbf{𝓗}\right)\mathrm{vec}\left[\mathit{𝓧}\left(n\right)\right],\end{array}$$

(6)

where

$$\begin{array}{c}\hfill \mathrm{vec}\left[\mathit{𝓧}\left(n\right)\right]=\left[\begin{array}{c}\mathrm{vec}\left[{\mathbf{X}}_{::\dots :1}\left(n\right)\right]\\ \mathrm{vec}\left[{\mathbf{X}}_{::\dots :2}\left(n\right)\right]\\ ⋮\\ \mathrm{vec}\left[{\mathbf{X}}_{::\dots :L_N}\left(n\right)\right]\end{array}\right],\end{array}$$

(7)

with ${\mathbf{X}}_{::\dots :l_i}\left(n\right)\in {\mathbb{R}}^{L_1\times L_2\times \cdots \times L_{N-1}}$ representing the frontal slices of $\mathit{𝓧}\left(n\right)$.

We can employ the notation:

$$\begin{array}{c}\hfill \mathbf{x}\left(n\right)=\mathrm{vec}\left[\mathit{𝓧}\left(n\right)\right]\end{array}$$

(8)

for the input vector of length L = $L_1{L}_2\cdots L_N$ and we can express the global impulse response of length $L_1{L}_2\cdots L_N$ as:

$$\begin{array}{c}\hfill \mathbf{g}=\mathrm{vec}\left(\mathbf{𝓗}\right)={\mathbf{h}}_N\otimes {\mathbf{h}}_{N-1}\otimes \cdots \otimes {\mathbf{h}}_1.\end{array}$$

(9)

Consequently, using (6), (8) and (9), we can write the output signal as:

$$\begin{array}{c}\hfill y\left(n\right)={\mathbf{g}}^T\mathbf{x}\left(n\right).\end{array}$$

(10)

The problem associated with the presented model is to indirectly find $\mathbf{g}$, by generating estimates for its individual components ${\mathbf{h}}_i$, using the so-called reference signal, given by the expression:

$$\begin{array}{c}\hfill d\left(n\right)={\mathbf{g}}^T\mathbf{x}\left(n\right)+w\left(n\right),\end{array}$$

(11)

where $w\left(n\right)$ is a noise signal uncorrelated with the output $y\left(n\right)$ [15,25]. Considering that an estimate of the global impulse response $\widehat{\mathbf{g}}$ is provided at every time index n, the corresponding a priori error signal can be written as:

$$\begin{array}{c}\hfill e\left(n\right)=d\left(n\right)-{\widehat{\mathbf{g}}}^T\mathbf{x}\left(n\right).\end{array}$$

(12)

The non-stationary nature of $\mathbf{g}$ requires a solution that is capable to update the estimate $\widehat{\mathbf{g}}$ with respect to any changes that might affect the global unknown system. Such condition eliminates the Wiener filter [6] as a valid option. Moreover, the adaptive systems that can be implemented in order to estimate $\mathbf{g}$ fail to provide satisfactory results when they are working with large adaptive filters (hundreds/thousands of coefficients). The limited convergence speeds reduce tracking capabilities, even when employing the RLS family of adaptive methods, which are designed to mitigate the correlation of the input signals better than the classical workhorse of the industry (i.e., the LMS family of adaptive systems).

In [18,26], the nature of the described MISO systems was exploited by splitting the problem of identifying $\mathbf{g}$ into multiple smaller problems, which target the separate estimation of the ${\mathbf{h}}_i$ systems. The tracking capabilities proved to be drastically improved using considerable less arithmetic effort, with respect to the classical one-filter configurations. Moreover, the RLS-based adaptive algorithms combined with the DCD iterations were employed, thus offering a numerically stable RLS version with more appealing properties in terms of required chip areas. Consequently, an important remaining development direction is to improve the proposed solution’s robustness in low SNR conditions.

3. Robust Tensor-Based RLS Adaptive Algorithms

Despite the tracking speed improvements brought by employing fast converging algorithms (such as the RLS-based ones) with the split approach described in the previous section, the robustness in scenarios with high interference level is still an important topic to be studied. We will describe the recently proposed combination between the RLS adaptive algorithm and the DCD iterations working within the tensor-based framework for the identification of MISO systems. Subsequently, we will propose a variable regularization method for the correlation matrices associated with the RLS algorithm, which will considerably slow (or even halt) the update process corresponding to the filter estimates ${\widehat{\mathbf{h}}}_i\left(n\right)$, when high noise levels bias the reference signal.

3.1. Tensor-Based Recursive Least-Squares Dichotomous Coordinate Descent Algorithm (RLS-DCD-T)

For the estimated impulse responses ${\widehat{\mathbf{h}}}_i\left(n\right)$ of the channels we can define the corresponding a priori error signals:

$$\begin{array}{cc}\hfill e_{{\widehat{\mathbf{h}}}_i}\left(n\right)& =d\left(n\right)-y_{{\widehat{\mathbf{h}}}_i}\left(n\right)\hfill \end{array}$$

(13)

where $y_{{\widehat{\mathbf{h}}}_i}\left(n\right)$ are the individual outputs of the N channels. We can employ the vector $\mathbf{x}\left(n\right)$ defined in (7) and (8) to write the individual inputs of the N channels as:

$$\begin{array}{c}\hfill {\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)={\left[{\widehat{\mathbf{h}}}_N(n-1)\otimes {\widehat{\mathbf{h}}}_{N-1}(n-1)\otimes \cdots \otimes {\widehat{\mathbf{h}}}_{i+1}(n-1)\otimes {\mathbf{I}}_{L_i}\otimes {\widehat{\mathbf{h}}}_{i-1}(n-1)\otimes \cdots \otimes {\widehat{\mathbf{h}}}_2(n-1)\otimes {\widehat{\mathbf{h}}}_1(n-1)\right]}^T\mathbf{x}\left(n\right),\end{array}$$

(14)

where ${\mathbf{I}}_{L_i}$ denotes the identity matrix of size $L_i\times L_i$.

Consequently, the individual output signals can be expressed as:

$$\begin{array}{cc}\hfill y_{{\widehat{\mathbf{h}}}_i}\left(n\right)& ={\widehat{\mathbf{h}}}_i^T(n-1){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right).\hfill \end{array}$$

(15)

Moreover, it can be demonstrated that $e_{{\widehat{\mathbf{h}}}_1}\left(n\right)=e_{{\widehat{\mathbf{h}}}_2}\left(n\right)=\cdots =e_{{\widehat{\mathbf{h}}}_N}\left(n\right)$ [17,26].

When applying the RLS algorithm in order to estimate $\mathbf{g}\left(n\right)$ (of length L) the cost function can be written as:

$$\begin{array}{c}\hfill J\left[\widehat{\mathbf{g}}\left(n\right)\right]=\sum _{k=1}^n{\lambda }^{n-k}{\left[d\left(k\right)-{\widehat{\mathbf{g}}}^T\left(n\right)\mathbf{x}\left(k\right)\right]}^2,\end{array}$$

(16)

with the forgetting factor $0<\lambda \le 1$. Considering that the MISO system can be split in concordance with the N channels, with the corresponding impulse responses ${\mathbf{h}}_i$ of lengths $L_i$, the cost function for each channel can be written as:

$$\begin{array}{c}\hfill J_{{\widehat{\mathbf{h}}}_N,{\widehat{\mathbf{h}}}_{N-1},\dots ,{\widehat{\mathbf{h}}}_{i+1},{\widehat{\mathbf{h}}}_{i-1},\dots ,{\widehat{\mathbf{h}}}_2,{\widehat{\mathbf{h}}}_1}\left[{\widehat{\mathbf{h}}}_i\left(n\right)\right]=\sum _{k=1}^n{\lambda }_i^{n-k}{\left[d\left(k\right)-{\widehat{\mathbf{h}}}_i^T\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(k\right)\right]}^2,\end{array}$$

(17)

where ${\lambda }_i$ are the individual forgetting factors. The minimization of every single cost function (17), with respect to ${\widehat{\mathbf{h}}}_i\left(n\right)$, can be performed by considering the other $N-1$ components fixed. The optimization is applied for the remaining one. When processing (17), we use the correlation matrices of each i channel:

$$\begin{array}{cc}\hfill {\mathbf{R}}_i\left(n\right)& =\sum _{k=1}^n{\lambda }_i^{n-k}{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(k\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(k\right)={\lambda }_i{\mathbf{R}}_i(n-1)+{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(n\right),\hfill \end{array}$$

(18)

respectively, the cross-correlation vectors between ${\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$ and $d\left(n\right)$:

$$\begin{array}{cc}\hfill {\mathbf{p}}_i\left(n\right)& =\sum _{k=1}^n{\lambda }_i^{n-k}{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(k\right)d\left(k\right)={\lambda }_i{\mathbf{p}}_i(n-1)+{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)d\left(n\right)\hfill \end{array}$$

(19)

and we obtain the set of normal equations [6]:

$$\begin{array}{c}\hfill {\mathbf{R}}_i\left(n\right){\widehat{\mathbf{h}}}_i\left(n\right)={\mathbf{p}}_i\left(n\right).\end{array}$$

(20)

We obtain the N update relations corresponding to the individual filters by solving Equation (20). The straightforward solution (and the most complex) is to directly compute the inverse matrices ${\mathbf{R}}_i^{-1}\left(n\right)$. However, even for filters with tens of coefficients, this approach would often require too many chip resources. The prohibitive nature of the obvious solution led to extensive research efforts, which tried to find the best compromise between complexity on one side, respectively, performance and numerical stability, on the other side. The solution provided by the matrix inversion lemma (a.k.a. Woodbury’s matrix identity) [6] was often used as a reference in recent literature (i.e., an acceptable step in the right direction) [16,22]. However, the method is still too costly for real-time implementations.

In recent literature [27,28], the combination between the RLS algorithm and the DCD iterations demonstrated good performance, with an arithmetic complexity proportional to the filter length. The method defines the residual vector:

$$\begin{array}{c}\hfill {\mathbf{r}}_i(n-1)={\mathbf{p}}_i(n-1)-{\mathbf{R}}_i(n-1){\widehat{\mathbf{h}}}_i(n-1),\end{array}$$

(21)

which is based on the consideration that a solution was provided at time index $n-1$ for (20). Several notations are also introduced to track the modifications brought to other elements of (20) at consecutive time indexes [18]:

$$\begin{array}{cc}\hfill \Delta {\mathbf{R}}_i\left(n\right)& ={\mathbf{R}}_i\left(n\right)-{\mathbf{R}}_i(n-1),\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill \Delta {\mathbf{p}}_i\left(n\right)& ={\mathbf{p}}_i\left(n\right)-{\mathbf{p}}_i(n-1),\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill \Delta {\mathbf{h}}_i\left(n\right)& ={\mathbf{h}}_i\left(n\right)-{\widehat{\mathbf{h}}}_i(n-1),\hfill \end{array}$$

(24)

where $\Delta {\mathbf{R}}_i\left(n\right)$ is a $L_i\times L_i$ matrix, and $\Delta {\mathbf{p}}_i\left(n\right)$, respectively, $\Delta {\mathbf{h}}_i\left(n\right)$, are $L_i$-valued column vectors.

The proposed method avoids computing a direct solution for the normal set of equations by writing (20) as:

$$\begin{array}{c}\hfill {\mathbf{R}}_i\left(n\right)\Delta {\mathbf{h}}_i\left(n\right)={\mathbf{p}}_i\left(n\right)-{\mathbf{R}}_i\left(n\right){\widehat{\mathbf{h}}}_i(n-1)={\mathbf{p}}_{0,i}\left(n\right),\end{array}$$

(25)

where $\Delta {\mathbf{h}}_i\left(n\right)$ are the solutions that will be estimated for each channel. Equation (25) can be modified to eliminate ${\mathbf{p}}_i\left(n\right)$, respectively, ${\mathbf{R}}_i\left(n\right)$, by using (22) and (23), then (21):

$$\begin{array}{c}\hfill {\mathbf{p}}_{0,i}\left(n\right)={\mathbf{r}}_i(n-1)+\Delta {\mathbf{p}}_i\left(n\right)-\Delta {\mathbf{R}}_i\left(n\right){\widehat{\mathbf{h}}}_i(n-1).\end{array}$$

(26)

Consequently, the new filter coefficients are obtained by adding the solutions vectors to the filter estimates from the previous time index:

$$\begin{array}{c}\hfill {\widehat{\mathbf{h}}}_i\left(n\right)={\widehat{\mathbf{h}}}_i(n-1)+\Delta {\widehat{\mathbf{h}}}_i\left(n\right).\end{array}$$

(27)

The updates for the filter estimates obtained from solving (25) are based on the filter coefficients ${\widehat{\mathbf{h}}}_i(n-1)$ from the previous time index. The approach is suitable for DCD iterations [27,29], which requires considerable less arithmetic resources (in comparison to the direct approaches) and also provides updated values for the residual vectors ${\mathbf{r}}_i\left(n\right)$. By employing (23) and (24), then ${\mathbf{p}}_{0,i}$, in

$$\begin{array}{c}\hfill {\mathbf{r}}_i\left(n\right)={\mathbf{p}}_i\left(n\right)-{\mathbf{R}}_i\left(n\right){\widehat{\mathbf{h}}}_i\left(n\right),\end{array}$$

(28)

we can write:

$$\begin{array}{c}\hfill {\mathbf{r}}_i\left(n\right)={\mathbf{p}}_{0,i}\left(n\right)-{\mathbf{R}}_i\left(n\right)\Delta {\widehat{\mathbf{h}}}_i\left(n\right).\end{array}$$

(29)

We will also use (22) in (18), respectively, (23) in (19), to obtain:

$$\begin{array}{cc}\hfill \Delta {\mathbf{R}}_i\left(n\right)& =({\lambda }_i-1){\mathbf{R}}_i(n-1)+{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(n\right),\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill \Delta {\mathbf{p}}_i\left(n\right)& =({\lambda }_i-1){\mathbf{p}}_i(n-1)+d\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right).\hfill \end{array}$$

(31)

Using (30) and (31), the matrix $\Delta {\mathbf{R}}_i\left(n\right)$ and the vector $\Delta {\mathbf{p}}_i\left(n\right)$ will be replaced in (26), then with (21) and (13), we can write:

$$\begin{array}{}& {\mathbf{p}}_{0,i}\left(n\right)& ={\mathbf{r}}_i(n-1)+({\lambda }_i-1){\mathbf{p}}_i(n-1)+d\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)\mathrm{(32)}& & -({\lambda }_i-1){\mathbf{R}}_i(n-1){\widehat{\mathbf{h}}}_i(n-1)-{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(n\right){\widehat{\mathbf{h}}}_i(n-1)\mathrm{(33)}& & ={\lambda }_i{\mathbf{r}}_i(n-1)+{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)e_{{\widehat{\mathbf{h}}}_i}\left(n\right).\hfill \end{array}$$

The last expression shows how the right side of (25) can be updated with minimum arithmetic effort, a task which is solved by the DCD portion of the algorithm. The overall RLS-DCD adaptive algorithm based on tensorial forms (also called RLS-DCD-T) is presented in Algorithm 1.

Algorithm 1 Exponential Weighted RLS-T Algorithm for a single channel.

$\mathrm{Initialization}$:

Set ${\widehat{\mathbf{h}}}_i\left(0\right)={\mathbf{0}}_{L_i\times 1},{\mathbf{r}}_i\left(0\right)={\mathbf{0}}_{L_i\times 1}$

0    ${\mathbf{R}}_i^{-1}\left(0\right)={\delta }_i{\mathbf{I}}_{L_i},{\delta }_i>0$

${\lambda }_i=1-\frac{1}{K_i{L}_i},K_i\ge 1$

$\mathrm{For}n=1,2,\dots ,\mathrm{number}\mathrm{of}\mathrm{iterations}:$

1   Compute ${\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$, based on (14)

2   ${\mathbf{R}}_i^{\left(0\right)}\left(n\right)={\lambda }_i{\mathbf{R}}_i^{\left(0\right)}(n-1)+{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^{\left(0\right)}\left(n\right)$

3   $y_{{\widehat{\mathbf{h}}}_i}\left(n\right)={\widehat{\mathbf{h}}}_i^T(n-1){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$

4   $e_{{\widehat{\mathbf{h}}}_i}\left(n\right)=d\left(n\right)-y_{{\widehat{\mathbf{h}}}_i}\left(n\right)$

5   ${\mathbf{p}}_{0,i}\left(n\right)={\lambda }_i{\mathbf{r}}_i(n-1)+e_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$

6   ${\mathbf{R}}_i\left(n\right)\Delta {\mathbf{h}}_i\left(n\right)={\mathbf{p}}_{0,i}\left(n\right)\stackrel{\mathrm{DCD}}{\to }\Delta {\widehat{\mathbf{h}}}_i\left(n\right),{\mathbf{r}}_i\left(n\right)$

7   ${\widehat{\mathbf{h}}}_i\left(n\right)={\widehat{\mathbf{h}}}_i(n-1)+\Delta {\widehat{\mathbf{h}}}_i\left(n\right)$

It can be noticed that the problem of determining the filter estimates ${\widehat{\mathbf{h}}}_i\left(n\right)$ is split into firstly computing each of the N increment vectors $\Delta {\widehat{\mathbf{h}}}_i\left(n\right)$ independently, then adding them to the estimates ${\widehat{\mathbf{h}}}_i(n-1)$ obtained from the previous adaptive system iteration. In step 2 of the algorithm, the update of matrix ${\mathbf{R}}_i\left(n\right)$ requires shifting all its values with the exception of the first column ${\mathbf{R}}_i^{\left(0\right)}\left(n\right)$ and the first row – both have the same set of values, which have to be updated [18,28]. The notation ${\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^{\left(0\right)}\left(n\right)$ represents the first value of the vector ${\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$. Moreover, step 6 shows that the residual vectors $r_i\left(n\right)$ are generated by the DCD method and they will be used at the next time index to generate the right side of the auxiliary sets of normal equations, which will again be processed by the DCD iterations.

In Algorithm 2 the DCD method with a $leading$ $element$ is presented. It works using a $greedy$ approach and it performs (for each channel) comparisons between the main diagonal of correlation matrix ${\mathbf{R}}_i\left(n\right)$ and the residual vector $\mathbf{r}\left(n\right)$ (equivalent positions) in order to determine at which indexes to perform modifications for the vector $\Delta {\widehat{\mathbf{h}}}_i\left(n\right)$. The corresponding value updates are equivalent to bits set to ’1’ for the estimated coefficients situated on various positions in the solution vector. In software validations, such operations are simulated using the step size ${\alpha }_i$ and the maximum expected amplitude of the solution values H, which is initialized using a power of two. The values in $\Delta {\widehat{\mathbf{h}}}_i\left(n\right)$ and ${\widehat{\mathbf{h}}}_i\left(n\right)$ are expected to be in the interval $\left[-H,H\right]$.

Algorithm 2 The DCD iterations with a leading element.

The content of the table is correct.

$\mathrm{Initialization}\text{:} {\alpha }_i=H_i/2,m_i=1$

$\Delta {\widehat{\mathbf{h}}}_i={\mathbf{0}}_{L_i\times 1},{\mathbf{r}}_i={\mathbf{0}}_{L_i\times 1}$

$\mathrm{For}(q=1...N_{u,i}):$

1    $p=arg{max}_{k=1,2,\dots ,L_i}\left\{|r_{k,i}|\right\}$

2    $\mathrm{While}((m_i\le M_{b,i})$ && $\left(\right|r_{p,i}|\ge ({\alpha }_i/2)R_{p,p,i})):$

$m_i++,{\alpha }_i={\alpha }_i/2$

3    $\mathrm{if}{m}_i>M_b\stackrel{}{\to }\mathrm{the}\mathrm{algorithm}\mathrm{stops}$

4    $\Delta {\widehat{h}}_{p,i}=\Delta {\widehat{h}}_{p,i}+\mathrm{sign}\left(r_{p,i}\right){\alpha }_i$

5    ${\mathbf{r}}_i={\mathbf{r}}_i-\mathrm{sign}\left(r_{p,i}\right){\alpha }_i{\mathbf{R}}_i^{\left(p\right)}$

The entire DCD process (i.e., search for the maximum value, verify conditions, update values and indexes) requires only additions and bit-shifts, which makes it an attractive choice for hardware implementations. The maximum number of $successful$ updates for any given RLS-DCD-T iteration is denoted in Algorithm 2 with $N_{u,i}$ (for channel i). The approach presented in Algorithm 1 demonstrated in the past, for relatively low values of $N_{u,i}$, performance comparable to the classical RLS algorithm. Excepting step 1, the overall complexity associated with the RLS-DCD-T is a sum of values approximately proportional to the individual filter lengths. More information about the RLS-DCD versions can be found in [18,22,27,28] and in corresponding references.

3.2. Robust RLS-DCD-T Adaptive Algorithm Based on Matrix Regularization

In this sub-section we concentrate on enhancing the robustness of the RLS-DCD-T algorithm by focusing on the regularization parameter ${\delta }_i$ for each of the N channels. Its usual value is a constant which traditionally influences the first iterations of the adaptive system in order to mitigate the ill-conditioned nature of the correlation matrices ${\mathbf{R}}_i\left(n\right)$. Our purpose is to change the scope of this parameter in order to have more significance in practice. By adjusting it in real-time, we aim to improve the performance of the RLS-DCD-T in low SNR conditions, when the reference signal is affected by an undesired additive noise.

We reconsider the least-squares criterion in order to include for the entire run of the algorithm the presence of the regularization parameter. Therefore, the individual cost functions will be rewritten as:

$$\begin{array}{c}\hfill {\widehat{J}}_{{\widehat{\mathbf{h}}}_N,{\widehat{\mathbf{h}}}_{N-1},\dots ,{\widehat{\mathbf{h}}}_{i+1},{\widehat{\mathbf{h}}}_{i-1},\dots ,{\widehat{\mathbf{h}}}_2,{\widehat{\mathbf{h}}}_1}\left[{\widehat{\mathbf{h}}}_i\left(n\right)\right]=J_{{\widehat{\mathbf{h}}}_N,{\widehat{\mathbf{h}}}_{N-1},\dots ,{\widehat{\mathbf{h}}}_{i+1},{\widehat{\mathbf{h}}}_{i-1},\dots ,{\widehat{\mathbf{h}}}_2,{\widehat{\mathbf{h}}}_1}\left[{\widehat{\mathbf{h}}}_i\left(n\right)\right]+{\delta }_i{∥{\widehat{\mathbf{h}}}_i\left(n\right)∥}_2^2,\end{array}$$

(34)

where ${∥·∥}_2$ denotes the ${\ell }_2$ norm. The update equation for each filter can accordingly be expressed as:

$$\begin{array}{cc}\hfill {\widehat{\mathbf{h}}}_i\left(n\right)& ={\widehat{\mathbf{h}}}_i(n-1)+{[{\mathbf{R}}_i\left(n\right)+{\delta }_i{\mathbf{I}}_{L_i}]}^{-1}{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)e_{{\widehat{\mathbf{h}}}_i}\left(n\right).\hfill \end{array}$$

(35)

By replacing (12) in (35), the latter becomes:

$$\begin{array}{}\mathrm{(36)}& {\widehat{\mathbf{h}}}_i\left(n\right)& ={[{\mathbf{R}}_i\left(n\right)+{\delta }_i{\mathbf{I}}_{L_i}]}^{-1}{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)d\left(n\right)+{\mathbf{I}}_{L_i}{\widehat{\mathbf{h}}}_i(n-1)+{[{\mathbf{R}}_i\left(n\right)+{\delta }_i{\mathbf{I}}_{L_i}]}^{-1}{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(n\right){\widehat{\mathbf{h}}}_i(n-1)\mathrm{(37)}& & ={\mathbf{P}}_{L_i}\left(n\right){\widehat{\mathbf{h}}}_i(n-1)+{\tilde{\mathbf{h}}}_i\left(n\right),\end{array}$$

where we denoted:

$$\begin{array}{cc}\hfill {\tilde{\mathbf{h}}}_i\left(n\right)& ={[{\mathbf{R}}_i\left(n\right)+{\delta }_i{\mathbf{I}}_{L_i}]}^{-1}{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)d\left(n\right),\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill {\mathbf{P}}_{L_i}\left(n\right)& ={\mathbf{I}}_{L_i}+{[{\mathbf{R}}_i\left(n\right)+{\delta }_i{\mathbf{I}}_{L_i}]}^{-1}{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(n\right).\hfill \end{array}$$

(39)

The contribution described in (38) is strongly connected to $d\left(n\right)$ and comprises the effects of the interference that we aim to mitigate. The expression for ${\tilde{\mathbf{h}}}_i\left(n\right)$ represents the solution for the system of equations:

$$\begin{array}{cc}\hfill [{\mathbf{R}}_i\left(n\right)+{\delta }_i{\mathbf{I}}_{L_i}]{\tilde{\mathbf{h}}}_i\left(n\right)& ={\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)d\left(n\right).\hfill \end{array}$$

(40)

Correspondingly, for each channel, we can define another error signal for the newly introduced system, as a difference between the reference signal and the output of the filter updates in (38):

$$\begin{array}{cc}\hfill {\tilde{e}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)& =d\left(n\right)-{\tilde{\mathbf{h}}}_i^T\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right).\hfill \end{array}$$

(41)

For the new structure we can define the goal of recovering/estimating the noise signal and we will find an expression for ${\delta }_i$ in order to have equality between the expected value of ${\tilde{e}}_{{\widehat{\mathbf{h}}}_i}^2\left(n\right)$ and the variance of the noise signal ${\sigma }_w^2$:

$$\begin{array}{c}\hfill E\left[{\tilde{e}}_{{\widehat{\mathbf{h}}}_i}^2\left(n\right)\right]=E\left[w^2\left(n\right)\right]={\sigma }_w^2.\end{array}$$

(42)

We will develop (41) by squaring it and applying the expectation operator on both sides of the equal sign. With this purpose, a few approximations will be explained, then used in the resulting computations. Firstly, for a large enough time index n (i.e., the RLS-DCD-T has functioned for a sufficient amount of time), the matrices ${\mathbf{R}}_i\left(n\right)$ defined in (18) can be written as:

$$\begin{array}{cc}\hfill {\mathbf{R}}_i\left(n\right)& \approx {\left(1-{\lambda }_i\right)}^{-1}E\left[{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(n\right)\right].\hfill \end{array}$$

(43)

For the previous expression we need to examine the value of the expectation operator. We start by using (14), then the transpose and the mixed-product properties of the Kronecker product. Only for this portion of the algorithm, we also consider ${\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$ to be uncorrelated with ${\widehat{\mathbf{h}}}_i\left(n\right)$, respectively, that ${\mathbf{R}}_i\left(n\right)$ are approximately diagonal matrices, so that $E\left[\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]\approx {\sigma }_x^2{\mathbf{I}}_{L_1{L}_2\dots L_N}$, where ${\sigma }_x^2$ is the variance of $\mathbf{x}\left(n\right)$. Thus, we obtain:

$$\begin{array}{}\mathrm{(44)}& \hfill E\left[{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(n\right)\right]& ={\sigma }_x^{2}E\left[{∥{\widehat{\mathbf{h}}}_N(n-1)∥}_2^2\cdots {∥{\widehat{\mathbf{h}}}_{i+1}(n-1)∥}_2^2{\mathbf{I}}_{L_i}{∥{\widehat{\mathbf{h}}}_{i-1}(n-1)∥}_2^2\cdots {∥{\widehat{\mathbf{h}}}_1(n-1)∥}_2^2\right]\mathrm{(45)}& & \approx {\sigma }_x^{2}E\left[{∥{\widehat{\mathbf{h}}}_N(n-1)∥}_2^2\right]\cdots E\left[{∥{\widehat{\mathbf{h}}}_{i+1}(n-1)∥}_2^2\right]{\mathbf{I}}_{L_i}E\left[{∥{\widehat{\mathbf{h}}}_{i-1}(n-1)∥}_2^2\right]\cdots E\left[{∥{\widehat{\mathbf{h}}}_1(n-1)∥}_2^2\right].\end{array}$$

Considering that we can approximate $E\left[{∥{\widehat{\mathbf{h}}}_i(n-1)∥}_2^2\right]\approx {∥{\widehat{\mathbf{h}}}_i(n-1)∥}_2^2$ we reach the expression:

$$\begin{array}{cc}\hfill E\left[{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^T\left(n\right)\right]& \approx {∥{\widehat{\mathbf{h}}}_N(n-1)∥}_2^2\cdots {∥{\widehat{\mathbf{h}}}_{i+1}(n-1)∥}_2^2{∥{\widehat{\mathbf{h}}}_{i-1}(n-1)∥}_2^2\cdots {∥{\widehat{\mathbf{h}}}_1(n-1)∥}_2^2{\sigma }_x^2{\mathbf{I}}_{L_i}.\hfill \end{array}$$

(46)

We also consider that the forgetting factors can be approximately chosen having the values ${\lambda }_i=1-1/L$, and together with (46), they can be used to rewrite (43) as

$$\begin{array}{cc}\hfill {\mathbf{R}}_i\left(n\right)& \approx {\alpha }_i{\mathbf{I}}_{L_i},\hfill \end{array}$$

(47)

where

$$\begin{array}{c}\hfill {\alpha }_i={∥{\widehat{\mathbf{h}}}_N(n-1)∥}_2^2\cdots {∥{\widehat{\mathbf{h}}}_{i+1}(n-1)∥}_2^2{∥{\widehat{\mathbf{h}}}_{i-1}(n-1)∥}_2^2\cdots {∥{\widehat{\mathbf{h}}}_1(n-1)∥}_2^2{\sigma }_x^{2}L.\end{array}$$

(48)

Moreover, we can express:

$$\begin{array}{c}\hfill {\mathbf{R}}_i\left(n\right)+{\delta }_i{\mathbf{I}}_{L_i}\approx ({\alpha }_i+{\delta }_i){\mathbf{I}}_{L_i},\end{array}$$

(49)

and proceed to squaring, respectively, applying the expectation operator on (41). We replace ${\tilde{\mathbf{h}}}_i\left(n\right)$ by using (38), and after some computations we find:

$$\begin{array}{c}\hfill {\sigma }_w^2={\sigma }_d^2-2{\sigma }_d^2\frac{{\alpha }_i}{{\alpha }_i+{\delta }_i}+{\sigma }_d^2{\left[\frac{{\alpha }_i}{{\alpha }_i+{\delta }_i}\right]}^2,\end{array}$$

(50)

where ${\sigma }_d^2$ denotes the variance of $d\left(n\right)$.

We can divide (50) by ${\sigma }_d^2$ and write a squared binomial on the right side of the equal, then use:

$$\begin{array}{c}\hfill {\sigma }_d^2={\sigma }_y^2+{\sigma }_w^2,\end{array}$$

(51)

respectively,

$$\begin{array}{c}\hfill \mathrm{SNR}=\frac{{\sigma }_y^2}{{\sigma }_w^2}.\end{array}$$

(52)

Finally, after a few more calculations, we reach a second degree equation:

$$\begin{array}{c}\hfill {\delta }_i^2-2\frac{{\alpha }_i{\delta }_i}{\mathrm{SNR}}-\frac{{\alpha }_i^2}{\mathrm{SNR}}=0,\end{array}$$

(53)

where we consider ${\delta }_i$ as the unknown variable. From this equation we take into consideration only the positive solution:

$$\begin{array}{c}\hfill {\delta }_i={\alpha }_i\frac{1+\sqrt{1+\mathrm{SNR}}}{\mathrm{SNR}}.\end{array}$$

(54)

3.3. Practical Considerations

From a practical point of view, once ${\delta }_i$ has been determined with (54), it is valid as long as the unknown systems comprising the MISO model remain the same and the statistical properties of the corresponding signals are unchanged. However, the practical applications associated with such presumptions are extremely limited. Consequently, we will adjust the proposed model in order to perform real-time estimations for all involved parameters.

Firstly, the variances ${\sigma }_x^2$, ${\sigma }_d^2$ and ${\sigma }_y^2$ have to be estimated at every time index n. We consider that the adaptive algorithm has converged to a certain degree and we will use $\widehat{y}\left(n\right)$, as the value for $y\left(n\right)$ is not directly available. We will employ the exponential window function:

$$\begin{array}{c}\hfill {\widehat{\sigma }}_u^2\left(n\right)={\gamma }_i{\widehat{\sigma }}_u^2(n-1)+(1-{\gamma }_i)u^2\left(n\right),\end{array}$$

(55)

where $0<{\gamma }_i\le 1$ determines the memory of the window, respectively, $u\left(n\right)$ represents in turn the signals $x\left(n\right)$, $d\left(n\right)$, and $\widehat{y}\left(n\right)$.

Secondly, working with the same assumption that the adaptive system has converged to a certain point, an estimate for the SNR can be determined based on (51), respectively, (52), for each channel as:

$$\begin{array}{c}\hfill \widehat{\mathrm{SNR}}\left(n\right)=\frac{{\widehat{\sigma }}_{\widehat{y}}^2\left(n\right)}{\left|{\widehat{\sigma }}_d^2\left(n\right)-{\widehat{\sigma }}_{\widehat{y}}^2\left(n\right)\right|}.\end{array}$$

(56)

Considering the above arguments, which proved correct when applied to other adaptive systems with variable parameters [16,22,30], the computation of ${\delta }_i\left(n\right)$ becomes straightforward for each channel. The values of ${\delta }_i\left(n\right)$ are used in the DCD portion of the RLS-DCD-T method in order to adjust the main diagonal of ${\mathbf{R}}_i\left(n\right)$ in the desired direction. More precisely, when the value of ${\delta }_i\left(n\right)$ increases, the comparisons performed in the DCD will reduce the number of filter updates—this is desirable in high interference scenarios. On the other hand, in high SNR situations, the successful updates should be performed as in Algorithm 1 for $\Delta \widehat{\mathbf{h}}\left(n\right)$, due to the negligible influence of ${\delta }_i\left(n\right)$.

With a small cost in arithmetic complexity, the algorithm gains much needed robustness. Moreover, it is expected to work properly in tracking scenarios, when one or more of the smaller unknown systems change. We call the new algorithm the variable regularized RLS-DCD-T algorithm, namely VR-RLS-DCD-T, which is summarized in Algorithm 3. It can be noticed in steps 5 a) to 5 d) that the extra computational requirements of the VR-RLS-DCD-T consist mostly of multiplications and additions. The overall arithmetic effort of the update process remains a sum of values proportional to each of the filter lengths. Two extra divisions and one square root operation have to be performed for each channel. From a practical point of view, when implementations are performed on Field Programmable Gate Array (FPGA) devices, a single Digital Signal Processing (DSP) block can be employed functioning in two distinct arithmetic modes.

Algorithm 3 VR-RLS-DCD-T.

$\mathrm{Initialization}$:

Set ${\widehat{\mathbf{h}}}_i\left(0\right)={\mathbf{0}}_{L_i\times 1},{\mathbf{r}}_i\left(0\right)={\mathbf{0}}_{L_i\times 1}$

0    ${\mathbf{R}}_i^{-1}\left(0\right)={\delta }_i{\mathbf{I}}_{L_i},{\delta }_i>0$

${\lambda }_i=1-\frac{1}{K_i{L}_i},K_i\ge 1$

${\gamma }_i=1-\frac{1}{{ϵ}_i{L}_i},{ϵ}_i>0;{\widehat{\sigma }}_{\widehat{y}}^2\left(0\right)=0;{\widehat{\sigma }}_d^2\left(0\right)=0;{\widehat{\sigma }}_x^2\left(0\right)=0$

$\mathrm{For}n=1,2,\dots ,\mathrm{number}\mathrm{of}\mathrm{iterations}:$

1    Compute ${\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$, based on (14)

2    ${\mathbf{R}}_i^{\left(0\right)}\left(n\right)={\lambda }_i{\mathbf{R}}_i^{\left(0\right)}(n-1)+{\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}^{\left(0\right)}\left(n\right)$

3    ${\widehat{y}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)={\widehat{\mathbf{h}}}_i^T(n-1){\mathbf{x}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$

4    $e_{{\widehat{\mathbf{h}}}_i}\left(n\right)=d\left(n\right)-{\widehat{y}}_{{\widehat{\mathbf{h}}}_i}\left(n\right)$

5 a)   ${\widehat{\sigma }}_u^2\left(n\right)={\gamma }_i{\widehat{\sigma }}_u^2(n-1)+(1-{\gamma }_i)u^2\left(n\right)$ for $u\left(n\right)\triangleq x\left(n\right),d\left(n\right)$ and $\widehat{y}\left(n\right)$

5 b)   ${\alpha }_i\left(n\right)={∥{\widehat{\mathbf{h}}}_N(n-1)∥}_2^2\cdots {∥{\widehat{\mathbf{h}}}_{i+1}(n-1)∥}_2^2{∥{\widehat{\mathbf{h}}}_{i-1}(n-1)∥}_2^2\cdots {∥{\widehat{\mathbf{h}}}_1(n-1)∥}_2^2{\widehat{\sigma }}_x^{2}L$

5 c)   $\widehat{\mathrm{SNR}}\left(n\right)=\frac{{\widehat{\sigma }}_{\widehat{y}}^2\left(n\right)}{\left|{\widehat{\sigma }}_d^2\left(n\right)-{\widehat{\sigma }}_{\widehat{y}}^2\left(n\right)\right|}$

5 d)   ${\delta }_i\left(n\right)={\alpha }_i\left(n\right)\frac{1+\sqrt{1+\widehat{\mathrm{SNR}}\left(n\right)}}{\widehat{\mathrm{SNR}}\left(n\right)}$

6    $\left[{\mathbf{R}}_i\left(n\right)+{\delta }_i\left(n\right){\mathbf{I}}_{L_i}\right]\Delta {\widehat{\mathbf{h}}}_i\left(n\right)={\mathbf{p}}_{0,i}\left(n\right)\stackrel{\mathrm{DCD}}{\to }\Delta {\widehat{\mathbf{h}}}_i\left(n\right),{\mathbf{r}}_i\left(n\right)$

7    ${\widehat{\mathbf{h}}}_i\left(n\right)={\widehat{\mathbf{h}}}_i(n-1)+\Delta {\widehat{\mathbf{h}}}_i\left(n\right)$

4. Simulation Results

Several configurations are simulated in order to validate the performance of the proposed algorithm in scenarios with low SNR conditions. We also verify that the VR-RLS-DCD-T works well in tracking situations, when the unknown global system $\mathbf{g}$ changes suddenly. We generate such a modification after the convergence is achieved, by altering the sign of one of the constituent systems, which are combined using the model described in (9). The experiments work with $N=3$, respectively, $N=4$, smaller systems of lengths $L_i,i=1,2,\dots ,N$ (depending on the case). The presented scenarios aim to demonstrate the tracking capabilities and the robustness of the proposed VR-RLS-DCD-T, with respect to the already studied RLS-T [17], respectively, RLS-DCD-T [18], which perform identification of the smaller impulse responses ${\widehat{\mathbf{h}}}_i\left(n\right)$. For the tracking situations we also use as reference the classical RLS method based on Woodburry’s identity for the direct identification of $\mathbf{g}$ [6].

Two of the smaller systems [${\mathbf{h}}_1\left(n\right)$, ${\mathbf{h}}_4\left(n\right)$] are comprised of samples taken from the impulse responses in the G.168 ITU-T Recommendation [31] – vectors with $L_1$, respectively, $L_4$, values. The other two systems systems [${\mathbf{h}}_2\left(n\right)$, ${\mathbf{h}}_3\left(n\right)$] are obtained using the functions $(-0.5^{j_1})/2$, respectively, $(0.8^{j_2})/2$, with $j_1=0,...,L_2-1$ and $j_2=0,...,L_3-1$. We use as input signals Gaussian sequences filtered through an auto-regressive AR(1) system with the pole 0.85, in order to have highly correlated input samples. For the output of the unknown global system $\mathbf{g}$, the reference signal $d\left(n\right)$ has an experimental SNR ratio set to 15 dB. In the intervals with low SNR conditions, the corresponding value is modified to −15 dB.

The forgetting factors for the adaptive filters are chosen as ${\lambda }_i=1-1/\left(2^{k_i}{L}_i\right)$ for the tensor-based methods, respectively, as $\lambda =1-1/\left(2^{k}L\right)$ for the classical RLS. The forms facilitate efficient hardware implementations of multiplications [20,32]. For a similar reason, the filter lengths have the form $L_i=2^{q_i}$, respectively, $L=2^q$. Furthermore, for the specific parameters of the DCD iterations, the maximum expected amplitudes of the filter coefficients are set to $H=1$, with the representation $M_b=15+1$ bits. In accordance with our team’s previously published or several referenced studies, the maximum number of allowed updates for each filter is set to $N_u=1$ [18,20,22,28].

In order to measure the accuracy of the unknown global system identification, we employ the normalized misalignment $20{\log }_{10}\left[{∥\mathbf{g}-\widehat{\mathbf{g}}\left(n\right)∥}_2/{∥\mathbf{g}∥}_2\right]$, where the global filter estimate $\widehat{\mathbf{g}}\left(n\right)$ is reconstituted using the smaller N filter estimates ${\widehat{\mathbf{h}}}_i\left(n\right)$. The tensor-based adaptive filters are initialized using the method recommended in [15] and the initial convergence is not shown, as it is not considered relevant [33].

The first experiment was performed for a tracking scenario with $L=1024$ coefficients for $\mathbf{g}$, which can be decomposed in three smaller systems with lengths $L_1=64$ and $L_2=L_3=4$. The global impulse response suffers a sudden change and all adaptive methods must go through a re-convergence process. The decomposition based algorithms use the same set of forgetting factors, respectively, the value of $\lambda$ for the RLS method was chosen to match the performance of the tensor-based algorithms at convergence state. It can be noticed in Figure 1 that the tensor-based methods benefit from the split structure working with smaller adaptive filters and converge considerably faster than the RLS method after the sudden change in the global system. Moreover, it is important to notice that the VR-RLS-DCD-T offers tracking performance similar with the RLS-DCD-T and the RLS-T.

In Figure 2, the same algorithms were simulated in the same type of scenario (i.e., a tracking situation) with a different configuration. The global system has the length $L=2048$, with a decomposition performed with 4 smaller impulse responses having $L_1=16$, $L_2=8$, respectively, $L_3=L_4=4$. Similar conclusions apply regarding the tracking capabilities when the global system is changed. The tensor-based variants show considerably better performance than the RLS. Moreover, the RLS-DCD-T and the VR-RLS-DCD-T methods manage to perform in the same manner as the RLS-T adaptive algorithm, while requiring lesser computational resources for the coefficients update process.

It is obvious that the direct approach of estimating $\mathbf{g}$ leads to an unwanted compromise between tracking speed and accuracy. If the same performance is targeted at convergence state, the RLS would be outperformed in any situation where going through the converging process is necessary. The conclusion includes the case when the filter coefficients are temporarily biased by low SNR conditions. On the other hand, if the tracking speed is matched between the RLS method and its tensor-based counterparts (by using for the former a smaller value of $\lambda$), the accuracy of the estimation is reduced at steady-state. Consequently, the next scenarios will only compare the tensor-based algorithms and show the improved robustness of the proposed VR-RLS-DCD-T for several configurations associated with the presented MISO model.

The global systems from the first two scenarios were employed for another validation type. In Figure 3 and Figure 4, for a duration of 10,000 samples, the SNR value is temporarily set to −15 dB. It can be noticed that the RLS-T and the RLS-DCD-T are strongly affected by the alteration of the reference signals. However, the change in conditions has diminished effects on the VR-RLS-DCD-T adaptive system. When the undesired phenomenon halts, the corresponding recovery process requires less iterations to return to the previous state.

The robustness validation was also performed for a third (and even longer) global system with $L=4096$ coefficients, which is decomposed using four impulse responses with equal lengths $L_1=L_2=L_3=L_4=8$. The results are illustrated in Figure 5. It is clear that the VR-RLS-DCD-T outperforms the other tensor-based algorithms and has the least affected filter coefficients during the high interference period. The tensor-based NLMS algorithm [11] was also tested for this scenario and the results were added in Figure 5. For one of the NLMS-T versions, the step-size $\mu$ was chosen to generate similar convergence with the RLS-based methods at steady-state. However, the recovery process requires the largest number of iterations, with respect to the RLS-based counterparts. Moreover, it can be noticed that attempting to match the convergence speed of the NLMS-T with the RLS-T versions (by using a larger $\mu$) leads to lower accuracy at steady-state.

5. Conclusions

This paper proposed a robust low-complexity RLS adaptive algorithm, which is designed on tensor-based decomposition and employs the DCD iterations to solve the set of normal equations specific to the RLS methods. The algorithm benefits from the reduced arithmetic requirements provided by splitting one major system identification problem into several smaller corresponding problems. Moreover, the DCD iterations require only bit-shifts and additions, which makes it attractive for practical applications. The VR-RLS-DCD-T has minimal additions to the arithmetic complexity and is designed for improved robustness in low SNR conditions, by influencing the update process through a continuous regularization process of the corresponding correlation matrices.

Simulations were performed in order to validate the proposed method in two types of scenarios. Firstly, we demonstrated that the tracking capabilities remain similar with other RLS tensor-based adaptive systems. Secondly, validations were completed with high noise levels affecting the reference signals. The results showed a robust behaviour in low SNR scenarios, when the VR-RLS-DCD-T filter coefficients suffer minimal modifications, which allow for minimal re-convergence periods after the interference signals cease.

In perspective, we aim to even further develop variable regularized RLS-DCD-T methods by introducing a version based on sliding windows [33]. Moreover, fixed-point FPGA implementations will be performed for MISO configurations as the next step towards our final goal of providing a better alternative for current practical applications.