Regularization Factor Selection Method for l₁-Regularized RLS and Its Modification against Uncertainty in the Regularization Factor

Featured Application

This algorithm can be applied to various kinds of sparse channel estimations, e.g., room impulse response, early reflection, and underwater channel response.

Abstract

This paper presents a new l₁-RLS method to estimate a sparse impulse response estimation. A new regularization factor calculation method is proposed for l₁-RLS that requires no information of the true channel response in advance. In addition, we also derive a new model to compensate for uncertainty in the regularization factor. The results of the estimation for many different kinds of sparse impulse responses show that the proposed method without a priori channel information is comparable to the conventional method with a priori channel information.

1. Introduction

Room impulse response (RIR) estimation is a problem in many applications that use acoustic signal processing. The RIR identification [1] is fundamental for various applications such as room geometry related spatial audio applications [2,3,4,5], acoustic echo cancellation (AEC) [6], speech enhancement [7], and dereverberation [8]. In [9], the RIR has relatively large magnitude values during the early part of the reverberation and fades to smaller values during the later part. This indicates that most RIR entries have values close to zero. Therefore, the RIR has a sparse structure. The sparse RIR model is useful for estimating RIRs in real acoustic environments when the source is given a priori [10]. There has been recent interest in adaptive algorithms for sparsity in various signals and systems [11,12,13,14,15,16,17,18,19,20,21,22]. Many adaptive algorithms based on least mean square (LMS) [11,12] and recursive least squares (RLS) [14,15,16,17] have been reported with different penalty functions. Sparse estimation research, such as that done by Eksioglu and Tanc [17], has proposed a sparse RLS algorithm, l₁-RLS, which is fully recursive like the plain RLS algorithm. The algorithm of l₁-RLS in [17] proposed a proper calculation method for the regularization factor. These recursive algorithms have the potential for sparse RIR estimation; however, the regularization factor should be established prior to applying these algorithms. The regularization factor calculation method requires information about a true sparse channel response for a good performance. The authors in [18,19] have also proposed recursive regularization factor selection methods; however, these methods still need the true impulse response in advance.

In this paper, we propose a new regularization factor calculation method for l₁-RLS algorithm in [17]. The new regularization factor calculation needs no information for the true channel response in advance. This makes it possible to apply l₁-RLS algorithm in various room environments. In addition, we derive a new model equation for l₁-RLS in [17] with uncertainty in the regularization factor and show that the new model is similar to the total least squares (TLS) model that compensates for uncertainty in the calculated regularization factor without the true channel response. For the performance evaluation, we simulate four different sparse channels and compare channel estimation performances. We show that, without any information of the true channel impulse response, the performance of the proposed algorithm is comparable to that of l₁-RLS with the information of the true channel impulse response.

This paper is organized as follows. In Section 2, we summarize l₁-RLS in [17]. In Section 3, we summarize the measure of sparsity. In Section 4, we propose a new method for the regularization calculation. In Section 5, we show that l₁-RLS with uncertainty in the regularization factor can be modeled as the TLS model. In Section 6, we summarize l₁-RTLS (recursive total least squares) algorithm as a solution for l₁-RLS with uncertainty in the regularization factor. In Section 7, we present simulation results to show the performance of the proposed algorithm. Finally, we give the conclusion in Section 8.

2. Summarize l₁-RLS

In the sparse channel estimation problem of interest, the system observes a signal represented by an $M\times 1$ vector $\mathbf{x}(k)={[x_k,\cdots ,x_{k-M+1}]}^T$ at time instant n, performs filtering, and obtains the output $y(i)={\mathbf{x}}^T(k){\mathbf{w}}_o(k)$, where ${\mathbf{w}}_o(k)={[w_k,\cdots ,w_{k-M+1}]}^T$ is the M dimensional actual system with finite impulse response (FIR) type. For system estimation, an adaptive filter system applies with M dimensional vector $\mathbf{w}(k)$ to the same signal vector $\mathbf{x}(k)$ and produces an estimated output $\widehat{y}(k)={\mathbf{x}}^T(k)\mathbf{w}(k)$, and calculates the error signal $e(k)=y(k)+n(k)-\widehat{y}(k)=\tilde{y}(k)-\widehat{y}(k)$, where $n(k)$ is the measurement noise, $y(k)$ is the output of the actual system, and $\widehat{y}(k)$ is the estimated output. In order to estimate the channel impulse response, an adaptive algorithm minimizes the cost function defined by

$$\mathbf{w}=\arg \underset{\mathbf{w}}{\min }\frac{1}{2}{\displaystyle \sum _{m=0}^k}{\lambda }^{k-m}{{\displaystyle \left(e\left(m\right)\right)}}^2.$$

(1)

From the gradient based minimization, Equation (1) becomes

$$\mathbf{R}\left(k\right)\mathbf{w}\left(k\right)=r(k),$$

(2)

where $\mathbf{R}\left(k\right)={\displaystyle \sum _{m=0}^k}{\lambda }^{k-m}\mathbf{x}(m){\mathbf{x}}^T(m)$ and $r(k)={\displaystyle \sum _{m=0}^k}{\lambda }^{k-m}\tilde{y}(m)\mathbf{x}(m)$. This equation is the normal equation for the least squares solution. Especially, ${\mathbf{w}}_o(k)$ is considered as a sparse system when the number of nonzero coefficients $K$ is less than the system order of $M$. In order to estimate the sparse system, most estimation algorithms exploit non-zero coefficients in the system [11,12,13,14,15,16,17]. In [17], Eksioglu proposed a full recursive l₁-regularized algorithm by the minimization of the object function as shown in Equation (3).

$$J_k=\frac{1}{2}{\epsilon }_k+{\gamma }_k{{\displaystyle \Vert \mathbf{w}\Vert }}_1,$$

(3)

where ${\epsilon }_k={\displaystyle \sum _{m=0}^k}{\lambda }^{k-m}{(e(m))}^2$. From the minimization of Equation (3), a modified normal equation was derived as shown in Equation (4).

$$\mathbf{R}(k)\mathbf{w}(k)=r(k)-{\gamma }_k{\nabla }^s{{\displaystyle \Vert \mathbf{w}(k)\Vert }}_1=\widehat{p}(k).$$

(4)

When we solve Equation (4), we should select the regularization factor as shown in Equation (5).

$${\gamma }_k=\frac{2\frac{tr\left({\mathbf{R}}^{-1}(k)\right)}{M}}{{{\displaystyle \Vert {\mathbf{R}}^{-1}(k){\nabla }^{s}f\left(\mathbf{w}(k)\right)\Vert }}_2^2}\times [\left(f\left(\mathbf{w}(k)\right)-\rho \right)+{\nabla }^{s}f\left(\mathbf{w}(k)\right)R^{-1}(k)\epsilon (k)],$$

(5)

where $f(\mathbf{w}(k))={{\displaystyle \Vert \mathbf{w}(k)\Vert }}_1$ and the subgradient of $f(\mathbf{w})$ is ${\nabla }^s{{\displaystyle \Vert \mathbf{w}\Vert }}_1=\mathrm{sgn}(\mathbf{w})$. In Equation (5), the regularization factor has the parameter, $\rho$, which should be set beforehand. In [17], the parameter was set as $\rho =f({\mathbf{w}}_{true})={{\displaystyle \Vert {\mathbf{w}}_{true}\Vert }}_1$ with ${\mathbf{w}}_{true}$ indicating the impulse response of the true channel. There was no further discussion about how to set $\rho$. However, it is not practical to know the true channel in advance.

3. Measure of Sparseness

In [20], the sparseness of a channel impulse response is measured by Equation (6).

$$\chi =\frac{L}{L-\sqrt{L}}\left(1-\frac{{{\displaystyle \Vert \widehat{\mathbf{w}}\Vert }}_1}{\sqrt{L}{{\displaystyle \Vert \widehat{\mathbf{w}}\Vert }}_2}\right),$$

(6)

where ${\Vert \widehat{\mathbf{w}}\Vert }_p$ is the p-norm of $\widehat{\mathbf{w}}$ and L is the dimension of $\widehat{\mathbf{w}}$. The range of $\chi$ is $0\le \chi \le 1$. That is dependent on the sparseness of $\widehat{\mathbf{w}}$. As $\widehat{\mathbf{w}}$ becomes sparser, the sparsity, $\chi$, comes close to 1, and as $\widehat{\mathbf{w}}$ becomes denser, $\chi$ comes close to 0. We often have small and none-zero value of $\chi$, even in a dense channel. For example, Figure 1 shows the relation of the value of $\chi$ and the percentage of none-zero components in $\widehat{\mathbf{w}}$ with L = 215. In Figure 1, we consider all possible cases of none-zero components in $\widehat{\mathbf{w}}$.

4. New $\rho$ Selection Method in the Sparsity Regularization Constant ${\gamma }_k$

Section 2 shows that the regularization constant ${\gamma }_k$ in Equation (5) needs $\rho$ to be set as $\rho ={\Vert \mathrm{true}\mathrm{system}\mathrm{impulse}\mathrm{response}\Vert }_1={\Vert {\mathbf{w}}_{true}\Vert }_1$. However, we need a new method in the constant selection because Equation (5) is not practical. Therefore, Section 4 proposes a new method to set this constant.

For a practical method for the constant selection, we can consider using the estimated vector $\widehat{\mathbf{w}}$ instead of using the true vector ${\mathbf{w}}_{true}$ because $\widehat{\mathbf{w}}$, the solution with l₁-norm, will be closer to the sparse true vector than the solution of the conventional RLS. The more iteration is repeated, the more $\widehat{\mathbf{w}}$ converges to the true value. Conventional RLS also converges to the true value; however, the solution with l₁-norm, is closer to the sparse true value. Therefore, we can use sparse estimate $\widehat{\mathbf{w}}$ instead of ${\mathbf{w}}_{true}$ when we set $\rho$, and the uncertainty arising from this is compensated through a TLS solution in the next section. When we determine $\rho$ using the estimated $\widehat{\mathbf{w}}$, we choose between the average $\rho$ and the current estimate ${\Vert \widehat{\mathbf{w}}\Vert }_1$.

Table 1. $\rho$ selection method in the sparsity regularization constant ${\gamma }_k$.

Step 1	Sparsity: $\chi =\frac{L}{L-\sqrt{L}}\left(1-\frac{{{\displaystyle \Vert \widehat{\mathbf{w}}\Vert }}_1}{\sqrt{L}{{\displaystyle \Vert \widehat{\mathbf{w}}\Vert }}_2}\right)$ [20] where $L$ is the length of the impulse response.
Step 2	$\rho (k)=0.99\rho (k-1)+0.01\left(e^{\chi -1}\times {{\displaystyle \Vert \widehat{\mathbf{w}}\Vert }}_1\right)$
Step 3	$\rho (k)=\{\begin{array}{c}min\left(\rho (k),0.98{{\displaystyle \Vert \widehat{\mathbf{w}}\Vert }}_1\right),\mathrm{if}x>0.75\\ min\left(\rho (k),0.999{{\displaystyle \Vert \widehat{\mathbf{w}}\Vert }}_1\right),\mathrm{otherwise}\end{array}$

summarizes the $\rho$ selection steps.

The determination method for $\rho$ value shown in Table 1 is as follows. In Step 1, the sparsity of the estimated $\widehat{\mathbf{w}}$ is calculated. The sparsity represents the sparseness of $\widehat{\mathbf{w}}$ as a number [23]. In Step 2, l₁-norm of the estimated $\widehat{\mathbf{w}}$ is scaled and the value is averaged with the previous $\rho$ value. The scaling value approaches 1 as the sparsity, $\chi$, gets close to 1. However, the scaling value gets close to $e^{-1}\simeq 0.37$ as the sparsity, $\chi$, gets close to 0. Therefore, the scaling does not change l₁-norm of $\widehat{\mathbf{w}}$ for the sparse $\widehat{\mathbf{w}}$. Instead the scaling changes the l₁-norm smaller for the dense $\widehat{\mathbf{w}}$. In Step 3, the smaller one between the averaged $\rho$ and the l₁-norm of the estimated $\widehat{\mathbf{w}}$ is selected as the new $\rho$ value. In this case, the $\rho$ value becomes completely new if the l₁-norm of the estimated $\widehat{\mathbf{w}}$ is selected, otherwise the previous trend is maintained. In Figure 1, the reference value 0.75 used in Step 3 means that less than 16% of all the impulse response taps are not zero.

5. New Modeling for l₁-RLS with Uncertainty in the Regularization Factor

If we set $\rho =\mathrm{constant}$, the regularization factor becomes

$$\begin{array}{ll}{{\displaystyle \tilde{\gamma }}}_k& =\frac{2\frac{tr\left({\mathbf{R}}^{-1}(k)\right)}{M}}{{{\displaystyle \Vert {\mathbf{R}}^{-1}(k){\nabla }^{s}f\left(\mathbf{w}(k)\right)\Vert }}_2^2}\times [\left(f\left(\mathbf{w}(k)\right)-\mathrm{constant}\right)+{\nabla }^{s}f\left(\mathbf{w}(k)\right)\times R^{-1}(k)\epsilon (k)]\\ & =\frac{2\frac{tr\left({\mathbf{R}}^{-1}(k)\right)}{M}}{{{\displaystyle \Vert {\mathbf{R}}^{-1}(k){\nabla }^{s}f\left(\mathbf{w}(k)\right)\Vert }}_2^2}\times \left(f\left(\mathbf{w}(k)\right)-{\Vert \mathbf{h}\Vert }_1+{\Vert \mathbf{h}\Vert }_1-\mathrm{constant}\right)+\frac{2\frac{tr\left({\mathbf{R}}^{-1}(k)\right)}{M}{\nabla }^{s}f\left(\mathbf{w}(k)\right)R^{-1}(k)\epsilon (k)}{{{\displaystyle \Vert {\mathbf{R}}^{-1}(k){\nabla }^{s}f\left(\mathbf{w}(k)\right)\Vert }}_2^2}.\end{array}$$

(7)

Then,

$${{\displaystyle \tilde{\gamma }}}_k={\gamma }_k+\frac{2\frac{tr\left({\mathbf{R}}^{-1}(k)\right)}{M}\left({\Vert \mathbf{h}\Vert }_1-\mathrm{constant}\right)}{{{\displaystyle \Vert {\mathbf{R}}^{-1}(k){\nabla }^{s}f\left(\mathbf{w}(k)\right)\Vert }}_2^2}={\gamma }_k+\Delta \gamma .$$

(8)

Using Equation (8), Equation (4) becomes

$$\mathbf{R}(k)\mathbf{w}(k)=r(k)-({\gamma }_k+\Delta \gamma ){\nabla }^s{{\displaystyle \Vert \mathbf{w}(k)\Vert }}_1.$$

(9)

${\nabla }^s{{\displaystyle \Vert \mathbf{w}\Vert }}_1=\mathrm{sgn}(\mathbf{w})$ is represented as

$${\nabla }^s{{\displaystyle \Vert \mathbf{w}(k)\Vert }}_1=\left[\begin{array}{ccc}\ddots & & \\ & \frac{1}{\left|{\mathbf{w}}_i\right|}& \\ & & \ddots \end{array}\right]\mathbf{w}(k).$$

(10)

By applying Equation (10) to Equation (9),

$$\left(R(k)+\Delta \gamma \left[\begin{array}{ccc}\ddots & & \\ & \frac{1}{\left|{\mathbf{w}}_i\right|}& \\ & & \ddots \end{array}\right]\right)\mathbf{w}(k)=r(k)-{\gamma }_k{{\displaystyle \Vert \mathbf{w}(k)\Vert }}_1,$$

(11)

where ${\mathbf{w}}_i$ is i-th element of $\mathbf{w}(k)$. Then it is simplified as

$$\left(R(k)+\Delta \gamma \left[\begin{array}{ccc}\ddots & & \\ & \frac{1}{\left|{\mathbf{w}}_i\right|}& \\ & & \ddots \end{array}\right]\right)\mathbf{w}(k)=\widehat{p}(k).$$

(12)

Equation (12) is very similar to the system model in Figure 2 that is contaminated by noise both in input and in output. Suppose that an example of the system in Figure 2 is represented as

$$\left[\begin{array}{ccc}{x}_k+n_{i,k}& \cdots & x_{k-N+1}+n_{i,k-N+1}\\ x_{k-1}+n_{i,k-1}& \cdots & x_{k-N}+n_{i,k-N}\\ ⋮& \ddots & ⋮\\ x_{k-N+1}+n_{i,k-N+1}& \cdots & x_{k-2N+2}+n_{i,k-2N+2}\end{array}\right]\times \mathbf{w}(k)=\left[\begin{array}{c}{y}_k+n_{o,k}\\ y_{k-1}+n_{o,k-1}\\ ⋮\\ y_{k-N+1}+n_{o,k-N+1}\end{array}\right],$$

(13)

where $x_k$ is $x(k)$, $n_{i,k}$ is $n_i(k)$, and $n_{o,k}$ is $n_o(k)$. Equation (13) is simplified as

$$A\mathbf{w}(k)=b.$$

(14)

If we multiply Equation (14) by A${}^H$ and average it, we get

$$E\left(A^{H}A\right)\mathbf{w}(k)=E\left(A^{H}b\right).$$

(15)

We can rewrite Equation (15) as follows

$$\left[\begin{array}{cccc}{r}_{xx}(0)+{\sigma }_n^2& r_{xx}(1)& \cdots & r_{xx}(N-1)\\ r_{xx}(1)& r_{xx}(0)+{\sigma }_n^2& \cdots & r_{xx}(N-2)\\ ⋮& ⋮& \ddots & ⋮\\ r_{xx}(N-1)& r_{xx}(N-2)& \cdots & r_{xx}(0)+{\sigma }_n^2\end{array}\right]\mathbf{w}(k)=\left[\begin{array}{c}{r}_{xy}(0)\\ r_{xy}(1)\\ ⋮\\ r_{xy}(N-1)\end{array}\right]$$

(16)

Then, it can be represented as

$$\left(R+{\sigma }_n^{2}I\right)\mathbf{w}(k)=\tilde{p}(k).$$

(17)

When we compare Equation (12) with Equation (17), the two system models have almost the same form. Therefore, it is feasible that the TLS method can be applied to Equation (12) [24,25,26,27,28,29,30]. Therefore, we expect to obtain almost the same performance as l₁-RLS with the true channel response if we apply the TLS method by the regularization factor with the new $\rho$ in Table 1. In the next section, we summarize l₁-RTLS (recursive total least squares) algorithm in [29].

6. Summarize l₁-RTLS for the Solution of l₁-RLS with Uncertainty in the Regularization Factor

Lim, one of the authors of this paper, has proposed the TLS solution for l₁-RLS known as l₁-RTLS [30]. In this section, we summarize l₁-RTLS in [30] for the solution of Equation (11).

The TLS system model assumes that both input and output are contaminated by additive noise as Figure 2. The output is given by

$$\tilde{y}(k)={{\displaystyle \tilde{\mathbf{x}}}}^T(k){\mathbf{w}}_o+n_o(k),$$

(18)

where the output noise $n_o(k)$ is the Gaussian white noise with variance ${\sigma }_o^2$. The noisy input vector in the system is modeled by

$$\tilde{\mathbf{x}}(k)=\mathbf{x}(k)+n_i(k)\in C^{M\times 1},$$

(19)

where $n_i(k)={{\displaystyle \left[n_i(k),n_i(k-1),\cdots n_i(k-M+1)\right]}}^T$ and the input noise $n_i(k)$ is the Gaussian white noise with variance ${\sigma }_i^2$. For the TLS solution, we set the augmented data vector as

$$\overline{\mathbf{x}}(k)={{\displaystyle \left[{{\displaystyle \tilde{\mathbf{x}}}}^T(k),\tilde{y}(k)\right]}}^T\in {\mathrm{R}}^{(M+1)\times 1}.$$

(20)

The correlation matrix is represented as

$$\overline{R}=\left[\begin{array}{cc}\tilde{R}& p\\ p^T& c\end{array}\right],$$

(21)

where $p=E\left\{\tilde{\mathbf{x}}(k)y(k)\right\}$, $c=E\left\{y(k)y(k)\right\}$, $\mathbf{R}=E\left\{\mathbf{x}(k){\mathbf{x}}^T(k)\right\}$ and $\tilde{R}=E\left\{\tilde{\mathbf{x}}(k){{\displaystyle \tilde{\mathbf{x}}}}^T(k)\right\}=\mathbf{R}+{\sigma }_i^{2}I$. In [27,28], the TLS problem becomes to find the eigenvector associated with the smallest eigenvalue of $\overline{R}$. Equation (22) is the typical cost function to find the eigenvector associated with the smallest eigenvalue of $\overline{R}$.

$$J(k)=\frac{1}{2}{{\displaystyle \tilde{\mathbf{w}}}}^T(k)\overline{R}(k)\tilde{\mathbf{w}}(k),$$

(22)

where $\overline{R}(k)$ is a sample correlation matrix at k-th instant, and $\tilde{\mathbf{w}}(k)={{\displaystyle \left[{{\displaystyle \widehat{\mathbf{w}}}}^T(k),-1\right]}}^T$ in which $\widehat{\mathbf{w}}(\mathrm{k})$ is the estimation result for the unknown system at k-th instant. We modify the cost function by adding a penalty function in order to reflect prior knowledge about the true sparsity system.

$$J(k)=\frac{1}{2}{{\displaystyle \tilde{\mathbf{w}}}}^T(k)\overline{R}(k)\tilde{\mathbf{w}}(k)+\lambda \left({{\displaystyle \tilde{\mathbf{w}}}}^T(k)\tilde{\mathbf{w}}(k-1)-1\right)+{\gamma }_{k}f\left(\tilde{\mathbf{w}}(k)\right),$$

(23)

where $\lambda$ is the Lagrange multiplier and ${\gamma }_k$ is the regularized parameter in [13]. We solve the equations by ${\nabla }_{\widehat{\mathbf{w}}}J(k)=0$ and ${\nabla }_{\lambda }J(k)=0$ simultaneously. ${\nabla }_{\widehat{\mathbf{w}}}J(k)=0:$

$$2\overline{R}(k)\tilde{\mathbf{w}}(k)+\lambda \tilde{\mathbf{w}}(k-1)+{\gamma }_k{\nabla }^{s}f\left(\tilde{\mathbf{w}}(k)\right)=0,$$

(24)

$${\nabla }_{\lambda }J(k)=0:{{\displaystyle \tilde{\mathbf{w}}}}^T(k)\tilde{\mathbf{w}}(k-1)=1,$$

(25)

where the subgradient of $f(\tilde{\mathbf{w}})={{\displaystyle \Vert \tilde{\mathbf{w}}\Vert }}_1$ is ${\nabla }_{\tilde{\mathbf{w}}}^s{{\displaystyle \Vert \tilde{\mathbf{w}}\Vert }}_1=\mathrm{sgn}(\tilde{\mathbf{w}})$. From (24), we obtain

$$\tilde{\mathbf{w}}(k)=-\frac{\lambda }{2}{{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)-{\gamma }_k{{\displaystyle \overline{R}}}^{-1}(k){\nabla }^{s}f\left(\tilde{\mathbf{w}}(k)\right).$$

(26)

Substituting Equation (26) in Equation (25), we get

$${{\displaystyle \left(-\frac{\lambda }{2}{{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)-{\gamma }_k{{\displaystyle \overline{R}}}^{-1}(k){\nabla }^{s}f\left(\tilde{\mathbf{w}}(k)\right)\right)}}^T\times \tilde{\mathbf{w}}(k-1)=1,$$

(27)

or

$$\lambda =-2\frac{1+{\gamma }_k{\nabla }^{s}f{{\displaystyle \left(\tilde{\mathbf{w}}(k)\right)}}^T{{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)}{{{\displaystyle \tilde{\mathbf{w}}}}^T(k-1){{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)}.$$

(28)

Substituting $\lambda$ in Equation (26) by Equation (28) leads to

$$\tilde{\mathbf{w}}(k)=\frac{1+{\gamma }_k{\nabla }^{s}f{{\displaystyle \left(\tilde{\mathbf{w}}(k)\right)}}^T{{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)}{{{\displaystyle \tilde{\mathbf{w}}}}^T(k-1){{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)}\times {{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)-{\gamma }_k{{\displaystyle \overline{R}}}^{-1}(k){\nabla }_{\tilde{\mathbf{w}}}f\left(\tilde{\mathbf{w}}(k)\right).$$

(29)

Equation (29) can be expressed in a simple form as

$$\tilde{\mathbf{w}}(k)=\alpha {{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)-{\gamma }_k{{\displaystyle \overline{R}}}^{-1}(k){\nabla }^{s}f\left(\tilde{\mathbf{w}}(k)\right),$$

(30)

where $\alpha =\frac{1+{\gamma }_k{\nabla }^{s}f{{\displaystyle \left(\tilde{\mathbf{w}}(k)\right)}}^T{{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)}{{{\displaystyle \tilde{\mathbf{w}}}}^T(k-1){{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)}$. Because asymptotically $\Vert \tilde{\mathbf{w}}(k)\Vert =1$ as $k\to \infty$, Equation (29) can be approximated as the following two equations.

$$\tilde{\mathbf{w}}(k)\simeq {{\displaystyle \overline{R}}}^{-1}(k)\tilde{\mathbf{w}}(k-1)-{\gamma }_k\left({{\displaystyle \tilde{\mathbf{w}}}}^T(k-1){{\displaystyle \overline{R}}}^{-1}(k-1)\tilde{\mathbf{w}}(k-1)\right){{\displaystyle \overline{R}}}^{-1}(k){\nabla }^{s}f\left(\tilde{\mathbf{w}}(k-1)\right).$$

(31)

$$\tilde{\mathbf{w}}(k)=\tilde{\mathbf{w}}(k)/\Vert \tilde{\mathbf{w}}(k)\Vert .$$

(32)

Finally, we obtain the estimated parameter of the unknown system as

$$\widehat{\mathbf{w}}(k)=-{{\displaystyle \tilde{\mathbf{w}}}}_{1:M}(k)/{{\displaystyle \tilde{\mathbf{w}}}}_{M+1}(k).$$

(33)

For Equation (23), we can use the modified regularization factor ${\gamma }_k$ in [30]

$${\gamma }_k=\frac{2\frac{tr\left({{\displaystyle \overline{R}}}^{-1}(k)\right)}{M}}{{{\displaystyle \Vert {{\displaystyle \overline{R}}}^{-1}(k){\nabla }^{s}f\left({{\displaystyle \widehat{\mathbf{w}}}}_{aug}(k)\right)\Vert }}_2^2}\times [\left(f\left({{\displaystyle \widehat{\mathbf{w}}}}_{aug}(k)\right)-\rho \right)+{\nabla }^{s}f\left({{\displaystyle \widehat{\mathbf{w}}}}_{aug}(k)\right){{\displaystyle \overline{R}}}^{-1}(k)\epsilon (k)],$$

(34)

where ${{\displaystyle \widehat{\mathbf{w}}}}_{aug}(k)={{\displaystyle \left[{{\displaystyle \widehat{\mathbf{w}}}}^T(k),-1\right]}}^T$, ${{\displaystyle \widehat{\mathbf{w}}}}_{aug,RLS}(k)={{\displaystyle \left[{{\displaystyle \widehat{\mathbf{w}}}}_{RLS}^T(k),-1\right]}}^T$, $\epsilon (k)={{\displaystyle \widehat{\mathbf{w}}}}_{aug}(k)-{{\displaystyle \widehat{\mathbf{w}}}}_{aug,RLS}(k)$, and ${{\displaystyle \widehat{\mathbf{w}}}}_{RLS}(k)$ is the estimated parameter by recursive least squares (RLS). As $f(\widehat{\mathbf{w}})={{\displaystyle \Vert \widehat{\mathbf{w}}\Vert }}_1$, the subgradient of $f({{\displaystyle \widehat{\mathbf{w}}}}_{aug}(k))$ is

$${\nabla }^s{{\displaystyle \Vert {{\displaystyle \widehat{\mathbf{w}}}}_{aug}(k)\Vert }}_1=\mathrm{sgn}({{\displaystyle \widehat{\mathbf{w}}}}_{aug}(k)).$$

(35)

As mentioned in Section 4, we apply new constant $\rho$ in Table 1, to the regularization factor ${\gamma }_k$ in Equation (34) instead of ${{\displaystyle \Vert {\mathbf{w}}_{true}\Vert }}_1$, where ${\mathbf{w}}_{true}$ is the true system impulse response.

7. Simulation Results

This section confirms the performance of the proposed algorithm in sparse channel estimation. In the first experiment, the channel estimation performance is compared with other algorithms using randomly generated sparse channels. In this simulation, we follow the same scenario in the experiments as [17]. The true system vector ${\mathbf{w}}_{true}$ is 64 dimensions. In order to generate the sparse channel, we set the number of the nonzero coefficients, S, in the 64 coefficients and randomly position the nonzero coefficients. The values of the coefficients are taken from an $N(0,1/\mathrm{S})$ distribution, where $N()$ is the normal distribution. In the simulation, we estimate the channel impulse response by the proposed algorithms that are l₁-RLS using the $\rho$ in Table 1 and l₁-RTLS using the $\rho$ in Table 1. For the comparison, we estimate the channel impulse response by l₁-RLS using the true channel response; in addition, we also execute the regular RLS algorithm in an oracle setting (oracle-RLS) where the positions of the true nonzero system parameters are assumed to be known. For the estimated channel results, we calculate the mean standard deviation (MSD), where $\mathrm{MSD}=E\left({\left|\widehat{\mathbf{w}}-{\mathbf{w}}_{true}\right|}^2\right)$, $\widehat{\mathbf{w}}$ is the estimated channel response and ${\mathbf{w}}_{true}$ is the true channel response. For the performance evaluation, we simulate the algorithms in the sparse channels for S = 4, 8, 16, and 32.

Figure 3 illustrates the MSD curves. For S = 4, Figure 3a shows that the estimation performance of l₁-RTLS using the regularization factor with the $\rho$ in Table 1 is almost the same as the l₁-RLS using regularization with a true channel impulse response. However, the performance of l₁-RLS using the regularization factor with the $\rho$ in Table 1 is gradually degraded and shows a kind of uncertainty accumulation effect. In the other cases of S, we can observe the same trend in the MSD curves. Therefore, we can confirm that the new regularization factor selection method and the new modeling for l₁-RLS can estimate the sparse channel as good as l₁-RLS using the regularization with the true channel impulse response. In all the simulation scenarios, oracle RLS algorithm produces the lowest MSD as expected.

Table 2. MSD (mean square deviation) comparison.

Sparsity (S)	Algorithm	MSD
4	l1-RLS with the true channel	−40.6 dB
	l1-RLS with the new $\rho$ method	−37.8 dB
	proposed l1-RTLS with the new $\rho$ method	−38.5 dB
	Oracle-RLS	−50.4 dB
8	l1-RLS with the true channel	−39.5 dB
	l1-RLS with the new $\rho$ method	−28.4 dB
	proposed l1-RTLS with the new $\rho$ method	−38.5 dB
	Oracle-RLS	−46.9 dB
16	l1-RLS with the true channel	−38.4 dB
	l1-RLS with the new $\rho$ method	−18.2 dB
	proposed l1-RTLS with the new $\rho$ method	−37.6 dB
	Oracle-RLS	−43.6 dB
32	l1-RLS with the true channel	−37.6 dB
	l1-RLS with the new $\rho$ method	−9.1 dB
	proposed l1-RTLS with the new $\rho$ method	−37.3 dB
	Oracle-RLS	−40.6 dB

summarizes the steady-state MSD values as varying S from 4 to 32. The results show that the proposed l₁-RTLS with the new $\rho$ is comparable to l₁-RLS with the true channel.

In the second experiment, we compare channel estimation performance using room impulse response. The size of the room is (7.49, 6.24, 3.88 m). The position of the sound source is (1.53, 0.96, 1.12 m) and the position of the receiver is (1.81, 5.17, 0.71 m), respectively. T60 is set to 100 ms and 400 ms. The impulse response of the room is generated using the program in [31]. We focus on the direct reflection part and the early reflection part in the RIR because the direct reflection and early reflection part of the RIR has a sparse property. This is the part that is estimated in the AEC applications [32]. This part is also related to localization and clarity in room acoustics [33,34,35]. Comparing the impulse response (IR) generated by setting T60 = 100 ms to the channel with 65 coefficients used in the first experiment, it is equivalent to S = 4 in the channel with 65 coefficients. In the same manner, the IR generated by setting T60 = 400 ms is equivalent to S = 10.

Table 3. MSD (mean square deviation) comparison in sparse RIR estimations.

Reverberation Time (T60)	Algorithm	MSD
100 ms	l1-RLS with the true channel	−38.5 dB
	l1-RLS with the new $\rho$ method	−34.7 dB
	proposed l1-RTLS with the new $\rho$ method	−35.4 dB
	Oracle-RLS	−45.3 dB
400 ms	l1-RLS with the true channel	−32.1 dB
	l1-RLS with the new $\rho$ method	−20.9 dB
	proposed l1-RTLS with the new $\rho$ method	−30.1 dB
	Oracle-RLS	−36.0 dB

summarizes the steady-state MSD values. The results also show the same trend as Table 2. In RIR estimation, the proposed l₁-RTLS with the new $\rho$ is also comparable to l₁-RLS with the true channel.

8. Conclusions

In this paper, we have proposed the regularization factor for recursive adaptive estimation. The regularization factor needs no prior knowledge of the true channel impulse response. We have also reformulated the recursive estimation algorithm as l₁-RTLS type. This formulation is robust to the uncertainty in the regularization factor without a priori knowledge of the true channel impulse response. Simulations show that the proposed regularization factor and l₁-RTLS algorithm provide good performance comparable to l₁- RLS with the knowledge of the true channel impulse response.