Norm Penalized Joint-Optimization NLMS Algorithms for Broadband Sparse Adaptive Channel Estimation

Abstract

A joint-optimization method is proposed for enhancing the behavior of the $l_1$-norm- and sum-log norm-penalized NLMS algorithms to meet the requirements of sparse adaptive channel estimations. The improved channel estimation algorithms are realized by using a state stable model to implement a joint-optimization problem to give a proper trade-off between the convergence and the channel estimation behavior. The joint-optimization problem is to optimize the step size and regularization parameters for minimizing the estimation bias of the channel. Numerical results achieved from a broadband sparse channel estimation are given to indicate the good behavior of the developed joint-optimized NLMS algorithms by comparison with the previously proposed $l_1$-norm- and sum-log norm-penalized NLMS and least mean square (LMS) algorithms.

1. Introduction

Recently, adaptive channel estimation has been extensively studied all over the world [1,2]. From these adaptive filtering frameworks, the LMS adaptive filtering algorithm has been deeply discussed for adaptive control, system identification, channel estimation applications owing to its low computational complexity and easy practical realization [3,4,5]. Although the LMS algorithm can effectively estimate the broadband multi-path channel, it has a sensitivity to the input signal scaling, and it is difficult to select an agreeable learning rate to achieve a stable and robust channel estimation behavior [6,7,8,9,10,11]. Subsequently, the NLMS has been proposed by using the normalization of the input training signal power in order to overcome the above addressed problem [12,13]. However, the conventional NLMS algorithm cannot perform well for dealing with sparse channel estimation.

Additionally, the broadband multi-path channel or underwater communication channel might be a sparse channel, which has been studied in recent decades [13,14,15,16,17]. From the measurement of the wireless channel, the channel impulse response can be regarded as sparse channel. This is to say that only a few channel impulse responses in most of the multi-path channels are dominant, while the major channel responses are zeros or their magnitudes are near-zeros. The traditional LMS- and NLMS-based channel estimation methods cannot make use of the inherent sparse properties of these broadband multi-path channels [6,7,8,9,10,11,12,13]. To utilize the sparse structures of the broadband sparse channel, compressed sensing (CS) methods have been introduced for developing various channel estimation algorithms used in sparse cases [18,19,20]. Although some of these CS-based sparse channel estimations can achieve robust estimation performance, these CS algorithms may have high complexity for dealing with time-varying channels or they have difficulty in constructing desired measurement matrices with the restricted isometry property limitation [21]. Thus, simple sparse channel estimation developments have attracted much more attentions in the recent decades.

Sparse LMSs have been presented under the inspiration of the CS techniques [22,23], which are realized by using a norm constraint term in the cost function of the LMS. The first sparse LMS algorithm motivated by the CS technique is carried out by introducing a $l_1$-norm constraint term into the basic LMS to exploit the in-nature sparse characteristics of broadband multi-path channel [24,25,26,27]. As a result, a zero attractor is given in the updating equation of the sparse LMS algorithm to put forward a zero-attracting (ZA) LMS (ZA-LMS) algorithm. Furthermore, a reweighting ZA-LMS (RZA-LMS) was reported by using a sum-log constraint instead of the $l_1$-norm penalty in the ZA-LMS algorithm [24,27]. Subsequently, the zero attracting techniques have been widely researched, and a great quantity of sparse LMS algorithms was exploited by using different norm constraints, such as $l_p$-norm and smooth approximation $l_0$-norm constraints [28,29,30,31,32]. Furthermore, the zero attracting (ZA) technique has also been expanded into the affine projection algorithm and the normalized NLMS algorithms to further exploit the applications of the ZA algorithms [12,33,34,35,36,37,38,39,40], which includes ZA-NLMS and RZA-NLMS algorithms. However, the affine projection algorithm has higher complexity than the NLMS algorithm, which limited its applications. Thus, the sparse NLMS algorithms have been extensively studied and have been used for sparse channel estimations. However, the behavior of these NLMS-based channel estimations was affected by the modified step size and the regularized parameters. The normalized step size has an important effect on the compromise between the channel estimation behavior and convergence speed, while the regularization parameter depends on the SNR of the systems [41]. From [41], we can see that the proposed joint-optimization method cannot utilize the sparsity of the multi-channels. In addition, the step size and the regularization parameters should be selected to address the conflict requirement between the channel estimation behavior and the convergence speed.

In this paper, a joint-optimization method is discussed to enhance the behavior of the ZA- and RZA-NLMS algorithms for estimating finite impulse response (FIR) sparse channels. Their estimation behaviors are discussed via a broadband multi-path channel, which is a prior known sparse channel from the previous studies [6,7,9,12,14,24,27,28,29,30,31,32,33,34,35,36,37,38,39,40]. The proposed algorithms are realized by implementing a joint-optimization on the regularization parameter and step size to achieve a minimization of the channel estimation misalignment, where the joint-optimization method is obtained from [41]. The joint-optimization scheme is derived in detail in the context of zero attraction theory, and it is incorporated into the previously-proposed ZA- and RZA-NLMS algorithms to form zero-attracting joint-optimization NLMS (ZAJO-NLMS) and reweighted ZAJO-NLMS (RZAJO-NLMS) algorithms. The parameter selection and the behavior of our ZAJO-NLMS and RZAJO-NLMS are evaluated though a sparse channel, which is similar to [6,7,9,12,14,24,27,28,29,30,31,32,33,34,35,36,37,38,39,40]. The computer simulations indicate that our ZAJO-NLMS and RZAJO-NLMS algorithms perform better than the previously-presented ZA-NLMS, RZA-NLMS, ZA-LMS, RZA-LMS, traditional LMS and NLMS algorithms for dealing with sparse channel estimation.

The structure is illustrated herein. We review the basic NLMS and the previously-reported ZA-NLMS, as well as the RZA-NLMS algorithms in Section 2 through estimating a broadband multipath wireless communication channel. Section 3 gives the derivation of the joint-optimization scheme and the proposed ZAJO-NLMS and RZAJO-NLMS algorithms. In Section 4, our ZAJO-NLMS and RZAJO-NLMS algorithms will be evaluated though a broadband sparse multipath channel, and their channel estimation behaviors are discussed and compared with the previous ZA-NLMS, RZA-NLMS, ZA-LMS, RZA-LMS, traditional LMS and NLMS algorithms. At last, a short summary is given in Section 5.

2. NLMS-Based Sparse Adaptive Channel Estimation Algorithm

We review the traditional NLMS and its sparse forms, which include the ZA-NLMS and RZA-NLMS, over a sparse multipath wireless channel, which is given in Figure 1.

$\mathbf{x}\left(n\right)={[x\left(n\right),x(n-1),\cdots ,x(n-N-1)]}^T$ is used as a training input signal, which is transmitted over an FIR channel $\mathbf{w}={[w_0,w_1,\cdots ,w_{N-1}]}^T$. In this paper, N denotes the length of a multipath channel, and ${(·)}^T$ is the transpose operation. The channel output is $y\left(n\right)$, which is denoted as $y\left(n\right)={\mathbf{x}}^T\left(n\right)\mathbf{w}$. The expected signal $d\left(n\right)$ is acquired at the receiver, and $d\left(n\right)=y\left(n\right)+v\left(n\right)$. $v\left(n\right)$ denotes an additional white Gaussian noise (AWGN). The NLMS-based channel estimation algorithms aim to get the unknown sparse channel $\mathbf{w}$ by utilizing $\mathbf{x}\left(n\right)$ and $\mathbf{y}\left(n\right)$ to minimize the instant error $e\left(n\right)$. We define $e\left(n\right)=d\left(n\right)-{\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)$. $\widehat{\mathbf{w}}\left(n\right)$ denotes the estimated vector. The cost function of the traditional NLMS is depicted as:

$$\begin{array}{c}\min {∥\widehat{\mathbf{w}}(n+1)-\widehat{\mathbf{w}}\left(n\right)∥}^2\hfill \\ \mathrm{s}.\mathrm{t}.:d\left(n\right)-{\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}(n+1)=0\hfill \end{array}.$$

(1)

By using the Lagrange multiplier method to carry out the desired minimization of (1) and introducing a controlling step size, the updating of the traditional NLMS is obtained [3,4,5]:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\frac{\mu e\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)+\delta }.$$

(2)

$\mu$ denotes a step size of the NLMS algorithm, and $\delta$ represents a regularization parameter with small value, which is to prevent from dividing by zero. Although the NLMS algorithm can give a good estimation of the sparse channel $\mathbf{w}$ by the use of the update Equation (2), it cannot use the sparsity property of the practical existing multi-path channels. Recently, the ZA technique has been introduced into the original cost function of the traditional NLMS for exploiting the sparseness of the channel. Then, the ZA-NLMS algorithm is presented by using an $l_1$-norm penalty to modify the original cost function of the traditional NLMS. Thus, the modified cost function for the reported ZA-NLMS is written as:

$$\begin{array}{c}\min {∥\widehat{\mathbf{w}}(n+1)-\widehat{\mathbf{w}}\left(n\right)∥}^2+{\gamma }_1{∥\widehat{\mathbf{w}}(n+1)∥}_1\hfill \\ \mathrm{s}.\mathrm{t}.:d\left(n\right)-{\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}(n+1)=0\hfill \end{array},$$

(3)

where ${\gamma }_1$ denotes a zero attracting strength parameter used to get a balance between the estimation behavior and the sparseness of ${∥\widehat{\mathbf{w}}(n+1)∥}_1$. Furthermore, the Lagrange multiplier method is utilized to find out a solution of (3). As a result, the updating equation of ZA-NLMS is described as [12]:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\frac{{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}e\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)+\delta }-{\rho }_{\mathrm{ZA}-\mathrm{NLMS}}\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\},$$

(4)

where ${\mu }_{\mathrm{ZA}-\mathrm{NLMS}}$ denotes a step size and ${\rho }_{\mathrm{ZA}-\mathrm{NLMS}}$ denotes the zero-attracting strength controlling factor for the ZA-NLMS algorithm, respectively. The ZA-NLMS algorithm utilizes the sparse characteristic of the multi-path wireless channel. However, it exerts a uniform zero attracting on all of the channel taps. Thus, the ZA-NLMS algorithm may reduce the estimation behavior when the designated channel is less sparse. To address this problem, a sum-log penalty [42] is utilized to form the RZA-NLMS algorithm, whose cost function is written as:

$$\begin{array}{c}\min {∥\widehat{\mathbf{w}}(n+1)-\widehat{\mathbf{w}}\left(n\right)∥}^2+{\gamma }_2{\displaystyle \sum _{i=1}^N}log(1+\epsilon \left|w_i\right|)\hfill \\ \mathrm{s}.\mathrm{t}.:d\left(n\right)-{\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}(n+1)=0\hfill \end{array}.$$

(5)

We also employ the Lagrange multiplier method to solve Equation (5). Then, the channel coefficients of the RZA-NLMS algorithm are updated by: [12]

$${\widehat{w}}_i(n+1)={\widehat{w}}_i\left(n\right)+\frac{{\mu }_{\mathrm{RZA}-\mathrm{NLMS}}e\left(n\right)x_i\left(n\right)}{x_i^T\left(n\right)x_i\left(n\right)+\delta }-{\rho }_{\mathrm{RZA}-\mathrm{NLMS}}\frac{\mathrm{sgn}\left\{{\widehat{w}}_i\left(n\right)\right\}}{\mathbf{1}+\epsilon \left|{\widehat{w}}_i\left(n\right)\right|},$$

(6)

or its vector form:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\frac{{\mu }_{\mathrm{RZA}-\mathrm{NLMS}}e\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)+\delta }-{\rho }_{\mathrm{RZA}-\mathrm{NLMS}}\frac{\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}}{\mathbf{1}+\epsilon \left|\widehat{\mathbf{w}}\left(n\right)\right|},$$

(7)

where ${\mu }_{\mathrm{RZA}-\mathrm{NLMS}}$ is a step size, while ${\rho }_{\mathrm{RZA}-\mathrm{NLMS}}$ denotes a zero-attracting strength controlling factor of the RZA-NLMS algorithm, respectively. Here, $\epsilon$ is a threshold that is used for controlling the reweighted factor $\mathbf{1}$/$(\mathbf{1}+\epsilon \left|\widehat{\mathbf{w}}\left(n\right)\right|)$.

From the above discussions, it is observed that the previously-proposed ZA-NLMS and RZA-NLMS exert the desired zero-attractor on each iteration. The proposed zero attractors attract the zero channel coefficients to zero quickly compared with the traditional NLMS algorithm. Thus, we can say that the sparse NLMS algorithms, namely ZA-NLMS and RZA-NLMS, utilize different sparse penalties to achieve various zero attractors. The traditional NLMS-based channel estimation algorithm is concluded as follows:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\mathrm{Error}\mathrm{updating}\mathrm{term}.$$

(8)

Comparing to the traditional NLMS method mentioned in (8), the mentioned sparse NLMS-based channel estimation algorithms provide amazing zero attractors for both ZA-NLMS and RZA-NLMS, and hence, their updated equation is:

$$\widehat{\mathbf{w}}(n+1)=\underset{\mathrm{sparse}\mathrm{NLMS}}{\underbrace{\underset{\mathrm{NLMS}}{\underbrace{\widehat{\mathbf{w}}\left(n\right)+\mathrm{Error}\mathrm{updating}\mathrm{term}}}+\mathrm{zero}\mathrm{attractor}}}.$$

(9)

3. Proposed Sparse Joint-Optimization NLMS Algorithms

Though the proposed sparse NLMS algorithms effectively utilize the in-nature properties of the wireless multi-path channel for achieving a superior channel estimation behavior, their performance will be affected by step size, regularization parameter $\delta$ and the zero-attracting strength controlling factor. After that, various techniques have been presented to enhance the behavior of the ZA-NLMS and RZA-NLMS algorithms, including variable step size methods and parameter-adjusting techniques [32]. Herein, we concentrate on constructing a joint-optimization method on the regularization parameter $\delta$ and step size.

Next, we will introduce our proposed ZAJO-NLMS algorithm in detail. Here, we consider the updating equation of the ZA-NLMS algorithm:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\frac{{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}e\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)+\delta }-{\rho }_{\mathrm{ZA}-\mathrm{NLMS}}\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\},$$

(10)

where ${\mu }_{\mathrm{ZA}-\mathrm{NLMS}}$ denotes a step size, $\delta$ represents a regularization parameter and:

$$e\left(n\right)=d\left(n\right)-{\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)$$

(11)

denotes the instantaneous channel estimation error at instant n. As we know, $\mathrm{E}\left\{{\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)\right\}=N{\delta }_x^2$ [41]; $\mathrm{E}\left\{·\right\}$ represents expectation, and ${\delta }_x^2$ represents the variance of $\mathbf{x}\left(n\right)$. For large N, we can get ${\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)\approx N{\delta }_x^2$ [43]. Then, we have:

$$\widehat{\mathbf{w}}(n+1)\approx \widehat{\mathbf{w}}\left(n\right)+\frac{{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}e\left(n\right)\mathbf{x}\left(n\right)}{N{\delta }_x^2+\delta }-{\rho }_{\mathrm{ZA}-\mathrm{NLMS}}\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}.$$

(12)

Next, we consider $\mathbf{w}\left(n\right)$ as a channel that can be modeled as a simplified first order Markov model [41],

$$\mathbf{w}(n+1)=\mathbf{w}\left(n\right)+\mathbf{g}\left(n\right),$$

(13)

where $\mathbf{g}\left(n\right)$ represents an AWGN signal with zero mean. Furthermore, we assume that $\mathbf{g}\left(n\right)$ is independent of the channel $\mathbf{w}\left(n\right)$. Therefore, we can get:

$${\mathbf{R}}_{\mathbf{g}}={\delta }_g^2{\mathbf{I}}_N,$$

(14)

where ${\mathbf{I}}_N$ is an $N\times N$ identity matrix. Thus, the variance of ${\delta }_g^2$ gives an important uncertainty on $\mathbf{w}\left(n\right)$. Here, we define a posteriori bias as:

$$\Delta \left(n\right)=\mathbf{w}\left(n\right)-\widehat{\mathbf{w}}\left(n\right).$$

(15)

Here, we can get:

$$\Delta (n+1)=\Delta \left(n\right)+\mathbf{g}\left(n\right)-\frac{{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}e\left(n\right)\mathbf{x}\left(n\right)}{N{\delta }_x^2+\delta }+{\rho }_{\mathrm{ZA}-\mathrm{NLMS}}\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}.$$

(16)

Taking the $l_2$-norm on (16), using expectation on its left and right sides and getting rid of the uncorrelated products based on the i.i.d. assumptions, we have:

$$\begin{array}{c}\mathrm{E}\left\{{∥\Delta (n+1)∥}_2^2\right\}=\mathrm{E}\left\{{∥\Delta \left(n\right)∥}_2^2\right\}+N{\delta }_g^2+{\rho }_{\mathrm{ZA}-\mathrm{NLMS}}^2\mathrm{E}\left\{{\mathrm{sgn}}^2\left\{\widehat{\mathbf{w}}\left(n\right)\right\}\right\}\hfill \\ -\frac{2{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}}{N{\delta }_x^2+\delta }\mathrm{E}\left\{{\mathbf{x}}^T\left(n\right)\Delta \left(n\right)e\left(n\right)\right\}+2{\rho }_{\mathrm{ZA}-\mathrm{NLMS}}\mathrm{E}\left\{\Delta \left(n\right)\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}\right\}-\frac{2{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}}{N{\delta }_x^2+\delta }\mathrm{E}\left\{{\mathbf{x}}^T\left(n\right)\mathbf{g}\left(n\right)e\left(n\right)\right\}\hfill \\ +\frac{{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}^2}{{(N{\delta }_x^2+\delta )}^2}\mathrm{E}\left\{e^2\left(n\right){\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)\right\}\hfill \end{array}.$$

(17)

Now, we concentrate on the last five terms in the Equation (17). From the above discussion, the instantaneous error is illustrated as:

$$e\left(n\right)={\mathbf{x}}^T\left(n\right)\mathbf{w}\left(n\right)+{\mathbf{x}}^T\left(n\right)\mathbf{g}\left(n\right)+v\left(n\right)$$

(18)

By taking the Equation (18) into account, we can get:

$$\begin{array}{c}\mathrm{E}\left\{{\mathbf{x}}^T\left(n\right)\Delta \left(n\right)e\left(n\right)\right\}=\mathrm{E}\left\{{\Delta }^T\left(n\right)\mathbf{x}\left(n\right)e\left(n\right)\right\}\hfill \\ =\mathrm{E}\left\{{\Delta }^T\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\Delta \left(n\right)\right\}=\mathrm{E}\left\{\mathrm{tr}\left[\Delta \left(n\right){\Delta }^T\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]\right\}\hfill \end{array},$$

(19)

where $\mathrm{tr}\left[·\right]$ represents the trace operation of a matrix. Assume that the misalignments at the instant n and $(n+1)$ are uncorrelated. In the stable state, a posteriori bias correlation matrix is approximated as a diagonal matrix [41], which is because the bias of each coefficient tends to be uncorrelated. Thus, we obtain:

$$\mathrm{E}\left\{{\Delta }^T\left(n\right)\mathbf{x}\left(n\right)e\left(n\right)\right\}\approx \mathrm{tr}\left\{\mathrm{E}\left[\Delta \left(n\right){\Delta }^T\left(n\right)\right]\mathrm{E}\left[\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]\right\}={\delta }_x^2\mathrm{E}\left\{{∥\Delta \left(n\right)∥}_2^2\right\}.$$

(20)

Similarly, the cross-correlation $\mathrm{E}\left\{{\mathbf{x}}^T\left(n\right)\mathbf{g}\left(n\right)e\left(n\right)\right\}$ can also be obtained based on (18). Therefore, the correlation matrix of $\mathbf{g}\left(n\right)$ can be regarded as a diagonal matrix. By eliminating the uncorrelated terms, we obtain:

$$\begin{array}{c}\mathrm{E}\left\{{\mathbf{x}}^T\left(n\right)\mathbf{g}\left(n\right)e\left(n\right)\right\}=\mathrm{E}\left\{{\mathbf{g}}^T\left(n\right)\mathbf{x}\left(n\right)e\left(n\right)\right\}=\mathrm{E}\left\{{\mathbf{g}}^T\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\mathbf{g}\left(n\right)\right\}\hfill \\ =\mathrm{E}\left\{\mathrm{tr}\left[\mathbf{g}\left(n\right){\mathbf{g}}^T\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]\right\}\approx \mathrm{tr}\left\{\mathrm{E}\left[\mathbf{g}\left(n\right){\mathbf{g}}^T\left(n\right)\right]\mathrm{E}\left[\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]\right\}=N{\delta }_x^2{\delta }_g^2\hfill \end{array}.$$

(21)

Then, the expectation term $\mathrm{E}\left\{e^2\left(n\right){\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)\right\}$ can also be calculated by taking Equation (18) into consideration. Similarly, we have:

$$\begin{array}{c}\mathrm{E}\left\{e^2\left(n\right){\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)\right\}=\mathrm{tr}\left\{\mathrm{E}\left[e^2\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]\right\}=\mathrm{tr}\left\{\mathrm{E}\left[v^2\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]\right\}+\hfill \\ \mathrm{tr}\left\{\mathrm{E}\left\{{\left[\Delta \left(n\right)\mathbf{x}\left(n\right)+{\mathbf{g}}^T\left(n\right)\mathbf{x}\left(n\right)\right]}^2\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right\}\right\}\hfill \end{array}.$$

(22)

Here, we assume that the correlation matrix of $\mathbf{x}\left(n\right)$ approximates a diagonal matrix, which has been widely used for simplifying the analysis [41,44]. Furthermore, this also motivates us to further develop the second term in Equation (22) based on the Gaussian moment factoring theory [45]. Then, we obtain:

$$\begin{array}{c}\mathrm{E}\left\{e^2\left(n\right){\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)\right\}\hfill \\ =N{\delta }_x^2{\delta }_v^2+\mathrm{tr}\left\{{\delta }_x^4\left\{N{\delta }_g^2+\mathrm{E}\left\{{∥\Delta \left(n\right)∥}_2^2\right\}\right\}{\mathbf{I}}_N+2{\delta }_x^4\left\{{\delta }_g^2{\mathbf{I}}_N+\mathrm{E}\left[\Delta \left(n\right){\Delta }^T\left(n\right)\right]\right\}\right\}\hfill \\ =N{\delta }_x^2{\delta }_v^2+(N+2){\delta }_x^4\left\{\mathrm{E}\left\{{∥\Delta \left(n\right)∥}_2^2\right\}+N{\delta }_g^2\right\}\hfill \end{array}.$$

(23)

As we know, at the stable status, we get:

$$\mathrm{E}\left\{\mathrm{sgn}\left[\widehat{\mathbf{w}}\left(n\right)\right]\right\}=\mathrm{sgn}\left[\widehat{\mathbf{w}}\left(n\right)\right],$$

(24)

and ${\rho }_{\mathrm{ZA}-\mathrm{NLMS}}$ is very small. Thus, Equation (17) can be approximated to be:

$$\begin{array}{c}\mathrm{E}\left\{{∥\Delta (n+1)∥}_2^2\right\}\approx \mathrm{E}\left\{{∥\Delta \left(n\right)∥}_2^2\right\}+N{\delta }_g^2-\frac{2{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}}{N{\delta }_x^2+\delta }\mathrm{E}\left\{{\mathbf{x}}^T\left(n\right)\Delta \left(n\right)e\left(n\right)\right\}\hfill \\ -\frac{2{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}}{N{\delta }_x^2+\delta }\mathrm{E}\left\{{\mathbf{x}}^T\left(n\right)\mathbf{g}\left(n\right)e\left(n\right)\right\}+\frac{{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}^2}{{(N{\delta }_x^2+\delta )}^2}\mathrm{E}\left\{e^2\left(n\right){\mathbf{x}}^T\left(n\right)\mathbf{x}\left(n\right)\right\}\hfill \end{array}$$

(25)

From the discussions above, we substitute Equations (20), (21) and (23) into Equation (25) and denote $m\left(n\right)=\mathrm{E}\left\{{∥\Delta (n+1)∥}_2^2\right\}$; we have:

$$m\left(n\right)=f({\mu }_{\mathrm{ZA}-\mathrm{NLMS}},\delta ,N,{\delta }_x^2)\mathrm{E}\left\{{∥\Delta \left(n\right)∥}_2^2\right\}+g({\mu }_{\mathrm{ZA}-\mathrm{NLMS}},\delta ,N,{\delta }_x^2,{\delta }_v^2,{\delta }_g^2),$$

(26)

where:

$$f({\mu }_{\mathrm{ZA}-\mathrm{NLMS}},\delta ,N,{\delta }_x^2)=1-\frac{2{\delta }_x^2}{N{\delta }_x^2+\delta }{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}+\frac{(N+2){\delta }_x^4}{{(N{\delta }_x^2+\delta )}^2}{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}^2,$$

(27)

and:

$$g({\mu }_{\mathrm{ZA}-\mathrm{NLMS}},\delta ,N,{\delta }_x^2,{\delta }_v^2,{\delta }_g^2)=\frac{N{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}^2{\delta }_x^2\left[{\delta }_v^2+(N+2){\delta }_x^2{\delta }_g^2\right]}{{(N{\delta }_x^2+\delta )}^2}-\frac{2N{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}{\delta }_x^2{\delta }_g^2}{N{\delta }_x^2+\delta }+N{\delta }_g^2.$$

(28)

From the result in Equation (26), we can see that the convergence speed and the misadjustment components are separated from each other. The first term of Equation (26) plays a significant role in the convergence speed of adaptive filters, which depends on ${\mu }_{\mathrm{ZA}-\mathrm{NLMS}}$, $\delta$, input signal power and filter length. It is worth noting that the convergence speed component does not rely on the ${\delta }_v^2$ and ${\delta }_g^2$ of the model [41]. Thus, the noise power ${\delta }_v^2$ and the uncertainties ${\delta }_g^2$ do not give any effect on the convergence. We can see that fastest convergence speed can be achieved when Equation (27) reaches its minimum. Then, we have:

$$\frac{\partial f({\mu }_{\mathrm{ZA}-\mathrm{NLMS}},\delta ,N,{\delta }_x^2)}{\partial {\mu }_{\mathrm{ZA}-\mathrm{NLMS}}}=0.$$

(29)

Therefore, we can get:

$${\mu }_{\mathrm{H}}=\frac{\delta +N{\delta }_x^2}{(N+2){\delta }_x^2},$$

(30)

where ${\mu }_{\mathrm{H}}$ is the fastest convergence speed controlling factor. As we know, the regularization parameter $\delta$ is small, and the length N is large. Thus, ${\mu }_{\mathrm{H}}\approx 1$ for achieving the fastest convergence speed, which is well known in [3,44]. Furthermore, the stability condition is obtained by letting $\left|f({\mu }_{\mathrm{ZA}-\mathrm{NLMS}},\delta ,N,{\delta }_x^2)\right|<1$, which results in:

$$0<{\mu }_{\mathrm{stable}}<2\frac{\delta +N{\delta }_x^2}{(N+2){\delta }_x^2}=2{\mu }_{\mathrm{H}}$$

(31)

Again, by considering $\delta =0$ and $N\gg 2$, the stability conditions of the NLMS and ZA-NLMS algorithms can be obtained, which is written as ${\mu }_{\mathrm{NLMS}/\mathrm{ZA}-\mathrm{NLMS}}$.

The second term of Equation (26) gives large effects on the misalignment of the proposed algorithm, which significantly depends on the noise power ${\delta }_v^2$ and the uncertainties ${\delta }_g^2$ of the model. With the increment of these two parameters, the misalignment is also increased [41]. The lowest misalignment is obtained when Equation (28) reaches the minimum. Furthermore, by taking the step size into account, we can get the lowest misalignment, which can be expressed as:

$${\mu }_{\mathrm{L}}=\frac{{\delta }_g^2(\delta +N{\delta }_x^2)}{{\delta }_v^2+(N+2){\delta }_x^2{\delta }_g^2}.$$

(32)

As we know, the broadband channel is always time-varying, and hence, ${\delta }_g^2=0$. In this case, ${\mu }_{\mathrm{L}}=0$. This is to say that the lowest misalignment is achieved as the step size approximates zero [3]. From the discussions mentioned above, we follow the optimization criterion to mimic the channel estimation misalignment to obtain the optimized sparse RZA-NLMS and ZA-NLMS algorithms, which is based on the convergence analysis above.

An ideal adaptive filtering algorithm needs low misalignment and a rapid convergence speed rate. Unluckily, the results in Equations (30) and (32) give opposite directions. Thus, we need to optimize the step size to enhance the channel estimation behavior. Furthermore, the regularization affects the behaviors on sparse NLMS algorithms. From (27), we can see that the convergence decreases when the regularization parameter $\delta$ increases, while the misalignment in Equation (28) always increases when regularization parameter $\delta$ decreases. Thus, we should control the step size and $\delta$ to mimic the effects on the performance of the channel estimation algorithms. Additionally, we also follow a minimization problem with respect to the channel estimation misalignment. According to Equation (26) and assuming these two parameters are dependent on time, we can impose [41]:

$$\frac{\partial m\left(n\right)}{\partial {\mu }_{\mathrm{ZA}-\mathrm{NLMS}}}=0,\frac{\partial m\left(n\right)}{\partial \delta }=0.$$

(33)

Then, we can get the same result, which is expressed as:

$$\frac{{\mu }_{\mathrm{ZA}-\mathrm{NLMS}}}{N{\delta }_x^2+\delta }=\frac{m(n-1)+N{\delta }_g^2}{N{\delta }_v^2+(N+2){\delta }_x^2\left[m(n-1)+N{\delta }_g^2\right]},$$

(34)

which gives a joint-optimization procedure. Then, we introduce Equation (34) into Equation (10) to obtain the updating equation of our ZAJO-NLMS algorithm. Then, we can get:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\frac{\left[m(n-1)+N{\delta }_g^2\right]\mathbf{x}\left(n\right)e\left(n\right)}{N{\delta }_v^2+(N+2){\delta }_x^2\left[m(n-1)+N{\delta }_g^2\right]}-{\gamma }_{\mathrm{ZAJO}}\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}.$$

(35)

Obversely, we should update the $m\left(n\right)$ in (35). Then, substituting (34) into (26) results in:

$$m\left(n\right)=\left\{1-\frac{{\delta }_x^2\left[m(n-1)+N{\delta }_g^2\right]}{N{\delta }_v^2+(N+2){\delta }_x^2\left[m(n-1)+N{\delta }_g^2\right]}\right\}\left[m(n-1)+N{\delta }_g^2\right].$$

(36)

When $n\to \infty$, we have:

$${\delta }_x^2\left(m(\infty )+N{\delta }_g^2\right)-N(N+2){\delta }_g^2{\delta }_x^2\left[m(\infty )+N{\delta }_g^2\right]-N^2{\delta }_g^2{\delta }_v^2=0,$$

(37)

whose solution is expressed as:

$$m(\infty )=\frac{N{\delta }_g^2}{2}\left[N+(N+2)\sqrt{1+\frac{4{\delta }_v^2}{{(N+2)}^2{\delta }_g^2{\delta }_x^2}}\right].$$

(38)

Similarly to the extraction of the ZAJO-NLMS, a reweighting factor is incorporating into the ZAJO-NLMS. As a result, the updating equation is obtained for realizing the RZAJO-NLMS algorithm:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\frac{\left[m(n-1)+N{\delta }_g^2\right]\mathbf{x}\left(n\right)e\left(n\right)}{N{\delta }_v^2+(N+2){\delta }_x^2\left[m(n-1)+N{\delta }_g^2\right]}-{\gamma }_{\mathrm{RZAJO}}\frac{\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}}{\mathbf{1}+\epsilon \left|\widehat{\mathbf{w}}\left(n\right)\right|}.$$

(39)

From the proposed ZAJO- and RZAJO-NLMS algorithms, we found that the regularization parameter $\delta$ and step size are joint-optimized, while the zero attractor keeps invariable. From the above discussions, we found that there are two additional zero attraction terms, namely $-{\gamma }_{\mathrm{ZAJO}}\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}$ and $-{\gamma }_{\mathrm{RZAJO}}\frac{\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}}{1+\epsilon \left|\widehat{\mathbf{w}}\left(n\right)\right|}$, in the ZAJO-NLMS and RZAJO-NLMS algorithms, which are different from [41] due to the zero attractors. In this paper, the proposed ZAJO-NLMS and RZAJO-NLMS algorithms are implemented by the combination of the zero-attraction-based NLMS and the joint-optimization method in [41]. As a result, the ZAJO-NLMS and RZAJO-NLMS algorithms are constructed to deal with the sparse channel estimation, which can give better performance due to the joint-optimization and the zero attractors $-{\gamma }_{\mathrm{ZAJO}}\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}$ and $-{\gamma }_{\mathrm{RZAJO}}\frac{\mathrm{sgn}\left\{\widehat{\mathbf{w}}\left(n\right)\right\}}{1+\epsilon \left|\widehat{\mathbf{w}}\left(n\right)\right|}$.

Our contributions can be summarized as follows:

(1)

Two optimized ZAJO-NLMS and RZAJO-NLMS algorithms with zero attractors have been proposed for sparse channel estimation, in the context of the state variable model.

(2)

The proposed ZAJO-NLMS and RZAJO-NLMS algorithms are realized by using the joint-optimization method and the zero attraction techniques to mimic the channel estimation misalignment.

(3)

The behaviors of the proposed ZAJO-NLMS and RZAJO-NLMS algorithms are evaluated for estimating sparse channels.

(4)

The ZAJO-NLMS and RZAJO-NLMS algorithms can achieve both faster convergence and lower misalignment than the ZA- and RZA-NLMS algorithms owing to the joint-optimization, which effectively adjusts the step size and the regularization parameter. In the future, we will develop an optimal algorithm to optimize the zero-attractor terms.

4. Results and Discussion

We construct several experiments to look into the estimation behavior of our ZAJO-NLMS and RZAJO-NLMS algorithms through a multi-path wireless communication channel, which is a general sparse channel model obtained from the measurement [14,17] and which has been widely used for verifying the estimation performance of NLMS-based channel estimations [6,7,9,12,14,24,27,28,29,30,31,32,33,34,35,36,37,38,39,40]. Moreover, the channel estimation behavior is evaluated using mean-square error, and the channel estimation performance is also compared with the traditional LMS, NLMS, ZA-LMS, RZA-LMS, ZA-NLMS and RZA-NLMS algorithms. Here, a multipath channel has a length of $N=16$. This channel with varying K is used for predicting the estimation behavior by means of the mean square error (MSE) standard. In the investigation, the channel estimation performance with different sparsity level K is also analyzed in detail. The K dominant coefficients are distributed randomly in the channel, and it is limited by ${∥\mathbf{w}∥}_2^2=1$. An example of a typical broadband sparse multi-path wireless channel is given in Figure 2.

Here, the sparse channel has a length of 16, and it has three dominant coefficients. The number of non-zero coefficients is denoted as sparsity level K. In all of the experiments, $\mathbf{x}\left(n\right)$ is a Gaussian signal, and $\mathbf{v}\left(n\right)$ is a Gaussian noise. The received signal has a power of $E_b=1$. The power of the noise $\mathbf{v}\left(n\right)$ is given by ${\delta }_v^2$. The estimation behaviors of our ZAJO-NLMS and RZAJO-NLMS algorithms are accessed by the MSE given by:

$$\mathrm{MSE}\left(n\right)=\mathrm{E}\left\{{∥\mathbf{w}\left(n\right)-\widehat{\mathbf{w}}\left(n\right)∥}^2\right\}.$$

(40)

Since the step size and the regularization parameter have been optimized based on the derivation of our ZAJO-NLMS and RZAJO-NLMS, only the zero-attracting parameter can affect the performance of these proposed algorithms. Thus, we investigated the effects of the ${\gamma }_{\mathrm{ZAJO}}$ and ${\gamma }_{\mathrm{RZAJO}}$ on the MSE. The performance is shown in Figure 3 and Figure 4 for ${\gamma }_{\mathrm{ZAJO}}$ and ${\gamma }_{\mathrm{RZAJO}}$, respectively.

We can see that both ${\gamma }_{\mathrm{ZAJO}}$ and ${\gamma }_{\mathrm{RZAJO}}$ have important effects on the sparse channel estimation. With a decrement of the ${\gamma }_{\mathrm{ZAJO}}$, the MSE of the ZAJO-NLMS algorithm shown in Figure 3 is getting better when ${\gamma }_{\mathrm{ZAJO}}$ ranges from $3\times {10}^{-3}$ to $1\times {10}^{-4}$. As for ${\gamma }_{\mathrm{ZAJO}}=1\times {10}^{-5}$, both the convergence speed and MSE are deteriorated. From Figure 4, it is observed that ${\gamma }_{\mathrm{RZAJO}}$ has the same trend as that of the ${\gamma }_{\mathrm{ZAJO}}$. Thus, we can properly select ${\gamma }_{\mathrm{ZAJO}}$ and ${\gamma }_{\mathrm{RZAJO}}$ to achieve desired channel estimation performance. According to the parameter effects in Figure 3 and Figure 4, we choose ${\gamma }_{\mathrm{ZAJO}}={\gamma }_{\mathrm{RZAJO}}=3\times {10}^{-4}$ to investigate the effects of the sparsity level on the sparse channel estimation performance.

Next, we set the sparsity level of the broadband sparse multi-path to $K=1,2,4,8$ to analyze the channel estimation performance. The simulation parameters are ${\mu }_{\mathrm{LMS}}={\mu }_{\mathrm{ZALMS}}$$={\mu }_{\mathrm{RZALMS}}=0.05$, ${\mu }_{\mathrm{NLMS}}$$={\mu }_{\mathrm{ZANLMS}}=$${\mu }_{\mathrm{RZANLMS}}=0.5$, ${\rho }_{\mathrm{ZALMS}}={\rho }_{\mathrm{RZALMS}}={\rho }_{\mathrm{ZANLMS}}={\rho }_{\mathrm{RZANLMS}}$$={\gamma }_{\mathrm{ZAJO}}={\gamma }_{\mathrm{RZAJO}}=3\times {10}^{-4}$. The results are comparatively shown in Figure 5, Figure 6, Figure 7 and Figure 8 for $K=1$, $K=2$, $K=4$ and $K=8$, respectively. It is observed from Figure 5 that our RZAJO-NLMS algorithm achieves the quickest convergence and smallest steady-state error compared with the traditional LMS, NLMS, ZA-LMS, RZA-LMS, ZA-, RZA- and ZAJO NLMS algorithms. The proposed RZAJO-NLMS algorithm has much more gains than the traditional RZA-NLMS algorithm. Additionally, our proposed ZAJO-NLMS algorithm achieves better channel estimation behaviors than those of the RZA-NLMS algorithm for $K=1$. When the sparsity level K increases from 2 to 8, the gain between the RZAJO-NLMS and RZA-NLMS algorithm is getting small. However, our proposed RZAJO-NLMS algorithm outperforms all of the other sparse channel estimation algorithms with reference to both the MSE and convergence. When $K=8$, our proposed ZAJO-NLMS algorithm achieves quicker convergence compared to the traditional NLMS to get the same MSE level. On the contrary, the behavior of the ZA-NLMS becomes worse than that of the traditional NLMS. Thereby, our proposed RZAJO- and ZAJO-NLMS algorithms are more useful for adaptive sparse channel estimation applications.

5. Conclusions

We proposed joint-optimization sparse NLMS algorithms, namely RZAJO-NLMS and ZAJO-NLMS. The joint-optimization was realized by using a state model to improve the channel estimation performance of both the ZA- and RZA-NLMS algorithms. Our RZAJO- and ZAJO-NLMS algorithms are based on the joint-optimization of step size and the regularization parameter. The proposed joint-optimization was derived in detail. Furthermore, the estimation behavior of our RZAJO- and ZAJO-NLMS algorithms is evaluated on a broadband sparse multi-path channel with different sparsity levels. The results verified that our RZAJO-NLMS algorithm provides the fastest convergence speed rate and lowest MSE. In addition, the proposed ZAJO-NLMS outperforms the previously-reported ZA-NLMS and traditional NLMS algorithms.

This study provided the RZAJO-NLMS and ZAJO-NLMS algorithms based on the zero attracting and the joint-optimization techniques. The proposed joint-optimization technique can be expanded to the $l_p$-norm $(0\le p\le 1)$ constrained NLMS, normalized LMF (NLMF) and normalized least mean mixed norm (NLMMN) to enhance the sparse channel estimation performance or sparse system identification. In addition, the proposed method can be used for exploiting the two-dimensional (2D) adaptive filters for imaging processing, which can be used for medical imaging denoising applications [46]. Moreover, our proposed RZAJO-NLMS and ZAJO-NLMS algorithms can be integrated into the orthogonal frequency-division multiplexing (OFDM) and multiple-input multiple-output (MIMO) OFDM systems to improve the quality of the communication systems [47,48].