A General Zero Attraction Proportionate Normalized Maximum Correntropy Criterion Algorithm for Sparse System Identification

Abstract

A general zero attraction (GZA) proportionate normalized maximum correntropy criterion (GZA-PNMCC) algorithm is devised and presented on the basis of the proportionate-type adaptive filter techniques and zero attracting theory to highly improve the sparse system estimation behavior of the classical MCC algorithm within the framework of the sparse system identifications. The newly-developed GZA-PNMCC algorithm is carried out by introducing a parameter adjusting function into the cost function of the typical proportionate normalized maximum correntropy criterion (PNMCC) to create a zero attraction term. The developed optimization framework unifies the derivation of the zero attraction-based PNMCC algorithms. The developed GZA-PNMCC algorithm further exploits the impulsive response sparsity in comparison with the proportionate-type-based NMCC algorithm due to the GZA zero attraction. The superior performance of the GZA-PNMCC algorithm for estimating a sparse system in a non-Gaussian noise environment is proven by simulations.

1. Introduction

Adaptive filtering techniques have been extensively exploited, and they have experienced a challenging problem as the impulse response (IR) of the unknown systems is sparse [1,2,3]. As a result, a concept for developing sparse adaptive filters to handle such problem is becoming a hot topic for all researchers [4,5,6,7,8,9,10,11,12]. For sparse systems, most of their coefficients take the values of zero or near-zeros, while only a few coefficients have significant values [13,14,15,16]. Such sparse systems are commonly encountered in our in-nature world and plenty of real-world engineering applications such as the multi-path channels in wireless communications [17,18], underwater communications [19,20] and acoustic channels [21]. For example, for the signal transmissions in wireless communication in hilly-terrain environments, the channel response always exhibits a multi-path fading selecting phenomenon due to the delays of the different propagation paths [22]. Thus, the measurement result shows a sparse characteristic for such channels [23]. Furthermore, this phenomenon happens over wireless voice transmission systems and networks, in which the echo paths are demonstrated as active ones and some unknown bulk delays occur because of the propagation of the network, buffer delays and encoding processing [24]. Usually, these bulk delay intervals have little energy, which makes the impulse response sparse [25]. For dealing with this class of sparse impulse responses, traditional adaptive filters might converge slowly since all the system coefficients have the same step sizes, such as the normalized least mean square (NLMS) algorithm [26,27,28]. To cancel out this drawback, proportionate-type techniques have been presented to exploit the natural sparseness characteristics of these impulse responses [11,29,30].

The proportionate NLMS (PNLMS) is a popular sparse signal processing algorithm for handling these sparse systems, which is realized by proportionately updating the filter coefficients in accordance with the magnitudes of these coefficients [11]. As a result, the PNLMS converges faster at its initial stage compared with the NLMS algorithm [29,30,31]. However, after the initial convergence, the PNLMS’s convergence is getting bad, even its convergence is worse than the NLMS algorithm. Several improved PNLMS algorithms have been reported to treat the medium sparsity systems, including the improved PNLMS (IPNLMS) and PNLMS++ algorithms [30,31]. However, these algorithms still cannot give the expected convergence achieved from the PNLMS at the beginning iteration stage for high sparseness impulse response applications. Recently, an information theoretic quantity has been presented to build the required cost function for adaptive signal processing applications [32,33,34,35,36,37,38]. To calculate the quantities, the entropy estimator was reported in detail. Then, a minimum error entropy (MEE) has been developed in the context of the adaptive filtering framework by using the entropy as a cost function to justify the signal structure well [32]. From the estimation performance of the MEE, we found that it was robust for processing the impulsive noise signals [39]. However, it has high computational complexity compared with the least mean square (LMS) algorithm, which is posing a problem for practical applications [39,40]. Moreover, the correntropy can be used as an efficient optimality criterion in robust and sparse adaptive filtering [37]. Recently, a maximum correntropy criterion (MCC) algorithm was reported based on a cost function with localized similarity [40]. Furthermore, the MCC has a similar robustness characteristics to the MEE algorithm, and it possesses a LMS-like complexity, making it suitable for practical engineering applications in impulsive noise environments. Additionally, Chen et al. proposed a novel similarity measure called generalized correntropy, which when used in robust adaptive filtering, can significantly outperform the existing methods [37]. However, these MCCs cannot utilize the sparseness characteristics of the unknown systems. Motivated by the zero attracting theory, zero attracting MCC (ZA-MCC), reweighted ZA-MCC (RZA-MCC), group-constrained MCC (GC-MCC) and reweighted GC-MCC (RGC-MCC) algorithms [41,42] have been reported for estimating sparse channels. However, these algorithms are realized based on the MCC algorithm to exploit the sparsity property of the multi-path channels, which might be affected by the input scalings.

In this paper, a general zero attraction proportionate normalized MCC (GZA-PNMCC) algorithm is developed for the application in the sparse system identifications. The devised GZA-PNMCC algorithm is proposed based on the proportionate adaptive filtering scheme and the zero attracting techniques. In order to get the presented GZA-PNMCC, the following steps are considered. Firstly, the normalized MCC (NMCC) is proposed, followed by the introduction of the proportionate-type technique into the NMCC to construct the PNMCC algorithm. Secondly, a parameter adjustment function is integrated into the PNMCC algorithm to create the proposed GZA-PNMCC algorithm, and the zero attracting can be well adjusted for meeting various applications. The proposed GZA-PNMCC algorithm is analyzed, and the influence of different parameters and sparsity levels is investigated. The achieved results demonstrate that the GZA-PNMCC algorithm provides a faster convergence and better sparse system estimation performance in a mixed Gaussian noise environment. Different from the GC-MCC and RGC-MCC, the proposed GZA-PNMCC is exploited based on the PNMCC that is a proportionate-type NMCC algorithm. The GZA-PNMCC algorithm utilizes the parameter adjustment function to adjust the penalties on the channel coefficients to approximate various norms, which is also different with the GC and RGC techniques that can only use the $l_0$-norm penalty on the large group and the $l_1$-norm penalty on the small group. The proposed GZA-PNMCC can exert various penalties on the channel coefficients. Since the proposed GZA-PNMCC and the previous GC-MCC algorithms are presented based on different basic algorithms and different penalties are used in these algorithms, we believe that the proposed GZA-PNMCC algorithm can further exploit the inherent property of the multi-path channels.

The paper consists of the following sections. The NMCC algorithm and its proportionate form, the PNMCC algorithm, are presented in Section 2. In Section 3, the GZA-PNMCC algorithm is mathematically derived within the framework of the sparse system identifications. The influence of key parameters and sparsity levels on the estimation performance of GZA-PNMCC is investigated in Section 4. Finally, conclusions are summarized in Section 5.

2. Past Works on NMCC and PNMCC Algorithms

2.1. NMCC Algorithm

We herein consider a sparse unknown system whose impulse responses are ${\mathbf{w}}_{\mathbf{o}}\left(n\right)={[w_0,w_1,w_2,\cdots ,w_{N-2},w_{N-1}]}^T$. We use the NMCC algorithm to get an estimation $\widehat{\mathbf{w}}\left(n\right)$ of the unknown system ${\mathbf{w}}_{\mathbf{o}}\left(n\right)$ by using a training signal $\mathbf{x}\left(n\right)={[x\left(n\right),x(n-1),x(n-2),\cdots ,x(n-N-2),x(n-N-1)]}^T$, and the estimated error is $e\left(n\right)$. Here, $e\left(n\right)$ denotes the differences between the expected signal $d\left(n\right)$ and the estimator output signal $y\left(n\right)$. Based on the adaptive filtering theory, we have:

$$d\left(n\right)={\mathbf{x}}^T\left(n\right){\mathbf{w}}_o\left(n\right)+v\left(n\right),$$

(1)

$$y\left(n\right)={\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right),$$

(2)

and:

$$e\left(n\right)=d\left(n\right)-{\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right),$$

(3)

where $v\left(n\right)$ denotes as an impulsive noise. NMCC-based system identification tries to find out a solution of the following problem [40]:

$$\begin{array}{c}{\displaystyle \min \frac{1}{2}{∥\widehat{\mathbf{w}}(n+1)-\widehat{\mathbf{w}}\left(n\right)∥}^2,}\hfill \\ \mathrm{subject}\mathrm{to}\widehat{e}\left(n\right)=\left[1-\xi \exp \left({\displaystyle -\frac{e^2\left(n\right)}{2{\sigma }^2}}\right)\right]e\left(n\right)\hfill \end{array},$$

(4)

where $\widehat{e}\left(n\right)=d\left(n\right)-{\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}(n+1)$, ${∥·∥}^2$ denotes the Euclidean vector norm and $\sigma >0$ is used to give a tradeoff effect between the convergence and the MSE, $\xi =\chi {∥\mathbf{x}\left(n\right)∥}^2$. By using the Lagrange multiplier method (LMM), we can get NMCC’s updating equation, which is [40]:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\chi \frac{\exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)}{{∥\mathbf{x}\left(n\right)∥}^2}e\left(n\right)\mathbf{x}\left(n\right).$$

(5)

From the derivation of the NMCC algorithm, we find that the updating equation in (5) of the NMCC algorithm is similar to the NLMS algorithm. However, there is an exponential term in (5), which renders the NMCC algorithm more powerful for preventing impulsive-like noises.

2.2. PNMCC Algorithm

A gain assignment matrix $\mathbf{G}\left(n\right)$ similar to that of the known PNLMS algorithm is used to devise the PNMCC algorithm. By introducing $\mathbf{G}\left(n\right)$ into (5), the PNMCC’s updating equation is obtained [11,43]:

$${\displaystyle \widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\chi \frac{{\displaystyle \mathbf{G}\left(n\right)\exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)}}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+\vartheta }e\left(n\right)\mathbf{x}\left(n\right),}$$

(6)

where $\vartheta >0$ denotes a small constant, which is to prevent the denominator from being divided by zero, and it can provide a stable solution. Similar to the PNLMS algorithm, $\mathbf{G}\left(n\right)$ denotes a diagonal matrix, which is used for modifying the step size of the NMCC algorithm under a certain rule. Generally speaking, $\mathbf{G}\left(n\right)$ is [11]:

$$\mathbf{G}\left(n\right)=\mathrm{diag}(g_0\left(n\right),g_1\left(n\right),\cdots ,g_{N-1}\left(n\right)),$$

(7)

where the elements $g_i\left(n\right)$ is described as:

$${\displaystyle g_i\left(n\right)=\frac{{\kappa }_i\left(n\right)}{{\displaystyle \sum _{i=0}^{N-1}}{\kappa }_i\left(n\right)},0\le i\le N-1}$$

(8)

with:

$${\kappa }_i\left(n\right)=max[{\gamma }_{g}max[{\rho }_p,\left|{\widehat{w}}_0\left(n\right)\right|,\left|{\widehat{w}}_1\left(n\right)\right|,\cdots ,\left|{\widehat{w}}_{N-1}\left(n\right)\right|],\left|{\widehat{w}}_i\left(n\right)\right|],$$

(9)

where ${\rho }_p>0$ and ${\gamma }_g>$ usually have typical values of ${\rho }_p=0.01$ and ${\gamma }_g=5/N$. ${\rho }_p$ aims to avoid the weight updating’s stalling at the initial iteration stage when all the system coefficients are set to be zero, and ${\gamma }_g$ aims to prevent ${\widehat{w}}_i\left(n\right)$ from stalling when it is much smaller than the largest coefficient.

3. Proposed GZA-PNMCC Algorithm

The GZA-PNMCC algorithm is based on a parameter adjustment function constraint and the PNMCC algorithm to enhance the convergence characteristic of the PNMCC algorithm after the initial stage. To realize the GZA-PNMCC algorithm, a parameter adjustment function is incorporated into PNMCC’s cost function, where the parameter adjustment function is given as follows:

$$S_{\beta }\left(\widehat{\mathbf{w}}\left(n\right)\right)=(1+{\beta }^{-1})(1-e^{-\beta \left|\widehat{\mathbf{w}}\left(n\right)\right|}),$$

(10)

where $\beta >0$ represents a regulation parameter to model the desired general zero attraction. The behavior of the parameter adjustment function is well investigated with different $\beta$ and is shown in Figure 1. It can be seen that the parameter adjustment function can be used for approximating an $l_0$-norm when a large $\beta$ is used. If we choose a very small $\beta$, the parameter adjustment function has a characteristic that is similar to the $l_1$-norm. Therefore, a proper $\beta$ should be selected to develop a flexible norm-like penalty. Based on the analysis of the performance of the parameter adjustment function, we can say that it can be used for implementing $l_1$-norm-like, $l_0$-norm-like, even $l_p$-norm-like penalties. Since the parameter adjustment function is flexible and useful for implementing the desired norms, it is used to derive the general zero attraction PNMCC algorithm. The proposed parameter adjustment function is integrated into the PNMCC to continue to use the sparseness for system identification. The GZA-PNMCC algorithm aims to solve the following problem:

$$\begin{array}{c}{(\widehat{\mathbf{w}}(n+1)-\widehat{\mathbf{w}}\left(n\right))}^T{\mathbf{G}}^{-1}\left(n\right)(\widehat{\mathbf{w}}(n+1)-\widehat{\mathbf{w}}\left(n\right))+{\gamma }_{\mathrm{GZA}}{\mathbf{G}}^{-1}\left(n\right)S_{\beta }\left(\widehat{\mathbf{w}}(n+1)\right)\hfill \\ \mathrm{subject}\mathrm{to}\widehat{e}\left(n\right)=\left[{\displaystyle 1-\xi \exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)}\right]e\left(n\right)\hfill \end{array}.$$

(11)

In Equation (11), ${\mathbf{G}}^{-1}\left(n\right)$ represents the inverse form of the gain assignment matrix $\mathbf{G}\left(n\right)$. ${\gamma }_{\mathrm{GZA}}$ denotes a small positive regularization to give a tradeoff between the convergence and the sparseness development. It can be seen that the GZA-PNMCC algorithm utilizes a ${\gamma }_{\mathrm{GZA}}{\mathbf{G}}^{-1}\left(n\right)S_{\beta }\left(\widehat{\mathbf{w}}(n+1)\right)$ to create an expected zero attraction term, which makes it different from the PNMCC algorithm. Additionally, ${\mathbf{G}}^{-1}\left(n\right)$ is integrated into Equation (11).

Then, we employ the LMM to find out the solution of Equation (11). As a result, we write the proposed GZA-PNMCC’S cost function as:

$$\begin{array}{c}{J}_{\mathrm{GZA}}(n+1)={(\widehat{\mathbf{w}}(n+1)-\widehat{\mathbf{w}}\left(n\right))}^T{\mathbf{G}}^{-1}\left(n\right)(\widehat{\mathbf{w}}(n+1)-\widehat{\mathbf{w}}\left(n\right))\hfill \\ {\displaystyle +{\gamma }_{\mathrm{GZA}}{\mathbf{G}}^{-1}\left(n\right)S_{\beta }\left(\widehat{\mathbf{w}}(n+1)\right)+\lambda \left(\widehat{e}\left(n\right)-\left[1-\xi \exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\right]e\left(n\right)\right).}\hfill \end{array}$$

(12)

where $\lambda$ represents the Lagrange multiplier. The gradients of $J_{\mathrm{GZA}}(n+1)$ with respect to $\widehat{\mathbf{w}}(n+1)$ and $\lambda$ are obtained as follows:

$$\frac{\partial J_{\mathrm{GZA}}(n+1)}{\partial \widehat{\mathbf{w}}(n+1)}=0\mathrm{and}\frac{\partial J_{\mathrm{GZA}}(n+1)}{\partial \lambda }=0,$$

(13)

and:

$$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}\left(n\right)+\lambda \mathbf{G}\left(n\right)\mathbf{x}\left(n\right)-{\gamma }_{\mathrm{GZA}}{S}_{\beta }^{\prime }\left(\widehat{\mathbf{w}}(n+1)\right),$$

(14)

where $S_{\beta }^{\prime }\left(\widehat{\mathbf{w}}(n+1)\right)$ is the derivative of the parameter adjustment function, which is:

$${\displaystyle s_{i,\beta }^{\prime }\left({\widehat{w}}_i(n+1)\right)=(\beta +1)e^{(-\beta \left|{\widehat{w}}_i(n+1)\right|)}\mathrm{sgn}\left(w_i(n+1)\right),}$$

(15)

and its vector form is:

$${\displaystyle S_{\beta }^{\prime }\left(\widehat{\mathbf{w}}(n+1)\right)=(\beta +1)e^{(-\beta \left|\widehat{\mathbf{w}}(n+1)\right|)}\mathrm{sgn}\left(\widehat{\mathbf{w}}(n+1)\right).}$$

(16)

By left multiplying by ${\mathbf{x}}^T\left(n\right)$ on both sides of Equation (14), we can obtain:

$$\begin{array}{c}{\displaystyle {\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}(n+1)={\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)+\lambda {\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)-{\gamma }_{\mathrm{GZA}}{\mathbf{x}}^T\left(n\right)S_{\beta }^{\prime }\left(\widehat{\mathbf{w}}(n+1)\right).}\hfill \end{array}$$

(17)

From (13), we can get:

$$\begin{array}{c}{\displaystyle \widehat{e}\left(n\right)=\left[1-\xi \exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\right]e\left(n\right).}\hfill \end{array}$$

(18)

Thus, Lagrange multiplier $\lambda$ is:

$$\begin{array}{c}{\displaystyle \lambda =\frac{\xi \exp \left({\displaystyle -\frac{e^2\left(n\right)}{2{\sigma }^2}}\right)e\left(n\right)+{\gamma }_{\mathrm{GZA}}{\mathbf{x}}^T\left(n\right)S_{\beta }^{\prime }\left(\widehat{\mathbf{w}}(n+1)\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}}\hfill \end{array}$$

(19)

By substituting (19) into (14), we can get:

$$\begin{array}{c}\widehat{\mathbf{w}}(n+1)\hfill \\ {\displaystyle =\widehat{\mathbf{w}}\left(n\right)+\frac{{\displaystyle \xi \exp \left({\displaystyle -\frac{e^2\left(n\right)}{2{\sigma }^2}}\right)e\left(n\right)+{\gamma }_{\mathrm{GZA}}{\mathbf{x}}^T\left(n\right)S_{\beta }^{\prime }\left(\widehat{\mathbf{w}}(n+1)\right)}}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}\mathbf{G}\left(n\right)\mathbf{x}(n+1)}\hfill \\ -{\gamma }_{\mathrm{GZA}}{S}_{\beta }^{\prime }\left(\widehat{\mathbf{w}}\left(n\right)\right)\hfill \\ {\displaystyle =\widehat{\mathbf{w}}\left(n\right)+\frac{{\displaystyle \xi e\left(n\right)\exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}}\hfill \\ -{\gamma }_{\mathrm{GZA}}\left\{{\displaystyle \mathbf{I}-\frac{\mathbf{G}\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}}\right\}{S}_{\beta }^{\prime }\left(\widehat{\mathbf{w}}\left(n\right)\right)\hfill \\ {\displaystyle =\widehat{\mathbf{w}}\left(n\right)+\frac{{\displaystyle \xi e\left(n\right)\exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}}\hfill \\ -{\gamma }_{\mathrm{GZA}}\left\{{\displaystyle \mathbf{I}-\frac{\mathbf{G}\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}}\right\}(\beta +1)e^{(-\beta \left|\widehat{\mathbf{w}}\left(n\right)\right|)}\mathrm{sgn}\left(\widehat{\mathbf{w}}\left(n\right)\right)\hfill \end{array}.$$

(20)

From the updating Equation (20), we found that the elements in $\mathbf{G}\left(n\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right){\left\{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)\right\}}^{-1}$ were smaller than one; therefore, they can be ignored [29]. Thereby, we rewrite the updating equation of the developed GZA-PNMCC algorithm as:

$$\begin{array}{cc}\hfill \widehat{\mathbf{w}}(n+1)=& \widehat{\mathbf{w}}\left(n\right)+\frac{{\displaystyle \xi e\left(n\right)\exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}\hfill \\ & -{\gamma }_{\mathrm{GZA}}(\beta +1)e^{(-\beta \left|\widehat{\mathbf{w}}\left(n\right)\right|)}\mathrm{sgn}\left(\widehat{\mathbf{w}}\left(n\right)\right)\hfill \end{array}.$$

(21)

To better control the convergence, a step size $\mu$ is introduced into the updating Equation (21), which is similar to the PNMCC algorithm. Additionally, we also employ a small positive constant to avoid dividing by zero. As a result, the updated equation of the developed GZA-PNMCC algorithm is modified to be:

$$\begin{array}{cc}\hfill \widehat{\mathbf{w}}(n+1)=& \widehat{\mathbf{w}}\left(n\right)+{\chi }_1\frac{{\displaystyle \exp \left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\mathbf{G}\left(n\right)e\left(n\right)\mathbf{x}\left(n\right)}}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{GZA}}}\hfill \\ & -{\rho }_{\mathrm{GZA}}(\beta +1)e^{(-\beta \left|\widehat{\mathbf{w}}\left(n\right)\right|)}\mathrm{sgn}\left(\widehat{\mathbf{w}}\left(n\right)\right)\hfill \end{array},$$

(22)

where ${\chi }_1=\xi \mu$, and ${\rho }_{\mathrm{GZA}}=\mu {\gamma }_{\mathrm{GZA}}$ is a regularization parameter that controls the ability of the zero attraction. The zero attraction term $-{\gamma }_{\mathrm{GZA}}(\beta +1)e^{(-\beta \left|\widehat{\mathbf{w}}\left(n\right)\right|)}\mathrm{sgn}\left(\widehat{\mathbf{w}}\left(n\right)\right)$ in Equation (22) is named as an expected zero attractor. In the developed GZA-PNMCC algorithm, a proper $\beta$ can be chosen to produce different penalties. The GZA zero attraction can give a high probability zero attraction on the zero coefficients or near-zero coefficients. What is more, a gain enhancement matrix is used for further assigning the large step-size to dominant coefficients. In fact, our developed GZA-PNMCC algorithm provides fast convergence for large coefficients by using the gain matrix scheme, while it speeds up the convergence for the small coefficients by using the proposed GZA zero attraction.

4. Behavior of the Proposed GZA-PNMCC Algorithm

We construct two examples to discuss the performance of the developed GZA-PNMCC algorithm within the framework of the sparse system identifications. We compare its performance with the MCC [39,40], NMCC [40], ZA-MCC [41], RZA-MCC [41] and PNLMS [11] algorithms. We use mean square deviation (MSD) to investigate the estimation behavior of the developed GZA-PNMCC algorithm. The definition of the MSD is:

$$\mathrm{MSD}\left(\widehat{\mathbf{w}}\left(n\right)\right)=\mathrm{E}[{∥{\mathbf{w}}_o\left(n\right)-\widehat{\mathbf{w}}\left(n\right)∥}^2].$$

(23)

We find that the parameter ${\chi }_1$ has an important effect on the estimation behavior of the GZA-PNMCC algorithm. Therefore, we create an experiment to give an illustration of the performance of the GZA-PNMCC in the presence of the impulsive noise. We use $(1-\theta )N({\iota }_1,{\nu }_1^2)+\theta N({\iota }_2,{\nu }_2^2)=(0,0.01,0,20,0.05)$ to model the desired impulsive noise, where $N({\iota }_i,{\nu }_i^2)(i=1,2)$ are Gaussian distributions with means of ${\iota }_i$ and variances of ${\nu }_i^2$ [41]. $\theta$ denotes a mixture parameter, which is to control the mixed performance of the two noises. In this paper, we assume $\mathbf{x}\left(n\right)$ is independent with $v\left(n\right)$. Furthermore, we use a sparse system whose length is $N=16$. There is only one dominant coefficient, which is randomly distributed within the impulse response of the system. The parameters $\vartheta =0.01$ and $\sigma =1000$ are used to investigate the effects of ${\chi }_1$, and the computer simulation results are shown in Figure 2. It is noted from Figure 2 that the developed GZA-PNMCC algorithm has a fast convergence when ${\chi }_1$ is large. For smaller ${\chi }_1$, the GZA-PNMCC algorithm converges slower in comparison with larger ${\chi }_1$. However, we can achieve lower estimation misalignment when we choose small ${\chi }_1$.

Furthermore, ${\rho }_{\mathrm{GZA}}$ plays an import role in the GZA-PNMCC algorithm since it controls the zero-attraction ability. We discuss the effects of ${\rho }_{\mathrm{GZA}}$ for estimation of a sparse system, which is the same as that in the ${\chi }_1$ investigation. The performance of ${\rho }_{\mathrm{GZA}}$ is given in Figure 3. It can be seen that ${\rho }_{\mathrm{GZA}}=9\times {10}^{-5}$ provides the fastest convergence speed rate and the best system estimation behavior. Generally speaking, the estimation misalignment becomes smaller when ${\rho }_{\mathrm{GZA}}$ decreases from $9\times {10}^{-4}$ to $9\times {10}^{-5}$. If we continue to reduce ${\rho }_{\mathrm{GZA}}$, the estimation misalignment is changed in the opposite direction, meaning that the estimation misalignment becomes large when ${\rho }_{\mathrm{GZA}}$ decreases from $5\times {10}^{-5}$ to $5\times {10}^{-7}$. Therefore, we should carefully select ${\rho }_{\mathrm{GZA}}$ and ${\chi }_1$ in order to get a good performance and a balance between the convergence speed rate and estimation misalignment.

Based on the key parameter investigations, we create an experiment to discuss the convergence characteristics of the GZA-PNMCC algorithm and compare its convergence with the PNLMS, PNMCC, NMCC, MCC, ZA-MCC and RZA-MCC algorithms. To compare the convergence at the same estimation misalignment level, we set all the simulation parameters as ${\chi }_{\mathrm{MCC}}=0.0052$, ${\chi }_{\mathrm{NMCC}}=0.085$, ${\mu }_{\mathrm{ZA}}=0.01$, ${\mu }_{\mathrm{RZA}}=0.015$, ${\rho }_{\mathrm{ZA}}=3\times {10}^{-5}$, ${\rho }_{\mathrm{RZA}}=7\times {10}^{-5}$, ${\mu }_{\mathrm{PNLMS}}=0.072$, $\chi =0.088$, ${\chi }_1=0.35$, ${\rho }_{\mathrm{GZA}}=8\times {10}^{-5}$, where ${\chi }_{\mathrm{MCC}}$, ${\chi }_{\mathrm{NMCC}}$, ${\mu }_{\mathrm{ZA}}$, ${\mu }_{\mathrm{RZA}}$, ${\mu }_{\mathrm{PNLMS}}$ are the step sizes for the MCC, NMCC, ZA-MCC, RZA-MCC and PNLMS algorithms. ${\rho }_{\mathrm{ZA}}$ and ${\rho }_{\mathrm{RZA}}$ are the zero attraction parameters for ZA-MCC and RZA-MCC algorithms. The convergence of the GZA-PNMCC algorithm is described in Figure 4. Comparing with the previously presented sparse adaptive filtering ZA-MCC, RZA-MCC, PNMCC and PNLMS algorithms, our GZA-PNMCC algorithm achieves the fastest convergence speed rate because it integrates a flexible GZA zero attractor that can accelerate the convergence of zero or near-zero coefficients.

Then, an experiment is set up to investigate the system estimation performance of the GZA-PNMCC algorithm in the context of the sparse system identifications with different effects of the sparsity levels K, where K denotes the number of the dominant coefficients in the impulse response of the unknown sparse system. Assuming that the IR of the unknown sparse system has a length of $N=16$. To achieve the same convergence at the early stage, the simulation parameters are set as ${\chi }_{\mathrm{MCC}}=0.03$, ${\chi }_{\mathrm{NMCC}}=0.4$, ${\mu }_{\mathrm{ZA}}={\mu }_{\mathrm{RZA}}=0.03$, ${\rho }_{\mathrm{ZA}}=8\times {10}^{-5}$, ${\rho }_{\mathrm{RZA}}=2\times {10}^{-4}$, ${\mu }_{\mathrm{PNLMS}}=0.27$, $\chi =0.24$, ${\chi }_1=0.3$, ${\rho }_{\mathrm{GZA}}=9\times {10}^{-5}$. The GZA-PNMCC’s estimation behavior is given in Figure 5, and its performance is compared with those of the MCC, NMCC, ZA-MCC, RZA-MCC and PNLMS algorithms. In this experiment, we choose $K=1,2,4,6$ to discuss the system estimation behavior of the GZA-PNMCC. From Figure 5a, we can see that the GZA-PNMCC algorithm has the fastest convergence and achieves the smallest estimation misalignment. The GZA-PNMCC algorithm provides much more gain than that of the PNMCC algorithm. Comparing to the previously reported ZA-MCC and RZA-MCC, our GZA-PNMCC algorithm has clear advantages with respect to both convergence speed rate and steady-state MSD. As K increase from one to six, it is found that the estimation misalignment becomes larger because of the reduced sparsity. However, the developed GZA-PNMCC is still better than the MCC, NMCC, ZA-MCC, RZA-MCC and PNLMS algorithms in terms of the MSD.

Finally, we set up an experiment to study the tracking behavior of our GZA-PNMCC algorithm for estimating a long-tap echo channel with two different sparsity levels and a length of 256. The sparsity measurement of the echo channel is ${\zeta }_{12}\left({\mathbf{w}}_{\mathbf{o}}\right)=\frac{N}{N-\sqrt{N}}\left(1-\frac{{∥{\mathbf{w}}_{\mathbf{o}}∥}_1}{\sqrt{N}{∥{\mathbf{w}}_{\mathbf{o}}∥}_2}\right)$ [44,45,46,47,48]. A typical echo channel is described in Figure 6. In this experiment, the sparsity is ${\zeta }_{12}\left({\mathbf{w}}_{\mathbf{o}}\right)=0.8222$ for the first 8000 iterations, while it is changed to be ${\zeta }_{12}\left({\mathbf{w}}_{\mathbf{o}}\right)=0.7362$ for the last 8000 iterations. Here, ${\zeta }_{12}\left({\mathbf{w}}_{\mathbf{o}}\right)$ can choose different values to get different channels. For ${\zeta }_{12}\left({\mathbf{w}}_{\mathbf{o}}\right)=0.8222$, the channel is sparser than that of ${\zeta }_{12}\left({\mathbf{w}}_{\mathbf{o}}\right)=0.7362$. The parameters are ${\chi }_{\mathrm{MCC}}=0.0055$, ${\chi }_{\mathrm{NMCC}}=1.3$, ${\mu }_{\mathrm{ZA}}={\mu }_{\mathrm{RZA}}=0.0055$, ${\rho }_{\mathrm{ZA}}=4\times {10}^{-6}$, ${\rho }_{\mathrm{RZA}}=1\times {10}^{-5}$, ${\mu }_{\mathrm{PNLMS}}=1$, $\chi =0.9$, ${\chi }_1=0.8$, ${\rho }_{\mathrm{GZA}}=1\times {10}^{-6}$. The behavior of the GZA-PNMCC algorithm for estimating an echo channel is shown in Figure 7. We note that our GZA-PNMCC algorithm provides the fastest convergence speed rate and smallest MSD for ${\zeta }_{12}\left({\mathbf{w}}_{\mathbf{o}}\right)=0.8222$. Furthermore, it can be seen that there are two convergence stages, which are caused by the proportionate and the zero attraction schemes, respectively. Even when ${\zeta }_{12}\left({\mathbf{w}}_{\mathbf{o}}\right)=0.7362$, the GZA-PNMCC algorithm still outperforms the PNMCC, PNLMS, MCC, NMCC, ZA-MCC and RZA-MCC algorithms. The GZA-PNMCC algorithm is less affected by the sparseness of the unknown system ${\mathbf{w}}_{\mathbf{o}}$, which renders it effective and useful for sparse system identification. In addition, the proposed GZA-PNMCC algorithm has a moderate computational complexity, including ($4N+5$) additions, ($7N+4$) multiplications, (N) divisions and ($N+1$) exponential calculations.

5. Conclusions

A GZA-PNMCC algorithm with zero attraction has been developed for estimating sparse system identifications. The derivation and the analysis of the GZA-PNMCC algorithm is achieved by the LMM. The GZA-PNMCC algorithm uses a zero attraction scheme to improve the convergence at the subsequent stage by quickly forcing the zero system coefficients or near-zero system coefficients to zero. The simulation results demonstrated that the developed GZA-PNMCC algorithm is effective and useful for sparse system identifications because it provides superior performance to that of the PNLMS and PNMCC algorithms. Thereby, our developed GZA-PNMCC is suitable for sparse signal processing in practical engineering applications such as sparse channel estimation and echo cancellation.