A Proportionate Normalized Maximum Correntropy Criterion Algorithm with Correntropy Induced Metric Constraint for Identifying Sparse Systems

Abstract

A proportionate-type normalized maximum correntropy criterion (PNMCC) with a correntropy induced metric (CIM) zero attraction terms is presented, whose performance is also discussed for identifying sparse systems. The proposed sparse algorithms utilize the advantage of proportionate schemed adaptive filter, maximum correntropy criterion (MCC) algorithm, and zero attraction theory. The CIM scheme is incorporated into the basic MCC to further utilize the sparsity of inherent sparse systems, resulting in the name of the CIM-PNMCC algorithm. The derivation of the CIM-PNMCC is given. The proposed algorithms are used for evaluating the sparse systems in a non-Gaussian environment and the simulation results show that the expanded normalized maximum correntropy criterion (NMCC) adaptive filter algorithms achieve better performance than those of the squared proportionate algorithms such as proportionate normalized least mean square (PNLMS) algorithm. The proposed algorithm can be used for estimating finite impulse response (FIR) systems with symmetric impulse response to prevent the phase distortion in communication system.

1. Introduction

Sparse adaptive filtering (AF) algorithms are becoming a boosted topic in recent decades and they are discussed and used for sparse system identifications, multi-path channel estimations, echo cancelation and underwater acoustic communications [1,2,3,4,5,6]. Additionally, the AF has been used for improving the electrocardiogram (ECG) detection during magnetic resonance imaging (MRI) [7]. Furthermore, most of the in-nature signals are sparse or can be regarded as sparse such as underwater channels, a multi-path channel in complex terrain environments, echoes in a high-definition television channel, in which only a few coefficients are dominant, while most of the coefficients are unimportant [8,9,10,11]. In this case, the insignificant coefficients are usually inactive, which are usually zeros or near zeros in a system. For such sparse signal processing, traditional least mean square (LMS), and normalized LMS (NLMS) might perform poorly by considering only the convergence, and steady-state error (SSE) since these conventional adaptive filters did not use the prior sparse information of the sparse systems. Thus, the demand for sparse signal processing has been becoming a hot topic. The fast convergence algorithms and sparse AF are desired in the sparse finite impulse response (FIR) environments, resulting in the rapid development of proportionate-type (Pt) adaptive filters, including the PNLMS algorithm [2]. The PNLMS algorithm exploits the sparsity characteristics in the sparse systems via proportionally assigning different gains to the magnitudes of the estimated coefficients [2,3]. By using this technique, the convergence speed at the initial stage can be improved. However, the PNLMS’s convergence speed may be slower than the traditional NLMS algorithm for estimating long sparse signals because the small coefficients are assigned small gains, which increases the convergence time for achieving a steady-state mean square error (MSE) [3]. Another drawback of the typical PNLMS algorithm is that the convergence speed rate may be slower in comparison with NLMS algorithm when the sparse systems are less sparse or dispersive. In the sequel, several improved PNLMS algorithms have been proposed to enhance its performance [3,4,12,13,14]. These PNLMS algorithms with low complexity were realized by using the squared error criterion to get optimal for processing Gaussian data. All of these PNLMS algorithms mainly update their favored adaptive filtering coefficients, which render these PNLMS algorithms suitable for sparse signal processing [12,13,14,15]. Moreover, the PNLMS algorithm is also an NLMS algorithm if all the coefficients are assigned the same gains. However, the PNLMS algorithm may perform worse when the data are non-Gaussian noises such as the low frequency atmospheric noise [16]. The PNLMS algorithm and its variants just take the second-order statistics into consideration to construct the cost function [2,3,4], which is not enough to capture all the information from the data. Therefore, the performance of these PNLMS algorithms will be poor if they are used in the non-Gaussian environments.

With the on-going AF algorithms, a minimum error entropy (MEE) was reported within information theoretic learning framework for handling the non-Gaussian signals [16,17,18,19,20,21,22,23]. In the information theoretic learning, a quadratic Renyi’s entropy has been used for data classification, channel modeling and data fusion [24,25,26] and it is used to provide an alternative method instead of MSE [16]. As a result, the entropy can access the samples well by using a non-parametric Parzen window method. Then, the MEE adaptive filters were used for identifying systems, which can obtain a better estimating performance in non-Gaussian environments [23]. This is because the entropy employs a high-order statistic and utilizes the contents of the information instead of the power of the signal power. After that, the proportionate MEE (PNMEE) algorithm is proposed to utilize the sparse property in nature systems [16]. However, the PMEE algorithm has a high computational burden, which makes it not good for practical engineering applications. To reduce the computational burden, the maximum correntropy criterion (MCC) was developed using a similarity function to act as a novel cost function [27]. Finally, the MCC has a comparable complexity compared with the basic LMS algorithm, while it can provide a robustness-like performance as that of the MEE algorithm. Thereby, the MCC algorithm is a proper selective method for practical engineering applications in non-Gaussian environments. However, the MCC does not use the sparse-structure information in the existing sparse system. Although a Pt technique was integrated into the normalized MCC (NMCC) to form a proportionate NMCC (PNMCC) algorithm to handle the sparse signals, the PNMCC’s convergence becomes slow after the early iterations and the PNMCC cannot fully use the sparse information of the natural sparse signals.

An improved proportionate normalized MCC (PNMCC) and a correntropy induced metric (CIM) penalized PNMCC (CIM-PNMCC) are considered for fully taking use of the inherent sparsity characteristics in the sparse systems. The developed PNMCC algorithm is realized by incorporating the proportionate-type technique, the normalization method and a generalized Gaussian density function (GGDF) into the MCC algorithm, while the CIM-PNMCC algorithm is implemented by means of the CIM theory which is integrated into the newly presented PNMCC algorithm to form an amazing zero attractor. The proposed PNMCC and the CIM-PMCC algorithms are mathematically given in detail and they are used for estimating sparse systems. These two algorithms perform better for identifying the sparse systems in non-Gaussian noises compared with the PNLMS algorithm.

The paper is well constructed as follows. We review the MCC and zero attraction methods in Section 2. In Section 3, the PNMCC and CIM-PNMCC algorithms will be proposed. The performance for sparse system identification of the developed algorithms is discussed in Section 4. Finally, we give a short summary of this paper.

Notations:

• $∥·∥$  $l_2$-norm
• ${(·)}^T$  Transpose operation for a matrix or a vector
• $\mathbf{x}$ with bold front  Vector or Matrix

2. Review of the MCC and Zero-Attraction (ZA) Technique

2.1. Conventional MCC

We consider a sparse system identification within the MCC framework shown in Figure 1. From Figure 1, we can see that the MCC-based adaptive filtering is to mimic the entropy error $e(n)$ that is a difference between output $y(n)$ and desired signal $d(n)$ under non-Gaussian noise $v(n)$. As we know, the entropy is to quantify the uncertainty with designated random variables, and, hence, the minimized errors are concentrated. Herein, we discuss the MCC algorithm based on a linear system given in Figure 1. From the system identification theory, $d(n)$ is depicted by:

$$d(n)={\mathbf{x}}^T(n){\mathbf{w}}_o(n)+v(n),$$

(1)

where $\mathbf{x}(n)={[x(n),x(n-1),\cdots ,x(n-N-1)]}^T$ represents an input vector, ${\mathbf{w}}_{\mathbf{o}}(n)={[w_0,w_1,\cdots ,w_{N-1}]}^T$ denotes an unknown system which is an FIR channel, ${}^T$ is the transpose operation, $v(n)$ denotes noise or interference signal which is non-Gaussian. Here, the memory size of the channel is N. In the AF framework, the estimated output is $y(n)={\mathbf{x}}^T(n)\tilde{\mathbf{w}}(n)$, and, hence, $e(n)$ is given by:

$$e(n)=d(n)-y(n).$$

(2)

$\tilde{\mathbf{w}}(n)$ represents the estimated system. The MCC algorithm is to calculate the equation [23,27]:

$$\begin{array}{c}min\frac{1}{2}{∥\tilde{\mathbf{w}}(n+1)-\tilde{\mathbf{w}}(n)∥}^2,\hfill \\ {\displaystyle \mathrm{subject}\mathrm{to}\widehat{e}(n)=\left[1-\xi exp\left(-\frac{e^2(n)}{2{\sigma }^2}\right)\right]e(n).}\hfill \end{array}$$

(3)

Herein, $\widehat{e}(n)=d(n)-{\mathbf{x}}^T(n)\tilde{\mathbf{w}}(n+1)$, $∥·∥$ represents $l_2$ norm, and we use $\xi ={\chi }_{\mathrm{MCC}}{∥\mathbf{x}(n)∥}^2$.

To get the solution of Equation (3), the Lagrange multiplier (LM) method is used in this paper. Therefore, the MCC’s cost function is [27]:

$$J_{\mathrm{MCC}}(n)=\frac{1}{2}{∥\tilde{\mathbf{w}}(n+1)-\tilde{\mathbf{w}}(n)∥}^2+\lambda \left(\widehat{e}(n)-\left[1-\xi exp\left(-\frac{e^2(n)}{2{\sigma }^2}\right)\right]e(n)\right),$$

(4)

where $\lambda$ denotes Lagrange multiplier. Then, we have:

$$\frac{\partial J_{\mathrm{MCC}}(n)}{\partial \tilde{\mathbf{w}}(n+1)}=\mathbf{0},$$

and

$$\frac{\partial J_{\mathrm{MCC}}(n)}{\partial \lambda }=\mathbf{0}.$$

Thus, we can get the following formula:

$$\tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}(n)+\lambda \mathbf{x}(n).$$

(5)

Here, $\lambda$ is obtained:

$${\displaystyle \lambda =\xi \frac{{\displaystyle exp\left(-\frac{e^2(n)}{2{\sigma }^2}\right)e(n)}}{{∥\mathbf{x}(n)∥}^2}.}$$

(6)

We substitute Equation (6) into Equation (5), the updating of the MCC is [23,27]:

$$\tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}(n)+{\chi }_{\mathrm{MCC}}exp\left(-\frac{e^2(n)}{2{\sigma }^2}\right)e(n)\mathbf{x}(n).$$

(7)

Compared to the basic LMS, an exponential weighting is considered and used in MCC, which is for eliminating the large errors. The MCC algorithm is robust for dealing with impulsive noises for system identifications.

2.2. Zero Attracting Technique

From the update equation in Equation (7), it is found that the MCC system identification method does not consider the sparse-structure information in the sparse system ${\mathbf{w}}_o(n)$. Furthermore, the MCC-based system identification method can be written as [23,27]

$$\tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}(n)+\mathrm{Adaption}\mathrm{error}\mathrm{updates}.$$

(8)

In recent years, the zero attracting methods [10,27,28,29,30,31,32,33,34,35,36,37,38,39,40] have been widely studied to use the sparsity information in the FIR channels. These zero attracting algorithms are realized by integrating norm penalties, such as $l_1$-norm and reweighted $l_1$-norm [10], into the traditional AF algorithms. Thus, the zero attracting sparse MCC-based adaptive filters can be summarized as [10,27,28,29,30,31,32,33,34,35,36,37,38,39,40]

$$\tilde{\mathbf{w}}(n+1)=\underset{\mathrm{Zero}-\mathrm{attracting}\mathrm{MCC}\mathrm{algorithms}}{\underbrace{\stackrel{\mathrm{MCC}\mathrm{algorithm}}{\overbrace{\tilde{\mathbf{w}}(n)+\mathrm{Adaptive}\mathrm{error}-\mathrm{update}}}+\mathrm{Sparsity}\mathrm{constraint}\mathrm{term}}}.$$

(9)

In Equation (9), the sparsity penalty term is usually given by introducing different norm penalties to utilize the prior sparse inform of these sparse systems [10,27,28,29,30,31,32,33,34,35,36,37,38,39,40]. Thus, the $l_1$- and reweighted $l_1$-norms are considered for exploiting the sparse MCCs to form ZA MCC (ZA-MCC) and reweighting ZA-MCC (RZA-MCC) algorithms [41]. Although the sparse MCCs achieve better estimation performance compared with the traditional MCC algorithm for identifying sparse systems, they are still needed to be improved for practical applications. Thus, we will develop a sparse NMCC algorithm by using the normalization method, proportionate-type method and the zero attracting techniques.

3. Proposed Proportionate NMCC Algorithms

3.1. Proportionate NMCC (PNMCC) Algorithm

In order to get the PNMCC, we reinvestigate the MCC in the AF framework again. Here, the NMCC is also rewritten in Equation (3). The differences between the MCC and NMCC algorithms are that we use $\xi =\chi$ in NMCC algorithm to replace the $\xi ={\chi }_{\mathrm{MCC}}{∥\mathbf{x}(n)∥}^2$ in the MCC. Similarly, the NMCC is to implement the following updating equation:

$${\displaystyle \tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}(n)+\chi \frac{{\displaystyle exp\left(-\frac{e^2(n)}{2{\sigma }^2}\right)}}{{∥\mathbf{x}(n)∥}^2}e(n)\mathbf{x}(n).}$$

(10)

Then, we use a reweighting controlling matrix $\mathbf{G}(n)$ for reassigning different weights to the magnitudes of the filter coefficients, where the idea is obtained from the known Pt algorithms. Introducing $\mathbf{G}(n)$ into (10), we get

$${\displaystyle \tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}(n)+\chi \frac{{\displaystyle \mathbf{G}(n)exp\left(-\frac{e^2(n)}{2{\sigma }^2}\right)}}{{\mathbf{x}}^T(n)\mathbf{G}(n)\mathbf{x}(n)+\vartheta }e(n)\mathbf{x}(n).}$$

(11)

Here, $\vartheta >0$ represents a constant to get a stable solution. We know that a diagonal matrix $\mathbf{G}(n)$ is used to give a modification of NMCC’s step size under a rule given as follows. In general, $\mathbf{G}(n)$ is [2,3,4]

$$\mathbf{G}(n)=\mathrm{diag}(g_0(n),g_1(n),\cdots ,g_{N-1}(n)),$$

(12)

where the individual gain $g_i(n)$ is

$${\displaystyle g_i(n)=\frac{{\kappa }_i(n)}{{\displaystyle \sum _{i=0}^{N-1}}{\kappa }_i(n)},0\le i\le N-1}$$

(13)

with

$${\kappa }_i(n)=max[{\gamma }_{g}max[{\rho }_p,\left|{\tilde{w}}_0(n)\right|,\left|{\tilde{w}}_1(n)\right|,\cdots ,\left|{\tilde{w}}_{N-1}(n)\right|],\left|{\tilde{w}}_i(n)\right|].$$

(14)

Here, ${\rho }_p>0$ and ${\gamma }_g>0$ are constants, and they are usually set to be ${\rho }_p=0.01$ and ${\gamma }_g=5/N$ [2]. ${\rho }_p$ is implemented to avoid the stalling. The parameter ${\gamma }_g$ is used for preventing ${\tilde{w}}_i(n)$ from stalling.

3.2. Proportionate NMCC with a CIM

Here, we first introduce the CIM theory [41,42,43,44,45,46,47,48,49,50]. As we know, the correntropy method is achieved from nonlinear measurement of random similar vectors $\mathbf{g}={\left[\begin{array}{cccc}{g}_1& g_2& \cdots & g_N\end{array}\right]}^T$ and $\mathbf{f}={\left[\begin{array}{cccc}{f}_1& f_2& \cdots & f_N\end{array}\right]}^T$, which are in kernel space. Thus, we have

$$L(\mathbf{g},\mathbf{f})=\mathbf{E}\left[\kappa (\mathbf{g},\mathbf{f})\right]=\int \kappa (\mathbf{g},\mathbf{f})d{F}_{gf}(g,f),$$

(15)

where $\kappa (\mathbf{g},\mathbf{f})$ represents the shift-invariant Mercer kernel, and $F_{gf}(g,f)$ represents the joint distribution of $\mathbf{g}$ and $\mathbf{f}$, and $\mathbf{E}\left[\mathbf{x}\right]$ is the expectation operation of $\mathbf{x}$. In general, the distribution of the data is usually unknown. Moreover, only a few samples of $\{g_i,f_i\}$ are available in practice. Thus, the correntropy can be obtained by [41,42,43,44,45,46,47,48,49,50]

$$\overline{L}(\mathbf{g},\mathbf{f})=\frac{1}{N}\sum _{i=1}^N\kappa (g_i,f_i).$$

(16)

According to the correntropy theory, the Gaussian kernel is a kind of typical kernel, which is given by [41,42,43,44,45,46,47,48,49,50]

$$\kappa (g,f)=\frac{1}{{\sigma }_1\sqrt{2\pi }}exp\left(-\frac{e^2}{2{\sigma }_1^2}\right),$$

(17)

where $e=g-f$, and ${\sigma }_1$ is the width of a kernel. The used correntropy has a boundary under various distributions and it can give robustness characteristics for non-Gaussian noise, which has been used for exploiting MCC algorithms.

Next, we give the CIM to better understand the correntropy theory. By considering $\mathbf{g}$ and $\mathbf{f}$ in a sampling space, we can define the CIM as [41,42,43,44,45,46,47,48,49,50]

$$\mathrm{CIM}(\mathbf{g},\mathbf{f})=\sqrt{(\kappa (0)-\overline{L}(\mathbf{g},\mathbf{f}))},$$

(18)

where $\kappa (0)=\frac{1}{{\sigma }_1\sqrt{2\pi }}$. Thus, we can get

$${\mathrm{CIM}}^2(\mathbf{g},0)=\frac{1}{{\sigma }_{1}N\sqrt{2\pi }}\sum _{i=1}^N\left(1-exp\left(-\frac{a_i^2}{2{\sigma }_1^2}\right)\right).$$

(19)

It can be seen that, if $\left|g_i\right|>\varsigma ,\forall g_i\ne 0$, then, as ${\sigma }_1=0$, we found that the CIM is an $l_0$-norm approximation, and $\varsigma >0$ is set to be a very small constant.

Finally, we propose a CIM penalized PNMCC (CIM-PNMCC) algorithm, which is realized by introducing CIM penalty into PNMCC’s cost function to construct a sparse PNMSS with a implementation of the CIM. We also take the gain controlling matrix into consideration to design the desired zero-attractor. According to the zero attracting sparse AF algorithms, the developed CIM-PNMCC is to calculate

$$\begin{array}{c}{(\tilde{\mathbf{w}}(n+1)-\tilde{\mathbf{w}}(n))}^T{\mathbf{G}}^{-1}(n)(\tilde{\mathbf{w}}(n+1)-\tilde{\mathbf{w}}(n))+{\gamma }_{\mathrm{CIM}}{\mathbf{G}}^{-1}(n){\mathrm{CIM}}^2(\tilde{\mathbf{w}}(n+1),0)\hfill \\ \mathrm{subject}\mathrm{to}\widehat{e}(n)=\left[{\displaystyle 1-\xi exp\left(-\frac{e^2(n)}{2{\sigma }^2}\right)}\right]e(n),\hfill \end{array}$$

(20)

where ${\mathbf{G}}^{-1}(n)$ denotes the inverse operation of $\mathbf{G}(n)$. ${\gamma }_{\mathrm{CIM}}>0$ is a regularization, which is a balance between the system identification errors and CIM penalty of $\tilde{\mathbf{w}}(n+1)$. Our devised CIM-PNMCC is different from the general zero attracting since it provides a scaling by ${\mathbf{G}}^{-1}(n)$ on the CIM penalty.

To get the minimization of (20), the LM method is employed. Therefore, the proposed CIM-PNMCC’s cost function is

$$\begin{array}{c}J(n+1)={(\tilde{\mathbf{w}}(n+1)-\tilde{\mathbf{w}}(n))}^T{\mathbf{G}}^{-1}(n)(\tilde{\mathbf{w}}(n+1)-\tilde{\mathbf{w}}(n))+{\gamma }_{\mathrm{CIM}}{\mathbf{G}}^{-1}(n){\mathrm{CIM}}^2(\tilde{\mathbf{w}}(n+1),0)\hfill \\ {\displaystyle +\lambda \left(\widehat{e}(n)-\left[1-\xi exp\left(-\frac{e^2(n)}{2{\sigma }^2}\right)\right]e(n)\right).}\hfill \end{array}$$

(21)

The gradients on $J(n+1)$ in Equation (20) by considering $\tilde{\mathbf{w}}(n+1)$ and $\lambda$ are given by

$$\frac{\partial J(n+1)}{\partial \tilde{\mathbf{w}}(n+1)}=0\mathrm{and}\frac{\partial J(n+1)}{\partial \lambda }=0$$

(22)

and

$$\tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}(n)+\lambda \mathbf{G}(n)\mathbf{x}(n)-{\gamma }_{\mathrm{CIM}}\frac{1}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}(n)exp\left(-\frac{{∥\tilde{\mathbf{w}}(n)∥}^2}{2{\sigma }_1^2}\right).$$

(23)

Multiplying ${\mathbf{x}}^T(n)$ on both sides of Equation (23), we get

$$\begin{array}{c}{\mathbf{x}}^T(n)\tilde{\mathbf{w}}(n+1)=\hfill \\ {\displaystyle {\mathbf{x}}^T(n)\tilde{\mathbf{w}}(n)+\lambda {\mathbf{x}}^T(n)\mathbf{G}(n)\mathbf{x}(n)-{\gamma }_{\mathrm{CIM}}{\mathbf{x}}^T(n)\frac{1}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}(n)exp\left(-\frac{{∥\tilde{\mathbf{w}}(n)∥}^2}{2{\sigma }_1^2}\right).}\hfill \end{array}$$

(24)

Considering Equation (22), we obtain

$$\begin{array}{c}{\displaystyle \widehat{e}\left(n\right)=\left[1-\xi exp\left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\right]e\left(n\right).}\hfill \end{array}$$

(25)

The Lagrange multiplier is obtained

$$\begin{array}{c}{\displaystyle \lambda =\frac{{\displaystyle \xi exp\left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)e\left(n\right)+\frac{{\gamma }_{\mathrm{CIM}}{\mathbf{x}}^T\left(n\right)}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}\left(n\right)exp\left(-\frac{{∥\tilde{\mathbf{w}}\left(n\right)∥}^2}{2{\sigma }_1^2}\right)}}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}.}\hfill \end{array}$$

(26)

Considering Equations (26) and (23), we get

$$\begin{array}{c}{\displaystyle \tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}(n)+\frac{{\displaystyle \xi exp\left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)e\left(n\right)+\frac{{\gamma }_{\mathrm{CIM}}{\mathbf{x}}^T\left(n\right)}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}\left(n\right)exp\left(-\frac{{∥\tilde{\mathbf{w}}\left(n\right)∥}^2}{2{\sigma }_1^2}\right)}}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)}\mathbf{G}(n)\mathbf{x}(n)}\hfill \\ {\displaystyle -{\displaystyle {\gamma }_{\mathrm{CIM}}}\frac{1}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}(n)exp\left(-\frac{{∥\tilde{\mathbf{w}}(n)∥}^2}{2{\sigma }_1^2}\right)}\hfill \\ {\displaystyle =\tilde{\mathbf{w}}(n)+\frac{{\displaystyle \xi e(n)exp\left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\mathbf{G}(n)\mathbf{x}(n)}}{{\mathbf{x}}^T(n)\mathbf{G}(n)\mathbf{x}(n)}}\hfill \\ {\displaystyle -{\gamma }_{\mathrm{CIM}}\left\{\mathbf{I}-\frac{\mathbf{G}(n)\mathbf{x}(n){\mathbf{x}}^T(n)}{{\mathbf{x}}^T(n)\mathbf{G}(n)\mathbf{x}(n)}\right\}\frac{1}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}(n)exp\left(-\frac{{∥\tilde{\mathbf{w}}(n)∥}^2}{2{\sigma }_1^2}\right).}\hfill \end{array}$$

(27)

From the updated Equation (27), we can see that the elements in $\mathbf{G}(n)\mathbf{x}(n){\mathbf{x}}^T(n){\left\{{\mathbf{x}}^T(n)\mathbf{G}(n)\mathbf{x}(n)\right\}}^{-1}$ are very small in comparison with 1. Thus, we can ignore this term, and the proposed CIM-PNMCC’s updating equation is

$${\displaystyle \tilde{\mathbf{w}}(n+1)=\tilde{\mathbf{w}}(n)+\frac{{\displaystyle \xi e(n)exp\left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\mathbf{G}(n)\mathbf{x}(n)}}{{\mathbf{x}}^T(n)\mathbf{G}(n)\mathbf{x}(n)}-{\gamma }_{\mathrm{CIM}}\frac{1}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}(n)exp\left(-\frac{{∥\tilde{\mathbf{w}}(n)∥}^2}{2{\sigma }_1^2}\right).}$$

(28)

We introduce a step-size $\beta$ and ${\epsilon }_{\mathrm{CIM}}={\delta }_x^2/N$. Then, the proposed CIM-PNMCC’s updating equation is changed to be

$$\begin{array}{c}\tilde{\mathbf{w}}(n+1)=\hfill \\ {\displaystyle \tilde{\mathbf{w}}(n)+{\chi }_1\frac{{\displaystyle exp\left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)\mathbf{G}(n)e(n)\mathbf{x}(n)}}{{\mathbf{x}}^T(n)\mathbf{G}(n)\mathbf{x}(n)+{\epsilon }_{\mathrm{CIM}}}-{\rho }_{\mathrm{CIM}}\frac{1}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}(n)exp\left(-\frac{{∥\tilde{\mathbf{w}}(n)∥}^2}{2{\sigma }_1^2}\right),}\hfill \end{array}$$

(29)

where ${\chi }_1=\xi \beta$, and ${\rho }_{\mathrm{CIM}}=\beta {\gamma }_{\mathrm{CIM}}$ is a regularization parameter used for controlling the zero attracting strength. Similar to the early reported zero attracting algorithms, there is an additional term ${\displaystyle -{\rho }_{\mathrm{CIM}}\frac{1}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}(n)exp\left(-\frac{{∥\tilde{\mathbf{w}}(n)∥}^2}{2{\sigma }_1^2}\right)}$ in the CIM-PNMCC algorithm, which is called a CIM zero-attractor. In the devised CIM-PNMCC, the constructed CIM zero-attractor attracts small coefficients to zero with a high probability. Furthermore, our proposed CIM-PNMCC algorithm employs a gain controlling matrix to assign a large step-size for the dominant coefficients, while the zero attractor mainly gives a strong zero-attraction to the inactive coefficients. Therefore, our CIM-PNMCC algorithm has the advantages of the proportionate-type algorithms, normalized adaptive filtering algorithms and zero attraction adaptive filters.

From the derivation of the CIM-PNMCC, we can put the remarks as follows:

• A PNMCC algorithm is devised by using a generalized Gaussian distribution function to utilize the prior-sparse-structure information in the in-nature systems.
• A CIM constraint is adopted and incorporated into the proposed PNMCC’s cost function to create a modified cost function.
• The derivation of the devised CIM-PNMCC algorithm is presented by the use of the LM method to further take the advantages of the prior-sparse-structure information.
• The convergence of the CIM-PNMCC is analyzed and its performance is discussed for identifying sparse systems, which is compared with the previous MCC algorithms.
• Our developed CIM-PNMCC outperforms the previous MCC algorithms in terms of the convergence and MSD.

4. Convergence Analysis of the Devised CIM-PNMCC

The mean and convergence for the CIM-PNMCC is performed based on an approximation approach. For simple description, our proposed CIM-PNMCC is rewritten as

$$\tilde{\mathbf{w}}\left(n+1\right)=\tilde{\mathbf{w}}\left(n\right)+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)e\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}m\left(n\right),$$

(30)

where $f\left(e\left(n\right)\right)=exp\left(-\frac{e^2\left(n\right)}{2{\sigma }^2}\right)$ and $m\left(n\right)=-\frac{1}{N{\sigma }_1^3\sqrt{2\pi }}\tilde{\mathbf{w}}\left(n\right)exp\left(-\frac{{∥\tilde{\mathbf{w}}\left(n\right)∥}^2}{2{\sigma }_1^2}\right)$. For simple convergence analysis, we give some assumptions that are used frequently in [51,52,53,54].

We have the following assumptions (A): A1: $\mathbf{x}\left(n\right)$ with zero mean is i. i. d.

A2: $v\left(n\right)$ is independent with $\mathbf{x}\left(n\right)$, and it is zero mean, and its variance is ${\sigma }_v^2$.

A3: $f\left(e\left(n\right)\right)$ is independent with $\mathbf{x}\left(n\right)$.

A4: $\tilde{\mathbf{w}}\left(n\right)$ and $m\left(n\right)$ are independent with $\mathbf{x}\left(n\right)$.

A5: The expectation of a ration of two random variables is equal to the ration of the expectation of each random variable. Moreover, the expectation is existing, and $\mathrm{E}\left[{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}\right]=N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}$, and $\mathrm{E}\left[{\left({\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}\right)}^2\right]={\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2$ [55].

4.1. Mean Convergence

A misalignment vector is defined by

$$\widehat{\mathbf{w}}\left(n\right)=\tilde{\mathbf{w}}\left(n\right)-{\mathbf{w}}_0\left(n\right).$$

(31)

Then, combining Equations (1), (2) and (30), we obtain

$$\begin{array}{c}{\displaystyle \widehat{\mathbf{w}}\left(n+1\right)=\widehat{\mathbf{w}}\left(n\right)+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)e\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}m\left(n\right)}\hfill \\ {\displaystyle =\widehat{\mathbf{w}}\left(n\right)+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}\left({\mathbf{x}}^T\left(n\right){\mathbf{w}}_0\left(n\right)+v\left(n\right)-{\mathbf{x}}^T\left(n\right)\tilde{\mathbf{w}}\left(n\right)\right)\mathbf{x}\left(n\right)+{\rho }_{\mathrm{CIM}}m\left(n\right)}\hfill \\ {\displaystyle =\widehat{\mathbf{w}}\left(n\right)+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)v\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}-\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\widehat{\mathbf{w}}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}m\left(n\right)}\hfill \\ {\displaystyle =\left[{\displaystyle \mathbf{I}-\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}}\right]\widehat{\mathbf{w}}\left(n\right)+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)v\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}m\left(n\right).}\hfill \end{array}$$

(32)

Taking the expectation on Equation (32) and using the A1, A2, A3 and A5, we get

$$\begin{array}{c}\mathrm{E}\left[\widehat{\mathbf{w}}\left(n+1\right)\right]=\left[{\displaystyle I-\frac{{\chi }_1\mathbf{G}\left(n\right)\mathrm{E}\left[f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]}{\mathrm{E}\left[{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}\right]}}\right]\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right]+{\rho }_{\mathrm{CIM}}\mathrm{E}\left[m\left(n\right)\right]\hfill \\ {\displaystyle =\left[I-\frac{{\chi }_1\mathbf{G}\left(n\right)\mathbf{R}}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}\right]\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right]+{\rho }_{\mathrm{CIM}}\mathrm{E}\left[m\left(n\right)\right],}\hfill \end{array}$$

(33)

where $\mathbf{R}=\mathrm{E}\left[f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)\right]$ represents the auto-correlation matrix. Based on the symmetric and positive semidefinite property of the $\mathbf{R}$, we have

$$\mathbf{R}=\mathbf{Q}\mathsf{\Lambda }{\mathbf{Q}}^T,$$

(34)

where $\mathsf{\Lambda }=\mathrm{diag}\left({\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_k\right)$ is the eigenvalue matrix. Next, premultiplying ${\mathbf{Q}}^T$ on both sides of Equation (33), and considering $\mathbf{Q}$ unitarians, we get

$${\widehat{\mathbf{w}}}^{\prime }\left(n+1\right)=\left[\mathbf{I}-\frac{{\chi }_1\mathbf{G}\left(n\right)\mathsf{\Lambda }}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}\right]{\widehat{\mathbf{w}}}^{\prime }\left(n\right)+{\rho }_{\mathrm{CIM}}{\mathbf{Q}}^T\mathrm{E}\left[m\left(n\right)\right],$$

(35)

where ${\widehat{\mathbf{w}}}^{\prime }\left(n\right)={\mathbf{Q}}^T\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right]$, and $\left[\mathbf{I}-\frac{{\chi }_1\mathbf{G}\left(n\right)\mathsf{\Lambda }}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}\right]$ represents a diagonal. $\mathrm{E}\left[m\left(n\right)\right]$ is bounded [41] and every element of ${\widehat{\mathbf{w}}}^{\prime }\left(n+1\right)$ is independent. Hence, ${\widehat{\mathbf{w}}}^{\prime }\left(n+1\right)$ converges to zero if and only if $\left|1-{\chi }_1{g}_k{\lambda }_k\right|<1$. As a result, our CIM-PNMCC algorithm converges only if it converges for the ${\lambda }_{max}$. Since ${\lambda }_{max}\le \mathrm{tr}\left[\mathbf{R}\right]$, where $\mathrm{tr}\left[·\right]$ is matrix trace operation, $\widehat{\mathbf{w}}\left(n\right)$ converges to zero when ${\chi }_1$ meets

$$0<{\chi }_1<\frac{2}{\mathrm{tr}\left[\frac{\mathbf{G}\left(n\right)\mathbf{R}}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}\right]}.$$

(36)

Using A1 and A3, we have $\mathbf{R}=\mathrm{E}\left[f\left(e\left(n\right)\right)\right]{\sigma }_x^2\mathbf{I}$. Moreover, considering $\mathrm{tr}\left[\mathbf{AB}\right]\le \mathrm{tr}\left[\mathbf{A}\right]\mathrm{tr}\left[\mathbf{B}\right]$, and $\mathrm{tr}\left[\mathbf{G}\left(n\right)\right]=N$, we obtain

$$0<{\chi }_1<\frac{2tr\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}{N{\sigma }_x^2\mathrm{E}\left[f\left(e\left(n\right)\right)\right]}.$$

(37)

4.2. Mean Square Convergence (MSC)

As for the MSC of our CIM-PNMCC, we give the auto-covariance of $\widehat{\mathbf{w}}\left(n\right)$

$$\mathrm{S}\left(n\right)=\mathrm{E}\left[z\left(n\right)z^T\left(n\right)\right],$$

(38)

where

$$z\left(n\right)=\widehat{\mathbf{w}}\left(n\right)-\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right].$$

(39)

Then, considering Equations (39), (32) and (33), we get

$$\begin{array}{c}{\displaystyle z\left(n+1\right)=\left[{\displaystyle \mathbf{I}-\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}}\right]\widehat{\mathbf{w}}\left(n\right)+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)v\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}m\left(n\right)}\hfill \\ {\displaystyle -\left[I-\frac{{\chi }_1\mathbf{G}\left(n\right)\mathbf{R}}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}\right]\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right]-{\rho }_{\mathrm{CIM}}\mathrm{E}\left[m\left(n\right)\right]}\hfill \\ {\displaystyle =\left[{\displaystyle \mathbf{I}-\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}}\right]z\left(n\right)+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)v\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}m\left(n\right)}\hfill \\ -{\rho }_{\mathrm{CIM}}\mathrm{E}\left[m\left(n\right)\right]+{\chi }_1\left[{\displaystyle \frac{\mathbf{G}\left(n\right)\mathbf{R}}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}-\frac{\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}}\right]\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right]\hfill \\ {\displaystyle =\mathsf{\Phi }\left(n\right)z\left(n\right)+{\chi }_1\mathrm{Y}\left(n\right)\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right]+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)v\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}\mathrm{B}\left(n\right),}\hfill \end{array}$$

(40)

where

$$\begin{array}{c}\mathsf{\Phi }\left(n\right)=\left[{\displaystyle \mathbf{I}-\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}}\right],\hfill \\ \mathrm{Y}\left(n\right)=\left[{\displaystyle \frac{\mathbf{G}\left(n\right)\mathbf{R}}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}-\frac{\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)\mathbf{x}\left(n\right){\mathbf{x}}^T\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}}\right],\hfill \\ \mathrm{B}\left(n\right)=m\left(n\right)-\mathrm{E}\left[m\left(n\right)\right].\hfill \end{array}$$

(41)

Under the assumptions 1–5, we can say that $\mathrm{Y}\left(n\right)$ and $\mathrm{B}\left(n\right)$ are both zero means. In addition, we know that $\widehat{\mathbf{w}}\left(n\right)$, $\mathbf{x}\left(n\right)$ and $v\left(n\right)$ are independent with each other. Putting Equation (40) into Equation (38), we have

$$\begin{array}{c}\mathrm{S}\left(n+1\right)=\mathrm{E}\left[\begin{array}{c}\left({\displaystyle \mathsf{\Phi }\left(n\right)z\left(n\right)+{\chi }_1\mathrm{Y}\left(n\right)E\left[\widehat{\mathbf{w}}\left(n\right)\right]+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)v\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}\mathrm{B}\left(n\right)}\right)\times \hfill \\ {\left({\displaystyle \mathsf{\Phi }\left(n\right)z\left(n\right)+{\chi }_1\mathrm{Y}\left(n\right)\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right]+\frac{{\chi }_1\mathbf{G}\left(n\right)f\left(e\left(n\right)\right)v\left(n\right)\mathbf{x}\left(n\right)}{{\mathbf{x}}^T\left(n\right)\mathbf{G}\left(n\right)\mathbf{x}\left(n\right)+{\epsilon }_{\mathrm{CIM}}}+{\rho }_{\mathrm{CIM}}\mathrm{B}\left(n\right)}\right)}^T\hfill \end{array}\right]\hfill \\ =\mathrm{E}\left[\mathsf{\Phi }\left(n\right)z\left(n\right)z^T\left(n\right){\mathsf{\Phi }}^T\left(n\right)\right]+{\rho }_{\mathrm{CIM}}\mathrm{E}\left[\mathsf{\Phi }\left(n\right)z\left(n\right){\mathrm{B}}^T\left(n\right)\right]\hfill \\ {\displaystyle +{\chi }_1^2\mathrm{E}\left[\mathrm{Y}\left(n\right)\phi \left(n\right){\phi }^T\left(n\right){\mathrm{Y}}^T\left(n\right)\right]+\frac{{\chi }_1{\mathbf{G}}^2\left(n\right)\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^2{\sigma }_v^2}{{\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2}+{\rho }_{\mathrm{CIM}}^2\mathrm{E}\left[\mathrm{B}\left(n\right){\mathrm{B}}^T\left(n\right)\right]}\hfill \\ +{\rho }_{\mathrm{CIM}}\mathrm{E}\left[\mathrm{B}\left(n\right)z^T\left(n\right){\mathsf{\Phi }}^T\left(n\right)\right],\hfill \end{array}$$

(42)

where $\phi \left(n\right)=\mathrm{E}\left[\widehat{\mathbf{w}}\left(n\right)\right]$. Since the Gaussian variable with fourth-order moment is three times the variable square [52], and we also know that $\mathrm{S}\left(n\right)$ is symmetric [52], we obtain

$$\begin{array}{c}\mathrm{E}\left[\mathsf{\Phi }\left(n\right)z\left(n\right)z^T\left(n\right){\mathsf{\Phi }}^T\left(n\right)\right]=\left[{\displaystyle 1-\frac{2{\chi }_1\mathbf{G}\left(n\right)\mathrm{E}\left[f\left(e\left(n\right)\right)\right]{\sigma }_x^2}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}+\frac{2{\chi }_1^2{\mathbf{G}}^2\left(n\right)\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^4}{{\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2}}\right]\mathrm{S}\left(n\right)\hfill \\ {\displaystyle +\frac{{\chi }_1^2{\mathbf{G}}^2\left(n\right)\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^4}{{\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2}tr\left[\mathrm{S}\left(n\right)\right]}\hfill \\ {\displaystyle \mathrm{E}\left[\mathrm{Y}\left(n\right)\phi \left(n\right){\phi }^T\left(n\right){\mathrm{Y}}^T\left(n\right)\right]=\frac{{\mathbf{G}}^2\left(n\right)\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^4}{{\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2}\left[\phi \left(n\right){\phi }^T\left(n\right)+tr\left[\phi \left(n\right){\phi }^T\left(n\right)\right]I_N\right].}\hfill \end{array}$$

(43)

Moreover, using Equations (31) and (39) and $\mathrm{B}\left(n\right)$, we have

$$\mathrm{E}\left[\mathsf{\Phi }\left(n\right)z\left(n\right){\mathrm{B}}^T\left(n\right)\right]=\mathrm{E}\left[\mathrm{B}\left(n\right)z^T\left(n\right){\mathsf{\Phi }}^T\left(n\right)\right]=\left[1-\frac{{\chi }_1\mathbf{G}\left(n\right)\mathrm{E}\left[f\left(e\left(n\right)\right)\right]{\sigma }_x^2}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}\right]\mathrm{E}\left[\tilde{\mathbf{w}}\left(n\right){\mathrm{B}}^T\left(n\right)\right].$$

(44)

According to Equations (42)–(44), we have

$$\begin{array}{c}tr\left[\mathrm{S}\left(n+1\right)\right]=\left[1-\frac{2{\chi }_{1}tr\left[\mathbf{G}\left(n\right)\right]\mathrm{E}\left[f\left(e\left(n\right)\right)\right]{\sigma }_x^2}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}+\left(N+2\right)\frac{{\chi }_1^{2}tr\left[{\mathbf{G}}^2\left(n\right)\right]\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^4}{{\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2}\right]tr\left[\mathrm{S}\left(n\right)\right]\hfill \\ +2{\rho }_{\mathrm{CIM}}\left[1-\frac{{\chi }_{1}tr\left[\mathbf{G}\left(n\right)\right]\mathrm{E}\left[f\left(e\left(n\right)\right)\right]{\sigma }_x^2}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}+\right]\mathrm{E}\left[\tilde{\mathbf{w}}\left(n\right){\mathrm{B}}^T\left(n\right)\right]\hfill \\ +{\chi }_1^2\left(N+1\right)\frac{tr\left[{\mathbf{G}}^2\left(n\right)\right]\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^4}{{\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2}\phi \left(n\right){\phi }^T\left(n\right)\hfill \\ +\frac{N{\chi }_1^2{\mathbf{G}}^2\left(n\right)\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^2{\sigma }_v^2}{{\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2}+{\rho }_{\mathrm{CIM}}^2\mathrm{E}\left[\mathrm{B}\left(n\right){\mathrm{B}}^T\left(n\right)\right].\hfill \end{array}$$

(45)

In Equation (45), $\phi \left(n\right)$, $\mathrm{E}\left[\tilde{\mathbf{w}}\left(n\right)\right]$, and $\mathrm{B}\left(n\right)$ are all bounded. Thus, $\mathrm{E}\left[\tilde{\mathbf{w}}\left(n\right){\mathrm{B}}^T\left(n\right)\right]$ is converged. Then, the CIM-PNMCC is stable when

$$\left|1-\frac{2{\chi }_{1}tr\left[\mathbf{G}\left(n\right)\right]\mathrm{E}\left[f\left(e\left(n\right)\right)\right]{\sigma }_x^2}{N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}}+\left(N+2\right)\frac{{\chi }_1^{2}tr\left[{\mathbf{G}}^2\left(n\right)\right]\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^4}{{\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)}^2}\right|<1.$$

(46)

By solving the Equation (46), we can get the range of ${\chi }_1$

$$0<{\chi }_1<\frac{2\left(N{\sigma }_x^2+{\epsilon }_{\mathrm{CIM}}\right)tr\left[\mathbf{G}\left(n\right)\right]\mathrm{E}\left[f\left(e\left(n\right)\right)\right]}{\left(N+2\right)tr\left[{\mathbf{G}}^2\left(n\right)\right]\mathrm{E}\left[f^2\left(e\left(n\right)\right)\right]{\sigma }_x^2}.$$

(47)

5. Results and Discussions of the PNMCC and CIM-PNMCC Algorithms

We will discuss the behavior of our proposed PNMCC and CIM-PNMCC for identifying sparse systems. Furthermore, their system identification performance is analyzed by comparing with NMCC, MCC, RZA-MCC and ZA-MCC algorithms. Additionally, the PNLMS algorithm is also employed for estimation performance comparison since the MCC is an LMS-like algorithm and the PNLMS is a proportionate-typed NLMS, which is also a PNMCC-like algorithm. The MSD is employed for evaluating the behaviors of the devised PNMCC and CIM-PNMCC algorithms, defining by

$$\mathrm{MSD}(\tilde{\mathbf{w}}(n))=\mathrm{E}\left[{∥{\mathbf{w}}_o(n)-\tilde{\mathbf{w}}(n)∥}^2\right].$$

(48)

Based on the derivation of the PNMCC and CIM-PNMCC algorithms, $\chi$ gives an important effect on the PNMCC algorithm, while ${\chi }_1$ and ${\rho }_{\mathrm{CIM}}$ play important effects on the CIM-PNMCC algorithm. Thereby, we first investigate their effects on the estimation behaviors over sparse systems with impulsive noise. In this paper, $(1-\theta )N({\iota }_1,{\nu }_1^2)+\theta N({\iota }_2,{\nu }_2^2)=(0,0.01,0,20,0.05)$ is used for generating the desired noise, where $N({\iota }_i,{\nu }_i^2)(i=1,2)$ are the Gaussian distributions with means of ${\iota }_i$ and variances of ${\nu }_i^2$, where $\theta$ is a mixture parameter. Herein, a sparse system with a length of $N=16$ is adopted. One non-zero tap randomly distributes in the system. To investigate the effects of $\chi$ on the PNMCC algorithm, the simulation parameters are $\vartheta =0.01$ and $\sigma =1000$ with corresponding results presented in Figure 2. The developed PNMCC converges faster when $\chi$ increases from $\chi =0.3$ to $\chi =1$, which has a similar function in the PNLMS algorithm to implement the step-size.

Similarly, the parameter ${\chi }_1$ is also selected to understand the performance of the derived CIM-PNMCC, whose performance is described in Figure 3. The CIM-PNMCC’s convergence speed rate becomes faster when ${\chi }_1$ increases from 0.2 to 0.7. However, the estimation misalignment is getting worse with an increment of ${\chi }_1$. Additionally, ${\rho }_{\mathrm{CIM}}$ is a zero attraction controlling parameter, which is designed to trade off the sparsity development and estimation bias. The effects of ${\rho }_{\mathrm{CIM}}$ on the CIM-PNMCC estimation behavior is illustrated in Figure 4. It is observed that the estimation bias of the CIM-PNMCC is reduced as ${\rho }_{\mathrm{CIM}}$ ranges from $9\times {10}^{-4}$ to $5\times {10}^{-5}$. If ${\rho }_{\mathrm{CIM}}$ is reduced from $5\times {10}^{-5}$ to $5\times {10}^{-7}$, the estimation bias rebounds towards the opposite direction. Therefore, we should choose proper parameter value to obtain good performance of the PNMCC and CIM-PNMCC algorithms.

Then, these parameters are used to obtain the convergence speed investigation of the developed PNMCC and CIM-PNMCC algorithms. Here, ${\chi }_{\mathrm{MCC}}=0.0052$, ${\chi }_{\mathrm{NMCC}}=0.085$, ${\mathsf{\mu }}_{\mathrm{ZA}}=0.01$, ${\mathsf{\mu }}_{\mathrm{RZA}}=0.015$, ${\rho }_{\mathrm{ZA}}=3\times {10}^{-5}$, ${\rho }_{\mathrm{RZA}}=7\times {10}^{-5}$, ${\mathsf{\mu }}_{\mathrm{PNLMS}}=0.072$, $\chi =0.088$, ${\chi }_1=0.4$, ${\rho }_{\mathrm{CIM}}=2\times {10}^{-5}$, where ${\chi }_{\mathrm{NMCC}}$, ${\chi }_{\mathrm{MCC}}$, ${\mathsf{\mu }}_{\mathrm{RZA}}$, ${\mathsf{\mu }}_{\mathrm{ZA}}$, ${\mathsf{\mu }}_{\mathrm{PNLMS}}$ are step-sizes for NMCC, MCC, RZA-MCC, ZA-MCC and PNLMS algorithms. ${\rho }_{\mathrm{RZA}}$ and ${\rho }_{\mathrm{ZA}}$ are the regularization parameters for RZA-MCC and ZA-MCC, respectively. The convergence comparisons for PNMCC and CIM-PNMCC algorithms are given in Figure 5. From Figure 5, we observe that the PNMCC converges faster than the PNLMS and NMCC algorithms. It is worth noting that our developed CIM-PNMCC gets the fastest convergence speed for adaptive system identifications. The CIM-PNMCC algorithm only needs 200 iterations to reach a steady-state MSD, while the previously reported RZA-MCC algorithm requires more than 400 iterations to get its steady-state MSD. Thereby, the CIM-PNMCC algorithm converges much quicker than the RZA-MCC algorithm for achieving the same MSD.

Next, the effects of the sparsity level are analyzed by using the non-zero coefficients K in the FIR system. Here, we still use a 16 length sparse system that has K dominant coefficients. The simulated parameters for the mentioned adaptive filtering algorithms are ${\chi }_{\mathrm{MCC}}=0.03$, ${\chi }_{\mathrm{NMCC}}=0.4$, ${\mathsf{\mu }}_{\mathrm{ZA}}={\mathsf{\mu }}_{\mathrm{RZA}}=0.03$, ${\rho }_{\mathrm{ZA}}=8\times {10}^{-5}$, ${\rho }_{\mathrm{RZA}}=2\times {10}^{-4}$, ${\mathsf{\mu }}_{\mathrm{PNLMS}}=0.27$, $\chi =0.24$, ${\chi }_1=0.3$, ${\rho }_{\mathrm{CIM}}=5\times {10}^{-5}$. The estimation behavior of our developed PNMCC and CIM-PNMCC algorithms is discussed for various K, and their estimation behaviors are given in Figure 6, Figure 7, Figure 8 and Figure 9, respectively. It is noted that our developed PNMCC achieves a little gain compared with the PNLMS because they have comparable complexity and similar updating equation except the exponential weighting. However, our PNMCC algorithm achieves lower steady-state bias for $K=1$. As for the CIM-PNMCC, it provides the lowest steady-state misalignment because it has a zero attraction term that can quickly force inactive coefficients to zero. With an increase of K, the MSD floor is increased. However, our developed CIM-PNMCC still provides the lowest steady-state misalignment, indicating that the CIM-PNMCC is more useful. Even for $K=8$, our CIM-PNMCC algorithm can still get an MSD level of ${10}^{-4}$, which is a very low estimation bias. As we know, the ZA-MCC is a regular zero attraction algorithm. However, with the increasing of K, we can see that the performance of the ZA-MCC has degraded because of its uniform zero attraction on all the channel coefficients.

Then, we construct an example to discuss the tracking behavior of our devised PNMCC and CIM-PNMCC algorithms. Herein, we investigate their tracking performance over network echo channels with different sparsities, and the channel length is set to be 256. We use ${\zeta }_{12}({\mathbf{w}}_{\mathbf{o}})=\frac{N}{N-\sqrt{N}}\left(1-\frac{{∥{\mathbf{w}}_{\mathbf{o}}∥}_1}{\sqrt{N}{∥{\mathbf{w}}_{\mathbf{o}}∥}_2}\right)$ to measure the sparsity of the designated network echo channel [37]. A typical network echo channel is given in Figure 10. The simulation parameters used in this experiment are ${\chi }_{\mathrm{MCC}}=0.0055$, ${\chi }_{\mathrm{NMCC}}=1.3$, ${\mathsf{\mu }}_{\mathrm{ZA}}={\mathsf{\mu }}_{\mathrm{RZA}}=0.0055$, ${\rho }_{\mathrm{ZA}}=4\times {10}^{-6}$, ${\rho }_{\mathrm{RZA}}=1\times {10}^{-5}$, ${\mathsf{\mu }}_{\mathrm{PNLMS}}=1$, $\chi =0.9$, ${\chi }_1=0.8$, ${\rho }_{\mathrm{CIM}}=6\times {10}^{-8}$. The tracking behavior for ${\zeta }_{12}({\mathbf{w}}_{\mathit{o}})=0.8222$ and ${\zeta }_{12}({\mathbf{w}}_{\mathbf{o}})=0.7362$ is illustrated in Figure 10. Our PNMCC algorithm outperforms the PNLMS, MCC and NMCC algorithms in terms of the estimation bias for ${\zeta }_{12}({\mathbf{w}}_{\mathbf{o}})=0.8222$. However, our PNMCC algorithm achieves the similar tracking behavior to the PNLMS algorithm because of the weighted step-size assignment scheme. For our developed CIM-PNMCC, it possesses the fastest convergence speed and lowest MSD for both ${\zeta }_{12}({\mathbf{w}}_{\mathbf{o}})=0.8222$ and ${\zeta }_{12}({\mathbf{w}}_{\mathbf{o}})=0.7362$. Therefore, a conclusion is given that our developed PNMCC and CIM-PNMCC algorithms can be used to effectively handle the sparse system identification. Furthermore, the CIM-PNMCC algorithm has little effect on the sparsity of ${\mathbf{w}}_{\mathbf{o}}$ compared with the mentioned algorithms above.

Finally, a practical speech is considered as the input signal to test the performance of the devised CIM-PNMCC. In this experiment, the used speech signal is a 2 s real speech, which has a sampling ratio of 8 kHz. The parameter used herein is the same as the last experiment. The speech signal, as well as the results, are shown in Figure 11 and Figure 12, respectively. The proposed CIM-PNMCC can still get the best convergence speed and lower estimation error for echo cancellation applications.

According to the aforementioned discussion of the proposed PNMCC and CIM-PNMCC algorithms, the CIM-PNMCC achieves the smallest estimation misalignment and quickest convergence compared with the mentioned adaptive system identification algorithms. For the proposed PNMCC algorithm, it has a similar complexity to the PNLMS, while it can provide a little better gain than the PNLMS in impulsive noise environments. Moreover, the CIM-PNMCC performs better than the PNMCC since it introduces a CIM zero-attractor to use the sparsity characteristics in the used systems. This is attributed to the CIM measurement which can account for the dominant non-zero coefficients. In addition, the CIM zero attractor forces the inactive coefficients to zero rapidly. Additionally, the developed CIM-PNMCC algorithm has an additional regularization parameter ${\rho }_{\mathrm{CIM}}$ that is used for controlling the zero attraction ability.

6. Conclusions

PNMCC and CIM-PNMCC algorithms were developed, and their behaviors were presented and discussed for sparse system identification. The CIM-PNMCC algorithm can provide the fastest convergence, while the PNMCC also performs better than the PNLMS algorithm. Since the CIM-PNMCC algorithm integrates a CIM zero attraction in its iteration, its convergence is faster and its estimation bias is smaller than the presented PNMCC algorithm. The obtained results showed that the CIM-PNMCC is stable, and it can obtain more gains. Therefore, the proposed CIM-PNMCC is a good candidate to consider the practical applications for sparse system identifications. For the future work, we will develop the block sparse PNMCC algorithm based on the block method in [56,57] and reduce the complexity of the proposed CIM-PNMCC algorithm.