## 1. Introduction

In nature, the impulse response of most unknown systems can be regarded as sparse, which consists of only a few dominant coefficients [

1,

2,

3,

4]. The prior known sparse information can be used for improving the estimation performance in signal processing. Thus, sparse signal processing has been garnering significant attention in recent decades [

1,

2,

3,

4,

5]. In particular, the developed sparse signal processing techniques have been used in wireless communications, speech signal processing and imaging processing, which include compressed sensing (CS) [

6,

7,

8] and sparse adaptive filtering [

9,

10,

11,

12,

13,

14,

15,

16,

17,

18,

19,

20,

21,

22,

23,

24,

25,

26,

27,

28,

29,

30]. Although compressed sensing algorithms are useful for dealing with sparse signals, they need to construct measurement matrices with the limitation of restricted isometry property (RIP) [

31]. Thus, the computational complexity might be higher in comparison with sparse adaptive filtering algorithms [

21,

22,

23].

Fueled by the demand of sparse signal processing, sparse adaptive filter algorithms have been widely studied for sparse system identification, echo cancellation and channel estimation [

9,

10,

11,

12,

13,

14,

15,

16,

17,

18,

19,

20,

21,

22,

23,

24,

25,

26,

27,

28,

29,

30]. These proposed sparse adaptive filtering algorithms can be mainly categorized into two groups: proportionate-type adaptive filters [

17,

18,

19,

20] and zero-attracting adaptive filters [

9,

10,

11,

12,

13,

14,

15,

16,

21,

22,

23,

24,

25,

26,

27,

28,

29,

30]. As we know, the proportionate-type adaptive filter algorithms assign proportional step sizes to different coefficients on the basis of their magnitudes, such as proportionate normalized least mean square (PNLMS) [

17,

18,

19], proportionate affine projection algorithms (PAPA) [

32] and their variants [

19]. Other effective sparse adaptive filters have been reported as motivated by the least absolute shrinkage and selection operator (LASSO) [

6] and compressed sensing [

7,

8], which are known as zero attracting adaptive filters. Firstly, an

${l}_{1}$-norm penalty has been incorporated into the cost function of the traditional least mean square (LMS) to form the zero-attracting LMS (ZA-LMS) algorithm [

9]. As a result, the ZA-LMS algorithm can produce an additional term which acts as a zero attractor to make the ZA-LMS algorithm converge quickly. However, the shrinkage of the ZA-LMS algorithm cannot distinguish between the zero coefficients and non-zero coefficients [

9]. Furthermore, the ZA-LMS algorithm gives a penalty to all the coefficients uniformly, and, hence, its performance may be deteriorated for less sparse systems. Then, a reweighted ZA-LMS (RZA-LMS) algorithm is presented by introducing a log-sum function into the cost function of the conventional LMS algorithm [

9]. By using this log-sum function, the zero attractor in the RZA-LMS algorithm can provide a selective shrink according to the magnitudes of the coefficients [

9]. However, these sparse LMS algorithms are sensitive to the scaling of the input training signal because of the drawback of the LMS algorithm. Recently, the proposed zero attracting techniques have been expanded to high-order-moment (HOM) algorithms to reduce the effects of the scaling of input signal, which include least mean fourth (LMF) algorithm [

12,

13,

23,

33], least mean square/fourth (LMS/F) algorithm [

24,

25,

26,

34], least-mean mixture-norm (LMMN) algorithm [

35,

36,

37] and affine projection (AP) algorithms [

14,

15,

16,

38,

39]. However, some of these sparse algorithms have high computational complexity. New sparse adaptive filtering algorithms should provide a balance parameter to give a trade-off between the convergence and steady-state performance. Another effective method to reduce the effect of the scaling of the input training signal is to utilize a normalized factor to form sparse normalized LMS (NLMS) algorithms [

40,

41]. In [

42], the sparse NLMS algorithm has been used for sparse channel estimation to achieve a better performance. In essence, these adaptive filter algorithms estimate the unknown system by minimizing the instantaneous error which is the difference between the desired signal and the output of the filters.

Recently, information theoretic quantities have been used as cost function to construct adaptive systems. An entropy estimator has been proposed to make the computation of the quantities reliable. After that, a great number of adaptive filtering algorithms have been proposed on the basis of the minimum error entropy (MEE) [

43]. By using entropy as a cost function, the signal structure can be well justified. The MEE cost function can provide robustness when dealing with non-Gaussian and impulsive noise signals, but its complexity is higher in comparison with the LMS-like algorithms. High complexity (

$O(NL)$), where

L is the length of a batch of error samples, is not suitable for practical engineering application such as noise cancellation. Then, a maximum correntropy criterion (MCC) algorithm has been proposed by using localized similarity as a cost function [

2,

44,

45]. As a result, the computational complexity (

$O(N)$) of the MCC is comparable to the LMS algorithm, while its robustness is similar to the MEE algorithm in terms of convergence and estimation behavior in impulsive noise environment [

2,

44,

45,

46,

47,

48]. Thus, the MCC is more suitable for practical engineering applications such as noise cancellation in speech data [

2,

44,

45]. However, the MCC algorithm cannot exploit the sparse structure of in-nature systems such as multi-path channels and network echo channels because it indiscriminately handles all the filter coefficients. Inspired by the ZA-LMS and RZA-LMS algorithms, the zero-attracting techniques have been introduced into the traditional MCC algorithm to construct zero-attracting MCC (ZA-MCC) and reweighted ZA-MCC (RZA-MCC) algorithms to exploit the sparsity of the in-nature systems [

49].

In this paper, a soft parameter function penalized normalized maximum correntropy criterion (SPF-NMCC) algorithm is proposed based on the normalized adaptive filtering theory, MCC algorithm and zero attracting techniques. The proposed SPF-NMCC algorithm is realized by normalizing the MCC algorithm to form a normalized MCC (NMCC) algorithm that is similar to the NLMS algorithm. Then, a soft parameter function is included into the cost function of the NMCC algorithm to give a desired zero attractor term in the update equation of the SPF-NMCC algorithm. The proposed SPF-NMCC algorithm is derived in detail based on the Lagrange multiplier method, and its performance is investigated for estimating a sparse system and an echo channel. The results obtained from the computer simulations show that the proposed SPF-NMCC algorithm is better than the traditional MCC, ZA-MCC, and RZA-MCC algorithms in terms of the convergence speed rate and the steady-state performance.

The rest of this paper is structured as follows. In

Section 2, the traditional MCC algorithm and its sparse forms are reviewed and discussed within the system identification framework. In

Section 3, the proposed SPF-NMCC algorithm is described mathematically in the framework of sparse system identification, which is based on the normalized adaptive filtering theory, MCC algorithm and zero attracting theories. In

Section 4, the estimation behaviors of the proposed SPF-NMCC algorithm are well investigated in the context of sparse system identifications. Finally, a conclusion of this work is given in

Section 5.

## 3. Proposed Sparse SPF-NMCC Algorithm

Although the ZA-MCC and RZA-MCC algorithms can well utilize the sparse property of the in-nature systems, they might be affected by scaling of the input signals. As is known to us, normalized adaptive filtering can give resistance to the effects on this scaling, and the zero attracting technique is an effective method to estimate sparse systems. Motivated by the advantages of the normalized adaptive filters and zero attracting techniques, we propose a SPF-NMCC algorithm to further exploit sparseness characteristics for sparse system identification.

Similar to the sparse MCC algorithms, a soft parameter function is used to exploit the sparsity characteristics of these systems, which is defined as

where

$\beta >0$. The performance of the soft parameter function is shown in

Figure 1. It can be seen that the soft parameter function is close to

${l}_{0}$-norm when

β is very large, while it can be regarded as a

${l}_{1}$-norm for small

β. We can choose a large

β to approximately implement the

${l}_{0}$-norm penalty, and we can use small

β to construct

${l}_{1}$-norm penalty to exploit the sparsity of existing sparse systems.

Then, we integrate this soft parameter function into the cost function of an NMCC algorithm to exploit the sparse characteristics of in-nature systems such as echo channels. Thus, the proposed SPF-NMCC algorithm is to find out a solution of the following problem

where

${\gamma}_{\mathrm{SPF}}$ is a regularization parameter that can be tuned to give a tradeoff between sparsity and estimation performance. Then, the cost function of the SPF-NMCC algorithm can be written as

By using Lagrange multiplier method on Equation (

20), we have

and

Then, we have

and

where

and its vector form is

Left multiplying by

${\mathbf{x}}^{T}\left(n\right)$ on the Equation (

23) and moving

$\widehat{\mathbf{w}}(n+1)$ to the left side, we can obtain

Combining with Equations (

24) and (

27),

If we take Equations (

23) and (

28) into consideration and consider

$\widehat{\mathbf{w}}(n+1)=\widehat{\mathbf{w}}(n)$ when the iteration is stable, we can obtain

Discarding the term

${\gamma}_{\mathrm{SPF}}\frac{{\mathbf{x}}^{T}(n){S}_{\beta}^{\prime}(\widehat{\mathbf{w}}(n))\mathbf{x}(n)}{{\u2225\mathbf{x}(n)\u2225}^{2}}$ since it is very small, we can get the update equation of the SPF-NMCC algorithm, which is given by

where

${\mu}_{\mathrm{SPF}}=\xi $ acts as a step-size. It is observed that Equation (

30) represents a unified form of the sparse MCC algorithm. The previously reported sparse MCC can be regarded as special cases of the proposed SPF-NMCC algorithm. If

${\gamma}_{\mathrm{SPF}}\ne 0$ and

$\xi =\chi {\u2225\mathbf{x}(n)\u2225}^{2}$, the proposed SPF-NMCC algorithm is a soft function parameter penalized MCC algorithm. In addition, we can choose different

β to control the SPF-NMCC algorithm. From the derivation of the SPF-NMCC algorithm, we can see that the proposed SPF-NMCC algorithm is a zero attracting algorithm, whose zero attracting ability can be controlled by the regularization parameter

${\gamma}_{\mathrm{SPF}}$. In the update Equation (

30),

$-{\gamma}_{\mathrm{SPF}}{S}_{\beta}^{\prime}(\widehat{\mathbf{w}}(n))$ is a zero attractor, which is used to speed up the convergence of the SPF-NMCC algorithm.

To better understand the optimization problem given in Equation (

19), we present an intuitive geometric explanation of the updated equation in Equation (

30), which is illustrated in

Figure 2. It is found that

$\widehat{\mathbf{w}}(n)$ converges to the hyperplane II that is defined by

$\overline{e}(n)=0$ through paths A1 and A2. Additionally, path A1 is orthogonal to the hyperplane. Path A2 is the zero-attracting path, which forces the zero or near-zero coefficients of

$\widehat{\mathbf{w}}(n)$ in the direction to zero. We can also see that the

$exp\left(-\frac{{e}^{2}(n)}{2{\sigma}^{2}}\right)$ approximates to

$\widehat{\mathbf{w}}(n+1)$ for large

$e(n)$, which helps to reduce the update magnitude to render the proposed algorithm robust in dealing with impulsive noise.

## 4. Performance of the SPF-NMCC Algorithm

In this section, we will investigate the estimation performance of the proposed SPF-NMCC algorithm in the context of sparse system identification. The key parameters of the proposed SPF-NMCC algorithm are discussed and the performance of the SPF-NMCC algorithm is evaluated with different sparsity levels. Furthermore, the convergence of the proposed SPF-NMCC algorithm is given and is compared with LMS, MCC, ZA-MCC and RZA-MCC algorithms. In all the simulations, 500 Monte Carlo runs are used to get each point for all the mentioned adaptive filtering algorithms. Herein, the sparsity level

K is defined as the number of non-zero coefficients and these

K dominant coefficients are randomly distributed within the length of the sparse system. In addition, the FIR system with a length of

$N=16$ and the sparsity levels of

$K=2$,

$K=4$ and

$K=8$ are considered to investigate the performance of the proposed SPF-NMCC algorithm. Since the MCC algorithms perform well in an impulsive noise environment, we employ an impulsive noise

$v(n)$ model, which is given as follows

where

$N({\iota}_{i},{\nu}_{i}^{2})(i=1,2)$ are the Gaussian distributions whose means are

${\iota}_{i}$ and variances are

${\nu}_{i}^{2}$, and

θ is a mixture parameter used for controlling the mixture of the two noises. In addition, we assume the mixed Gaussian noise is independent of

$\mathbf{x}(n)$. In all of the simulations, the mixed impulsive noise

$({\iota}_{i},{\nu}_{i}^{2},{\iota}_{2},{\nu}_{2}^{2},\theta )$ is set to be

$(0,0.05,0,20,0.05)$. The sparse system estimation performance is given by mean square deviation (MSD) whose definition is

Firstly, the effects of parameters

β,

${\mu}_{\mathrm{SPF}}$, and

${\gamma}_{\mathrm{SPF}}$ are investigated in detail. The effects of

β on the proposed SPF-NMCC algorithm are shown in

Figure 3. It is found that the steady-state error floor is reduced with an increment of

β because large

β can well approximate

${l}_{0}$-norm. The steady-state behavior of the proposed SPF-NMCC algorithm with various

${\mu}_{\mathrm{SPF}}$ is demonstrated in

Figure 4. It is noted that the convergence of the proposed SPF-NMCC algorithm becomes faster when

${\mu}_{\mathrm{SPF}}$ increases from 0.2 to 0.9, which is similar to the MCC algorithm. Thus, parameters

${\gamma}_{\mathrm{SPF}}$,

β and

${\mu}_{\mathrm{SPF}}$ of the proposed SPF-NMCC algorithm should be properly selected to achieve a good performance. The effects of the regularization parameter

${\gamma}_{\mathrm{SPF}}$ is investigated to evaluate the steady-state performance of the proposed SPF-NMCC algorithm. In this experiment, the simulation parameters are

$\sigma =1000$,

$\beta =6$ and

$\xi =0.2$. The simulated results of the parameter

${\gamma}_{\mathrm{SPF}}$ are shown in

Figure 5. It can be seen that the regularization parameter

${\gamma}_{\mathrm{SPF}}$ has an important effect on the steady-state performance of the proposed SPF-NMCC algorithm. When

${\gamma}_{\mathrm{SPF}}$ decreases from

$8\times {10}^{-4}$ to

$5\times {10}^{-5}$, the steady-state error floor is reduced. If we continue to decrease the value of

${\gamma}_{\mathrm{SPF}}$, the steady-state error floor is deteriorated and is rebounded to an opposite direction. This is because the large

${\gamma}_{\mathrm{SPF}}$ can provide a strong zero attraction to attract the zero or near zero coefficients to zero quickly, while the small

${\gamma}_{\mathrm{SPF}}$ exerts a weak zero attraction on the zero or near zero coefficients [

26,

28]. Thus, the regularization parameter

${\gamma}_{\mathrm{SPF}}$ can well control the zero attracting ability to provide an excellent performance for sparse system identification.

Secondly, we perform an experiment to investigate the convergence speed of the proposed SPF-NMCC algorithm. The convergence of the SPF-NMCC algorithm is illustrated in

Figure 6 and its convergence is also compared with the existing LMS, MCC, NLMS, NMCC, ZA-LMS, ZA-MCC, RZA-LMS, and RZA-MCC algorithms. Here, the simulation parameters are

${\mu}_{\mathrm{LMS}}=0.005$,

$\chi =0.0052$,

${\mu}_{\mathrm{NLMS}}=0.082$,

${\chi}_{\mathrm{NMCC}}=0.087$,

${\mu}_{\mathrm{ZALMS}}=0.008$,

${\rho}_{\mathrm{ZALMS}}=9\times {10}^{-6}$,

${\xi}_{\mathrm{ZA}}=0.008$,

${\rho}_{\mathrm{ZA}}=3\times {10}^{-5}$,

${\mu}_{\mathrm{RZALMS}}=0.013$,

${\rho}_{\mathrm{RZALMS}}=5\times {10}^{-5}$,

${\xi}_{\mathrm{RZA}}=0.018$,

${\rho}_{\mathrm{RZA}}=7\times {10}^{-5}$,

${\mu}_{\mathrm{SPF}}=0.3$, and

${\gamma}_{\mathrm{SPF}}=2.5\times {10}^{-5}$, which result in nearly the same steady-state error floor. Here,

${\mu}_{\mathrm{LMS}}$,

${\mu}_{\mathrm{NLMS}}$,

${\mu}_{\mathrm{ZALMS}}$ and

${\mu}_{\mathrm{RZALMS}}$ represent the step-sizes of the LMS, NLMS, ZA-LMS and RZA-LMS algorithms, while

${\rho}_{\mathrm{ZALMS}}$ and

${\rho}_{\mathrm{RZALMS}}$ are the regularization parameters of the ZA-LMS and RZA-LMS algorithms. It is observed that our proposed SPF-NMCC algorithm achieves the fastest convergence speed rate. It is worth noting that the NMCC algorithm also converges faster than the MCC algorithm. Another interesting result of this figure is that zero attracting algorithms increase convergence speed, and further increments on it can be obtained by reweighing the coefficients. It is also worth noting that algorithms using MCC converge faster than those using LMS criterion. Furthermore, the proposed SPF scheme can provide a flexible zero attraction to switch the norm penalties between

${l}_{0}$-norm and

${l}_{1}$-norm, resulting in the fastest convergence.

Thirdly, our proposed SPF-NMCC algorithm is analyzed under different sparsity level

K, namely,

$K=2$,

$K=4$ and

$K=8$. In this experiment, the related parameters are set as

${\mu}_{\mathrm{LMS}}={\mu}_{\mathrm{ZALMS}}={\mu}_{\mathrm{RZALMS}}=0.06$,

$\chi =0.05$,

${\mu}_{\mathrm{NLMS}}=0.6$,

${\chi}_{\mathrm{NMCC}}=0.5$,

${\rho}_{\mathrm{ZALMS}}=8\times {10}^{-5}$,

${\xi}_{\mathrm{ZA}}={\xi}_{\mathrm{RZA}}=0.05$,

${\rho}_{\mathrm{ZA}}=5\times {10}^{-5}$,

${\rho}_{\mathrm{RZALMS}}={\rho}_{\mathrm{RZA}}=8\times {10}^{-4}$,

${\mu}_{\mathrm{SPF}}=0.45$, and

${\gamma}_{\mathrm{SPF}}=8\times {10}^{-5}$. The system estimation behaviors of the SPF-NMCC algorithm for

$K=2$,

$K=4$ and

$K=8$ are demonstrated in

Figure 7,

Figure 8 and

Figure 9, respectively. It is found that our proposed SPF-NMCC algorithm has the lowest steady-state error floor for

$K=2$. In addition, the proposed NMCC algorithm with the same initial convergence is better than the MCC algorithm in terms of the steady-state behavior. When

K increases from 2 to 8, the steady-state error floor increases. However, our proposed SPF-NMCC algorithm outperforms the previously presented algorithms with respect to the steady-state behavior. From

Figure 7,

Figure 8 and

Figure 9, the convergence speed of the proposed SPF-NMCC algorithm is a bit larger than that of the other mentioned algorithms, which is to obtain a lower steady-state behavior.

As we know, adaptive filtering algorithms have been widely used for echo cancellation. Therefore, we also construct an experiment to discuss the steady-state behavior of the proposed algorithm over an acoustic echo channel. Here, we define the measurement of the sparseness of the used acoustic echo channel as

${\zeta}_{12}({\mathbf{w}}_{\mathbf{o}})=\frac{N}{N-\sqrt{N}}\left(1-\frac{{\u2225{\mathbf{w}}_{\mathbf{o}}\u2225}_{1}}{\sqrt{N}{\u2225{\mathbf{w}}_{\mathbf{o}}\u2225}_{2}}\right)$.

Figure 10 gives a typical acoustic echo channel whose length is 256, and there are 16 dominant coefficients. In order to better understand the effects of the sparsity,

${\zeta}_{12}({\mathbf{w}}_{\mathbf{o}})=0.8222$ and

${\zeta}_{12}({\mathbf{w}}_{\mathbf{o}})=0.7362$ are employed to evaluate the tracking behavior of the proposed SPF-NMCC algorithm. The parameters are

$\chi =0.0045$,

${\mu}_{\mathrm{NLMS}}=1$,

${\chi}_{\mathrm{NMCC}}=0.9$,

${\mu}_{\mathrm{ZALMS}}={\mu}_{\mathrm{RZALMS}}=0.005$,

${\rho}_{\mathrm{ZALMS}}={\rho}_{\mathrm{RZALMS}}=2\times {10}^{-5}$,

${\xi}_{\mathrm{ZA}}=0.0035$,

${\rho}_{\mathrm{ZA}}={\rho}_{\mathrm{RZA}}=5\times {10}^{-6}$,

${\xi}_{\mathrm{RZA}}=0.003$,

${\mu}_{\mathrm{SPF}}=0.9$, and

${\gamma}_{\mathrm{SPF}}=3\times {10}^{-6}$. The simulation results for tracking the acoustic echo channel are shown in

Figure 11. It can be seen that our proposed SPF-NMCC algorithm outperforms the conventional LMS and MCC algorithms and their related sparse forms in terms of both the convergence speed rate and steady-state behavior. Even for a reduced sparsity of

${\zeta}_{12}({\mathbf{w}}_{\mathbf{o}})=0.7362$, our proposed SPF-NMCC algorithm still has the best steady-state performance. Thus, we can say that the proposed SPF-NMCC algorithm shows little dependence on the sparsity of the unknown system

${\mathbf{w}}_{\mathbf{o}}$ in comparison with the mentioned previously presented adaptive filtering algorithms. Thus, the proposed SPF-NMCC algorithm is more effective for dealing with sparse system identification.

At last, we give the complexity of the SPF-NMCC algorithm and compare it with the mentioned algorithms above in

Table 1. We use the additions, multiplications, divisions and exponential calculation to give a comparison of the computational complexity. The comparisons of the related computational complexity are given in

Table 1. We can see that our SPF-NMCC algorithm has (

$N+4K+1$) additions, (

$N+5K$) multiplications, (

N) divisions and (

$N+1$) exponential calculations, which provides a moderate computational complexity for sparse system identification.

On the basis of the discussions and analysis aforementioned, we can give a short summary about our proposed SPF-NMCC algorithm for handling sparse system identification. It is found that our proposed SPF-NMCC algorithm has the fastest convergence speed rate for achieving the same steady-state error floor. Moreover, it can also provide the smallest system estimation misalignment in comparison with the LMS and MCC algorithm and their related zero attracting forms. This is because the proposed SPF-NMCC algorithm employs the SPF function to form a desired zero attractor to exploit the sparseness of the systems, and it uses the normalization to give a resistance to the scaling of the input. In addition, the zero attracting abilities of the proposed SPF-NMCC algorithm can be controlled by ${\gamma}_{\mathrm{SPF}}$. Our SPF-NMCC algorithm introduced two extra parameters to exploit the sparsity of the traditional MCC algorithm by introducing a zero attractor. Although two extra parameters have been added, which slightly increases the computational complexity, the performance of the SPF-NMCC algorithm is better than the MCC and NMCC algorithms. Even compared with the previously reported ZA-MCC and RZA-MCC algorithms, the SPF-NMCC algorithm has the fastest convergence and lowest estimation error in terms of MSD. However, the proposed SPF-NMCC algorithm has a cost of selecting the proper hyper-parameter values in practical engineering applications because it requires more time and data to select the parameters β and ${\rho}_{\mathrm{SPF}}$ in comparison with the MCC algorithm. Compared with the ZA-MCC and RZA-MCC algorithms, the proposed SPF-NMCC algorithm needs to find a proper β.