# A Proportionate Normalized Maximum Correntropy Criterion Algorithm with Correntropy Induced Metric Constraint for Identifying Sparse Systems

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- $\u2225\xb7\u2225$ ${l}_{2}$-norm
- ${(\xb7)}^{T}$ Transpose operation for a matrix or a vector
- $\mathbf{x}$ with bold front Vector or Matrix

## 2. Review of the MCC and Zero-Attraction (ZA) Technique

#### 2.1. Conventional MCC

#### 2.2. Zero Attracting Technique

## 3. Proposed Proportionate NMCC Algorithms

#### 3.1. Proportionate NMCC (PNMCC) Algorithm

#### 3.2. Proportionate NMCC with a CIM

- A PNMCC algorithm is devised by using a generalized Gaussian distribution function to utilize the prior-sparse-structure information in the in-nature systems.
- A CIM constraint is adopted and incorporated into the proposed PNMCC’s cost function to create a modified cost function.
- The derivation of the devised CIM-PNMCC algorithm is presented by the use of the LM method to further take the advantages of the prior-sparse-structure information.
- The convergence of the CIM-PNMCC is analyzed and its performance is discussed for identifying sparse systems, which is compared with the previous MCC algorithms.
- Our developed CIM-PNMCC outperforms the previous MCC algorithms in terms of the convergence and MSD.

## 4. Convergence Analysis of the Devised CIM-PNMCC

#### 4.1. Mean Convergence

#### 4.2. Mean Square Convergence (MSC)

## 5. Results and Discussions of the PNMCC and CIM-PNMCC Algorithms

## 6. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Naylor, P.A.; Cui, J.; Brookes, M. Adaptive algorithms for sparse echo cancellation. Signal Process.
**2009**, 86, 1182–1192. [Google Scholar] [CrossRef] - Duttweiler, D.L. Proportionate normalized least-mean-squares adaptation in echo cancelers. IEEE Trans. Speech Audio Peocess.
**2000**, 8, 508–518. [Google Scholar] [CrossRef] - Li, Y.; Hamamura, M. An improved proportionate normalized least-mean-square algorithm for broadband multipath channel estimation. Sci. World J.
**2014**, 2014, 572969. [Google Scholar] [CrossRef] [PubMed] - Benesty, J.; Gay, S.L. An improved PNLMS algorithm. In Proceedings of the 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP’02, Orlando, FL, USA, 13–17 May 2002; Volume II, pp. 1881–1884. [Google Scholar]
- Hunt, K.J.; Sbarbaro, D. Adaptive filtering and neural networks for realisation of internal model control. Intell. Syst. Eng.
**1993**, 2, 67–76. [Google Scholar] [CrossRef] - Kumar, A.M.P.; Ramesha, K. Adaptive filter algorithms based noise cancellation using neural network in mobile applications. In Proceedings of the International Conference on Intelligent Computing and Applications, Canberra, Australia, 20–23 January 2017; pp. 67–68. [Google Scholar]
- Kakareka, J.W.; Faranesh, A.Z.; Pursley, R.H.; Campbell-Washburn, A.; Herzka, D.A.; Rogers, T.; Kanter, J.; Ratnayaka, K.; Lederman, R.J.; Pohida, T.J. Physiological recording in the MRI environment (PRiME): MRI-compatible hemodynamic recording system. IEEE J. Transl. Eng. Health Med.
**2018**, 2018, 4100112. [Google Scholar] [CrossRef] [PubMed] - Gui, G.; Adachi, F. Improved least mean square algorithm with application to adaptive sparse channel estimation. EURASIP J. Wirel. Commun. Netw.
**2013**, 2013, 204. [Google Scholar] [CrossRef] [Green Version] - Li, Y.; Zhang, C.; Wang, S. Low complexity non-uniform penalized affine projection algorithm for sparse system identification. Circuit Syst. Signal Process.
**2016**, 35, 1611–1624. [Google Scholar] [CrossRef] - Chen, Y.; Gu, Y.; Hero, A.O. Sparse LMS for system identification. In Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’09, Taipei, Taiwan, 19–24 April 2009; pp. 3125–3128. [Google Scholar]
- Cotter, S.F.; Rao, B.D. Sparse channel estimation via matching pursuit with application to equalization. IEEE Trans. Commun.
**2002**, 50, 374–377. [Google Scholar] [CrossRef] - Deng, H.; Doroslovacki, M. Improving convergence of the PNLMS algorithm for sparse impulse response identification. IEEE Signal Process. Lett.
**2005**, 12, 181–184. [Google Scholar] [CrossRef] - Das, R.L.; Chakraborty, M. A zero attracting proportionate normalized least mean square algorithm. In Proceedings of the 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC’12, Hollywood, CA, USA, 3–6 December 2012; pp. 1–4. [Google Scholar]
- Deng, H.; Doroslovacki, M. Proportionate adaptive algorithms for network echo cancellation. IEEE Trans. Signal Process.
**2006**, 54, 1794–1803. [Google Scholar] [CrossRef] - Paleolohu, C.; Ciochina, S.; Benesty, J. An efficient proportionate affine projection algorithm for echo cancellation. IEEE Signal Process. Lett.
**2010**, 17, 165–168. [Google Scholar] [CrossRef] - Wu, Z.; Peng, S.; Chen, B.; Zhao, H.; Principe, J.C. Proportionate minimum error entropy algorithm for sparse system identification. Entropy
**2015**, 17, 5995–6006. [Google Scholar] [CrossRef] - Chen, B.; Zhu, Y.; Hu, J.; Principe, J.C. System Parameter Identification: Information Criteria and Algorithms; Elsevier: Amsterdam, The Netherlands, 2013. [Google Scholar]
- Wolsztynski, E.; Thierry, E.; Pronzato, L. Minimum-entropy estimation in semi-parametric models. Signal Process.
**2005**, 85, 937–949. [Google Scholar] [CrossRef] [Green Version] - Chen, B.; Principe, J.C. Some further results on the minimum error entropy estimation. Entropy
**2012**, 14, 966–977. [Google Scholar] [CrossRef] - Chen, B.; Principe, J.C. On the smoothed minimum error entropy Criterion. Entropy
**2012**, 14, 2311–2323. [Google Scholar] [CrossRef] - Xue, Y.; Zhu, X. The minimum error entropy based robust wireless channel tracking in impulsive noise. IEEE Commun. Lett.
**2002**, 6, 228–230. [Google Scholar] - Liu, W.F.; Pokharel, P.P.; Principe, J.C. Correntropy: Properties and applications in nonGaussian signal processing. IEEE Trans. Signal Process.
**2007**, 55, 5286–5298. [Google Scholar] [CrossRef] - Singh, A.; Principe, J.C. Using correntropy as a cost function in linear adaptive filters. In Proceedings of the 2009 International Joint Conference on Neural Networks, IJCNN’09, Atlanta, GA, USA, 14–19 June 2009; pp. 2950–2955. [Google Scholar]
- Palmieri, F.A.N.; Ciuonzo, D. Objective priors from maximum entropy in data classification. Inf. Fusion
**2013**, 14, 186–198. [Google Scholar] [CrossRef] [Green Version] - Debbah, M.; Muller, R.R. MIMO channel modeling and the principle of maximum entropy. IEEE Trans. Inf. Theory
**2005**, 51, 1667–1690. [Google Scholar] [CrossRef] - Palmieri, F.; Ciuonzo, D. Data fusion with entropic priors. In Proceedings of the 20th Italian Workshop on Neural Nets, Vietri sul Mare, Salerno, Italy, 27–29 May 2011; Volume 226, pp. 107–114. [Google Scholar]
- Li, Y.; Wang, Y.; Yang, R.; Albu, F. A soft parameter function penalized normalized maximum correntropy criterion algorithm for sparse system identification. Entropy
**2017**, 19, 45. [Google Scholar] [CrossRef] - Gu, Y.; Jin, J.; Mei, S. L
_{0}norm constraint LMS algorithms for sparse system identification. IEEE Signal Process. Lett.**2009**, 16, 774–777. [Google Scholar] - Taheri, O.; Vorobyov, S.A. Sparse channel estimation with L
_{p}-norm and reweighted L_{1}-norm penalized least mean squares. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, Prague, Czech Republic, 22–27 May 2011; pp. 2864–2867. [Google Scholar] - Li, Y.; Wang, Y.; Jiang, T. Sparse channel estimation based on a p-norm-like constrained least mean fourth algorithm. In Proceedings of the 2015 International Conference on Wireless Communications & Signal Processing (WCSP) WCSP’15, Nanjing, China, 15–17 October 2015. [Google Scholar]
- Li, Y.; Hamamura, M. Zero-attracting variable-step-size least mean square algorithms for adaptive sparse channel estimation. Int. J. Adapt. Control Signal Process.
**2015**, 29, 1189–1206. [Google Scholar] [CrossRef] - Li, Y.; Li, W.; Yu, W.; Wan, J.; Li, Z. Sparse adaptive channel estimation based on l
_{p}-norm-penalized affine projection algorithm. Int. J. Antennas Propag.**2014**, 2014, 434659. [Google Scholar] [CrossRef] - Li, Y.; Jin, Z.; Wang, Y.; Yang, R. A robust sparse adaptive filtering algorithm with a correntropy induced metric constraint for broadband multi-path channel estimation. Entropy
**2016**, 18, 380. [Google Scholar] [CrossRef] - Gui, G.; Peng, W.; Adachi, F. Sparse least mean fourth algorithm for adaptive channel estimation in low signal-to-noise ratio region. Int. J. Commun. Syst.
**2014**, 27, 3147–3157. [Google Scholar] [CrossRef] - Li, Y.; Wang, Y.; Jiang, T. Norm-adaption penalized least mean square/fourth algorithm for sparse channel estimation. Signal Process.
**2016**, 128, 243–251. [Google Scholar] [CrossRef] - Gui, G.; Xu, L.; Matsushita, S. Improved adaptive sparse channel estimation using mixed square/fourth error criterion. J. Frankl. Inst.
**2015**, 352, 4579–4594. [Google Scholar] [CrossRef] [Green Version] - Li, Y.; Wang, Y.; Jiang, T. Sparse-aware set-membership NLMS algorithms and their application for sparse channel estimation and echo cancelation. AEU-Int. J. Electron. Commun.
**2016**, 70, 895–902. [Google Scholar] [CrossRef] - Gui, G.; Mehbodniya, A.; Adachi, F. Least mean square/fourth algorithm for adaptive sparse channel estimation. In Proceedings of the 2013 IEEE 24th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), London, UK, 8–11 September 2013; pp. 296–300. [Google Scholar]
- Albu, F.; Gully, A.; de Lamare, R.C. Sparsity-aware pseudo affine projection algorithm for active noise control. In Proceedings of the 2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Siem Reap, Cambodia, 9–12 December 2014; pp. 1–5. [Google Scholar]
- Li, Y.; Wang, Y.; Jiang, T. Sparse least mean mixed-norm adaptive filtering algorithms for sparse channel estimation application. Int. J. Commun. Syst.
**2016**, 30. [Google Scholar] [CrossRef] - Ma, W.; Qu, H.; Gui, G.; Xu, L.; Zhao, J.; Chen, B. Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-Gaussian environments. J. Frankl. Inst.
**2015**, 352, 2708–2727. [Google Scholar] [CrossRef] [Green Version] - Chen, B.; Principe, J.C. Maximum correntropy estimation is a smoothed MAP estimation. IEEE Signal Process. Lett.
**2012**, 19, 491–494. [Google Scholar] [CrossRef] - Chen, B.; Xing, L.; Liang, J.; Zheng, N.; Principe, J.C. Steady-state mean-square error analysis for adaptive filtering under the maximum correntropy criterion. IEEE Signal Process. Lett.
**2014**, 21, 880–884. [Google Scholar] - Wu, Z.; Peng, S.; Chen, B.; Zhao, H. Robust Hammerstein adaptive filtering under maximum correntropy criterion. Entropy
**2015**, 17, 7149–7166. [Google Scholar] [CrossRef] - Huijse, P.; Estevez, P.A.; Zegers, P.; Principe, J.C.; Protopapas, P. Period estimation in astronomical time series using slotted correntropy. IEEE Signal Process. Lett.
**2011**, 18, 371–374. [Google Scholar] [CrossRef] - Li, Y.; Wang, Y. Sparse SM-NLMS algorithm based on correntropy criterion. IET Electron. Lett.
**2016**, 52, 1461–1463. [Google Scholar] [CrossRef] - Zhao, S.; Chen, B.; Principe, J.C. Kernel adaptive filtering with maximum correntropy criterion. In Proceedings of the 2011 International Joint Conference on Neural Networks IJCNN, San Jose, CA, USA, 31 July–5 August 2011; pp. 2012–2017. [Google Scholar]
- Chen, B.; Xing, L.; Zhao, H.; Zheng, N.; Principe, J.C. Generalized correntropy for robust adaptive filtering. IEEE Trans. Signal Process.
**2016**, 64, 3376–3387. [Google Scholar] [CrossRef] - Chen, B.; Wang, J.; Zhao, H.; Zheng, N.; Principe, J.C. Convergence of a fixed-point algorithm under maximum correntropy criterion. IEEE Signal Process. Lett.
**2015**, 22, 1723–1727. [Google Scholar] [CrossRef] - Seth, S.; Principe, J.C. Compressed signal reconstruction using the correntropy induced metric. In Proceedings of the 2008 IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP’08, Las Vegas, NV, USA, 31 March–4 April 2008; pp. 3845–3848. [Google Scholar]
- Aliyu, M.L.; Alkassim, M.A.; Salman, M.S. A l
_{p}-norm variable step-size LMS algorithm for sparse system identification. Signal Image Video Process.**2015**, 9, 1559–1565. [Google Scholar] [CrossRef] - Shi, K.; Shi, P. Convergence analysis of sparse LMS algorithms with l
_{1}-norm penalty based on white input signal. Signal Process.**2010**, 90, 3289–3293. [Google Scholar] [CrossRef] - Salman, M.S. Sparse leaky-LMS algorithm for system identification and its convergence analysis. Int. J. Adapt. Control Signal Process.
**2015**, 28, 1065–1072. [Google Scholar] [CrossRef] - Ma, W.; Zheng, D.; Zhang, Z. Robust proportionate adaptive filter based on maximum correntropy criterion for sparse system identification in impulsive noise environments. Signal Image Video Process.
**2018**, 12, 117–124. [Google Scholar] [CrossRef] - Petraglia, M.R.; Haddad, D.B. Transient and steady-state MSE analysis of the IMPNLMS algorithm. Digit. Signal Process.
**2014**, 33, 50–59. [Google Scholar] - Li, Y.; Jiang, Z.; Osman, O.; Han, X.; Yin, J. Mixed norm constrained sparse APA algorithm for satellite and network channel estimation. IEEE Access
**2018**, 6, 65901–65908. [Google Scholar] [CrossRef] - Li, Y.; Jiang, Z.; Jin, Z.; Han, X.; Yin, J. Cluster-sparse proportionate NLMS algorithm with the hybrid norm constraint. IEEE Access
**2018**, 6, 47794–47803. [Google Scholar] [CrossRef]

**Figure 10.**Tracking behaviors of the PNMCC and CIM-PNMCC algorithms for estimating an echo channel (An example of typical echo channel is also included on the top of this figure.).

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Li, Y.; Wang, Y.; Sun, L.
A Proportionate Normalized Maximum Correntropy Criterion Algorithm with Correntropy Induced Metric Constraint for Identifying Sparse Systems. *Symmetry* **2018**, *10*, 683.
https://doi.org/10.3390/sym10120683

**AMA Style**

Li Y, Wang Y, Sun L.
A Proportionate Normalized Maximum Correntropy Criterion Algorithm with Correntropy Induced Metric Constraint for Identifying Sparse Systems. *Symmetry*. 2018; 10(12):683.
https://doi.org/10.3390/sym10120683

**Chicago/Turabian Style**

Li, Yingsong, Yanyan Wang, and Laijun Sun.
2018. "A Proportionate Normalized Maximum Correntropy Criterion Algorithm with Correntropy Induced Metric Constraint for Identifying Sparse Systems" *Symmetry* 10, no. 12: 683.
https://doi.org/10.3390/sym10120683