# Accelerated Conjugate Gradient for Second-Order Blind Signal Separation

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Second Order Blind Signal Separation

## 3. Accelerated Conjugate Gradient Algorithm

**Procedure 1**: Accelerated conjugate gradient algorithm for BSS

- Step 1: Initialize the iteration $k=1$ and the time domain unmixing weight matrices ${\mathbf{w}}^{\left(1\right)}\left(\tau \right)$ as$${\mathbf{w}}^{\left(1\right)}\left(\tau \right)=\left\{\begin{array}{c}{\mathbf{I}}_{L\times L},\phantom{\rule{3.33333pt}{0ex}}\mathrm{if}\phantom{\rule{3.33333pt}{0ex}}\tau =0\hfill \\ {\mathbf{0}}_{L\times L},\phantom{\rule{3.33333pt}{0ex}}\tau \ge 1\hfill \end{array}\right.$$
- Step 2: Obtain ${\mathbf{V}}^{\left(k\right)}$ as in (7). As ${\mathbf{V}}^{\left(k\right)}$ can have a higher objective function than ${\mathbf{W}}^{\left(k\right)}$, we can compare the cost function of ${\mathbf{V}}^{\left(k\right)}$ and ${\mathbf{W}}^{\left(k\right)}$. If $f({\mathbf{V}}^{\left(k\right)})>\rho \phantom{\rule{0.166667em}{0ex}}f({\mathbf{W}}^{\left(k\right)})$ where $\rho $ is a constant greater than 1, then we can reset ${\mathbf{V}}^{\left(k\right)}={\mathbf{W}}^{\left(k\right)}$. The accelerated gradient algorithm “slides” slightly further than the gradient descent direction obtained from ${\mathbf{W}}^{\left(k\right)}$. The projected search direction $\widehat{\Delta}{\mathbf{V}}^{\left(k\right)}$ is obtained from $\Delta {\mathbf{V}}^{\left(k\right)}$ by transforming the direction $\Delta {\mathbf{V}}^{\left(k\right)}$ to the time domain, truncating to length D and transforming it back to the frequency domain. Next, we obtain the conjugate gradient descent direction ${\mathbf{s}}^{\left(k\right)}$ as in (8).
- Step 4: Calculate the cost function ${f}^{(k+1)}\left(\mathbf{W}\right)$ at the $k+1$ iteration. If$$|10{log}_{10}{f}^{(k+1)}\left(\mathbf{W}\right)-10{log}_{10}{f}^{\left(k\right)}\left(\mathbf{W}\right)|<\u03f5$$
- Step 5: Stop the procedure. The optimum unmixing matrix is ${\mathbf{W}}^{(k+1)}$.

**Procedure 2**: Search for an optimum step size ${\mu}^{\left(k\right)}$ that minimizes the cost function (10).

- Step 1: Initialize a step size $\mu >0$, a constant $c>1$, and an accuracy level ${\u03f5}_{1}$. Set ${\mu}_{0}=0$, $s=\mu $, and ${\mu}_{1}=s$.
- Step 2: Obtain the cost functions $f\left({\mu}_{0}\right)$ and $f\left({\mu}_{1}\right)$. If $f\left({\mu}_{0}\right)\le f\left({\mu}_{1}\right)$, then reduce the initial step size by setting $s/c\to s$. Let ${\mu}_{1}=s$ and proceed to the beginning of Step 2. Otherwise, $f\left({\mu}_{0}\right)>f\left({\mu}_{1}\right)$ and continue to Step 3.
- Step 3: Increase s by setting $cs\to s$ and let ${\mu}_{2}={\mu}_{1}+s$. Calculate the cost function $f\left({\mu}_{2}\right)$. If $f\left({\mu}_{1}\right)>f\left({\mu}_{2}\right)$, then set ${\mu}_{0}={\mu}_{1}$, ${\mu}_{1}={\mu}_{2}$ and return to the beginning of Step 3. Otherwise, proceed to Step 4.
- Step 4: We now have three points, ${\mu}_{0}$, ${\mu}_{1}$, and ${\mu}_{2}$, satisfying$$f\left({\mu}_{0}\right)>f\left({\mu}_{1}\right),\phantom{\rule{3.33333pt}{0ex}}f\left({\mu}_{1}\right)\le f\left({\mu}_{2}\right),\phantom{\rule{3.33333pt}{0ex}}\mathrm{and}\phantom{\rule{3.33333pt}{0ex}}{\mu}_{0}<{\mu}_{1}<{\mu}_{2}.$$Thus, there exists a local minimum in the interval $[{\mu}_{0},{\mu}_{2}]$. As such, a parabola is fitted among three points $[{\mu}_{0},f\left({\mu}_{0}\right)]$, $[{\mu}_{1},f\left({\mu}_{1}\right)]$, and $\left[{\mu}_{2},f\left({\mu}_{2}\right)\right]$. The minimum a of this parabola in the interval $[{\mu}_{0},{\mu}_{2}]$ can be calculated using the parabola approximation [7]. Update ${\mu}_{0},\phantom{\rule{0.166667em}{0ex}}{\mu}_{1},\phantom{\rule{0.166667em}{0ex}}{\mu}_{2}$; stop the procedure if $|{\mu}_{2}-{\mu}_{1}|$ is small enough and output the optimal step size.

## 4. Design Examples

#### 4.1. Case 1: Simulated Room Environment

^{3}with a sampling rate ${f}_{s}=16,000$ Hz (see Figure 1). The inter-element distance for the microphone array is 0.04 m. We have three microphones at the $[x,y,z]$ positions $[1.96,\phantom{\rule{0.166667em}{0ex}}0.5,\phantom{\rule{0.166667em}{0ex}}1]$, $[2.0,\phantom{\rule{0.166667em}{0ex}}0.5,\phantom{\rule{0.166667em}{0ex}}1]$, and $[2.04,\phantom{\rule{0.166667em}{0ex}}0.5,\phantom{\rule{0.166667em}{0ex}}1]$, respectively. The speech signal is from the TIMIT library and the noise is from the NOISEX-92 library. The signal-to-interference ratio (SIR) is 0 dB, while the signal-to-noise ratio (SNR) is 0 dB and 10 dB. The length of the data is 8 seconds. The outputs are obtained using a fast-ISM room simulator [18] with reverberation time ${T}_{60}=0.15$ s.

#### 4.2. Case 2: Real Car Recording Data

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Kasak, P.; Jarina, R. Evaluation of Blind Source Separation Algorithms Applied to Music Signals. In Proceedings of the ELEKTRO, Krakow, Poland, 23–26 May 2022; pp. 1–8. [Google Scholar]
- Zhou, R.; Han, J.; Li, T.; Guo, Z. Fast Independent Component Analysis Denoising for Magnetotelluric Data Based on a Correlation Coefficient and Fast Iterative Shrinkage Threshold Algorithm. IEEE Trans. Geosci. Remote Sens.
**2022**, 60, 1–15. [Google Scholar] [CrossRef] - Ezilarasan, M.R.; Pari, J.B.; Aanandhasaravanan, K.; Prasanna, N.V.; Balaji, D. Performance analysis of ICA algorithm for Blind Source Separation. In Proceedings of the International Conference on Smart Structures and Systems (ICSSS), Chennai, India, 23–24 July 2020. [Google Scholar]
- Parra, L.; Spence, C. Convolutive blind separation for non-stationary sources. IEEE Trans. Speech Audio Process.
**2000**, 6, 320–327. [Google Scholar] [CrossRef] [Green Version] - Yong, P.C.; Nordholm, S.; Dam, H.H. Noise Estimation Based on Soft Decisions and Conditional Smoothing for Speech Enhancement. In Proceedings of the International Workshop on Acoustic Signal Enhancement (IWAENE 12), Aachen, Germany, 4–6 September 2012; pp. 4640–4643. [Google Scholar]
- Yong, P.C.; Nordholm, S.; Dam, H.H. Optimization and evaluation of sigmoid function with a priori SNR estimate for real-time speech enhancement. Speech Commun.
**2013**, 55, 358–376. [Google Scholar] [CrossRef] - Dam, H.H.; Rimantho, D.; Nordholm, S. Second-Order Blind Signal Separation with Optimal Step Size. Speech Commun.
**2013**, 55, 535–543. [Google Scholar] [CrossRef] - Dam, H.H.; Nordholm, S. Accelerated gradient with optimal step size for second-order blind signal separation. Multidimens. Syst. Signal Process.
**2018**, 29, 903–919. [Google Scholar] [CrossRef] - Dam, H.H.; Nordholm, S.; Low, S.Y.; Cantoni, A. Blind Signal Separation using Steepest Descent Method. IEEE Trans. Signal Process.
**2007**, 55, 4198–4207. [Google Scholar] - Dam, H.H.; Cantoni, A.; Nordholm, S.; Teo, K.L. Second-Order Blind Signal Separation for Convolutive Mixtures Using Conjugate Gradient. IEEE Signal Process. Lett.
**2008**, 15, 79–82. [Google Scholar] - Li, X.; Wang, W.; Xiang, S.Z.W.; Wu, X. Generalized Nesterov Accelerated Conjugate Gradient Algorithm for a Compressively Sampled MR Imaging Reconstruction. IEEE Access.
**2020**, 157130–157139. [Google Scholar] [CrossRef] - Yi, D.; Ji, S.; Bu, S. An Enhanced Optimization Scheme based on Gradient Descent Method for Machine Learning. Symmetry
**2019**, 11, 942. [Google Scholar] [CrossRef] [Green Version] - Beck, A.; Teboulle, M. A fast iterative shrinkage-thresholding algorithm for linear inverse problem. SIAM J. Imag. Sci.
**2009**, 2, 183–203. [Google Scholar] [CrossRef] [Green Version] - Tseng, P. On accelerated proximal gradient methods for convex concave optimization. Siam J. Optim. 2008; submitted for publication. Available online: http://www.math.washington.edu/tseng/papers/apgm.pdf(accessed on 21 September 2022).
- Giselsson, P.; Doanb, M.D.; Keviczky, T.; Schutter, B.D.; Rantzer, A. Accelerated gradient methods and dual decomposition in distributed model predictive control. Automatica
**2013**, 49, 829–833. [Google Scholar] [CrossRef] [Green Version] - McCormick, G.P. Nonlinear Programming. Theory, Algorithms, and Applications; John Wiley & Sons: New York, NY, USA, 1983. [Google Scholar]
- Fletcher, R.; Reeves, C.M. Functions minimization by conjugate gradient. Comput. J.
**1964**, 7, 149–154. [Google Scholar] [CrossRef] - Lehmann, E.; Johansson, A. Prediction of energy decay in room impulse responses simulated with an image-source model. J. Acoust. Soc. Am.
**2008**, 124, 269–277. [Google Scholar] [CrossRef] [PubMed]

**Table 1.**Second order BSS in a simulated room environment with two speech sources, noise, and three microphones for the dominant speech output detected by the kurtosis. The SNR is 0 dB.

Output 1 | Output 2 | |||
---|---|---|---|---|

Methods | SIR [dB] | SNR [dB] | SIR [dB] | SNR [dB] |

Steepest descent | ||||

with optimum step size | 7.1859 dB | 6.8365 dB | 7.1438 dB | 6.2152 dB |

Accelerated gradient | ||||

with optimum step size | 10.2324 dB | 9.4395 dB | 12.3647 dB | 10.3396 dB |

Accelerated conjugate gradient | ||||

with optimum step size | 10.9354 dB | 9.6536 dB | 12.4842 dB | 11.0761 dB |

**Table 2.**Second order BSS in a simulated room environment with two speech sources, noise, and three microphones for the dominant speech output detected by the kurtosis. The SNR is 10 dB.

Output 1 | Output 2 | |||
---|---|---|---|---|

Methods | SIR [dB] | SNR [dB] | SIR [dB] | SNR [dB] |

Steepest descent | ||||

with optimum step size | 7.8648 dB | 16.8737 dB | 8.0902 dB | 16.9402 dB |

Accelerated gradient | ||||

with optimum step size | 10.2414 dB | 18.9972 dB | 12.1539 dB | 20.0236 dB |

Accelerated conjugate gradient | ||||

with optimum step size | 10.9882 dB | 19.0562 dB | 13.7579 dB | 20.8592 dB |

**Table 3.**Second order BSS in a real car recording environment with two speech sources, noise, and three microphones for the dominant speech output detected by the kurtosis. The SNR is 0 dB.

Output 1 | Output 2 | |||
---|---|---|---|---|

Methods | SIR [dB] | SNR [dB] | SIR [dB] | SNR [dB] |

Steepest descent | ||||

with optimum step size | 7.6236 dB | 7.5242 dB | 0.8961 dB | 4.1718 dB |

Accelerated gradient | ||||

with optimum step size | 8.8505 dB | 8.6473 dB | 2.4538 dB | 6.8649 dB |

Accelerated conjugate gradient | ||||

with optimum step size | 10.3518 dB | 10.6519 dB | 4.9757 dB | 9.5280 dB |

**Table 4.**Second order BSS in a real car recording environment with two speech sources, noise, and three microphones for the dominant speech output detected by the kurtosis. The SNR is 10 dB.

Output 1 | Output 2 | |||
---|---|---|---|---|

Methods | SIR [dB] | SNR [dB] | SIR [dB] | SNR [dB] |

Steepest descent | ||||

with optimum step size | 7.1142 dB | 16.1042 dB | 0.8555 dB | 14.0435 dB |

Accelerated gradient | ||||

with optimum step size | 8.3185 dB | 16.8156 dB | 2.1784 dB | 16.6476 dB |

Accelerated conjugate gradient | ||||

with optimum step size | 9.8636 dB | 18.5544 dB | 4.5940 dB | 19.5139 dB |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Dam, H.H.; Nordholm, S.
Accelerated Conjugate Gradient for Second-Order Blind Signal Separation. *Acoustics* **2022**, *4*, 948-957.
https://doi.org/10.3390/acoustics4040058

**AMA Style**

Dam HH, Nordholm S.
Accelerated Conjugate Gradient for Second-Order Blind Signal Separation. *Acoustics*. 2022; 4(4):948-957.
https://doi.org/10.3390/acoustics4040058

**Chicago/Turabian Style**

Dam, Hai Huyen, and Sven Nordholm.
2022. "Accelerated Conjugate Gradient for Second-Order Blind Signal Separation" *Acoustics* 4, no. 4: 948-957.
https://doi.org/10.3390/acoustics4040058