An Expectation–Maximization-Based IVA Algorithm for Speech Source Separation Using Student’s t Mixture Model Based Source Priors

Abstract

The performance of the independent vector analysis (IVA) algorithm depends on the choice of the source prior to better model the speech signals as it employs a multivariate source prior to retain the dependency between frequency bins of each source. Identical source priors are frequently used for the IVA methods; however, different speech sources will generally have different statistical properties. In this work, instead of identical source priors, a novel Student’s t mixture model based source prior is introduced for the IVA algorithm that can adapt to the statistical properties of different speech sources and thereby enhance the separation performance of the IVA algorithm. The unknown parameters of the source prior and unmixing matrices are estimated together by deriving an efficient expectation maximization (EM) algorithm. Useful improvement in the separation performance in different realistic scenarios is confirmed by experimental studies on real datasets.

1. Introduction

The process of automated separation of acoustic sources from measured mixtures is known as acoustic blind source separation (BSS) [1]. The typical application of blind source separation is to handle the cocktail party problem, which is the process of focusing on one particular acoustic source of interest in the presence of multiple sound sources [2,3,4]. Human beings can easily pay attention to one of the speakers in the presence of multiple active speakers; however, it is much more difficult to replicate the same ability in machines [5]. In the past few decades, much research has been conducted to study different aspects of the cocktail party problem. This research includes the study of the geometry of the microphone array [6], room impulse response identification [7], localisation of speech sources [8] and statistical estimation of speech sources [9]. Independent component analysis (ICA) is one of the fundamental techniques to solve the cocktail party problem. The ICA algorithm was proposed by Herault and Jutten [10,11]; however, it has limitations such as permutation and scaling problems [12,13,14]. The IVA algorithm is an extension of the ICA algorithm which was proposed to theoretically mitigate the permutation problem of the ICA method that is inherent to most of the BSS algorithms [15].

The IVA algorithm is based on a dependency model which retains inter-frequency dependencies within each source vector. The dependent sources are arranged together as a multivariate variable in the frequency domain components of a signal. When the IVA algorithm is compared with the ICA algorithm, the inter-frequency dependencies within each source depend on the modified prior of the source signal. In the ICA algorithm, independence for each frequency component is measured separately at each frequency bin. On the other hand, the IVA method formulates the problem by considering that the dependencies exist between frequency bins rather than assuming the independence for frequency bins. The source priors in conventional algorithms were defined as independent priors; however, within the IVA algorithm each source prior is defined as a multivariate super-Gaussian distribution. Therefore, the cost function for the IVA algorithm is minimised only when the dependency between the source vectors is removed but the dependency among the components of each vector is preserved [16]. Hence it measures the dependence across the whole multivariate source and it can retain the higher order inter-frequency dependencies and structure of frequency components. Therefore, choosing an appropriate dependency model for the IVA algorithm is crucial to the performance of the algorithm.

Statistically the process of human speech production is highly complex [4,17,18] and the human speech signal is non-stationary in nature. Furthermore, the human speech signal is difficult to model with one fixed model as there can be wide variations in human speech, i.e., properties of natural speech vary from person-to-person and depend on which language is being spoken as the pronunciation rates and phonemes can be completely different in different parts of the world. Moreover, recorded speech is dependent on variations in room acoustics and microphone characteristics, e.g., different rooms will have different reverberation effects and different microphones will have variable frequency responses [3]. All of these factors can change the observed human speech signal and thereby different speech signals generally have different statistical properties. Hence, it is important that the BSS algorithms adapt their statistical structure according to the characteristics of the observed speech signals.

The IVA algorithm preserves the inter-frequency dependency between the individual sources in the frequency domain. The IVA method uses the score function and its form is crucial to the performance of the IVA algorithm. The score function is derived by statistical modelling of the speech sources by selecting an appropriate source prior. Speech signals are often characterized with single distribution statistical models that does not change according to the nature of speech signals. In the original IVA [15] method all the speech sources were modelled by identical multivariate Laplacian distributions. Different sources can have different statistical properties and modelling all the sources with identical distribution may not be the most appropriate solution. As a result a novel potential approach is to adopt the Student’s t mixture model (SMM) as a source prior for the IVA algorithm, instead of the conventional identical multivariate distributions. The probability density function of the Student’s t mixture model (SMM) has heavier tails as compared to other super Gaussian models and therefore it can model outliers in the data [19,20,21]. As human speech is highly random, the spread of samples can be very wide, and the SMMs, due to its heavier tails, can generally model high amplitude information in human speech more accurately [22].

The new framework of the expectation-maximization (EM) algorithm is implemented efficiently for the proposed IVA algorithm to estimate the unmixing matrices. The EM algorithm is a two step iterative approach which efficiently estimates the unknown parameters of the source prior and unmixing matrices. The EM method overcomes non-analytically solvable problems and it has been commonly used in the field of statistics, signal processing and machine learning [23]. By using SMMs as the source prior and implementing the new framework of EM, the proposed IVA algorithm shows performance improvement when compared with previous approaches [15,21,24,25]. To the best of our knowledge, there are no other studies that are using SMMs as source prior for the IVA algorithm to achieve robust and improved performance for different speech mixtures in realistic scenarios.

The rest of the paper is organized as follows. We begin by explaining the related work in the next section. We begin by explaining the IVA algorithm and related work in Section 2. It is followed by detailed description of the proposed EM framework for the IVA algorithm with SMMs as source priors in Section 3. Experimental results in realistic scenarios and comparisons between the proposed approach and other state-of-the-art methods are presented in Section 4. Finally, some concluding remarks are included in Section 5.

2. Related Work

In order to implement the IVA algorithm for the convolutive BSS, the short time Fourier transfer (STFT) is used to convert the problem from the time domain to the frequency domain as it eases the computational complexity of the time domain method. The basic noise free BSS model for the IVA method in the frequency domain can be defined as follows:

$$\mathbf{x}\left(k\right)=\mathbf{A}\left(k\right)\mathbf{s}\left(k\right)$$

(1)

where $\mathbf{A}$ is a mixing matrix of dimensions $m\times n$. The index k represents the k-th frequency of this multivariate method. In order to separate the source signals from the observed mixtures, an unmixing matrix must be estimated to retrieve the estimate of the original sources, as

$$\begin{array}{c}\hfill \widehat{\mathbf{s}}\left(k\right)=\mathbf{W}\left(k\right)\mathbf{x}\left(k\right)\end{array}$$

(2)

where $\widehat{\mathbf{s}}\left(k\right)$ is the estimated source signal, $\widehat{\mathbf{s}}\left(k\right)={[{\widehat{\mathbf{s}}}_1,{\widehat{\mathbf{s}}}_2\left(k\right)\cdots {\widehat{\mathbf{s}}}_n\left(k\right)]}^T$, $\mathbf{W}\left(k\right)$ is the unmixing matrix of dimensions $n\times m$. In this paper, focus is on the exactly determined case, so the number of sources is considered equal to the number of microphones, i.e., $n=m$.

In order to model the independence between sources, the IVA method uses the Kullback-Leibler divergence. So a cost function can be derived as follows [15]:

$$J_{IVA}=\mathcal{KL}(p\left(\widehat{\mathbf{s}}\right)\left|\right|\prod q\left(\widehat{\mathbf{s}}\right))$$

(3)

$$=\int p({\widehat{s}}_1\cdots {\widehat{s}}_n)\log {\displaystyle \frac{p({\widehat{s}}_1\cdots {\widehat{s}}_n)}{\prod q\left(\widehat{\mathbf{s}}\right)}}d{\widehat{s}}_1\cdots d{\widehat{s}}_n$$

(4)

$$=\mathrm{const}-\sum _{k=1}^K\log |\det (W^{\left(k\right)})|-\sum _{i=1}^{n}E\left[\log q\left({\widehat{s}}_i\right)\right]$$

(5)

where $\det (.)$ represents the matrix determinant and $E(.)$ shows the expectation operator. All the sources in the cost function of the IVA algorithm are multivariate and the cost function will be minimised when different vector sources become independent of each other and the dependency within each source vector is retained. Hence this cost function can be used to eliminate the dependency between the vector sources and preserve the frequency dependency within each vector source.

Previously, in the IVA method, speech signals have been modelled with various superGaussian distributions, e.g., the Laplacian distribution [15] or generalized Gaussian distribution [26] but speech signals can have very high and low amplitudes and other superGaussian distributions may not be able to accurately model the high amplitudes in the speech signals [27,28,29]. Therefore, new source priors are still needed to improve the performance of the IVA algorithm.

3. Proposed Method

In order to model the speech signals with low and high amplitude, the Student’s t distribution is adopted as a source prior for the IVA method. The multivariate Student’s t distribution is given as: A K-dimensional random separated source vector $\mathbf{s}={(s\left(1\right),\dots ,s\left(K\right))}^T$ can have a K-variate t distribution with degree of freedom $\nu$, precision matrix $\mathbf{\Lambda }$ and mean $\mathit{\mu }$, if its joint pdf is given by [27]:

$$St\left(\mathbf{s}\right|\mathit{\mu },\mathbf{\Lambda },\nu )={\displaystyle \frac{({\displaystyle \frac{\nu +d}{2}}){|\mathbf{\Lambda }|}^{1/2}}{\sqrt{\nu \pi }({\displaystyle \frac{\nu }{2}})}}{\left(1+{\displaystyle \frac{{(\mathbf{s}-\mathit{\mu })}^T\mathbf{\Lambda }(\mathbf{s}-\mathit{\mu })}{\nu }}\right)}^{-{\displaystyle \frac{\nu +d}{2}}}$$

(6)

In the joint pdf of the Student’s t distribution, the leptokurtic nature and the variance of distribution can be adjusted by tuning the degrees of freedom parameter $\nu$ [30]. When the $\nu$ parameter is set to a lower value, the tails of the distribution becomes heavier and if $\nu$ is increased to infinity, the Student’s t distribution tends to a Gaussian distribution [19,20]. Since different sources can have different statistical properties, so instead of using identical Student’s t source prior for all sources, the Student’s t mixture model (SMM) is adopted as a source prior in this work. By adopting the SMM as a source prior, different speech sources can be modelled with different SMMs thereby enabling the IVA method to adapt to the statistical properties of different types of signals. Assuming the sources are statistically independent, for a 2 × 2 case, an SMM can be represented as:

$$\begin{array}{c}\hfill p(\mathbf{s}\left(1\right)\cdots \mathbf{s}\left(K\right))=\prod _{i=1}^{2}p({\mathrm{s}}_i\left(1\right)\cdots {\mathrm{s}}_i\left(K\right))\\ \hfill p({\mathrm{s}}_i\left(1\right)\cdots {\mathrm{s}}_i\left(K\right))=\sum _{q_i}p\left(q_i\right)\prod _k{\mathit{S}}_t\left({\mathbf{s}}_i\left(k\right)\right|{\mathit{\mu }}_i\left(k\right),{\mathbf{\Lambda }}_i\left(k\right))\end{array}$$

(7)

where $q_i$ is the weight of the mixture component of the SMM source prior for source i and K represents the total number of frequency bins in the multivariate model. The precision matrix $\mathbf{\Lambda }$ is defined as the inverse of the covariance matrix and its $ik$-th element satisfies $1/{\mathbf{\Lambda }}_i\left(k\right)=E\{{\mid {\mathbf{s}}_i\left(k\right)\mid }^2{q}_i\}$. With appropriate normalisation and zero mean assumption, the Student’s t distribution can be written as:

$$\begin{array}{c}\hfill St\left({\mathbf{s}}_i\left(k\right)\right|0,{\mathbf{\Lambda }}_i\left(k\right))={\displaystyle \frac{{\mathbf{\Lambda }}_i\left(k\right)}{\pi }}{\left(1+{\displaystyle \frac{{\mathbf{\Lambda }}_i\left(k\right){\mid {\mathbf{s}}_i\left(k\right)\mid }^2}{\nu }}\right)}^{-{\displaystyle \frac{\nu +d}{2}}}\end{array}$$

(8)

When the vector of frequency components is considered from the same source i, the interdependency between these frequency components is preserved whereas the vectors that originate from different sources are independent of each other. Therefore, by adopting this inter-frequency dependency model, the IVA method prevents the permutation problem that is inherent to most BSS methods [4].

In the IVA algorithm, the scaling of mixture signal $\mathbf{x}\left(k\right)$ and mixing matrix $\mathbf{A}\left(k\right)$ cannot be determined by the separated source signals $\mathbf{s}\left(k\right)$, therefore observations are prewhitened. Because of the prewhitening process, both the mixing matrix $\mathbf{A}\left(k\right)$ and the unmixing matrix $\mathbf{W}\left(k\right)$ are unitary matrices. In this work, the 2 × 2 case has been considered, so the Cayley Klein parameterizations [31] for the unitary matrix $\mathbf{W}\left(k\right)$ are as follows:

$$\mathbf{W}\left(k\right)=\left(\begin{array}{cc}{a}_k& b_k\\ -b_k^{*}& a_k^{*}\end{array}\right)\therefore \left|\mathbf{W}\left(k\right)\right|=a_k{a}_k^{*}+b_k{b}_k^{*}=1$$

(9)

In the next section, the maximum likelihood estimate is derived for the IVA algorithm.

3.1. Maximum Likelihood Estimation of SMM

The maximum likelihood estimate is a well-known method that is usually used to estimate the mixture parameters. Based on the maximum likelihood method, the mixture parameters can be effectively estimated iteratively via the EM algorithm [23]. The log likelihood function for t components mixture of Student’s t distributions is considered and it is given as:

$$\mathcal{L}(\mathbf{x},\theta )=\sum _{i=1}^t\log p({\mathbf{x}}_i\left(1\right),\cdots ,{\mathbf{x}}_i\left(K\right))=\sum _{i=1}^t\log \left(\sum _{q_i}\prod _{k=1}^{K}p\left({\mathbf{x}}_i\left(k\right)\right|q_i)p\left(q_i\right)\right)$$

(10)

where $\theta =\{{\mathbf{W}}_i,{\mathbf{\Lambda }}_i,p\left(q_i\right)\}$ consists of the model parameters for the log likelihood function; $p({\mathbf{x}}_i\left(1\right),\cdots ,{\mathbf{x}}_i\left(K\right))$ is the PDF of the observed source mixture signals which is an SMM as it is generated by the SMM source priors. The ${\mathbf{W}}_i$ denotes the unmixing matrix, ${\mathbf{\Lambda }}_i$ represents the precision matrix and $q_i=[q_1,q_2]$ is the collective mixture index of the SMMs for the source prior. In the maximum likelihood estimation, the best fitting model helps to estimate parameters that can maximize the log-likelihood function, which is usually performed by using the EM algorithm [23]. Therefore, the model parameters set $\theta =\{{\mathbf{W}}_i,{\mathbf{\Lambda }}_i,p\left(q_i\right)\}$ can be estimated by training the SMM and maximizing the log likelihood function by using the EM algorithm. The detailed method for estimating the model parameters by the EM algorithm is explained in the next section.

3.2. The Expectation-Maximization Algorithm

The EM algorithm is suitable in finding latent parameters in probabilistic models by using an iterative optimization technique [23]. The EM algorithm is implemented by introducing discrete random variables $z\left(q_i\right)$ which are dependent on the observations $({\mathbf{x}}_i\left(1\right),\cdots ,{\mathbf{x}}_i\left(K\right))$ and the model parameter set $\theta$. The log likelihood function with these variables is given by

$$\begin{array}{c}\hfill \mathcal{L}(\mathbf{x},\theta )=\sum _{i=1}^t\log \left(\sum _{x_i}\prod _{k=1}^{K}p\left({\mathbf{x}}_i\left(k\right)\right|{\mathbf{q}}_i)p\left(q_i\right)\right)\\ \hfill =\sum _{i=1}^t\log \left(\sum _{q_i}{\displaystyle \frac{z\left(q_i\right){\prod }_{k=1}^{K}p\left({\mathbf{x}}_i\left(k\right)\right|q_i)p\left(q_i\right)}{z\left(q_i\right)}}\right)\end{array}$$

(11)

and can be used to optimise the model parameters. In the case of an increasing log likelihood function, a lower bound is formed on the increasing log likelihood for the observations $({\mathbf{x}}_i\left(1\right),\cdots ,{\mathbf{x}}_i\left(K\right))$. So the new parameters that increase the log likelihood function of the complete data with respect to current parameters, can be found. Hence there is an increase in the expected log likelihood of the complete data with respect to current parameters and it is produced by the updated parameters. Therefore, an auxiliary function can be used to represent the expected log likelihood function. There will be a definite increase in the log likelihood function when the auxiliary function is optimised but it does not necessarily yield a maximum likelihood solution [23]. Therefore, it is important to iteratively calculate and maximize the auxiliary function until convergence. Hence a local approximation is made which is the lower bound to the objective function. By using the Jensen’s inequality [32], the lower bound for the log likelihood function in Equation (11) can be calculated as follows:

$$\begin{array}{c}\hfill \mathcal{L}(\mathbf{x},\theta )\ge \sum _{i=1}^{t}z\left(q_i\right)\log \left({\displaystyle \frac{{\prod }_{k=1}^{K}p\left({\mathbf{x}}_i\left(k\right)\right|q_i)p\left(q_i\right)}{z\left(q_i\right)}}\right)\\ \hfill =\mathfrak{F}(z,\theta )\end{array}$$

(12)

The EM algorithm will run until convergence and it will iteratively maximize $\mathcal{L}(\mathbf{s},\theta )$ in two steps. The first step is the expectation step in which the posterior probability of the hidden variable $\mathfrak{F}(z,\theta )$ is calculated over $z\left(x_i\right)$ and in the second step, the $\theta$ is updated.

3.3. The Expectation Step

In the expectation step, $\theta$ is fixed and $\mathfrak{F}(z,\theta )$ is maximised over $z\left(q_i\right)$. In order to maximise $\mathfrak{F}(z,\theta )$, the derivative of the log-likelihood equation with respect to $z\left({\mathbf{q}}_i\right)$ is calculated as follows:

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial }{\partial z\left(q_i\right)}}\left(\mathcal{L}(\mathbf{x},\theta )\right)={\displaystyle \frac{\partial }{\partial z_i}}\left(\sum _{i=1}^{t}z\left(q_i\right)\log {\displaystyle \frac{{\prod }_{k=1}^{K}p\left({\mathbf{x}}_i\left(k\right)\right|{\mathbf{q}}_i)p\left(q_i\right)}{z\left(q_i\right)}}\right)\end{array}$$

(13)

In order to maximize $\theta$ for fixed $\mathfrak{F}(z,\theta )$, Equation (13) is equalised to zero and with appropriate normalization,

$$z\left(q_i\right)={\displaystyle \frac{{\prod }_{k=1}^{K}p\left({\mathbf{x}}_i\left(k\right)\right|q_i)p\left(q_i\right)}{p({\mathbf{x}}_i\left(1\right),\cdots ,{\mathbf{x}}_i\left(K\right))}}$$

(14)

Now by using $\mathbf{x}\left(k\right)=\mathbf{A}\left(k\right)\mathbf{s}\left(k\right)$,

$$p\left({\mathbf{s}}_i\left(k\right)\right|{\mathbf{q}}_i)=St\left({\mathbf{s}}_i\left(k\right)\right|0,{\mathbf{\Lambda }}_i\left(k\right))$$

(15)

and the precision matrix for the 2 × 2 case can be written as:

$${\mathbf{\Lambda }}_{ik}=\mathbf{W}{\left(k\right)}^{†}{\mathbf{\Phi }}_i\left(k\right)\mathbf{W}\left(k\right);{\mathbf{\Phi }}_i\left(k\right)=\left(\begin{array}{cc}{v}_1\left(k\right)& 0\\ 0& v_2\left(k\right)\end{array}\right)$$

(16)

As $\mathbf{W}\left(k\right)$ is a unitary matrix, therefore $\det \left({\mathbf{\Lambda }}_i\left(k\right)\right)=v_1\left(k\right)v_2\left(k\right)$ and from Equation (10), the function $f\left(x_i\right)$ can be defined as:

$$f\left(q_i\right)=\sum _k\log p\left({\mathbf{x}}_i\left(k\right)\right|{\mathbf{q}}_i)+\log p\left(q_i\right)$$

(17)

By using Equation (14), function $f\left(x_i\right)$ can be rewritten as $z\left(x_i\right)\propto e^{f\left(x_i\right)}$, therefore:

$$\begin{array}{c}\hfill j_i=\sum _{s_i}{e}^f\left(x_i\right);\\ \hfill z\left(x_i\right)={\displaystyle \frac{1}{j_i}}{e}^f\left(x_i\right)\end{array}$$

(18)

Next, the maximization step is considered.

3.4. The Maximization Step

In the maximization step (M-step) the parameters $\theta =\{{\mathbf{W}}_i,{\mathbf{\Lambda }}_i,p\left(q_i\right)\}$ can be estimated by maximising the cost function. In this step, each parameter is estimated separately. In the first step, the maximisation of ${\mathbf{W}}_i$ over the unitary constraint is considered. In order to maximize the ${\mathbf{W}}_i$, the precision matrix for the 2 × 2 case can take the following form:

$${\mathbf{\Phi }}_{ik}=\left(\begin{array}{cc}{v}_1\left(k\right)-v_2\left(k\right)& 0\\ 0& 0\end{array}\right)+\left(\begin{array}{cc}{v}_2\left(k\right)& 0\\ 0& v_2\left(k\right)\end{array}\right)$$

(19)

When Equation (15) is rearranged and $(p\left({\mathbf{x}}_i\left(k\right)\right|{\mathbf{q}}_i)$ is replaced in the log likelihood Equation (12), it will take the following form:

$$=\sum _{i=1}^{t}z\left(q_i\right)\left\{\lambda \right\}\left\{(-\nu /2-d/2)\log \left(1+{\displaystyle \frac{{\mathbf{\Lambda }}_i\left(k\right){\mid {\mathbf{x}}_i\left(k\right)\mid }^2}{\nu }}\right)\right\}$$

(20)

By using the log approximation $\log (1+a)\approx a$, where a is a small value, the above mentioned equation can take the following form, wherein equality is assumed for convenience.

$$=\sum _{i=1}^{t}z\left(q_i\right)\left\{\lambda \right\}\left\{(-\nu /2-d/2)\left({\displaystyle \frac{{\mathbf{\Lambda }}_i\left(k\right){\mid {\mathbf{x}}_i\left(k\right)\mid }^2}{\nu }}\right)\right\}$$

(21)

By further manipulating the above mentioned equation, details of which are given in Appendix A, the parameters of the unmixing matrix can be calculated as follows:

$$\left(\begin{array}{c}{a}_k^{*}\\ b_k^{*}\end{array}\right)={\displaystyle \frac{1}{\sqrt{1+{({\displaystyle \frac{{\beta }_k-M_{11}}{M_{12}}})}^2}}}\left(\begin{array}{c}1\\ {\displaystyle \frac{{\beta }_k-M_{11}}{M_{12}}}\end{array}\right)$$

(22)

Since the unmixing matrix ${\mathbf{W}}_{\mathbf{i}}\left(\mathbf{k}\right)=\left(\begin{array}{cc}{a}_k& b_k\\ -b_k^{*}& a_k^{*}\end{array}\right)$, it can be estimated by using the above mentioned analytical solution. It is an efficient method to estimate the unmixing matrix as the above mentioned method avoids the matrix calculations.

The model parameters $\theta =\{\mathbf{W}\left(k\right),\mathbf{\Lambda }\left(k\right),p\left(q_i\right)\}$ are estimated by maximizing the log likelihood function. Therefore, now $\mathfrak{F}(z,\theta )$ will be maximized over ${\mathbf{\Lambda }}_{iK}$ and $p\left(q_i\right)$ and they are given as follows.

$${\displaystyle \frac{1}{v_{i{k}_{j=r}}}}=\left({\displaystyle \frac{-\nu /2-d/2}{\nu }}\right){\displaystyle \frac{{\left[{\sum }_{i=1}^{t}z\left(q_{ij=r}\right)({{\mathbf{x}}_i\left(k\right)}^{†}\mathbf{W}{\left(k\right)}^{†}\mathbf{W}\left(k\right){\mathbf{x}}_i\left(k\right))\right]}_{jj}}{{\sum }_{i=1}^{t}z\left(q_{i=jr}\right)}}$$

(23)

where ${[.]}_{jj}$ denotes the $(j,j)$ element of the matrix. So $\mathfrak{F}(\mathbf{z},\theta )$ over ${\mathbf{\Lambda }}_i$ using the above mentioned solution.

Now, maximisation of $\mathfrak{F}(\mathbf{z},\theta )$ over $p\left(q_i\right)$ is performed and it is given as:

$$p(q_i=r)={\displaystyle \frac{{\sum }_{i=1}^{t}z(q_i=r)}{t}}$$

(24)

Hence the weighting parameter can be calculated by using the above mentioned equation. Detailed derivation for maximization of $\theta$ is included in Appendix A.

It can be seen that the EM algorithm effectively estimates all the model parameters $\theta =\{{\mathbf{W}}_i,{\mathbf{\Lambda }}_i,p\left(q_i\right)\}$. The E-step updates the $z\left(q_i\right)$, while the M-step effectively estimates the model parameters. In the EM algorithm the degrees of freedom parameter $\nu$ is fixed in advance for all the sources, then the M-step exists in the closed form (Algorithm 1). The value for the degrees of freedom can be estimated empirically for different source signals. The complete EM framework for the SMIVA algorithm is summarized as follows.

Algorithm 1 EM algorithm for Student’s t Mixtures

Require: Given a Student’s t mixture model, the aim is to maximize the log likelihood function with respect to the parameters $\theta =\{{\mathbf{W}}_i,{\mathbf{\Lambda }}_i,p\left(q_i\right)\}$. 1: Initialize the model parameters, the unmixing matrix ${\mathbf{W}}_i$, the precision ${\mathbf{\Lambda }}_i$ and the weight coefficients $p\left(q_i\right)$ and evaluate the initial value of the log likelihood. 2: Expectation Step: Evaluate the probabilities using the current parameter values $$z\left(x_i\right)={\displaystyle \frac{{\prod }_{k=1}^{K}p\left({\mathbf{x}}_i\left(k\right)\right|q_i)p\left(q_i\right)}{p(x_i\left(1\right),\cdots ,x_i\left(K\right))}}$$ (25) 3: Maximization Step: Using the current probabilities, re-estimate the parameters Coefficients of the unmixing matrices ${\mathbf{W}}_i$ are estimated by: $$\left(\begin{array}{c}{a}_k^{*}\\ b_k^{*}\end{array}\right)={\displaystyle \frac{1}{\sqrt{1+{({\displaystyle \frac{{\beta }_k-M_{11}}{M_{12}}})}^2}}}\left(\begin{array}{c}1\\ {\displaystyle \frac{{\beta }_k-M_{11}}{M_{12}}}\end{array}\right)$$ (26) Coefficients of the precision matrix ${\mathbf{\Lambda }}_i$ are estimated by $${\displaystyle \frac{1}{v_{i{k}_{j=r}}}}=\left({\displaystyle \frac{-\nu /2-d/2}{\nu }}\right){\displaystyle \frac{{\left[{\sum }_{i=1}^{t}z\left(x_{ij=r}\right)({{\mathbf{x}}_i\left(k\right)}^{†}\mathbf{W}{\left(k\right)}^{†}\mathbf{W}\left(k\right){\mathbf{x}}_i\left(k\right)\right]}_{jj}}{{\sum }_{i=1}^{t}z\left(x_{i=jr}\right)}}$$ (27) The weighting coefficients can be estimated as $$p(q_i=r)={\displaystyle \frac{{\sum }_{i=1}^{t}z(q_i=r)}{t}}$$ (28) 4: Evaluate the log likelihood $$\mathcal{L}(\mathbf{x},\theta )=\sum _{i=1}^t\log \left(\sum _{x_i}\prod _{k=1}^{K}p\left({\mathbf{x}}_i\left(k\right)\right|q_i)p\left(q_i\right)\right)$$ (29) and check for convergence of the log likelihood function, if the criterion for convergence is not fulfilled, return to step 2.

The separation performance of this EM framework for the SMIVA method will be evaluated in the next section.

4. Experimentations and Results

In this section, the separation performance of the SMIVA algorithm will be tested in three different experimental setups. Firstly, the new framework for the IVA algorithm is tested in a simulated environment and then in order to evaluate the performance in real scenarios, it is tested with real room impulse responses (RIRs), which can depict the performance of the proposed method in changing realistic settings. The results from all three sets of experiments for the proposed algorithm will be compared with the original IVA algorithm with different source priors.

4.1. Case I: Simulations with the Image Method

Firstly, the proposed method will be tested with RIRs that were generated by using the image method. The speech signals were selected randomly from the whole of the TIMIT dataset [33] and the length of speech signals was approximately 4 s. A 2 × 2 case was considered and the room has the $R{T}_{60}$ = 200 ms and it provides a good setup for comparing the separation performance of different algorithms. The positions of microphones in the room were set to [3.44, 2.50, 1.50] and [3.48, 2.50, 1.50] with azimuth angles of ${60}^{\circ }$ and ${30}^{\circ }$, respectively with reference to the normal of the microphone position. The STFT length is 1024 and sampling frequency is 8 kHz. The separation performance of the algorithm was evaluated with the objective measure of SDR [34]. The common parameters used in these experiments are summarised in

Table 1. Summary of parameters used in experiments.

Sampling rate	8 kHz
STFT frame length	1024
Reverberation time	200 ms
Room dimensions	7 m × 5 m × 3 m
Source signal duration	4 s (TIMIT)
Room impulse responses	Image method
Objective measure	Signal to Distortion Ratio (SDR)

.

The speech signals were convolved into mixtures in the above mentioned room settings. These speech mixtures were then separated by using the proposed SMIVA method and the separation results for different mixtures were compared with the separation performance of the original IVA method with the original super Gaussian source prior [15] and also with the IVA method with Student’s t source prior [21] and the results are shown in

Table 2. SDR (dB) values for different source priors for the IVA method with an image room impulse response [35]. The SMM source prior shows improvement for all mixtures.

	Original Super Gaussian	Student’s t Distribution	SMM Source Prior
Set-1	9.09	9.84	10.27
Set-2	8.98	9.72	10.24
Set-3	9.26	10.11	10.87
Set-4	9.02	9.95	10.49
Set-5	9.53	10.21	10.62
Set-6	9.51	10.14	10.74
Set-7	8.91	9.67	10.09
Set-8	9.86	10.48	11.05
Set-9	9.94	10.66	11.24
Set-10	10.02	10.56	11.01

and all the values shown for SDR are in dB. For each mixture SDR performance shown in the Table 2 is the average of two speech signals. It is evident from the Table 2 that when the SMM is adopted as a source prior, the average SDR improvement is approximately 1.1 dB for all the mixtures as compared to the original super Gaussian source prior for the IVA method. It is evident from Table 2 that the SMM source prior based SMIVA algorithm enhances the separation performance of the IVA method with a single distribution source prior, such as the Student’s t distribution and also the original super Gaussian.

For the derivation purposes, the unmixing matrix W is assumed to be unity in this study. In order to measure the effect of this assumption on the performance of the SMIVA algorithm, its separation performance is compared with the algorithms that do not place the same restriction on the solution such as [36,37]. Since the latent variables are modelled with a single distribution in these methods, the SMIVA algorithm is also adjusted to a uni-model case by using a single Student’s t distribution. For the sake of consistency in results, same experimental settings are used as in the previous case and the SDR measure is used to estimate the separation performance of the algorithms and the results are shown in Figure 1. It is clear from results that the SMIVA algorithm with the assumptions placed on the solution space still consistently perform better than the algorithms that does not have the same assumptions on the solution space. These results shows the significance of the modelling of the high and low amplitude information within the speech signals by using the Student’s t distribution.

In order to further investigate the separation performance of the SMIVA algorithm, its separation results are compared with the other mixture model source prior such as Gaussian mixture model [24] in the next section with real RIRs.

4.2. Case II: Simulations with Real RIRs

In the second set of experiments, the proposed SMIVA algorithm is tested with real RIRs. These real RIRs are obtained from [38] and these are recorded in different rooms with different acoustic properties. Three different room types (A, B, D) have been used with $R{T}_{60}$ of 320 ms, 470 ms and 890 ms, respectively. By using these RIRs the proposed method can be tested with the range of reverberation time. Therefore, these simulations show the performance of the proposed algorithm in real life scenarios as the $R{T}_{60}$ can vary drastically in realistic environments [39]. Different source location azimuth angles are available, which range from ${15}^{\circ }$ to ${90}^{\circ }$, relative to the second source.

Firstly, the proposed algorithm is tested in Room A, which is a typical medium sized office and it has the $R{T}_{60}$ of 320 ms, which is relatively small for a medium size office. In the experiments two speech signals are randomly chosen from the whole of the TIMIT dataset and the source location azimuth angles are set to be from $({15}^{\circ }$ to ${90}^{\circ }$ with a step of ${15}^{\circ })$. The mixed sources are separated by using the proposed SMIVA method and the separation performance in terms of SDR is compared with the IVA using the identical Student’s t source prior [21] and also with the original super Gaussian source prior based IVA method [15]. The separation performance for both methods is evaluated for six different angles varying from ${15}^{\circ }$ to ${90}^{\circ }$ with a step of ${15}^{\circ }$. At all the angles separation performance is averaged over six different speech mixture and the results are presented in Figure 2.

It is evident from Figure 2 that when proposed algorithm is used to separate the mixtures and the performance is compared with identical distribution source prior for the IVA, it consistently has a better separation performance at all the selected azimuths angles and approximately 1 dB of improvement in SDR values is recorded at all the angles as compared to the original IVA method [15].

The same set of experiments are then repeated for room-B, which evaluate the performance of the algorithms in more challenging and realistic scenarios as it is a medium size classroom with $R{T}_{60}$ of 470 ms. The separation performance in terms of SDR of both methods for six different azimuth angles is showed in Figure 3. It is evident from the result that the EM framework SMIVA performs better than the identical source priors for the original IVA method at all separation angles in ths reverberant real room environment.

Finally, the separation performance of the proposed EM framework for the IVA method is evaluated in a highly reverberant realistic environment that can depict the performance of the algorithm in the real life scenarios. For the highly reverberant environment, Room D was used which is a medium size seminar and presentation hall with a very high ceiling. The $R{T}_{60}$ for this seminar hall is 890 ms, which is high reverberation time and therefore it provides a good insight into the algorithm’s performance in an extremely difficult real life situations. The experimental setup in this highly reverberant room D is similar to the previous two rooms. The mixtures were separated with the IVA method with different source priors and the separation performance in terms of SDR for all methods is shown in Figure 4 for all six angles varying from ${15}^{\circ }$ to ${90}^{\circ }$. The SDR values in room D is lower in comparison with the SDR values for Room-A and Room-B, it is mainly because the $R{T}_{60}$ for Room D is really high as compared to the other two rooms. Also, it is evident from the Figure 4 that even in this highly reverberant environment the IVA method with SMM source prior performs better than the identical distribution source priors for the original IVA with Student’s t source prior and improves the average SDR performance of the original IVA method by approximately 0.8 dB.

The separation performance of the proposed EM framework for the IVA algorithm with SMMs as a source prior is also compared with the IVA algorithm with a Gaussian mixture model (GMM) as a source prior [24]. Since the mixture model is adapted as a source prior for the IVA algorithm, the comparison with other mixture models, i.e., GMM can provide better understanding of the separation performance of the proposed source prior. In this set of experiments, a mixture of two Gaussians is adapted as source prior for the IVA algorithm and the rest of the parameters are adjusted similar to the SMIVA algorithm and simulations are performed in the real room settings. For the experimental setup, same settings for Room A, B and D are used as in the previous case and speech signals are randomly chosen from TIMT dataset. Initially, experiments are performed in room A, which has $R{T}_{60}$ of 320 ms and it is repeated for six different source location varying from ${15}^{\circ }$ to ${90}^{\circ }$. Similarly, the same experimental setup is used for room B with $R{T}_{60}$ of 470 ms and for room D with $R{T}_{60}$ of 890 ms. In all the rooms mixtures are separated by using the EM framework IVA with both SMM and GMM source priors and the separation performance in terms of SDR is compared with the proposed method at six different source azimuth angles varying from ${15}^{\circ }$ to ${90}^{\circ }$. All the SDR values at all the angles are the average of separation performance of six different mixtures. The separation performance of both methods for all three rooms with the range of $R{T}_{60}$ is shown is Figure 5 and it is evident that the IVA method with SMM as a source prior has better separation performance than IVA with GMM as a source prior.

4.3. Case III: Simulations with Binaural Room Impulse Responses

The proposed algorithm is further tested with binaural room impulse response (BRIRs) obtained from [40]. These BRIRs are recorded in a real classroom which roughly has dimensions of 5 × 9 × 3.5 m ${}^3$. The six source location azimuths $({15}^{\circ },{30}^{\circ },{45}^{\circ },{60}^{\circ },{75}^{\circ },{90}^{\circ })$ relative to the right of listener were used for the experimentation. Also the distance between the source were changed three times $(0.15,0.40$ and 1 m). The measurements for the BRIRs are taken at four different listener locations (back, ear, corner and center) and the distance between the floor and ears was approximately 1.50 m. In these experiments only center location is used and the $R{T}_{60}$ at the center location for the classroom was 565 ms. All the measurements are repeated on three different occasions by taking down the equipment and reassembling it which improves the reliability of the measurements. Therefore, these BRIRs have been used in the experiments as they are reliable and also provide accurate estimate of the separation performance of the BSS algorithms in highly reverberant room environments. A summary of different parameters used in this set of experiments is given in

Table 3. Summary of parameters used in experiments.

Sampling rate	8 kHz
STFT frame length	1024
Velocity of sound	343 m/s
Reverberation time	565 ms (BRIRs)
Room dimensions	9 m × 5 m × 3.5 m
Source signal duration	3.5 s (TIMIT)

.

The 2 × 2 case was considered for the experiments and speech signals were randomly chosen from the whole TIMIT dataset and mixtures were created by using BRIRs. The length of the speech signals was approximately four seconds. The speech signals were then separated from the mixtures by using the proposed EM framework for the IVA algorithm with SMM as source prior. The separation performance of the proposed algorithm is compared with the separation performance of the IVA with GMM as source prior for the IVA algorithm. It provides a good estimate for the separation performance of the proposed algorithm and source prior as comparison is drawn with mixture model source priors. The separation performance in terms of SDR is shown in

Table 4. Comparison between SMM source prior and GMM source prior for the EM framework IVA algorithm with BRIRs ($R{T}_{60}$ = 565 ms). The separation performance at each angle is averaged over eighteen different speech mixtures. The IVA algorithm with proposed mixture model Student’s t source prior perform significantly better at all the separation angles than the GMM source prior.

	GMM Source Prior	SMM Source Prior	Percentage Improvement
Angle-${15}^{\circ }$	4.51	4.82	6.87%
Angle-${30}^{\circ }$	4.62	4.97	7.56%
Angle-${45}^{\circ }$	4.77	5.09	6.70%
Angle-${60}^{\circ }$	4.97	5.32	7.04%
Angle-${75}^{\circ }$	4.91	5.28	7.73%
Angle-${90}^{\circ }$	4.77	5.12	7.34%

for the six different source location $({15}^{\circ },{30}^{\circ },{45}^{\circ },{60}^{\circ },{75}^{\circ },{90}^{\circ })$. All the experiments are repeated three times and at each source location six different speech mixtures are separated. In order to improve the reliability of results, all the SDR values are the average of separation performance of the algorithms over eighteen different speech mixture.

From Table 4 it is evident that when the SMM is used as a source prior for the IVA algorithm it performs better as compared with the GMM as a source prior. Since speech signals are highly non-stationary in nature and there can be many useful samples in outliers which might not be properly modelled with the Gaussian mixtures but Student’s t mixtures because of its heavy tails can model the outlier information and therefore enhance the separation results of the IVA method. When the SMM is adopted as a source prior for the IVA method, at all the source location azimuths it improves the average separation performance for the IVA method by more than $6\%$ for all the angles as shown in Table 4.

Furthermore, the separation performance is evaluated with the subjective measure of PESQ [41]. This subjective measure compares the original signals and separated signals and gives a score from 0 to 4.5, 0 for the poor separation performance and 4.5 being the excellent separation performance. This measure therefore provides a good estimate about the similarity between the original and separated sources. So the speech mixtures made with BRIRs are separated with the proposed SMM source prior for the EM framework IVA and also with the GMM source prior IVA and the PESQ score is calculated for both the methods. The PESQ score for the IVA method with both source priors is shown in

Table 5. PESQ score for GMM and SMM source prior for the IVA algorithm.

	GMM Source Prior	SMM Source Prior
Set-1	1.85	2.02
Set-2	1.98	2.11
Set-3	1.96	2.13
Set-4	2.02	2.19
Set-5	1.93	2.14
Set-6	2.08	2.21

and the IVA method with SMM source prior consistently has the better PESQ score as compared with the GMM source prior for the IVA algorithm. The PESQ score was generally low in this experiment because of difficult room environment. However, it is evident from the table that when SMM is adapted as a source prior, it improves the separation performance for the IVA method.

Finally, the separation performance of the proposed EM framework for the IVA method with SMM as source prior is compared with the original IVA with identical source priors. BRIRs with $R{T}_{60}$ of 565 ms are used to evaluate the algorithms in highly reverberant environment that can depict the performance of the algorithms in the realistic scenarios. The Same experimental settings are used as in the first experiments and the source location is varied six times from $({15}^{\circ },{30}^{\circ },{45}^{\circ },{60}^{\circ },{75}^{\circ },{90}^{\circ })$. All the measurements are repeated three times and six different speech mixtures are separated at each angle by using IVA method with SMM as source prior and the results are compared with the separation performance of IVA method with multivariate Student’s t distribution as source prior, the IVA method with original multivariate super Gaussian source prior and also with IVA method with the mixed Student’s t and original super Gaussian source prior. This provides an overall comparison of the separation performance of different source prior and the framework for the IVA method. The results in terms of SDR (dB) for six different source location are shown in Figure 6.

It is evident from Figure 6 that the mixture model source prior performs better than the identical distribution source prior at all the source locations. Since different speech sources can have different statistical properties and the mixture model such as SMM source prior can model different sources with different Student’s t distribution in the mixture model while identical source prior model all the sources with the identical distribution and therefore their separation performance suffers as compared to the mixture model source priors.

5. Conclusions

This work presented the EM framework for the IVA method that uses the mixture of Student’s t distribution as a source prior in order to better model the different statistical properties in different speech sources. The mixture of Student’s t distribution source prior made use of the heavy tails nature of the Student’s t distribution to effectively model the high amplitude information in the speech signal. The complete EM framework was derived efficiently to estimate the model parameters for the IVA method. The separation performance for the proposed method was tested with image room impulse method and it confirms the advantage of using the proposed framework for the IVA method as it shows the SDR improvement of approximately 1 dB as compared to the original IVA method. Further experiments were conducted in real room environments with different reverberation times. In varying reverberant environment, new SMIVA consistently performed better as compared to other source priors and provides average SDR improvement of 0.8 dB. In order to further test the new SMIVA algorithm, it was tested with BRIRs and the SMM source prior improves the separation performance as compared to the GMM source prior and in all case, improvement of more than $6\%$ in SDR performance was recorded. This improvement in the separation performance can be further verified and used by implementing the algorithm in real time practical scenarios, which remain the topic for future work in this study. All the simulations performed with different real room environments confirmed that the proposed EM framework for the IVA algorithm that make use of the SMM source prior improves the separation performance even in highly reverberant real room environments.