Unsupervised Learning for Monaural Source Separation Using Maximization–Minimization Algorithm with Time–Frequency Deconvolution ^†

Abstract

This paper presents an unsupervised learning algorithm for sparse nonnegative matrix factor time–frequency deconvolution with optimized fractional $\beta$-divergence. The $\beta$-divergence is a group of cost functions parametrized by a single parameter $\beta$. The Itakura–Saito divergence, Kullback–Leibler divergence and Least Square distance are special cases that correspond to $\beta =0, 1, 2$, respectively. This paper presents a generalized algorithm that uses a flexible range of $\beta$ that includes fractional values. It describes a maximization–minimization (MM) algorithm leading to the development of a fast convergence multiplicative update algorithm with guaranteed convergence. The proposed model operates in the time–frequency domain and decomposes an information-bearing matrix into two-dimensional deconvolution of factor matrices that represent the spectral dictionary and temporal codes. The deconvolution process has been optimized to yield sparse temporal codes through maximizing the likelihood of the observations. The paper also presents a method to estimate the fractional $\beta$ value. The method is demonstrated on separating audio mixtures recorded from a single channel. The paper shows that the extraction of the spectral dictionary and temporal codes is significantly more efficient by using the proposed algorithm and subsequently leads to better source separation performance. Experimental tests and comparisons with other factorization methods have been conducted to verify its efficacy.

1. Introduction

Blind source separation (BSS) [1,2,3,4,5,6,7,8] is an ill-posed problem that cannot be totally solved without some prior information. This entails a certain number of assumptions have to be imposed to render the problem solvable such as channel type (linear [1] versus nonlinear [2]), mutual statistical independence among the sources [3], the number of sources [4], how the sources are mixed (instantaneous [5] versus convolutive [6]), and the location of the sources with respect to the microphones. Several recent solutions have been developed to mitigate some of these constraints. In the work [7], it was previously shown that non-Gaussian stationary process can be approximated as non-stationary Gaussian process which enabled separation involving mixtures of non-Gaussian sources. Of similar concept, a method is proposed for separation by decorrelating multiple non-stationary stochastic sources using a multivariable crosstalk-resistant adaptive noise canceller [8]. In a related method, the problem of speech quality enhancement is tackled using adaptive and non-adaptive filtering algorithms [9]. A two-microphone Gauss–Seidel pseudo affine projection algorithm combined with forward blind source separation is proposed. A higher efficiency in speech enhancement in noisy environment has been attained. The paper [10] proposes rational polynomial functions to replace the original score functions used in standard independent component analysis (ICA). The rational polynomials are derived by the Pade approximant from Taylor series expansion of the original nonlinearities which can be quickly evaluated to enable large-scale multidimensional sets of data characterized by super-Gaussian distribution to be separated within a short period of time. Recently, a bi-variate empirical mode decomposition algorithm combined with complex ICA by entropy bound minimization technique is proposed for convolutive signal separation [11]. In telecommunication problems, neither the direction of arrival (DOA) nor a training sequence is assumed to be available at the receiver. The only assumption is that the transmitted signals satisfy the constant modulus property. In the work [12], a multistage space–time equalizer is proposed to blindly separate signals received by an antenna array from different sources simultaneously. In the algorithm, each stage consists of an adaptive beamformer, a DOA estimator and an equalizer which are jointly optimized using the constant modulus property of the sources. Other than statistical independence and non-Gaussianity, signal separation approach based on second-order statistics of the speech signals using canonical correlation approach [13] has also been proposed. The work [14] considers complex-valued mixing matrix estimation and direction-of-arrival estimation of synchronous orthogonal frequency hopping signals in the underdetermined blind source separation (UBSS). A mixing matrix estimation algorithm is proposed by detecting single source points where only one source contributes its power. While traditional algorithms are usually applied in the ideal sparse environment, the work [15] proposes a solution where multiple input multiple output mixed signals are insufficiently sparse in both time and frequency domains under noisy conditions. The work [16] demonstrates the application of UBSS in addresses the mixing of pipe abrasive debris problem and focuses on the superimposed abrasive debris separation of a radial magnetic field abrasive sensor. Through accurately separating and calculating the morphology and amount of the abrasive debris, the abrasive sensor has provided the system with wear trend and sizes estimation of the wear particles.

In recent years, an alternate class of solutions for BSS based on nonnegative matrix factorization (NMF) [17] has been proposed. Compared to ICA, NMF gives a more part based decomposition and the decomposition is unique under certain conditions, making it unnecessary to impose the constraints in the form of orthogonality and independence [18]. These properties have led to a significant interest in NMF lately for its application in areas of BSS [5,19,20,21,22,23,24], pattern recognition [25], and dimensionality reduction [26]. Multiplicative update-based families of parameterized cost functions such as the Csiszar’s divergences [27,28] were also presented. The NMF is a matrix decomposition technique. Let the data matrix V be a nonnegative matrix of dimensions $I\times J$. The aim of NMF is to find two matrices W and H such that:

$$V=WH$$

(1)

or in scalar form,

$$V_{i,j}={\displaystyle \sum }_k{W}_{i,k}{H}_{k,j}$$

(2)

where i = 1, 2, ..., $k=1, 2,\dots , K$, and $j=1, 2, \dots , J$. When W and H are nonnegative matrices of dimensions $I\times K$ and $K\times J$, then is usually chosen such that

$$I\times K+K\times J\ll I\times J$$

(3)

A sparseness constraint can be added to the cost function [26,27,28,29,30,31], and this can be achieved by regularization using the L₁-norm leading to Sparse NMF (SNMF). Here, “sparseness” refers to a representational scheme where only a few units (out of a large population) are effectively used to represent typical data vectors. In effect, this implies most units taking values close to zero while only few take significantly non-zero values. Several other types of prior distribution over W and H can be defined, e.g., it is assumed that the prior of W and H satisfy the exponential density and the prior for the noise variance is chosen as an inverse gamma density [27]. In the work [28], Gaussian distributions are chosen for both W and H. The model parameters and hyper parameters are adapted by using the Markov chain Monte Carlo (MCMC) [32]. In all cases, a fully Bayesian treatment is applied to approximate inference for both model parameters and hyper parameters. While these approaches increase the accuracy of matrix factorization, it only works efficiently when a large sample dataset is available. Moreover, it consumes significantly high computational complexity at each iteration to adapt the parameters and its hyper parameters. The NMF with the β-divergence has been previously used in music signal processing [33,34]. In our previous paper [35], we investigated β-divergence for source separation problem. It was shown that improved performance has been attained over integer-based β-divergence. Thus, this motivates research of using β-divergence for music signal processing and source separation. However, all of these works fixed $\beta$ to some constant values within 0–2, and have not presented any method to determine the desired $\beta$ value. This significantly constrains the performance of matrix factorization and its ability in separating mixed sources. In addition, these works do not consider the issue of sparsity of the temporal codes which would undermine the quality of matrix factorization when the $\beta$ value is inappropriately chosen. The selection of the $\beta$ value should consider the sparseness constraint used in the cost function.

Regardless of the cost function and sparseness constraint being used, the standard NMF or SNMF models are only satisfactory for solving source separation provided that the spectral frequencies of the analyzed audio signal do not change over time. However, this is not the case for many realistic signals such as music and speech. As a result, the spectral dictionary obtained via the NMF or SNMF decomposition is not adequate to capture the temporal dependency of the frequency patterns within the signal. To remedy the situation, a pragmatic approach is to work on a more holistic model based on matrix factor deconvolution [21,22,23,24]. In this paper, we work with NMF model extended to two-dimensional time–frequency deconvolution of W and H where (W, H) are considered as the matrix factors [22]. Mathematically, this is expressed as

$$\begin{gathered} {\mathrm{V}}_{i,j}={\displaystyle \sum }_{k=1}^K{\displaystyle \sum }_{\tau =0}^{{\tau }_{max}}{\displaystyle \sum }_{\varphi =0}^{{\varphi }_{max}}{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi } \\ V={\displaystyle \sum }_{\tau =0}^{{\tau }_{max}}{\displaystyle \sum }_{\varphi =0}^{{\varphi }_{max}}\stackrel{\downarrow \varphi }{W^{\tau }}\stackrel{\to \tau }{H^{\varphi }} \end{gathered}$$

(4)

where $i$ and $i$ represent the frequency and time index, respectively, $k$ indicates the factor number, $\tau$ represents the temporal shift and $\varphi$ is the frequency shift. The terms ${\tau }_{max}$ and ${\varphi }_{max}$ are the maximum temporal and frequency shift, respectively. With this definition, both $W_{i,k}^{\tau }$ and $H_{k,j}^{\varphi }$ have tensorial structures with dimension $I\times K\times {\tau }_{max}$ and $K\times J\times {\varphi }_{max}$, respectively. Thus, $W_{i,k}^{\tau }$ represents the $\tau \mathrm{th}$-slice of the $k\mathrm{th}$-spectral basis while $H_{k,j}^{\varphi }$ represents the associated $\varphi \mathrm{th}$-slice of the $k\mathrm{th}$-temporal code. The downward and rightward arrow signs denote the corresponding shifting direction of each column in $W^{\tau }$ and each row in $H^{\varphi }$ by the amount indicated by $\tau$ and $\varphi$, respectively.

Model (4) represents both temporal structure and the pitch change which occur when an instrument plays different notes. In the log-frequency spectrogram, the pitch change corresponds to a displacement on the frequency axis. Where previous NMF methods needed one component to model each note for each instrument, Model (4) represents each instrument compactly by a single time–frequency profile convolved in both time and frequency by a time–pitch weight matrix. This model dramatically decreases the number of components needed to model various instruments and effectively solves the blind single channel source separation problem for certain classes of musical signals. When polyphonic music is modeled by factorizing the magnitude spectrogram with NMF, each instrument is modeled by an instantaneous frequency signature which can vary over time. However, the NMF requires multiple basis functions to represent tones with different pitch values. The two-dimensional time–frequency deconvolution model implicitly solves the problem of grouping notes. Thus, all notes for an instrument is an identical pitch shifted time–frequency signature, Model (4) will give better estimates of these signatures, because more examples of different notes are used to compute each time–frequency signature. In the event when this assumption does not hold, it might still hold in a region of notes for an instrument. Furthermore, the two-dimensional time–frequency deconvolution model can explain the spectral differences between two notes of different pitch by the two-dimensional deconvolution of the time–frequency signature.

The novelty of this paper can be summarized as follows: Firstly, a new algorithm is developed for sparse nonnegative matrix factor time–frequency deconvolution optimized with fractional β-divergence. Secondly, the maximization–minimization algorithm is developed to derive the auxiliary cost function which caters for any $\beta$ value. The paper shows that the optimal $\beta$ that leads to the desired performance is not necessarily limited to the special cases of integer $\beta$ but extends to fractional values. Thirdly, it is analytically shown that the convergence of the proposed algorithm is guaranteed under the auxiliary function. Fourthly, a method is proposed to estimate the fractional $\beta$ within the context of monoaural source separation. Finally, the paper proposes an adaptive method to estimate the sparsity parameter for each of the individual temporal code.

The remainder of the paper is organized as follows: In Section 2, the new algorithm for matrix factor time–frequency deconvolution model with β-divergence based on the maximization–minimization algorithmic framework is derived. Real application of blind source separation using the proposed method and comparisons with other matrix factorization methods are presented in Section 3. Finally, Section 4 concludes the paper.

2. Background

2.1. $\beta$-Divergence Cost Function

The NMF problem can be written as the minimization of an objective function:

$$\mathrm{D}\left(V|WH\right)={\displaystyle \sum }_{i,j}{d}_{\beta }\left(V_{i,j}{|\mathsf{\Lambda }}_{i,j}\right)$$

(5)

The general β-divergence [24,31] is defined as:

$$d_{\beta }(y|x)=\{\begin{array}{cc}\frac{y^{\beta }}{\beta (\beta -1)}+\frac{x^{\beta }}{\beta }-\frac{y{x}^{\beta -1}}{\beta -1},\hfill & \hfill \beta \to \mathbb{R}/\{0,1\}\\ y(\log y-\log x)+(x-y),\hfill & \hfill \beta =1\\ \frac{y}{x}-\log \frac{y}{x}-1\hfill & \hfill \beta =0\end{array}$$

(6)

when $\beta =2$, this matches with the first β-divergence and the update algorithm is referred to as the “Least Square” [17]. When we use the second β-divergence with $\beta =1$, the update algorithm is referred to as the “Kullback–Leibler” [17]. When the third β-divergence with $\beta =0$ is used, the update algorithm is referred to as the “Itakura–Saito” [33]. These algorithms have their own advantages and disadvantages. If the sources have large dynamic difference in the power, the Itakura–Saito divergence would have better performance than other NMF algorithms. The Least Square and Kullback–Leibler NMFs are more suited when the power of sources are close to other. However, it is difficult to define the difference of power between the sources, and therefore it is difficult to choose the algorithms. In this paper, we present the results to show that the best results are not necessarily limited to the above integer $\beta$ special cases. The use the fractional β-divergence is expected to yield more realistic and optimized results than the previous NMF algorithms. For completeness of presentation, the following section briefly reviews the update function based on the Least Square and Kullback–Leibler criterion.

2.1.1. Least Square Distance

The Least Square NMF algorithm introduced by Lee and Seung [17] defines the β-divergence as Least Squares divergence when $\beta =2$. First, we consider the least square cost function:

$$\begin{array}{cc}{C}_{LS}=\frac{1}{2}||V-{\mathsf{\Lambda }||}_F^2& ={\displaystyle {\displaystyle \sum }_{i,j}}{d}_2(V_{i,j}{|\mathsf{\Lambda }}_{i,j})\hfill \\ & =\frac{1}{2}{\displaystyle {\displaystyle \sum }_{i,j}}{\left(V_{i,j}-{\mathsf{\Lambda }}_{i,j}\right)}^2\hfill \end{array}$$

(7)

Differentiating $C_{LS}$ with respect to $W_{i,k}^{\tau }$ and $H_{k,j}^{\varphi }$, and plugging the multiplicative update algorithm for $\theta =\left\{{(W_{i,k}^{\tau })}_{I\times K\times {\tau }_{max}},{(H_{k,j}^{\varphi })}_{K\times J\times {\varphi }_{max}}\right\}$:

$$\theta \leftarrow \theta ·\left(\frac{{\left[\nabla d_{\beta }\left(y|x\right)\right]}_{-}}{{\left[\nabla d_{\beta }\left(y|x\right)\right]}_{+}}\right)$$

where $\partial d_{\beta }\left(y|x\right)/\partial \theta ={\left[\nabla d_{\beta }\left(y|x\right)\right]}_{+}-{\left[\nabla d_{\beta }\left(y|x\right)\right]}_{-}$, which leads to the following $W^{\tau }$ and $H^{\varphi }$ updates:

$$W^{\tau }=W^{\tau }·\frac{{\sum }_{\varphi }\stackrel{\uparrow \varphi \leftarrow \tau }{V}·{\stackrel{\to \tau }{H^{\varphi }}}^T}{{\sum }_{\varphi }\stackrel{\uparrow \varphi \leftarrow \tau }{\mathsf{\Lambda }}·{\stackrel{\to \tau }{H^{\varphi }}}^T}$$

(8)

$$H^{\varphi }=H^{\varphi }·\frac{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{W^{\tau }}}^T·\stackrel{\uparrow \varphi \leftarrow \tau }{V}}{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{W^{\tau }}}^T·\stackrel{\uparrow \varphi \leftarrow \tau }{\mathsf{\Lambda }}}$$

(9)

where “$A·B$” represents element-wise multiplication.

2.1.2. Kullback–Liebler Divergence

When $\beta =1$, the β-divergence is identical to the Kullback–Leibler divergence. The Kullback–Leibler divergence is expressed as:

$$C_{KL}={\displaystyle \sum }_{i,j}{d}_1\left(V_{i,j}|{\mathsf{\Lambda }}_{i,j}\right)={\displaystyle \sum }_{i,j}{V}_{i,j}\log \frac{V_{i,j}}{{\mathsf{\Lambda }}_{i,j}}-V_{i,j}+{\mathsf{\Lambda }}_{i,j}$$

(10)

By following similar steps as the Least Square, we can derive the update function as follow:

$$W^{\tau }=W^{\tau }·\frac{{\sum }_{\varphi }\left(\frac{\stackrel{\uparrow \varphi \leftarrow \tau }{V}}{\mathsf{\Lambda }}\right)·{\stackrel{\to \tau }{H^{\varphi }}}^T}{{\sum }_{\varphi }1·{\varphi }^T}$$

(11)

$$H^{\varphi }=H^{\varphi }·\frac{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{W^{\tau }}}^T·\left(\frac{\stackrel{\uparrow \varphi \leftarrow \tau }{V}}{\mathsf{\Lambda }}\right)}{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{W^{\tau }}}^T·1}$$

(12)

where “$\frac{A}{B}$” represents element-wise division and “1” is a column vector of unit elements.

2.2. Auxiliary Cost Function of Fractional β-Divergence for Matrix Factors Time–Frequency Deconvolution

In this subsection, we introduce the cost function for the fractional $\beta$-divergence matrix factors time–frequency deconvolution model. The algorithm allows the user to choose a fractional β value instead of using the previous NMF algorithms which constrain β to special cases of integer value. After the derivation, this paper shows the steps on how the update function of the fractional β-divergence is obtained for the parameters. Firstly, the first derivative of $d_{\beta }\left(y|x\right)$ are given by

$$d_{\beta }^{\prime \left(y|x\right)}=y^{\beta -2}\left(y-x\right)$$

(13)

This shows that $y$ is continuous in $\beta$ and thus the second derivative of $d_{\beta }\left(y|x\right)$ is given by

$$d_{\beta }^{″\left(y|x\right)}=y^{\beta -3}\left[\left(\beta -1\right)y-\left(\beta -2\right)x\right]$$

(14)

The second derivative shows that the β-divergence is convex for $y$ in $\beta \in \left[1,2\right]$. Outside of this range, $d_{\beta }\left(y|x\right)$ can be expressed as:

$$d_{\beta }\left(y|x\right)=\check{d}\left(y|x\right)+\widehat{d}\left(y|x\right)+\overline{d}\left(x\right)$$

(15)

where $\check{d}\left(y|x\right)$ is a convex function of $y$, $\widehat{d}\left(y|x\right)$ is a concave function of y, and $\overline{d}\left(x\right)$ is a constant of y.

Table 1. Differentiable Convex-Concave-Constant Decomposition of $\beta$-Divergence.

Range	$\check{\mathit{d}}\left(\mathit{y}|\mathit{x}\right)$	$\widehat{\mathit{d}}\left(\mathit{y}|\mathit{x}\right)$	$\overline{\mathit{d}}\left(\mathit{x}\right)$
$\beta <1$ and $\beta \ne 0$	$-\frac{1}{\beta -1}x{y}^{\beta -1}$	$\frac{1}{\beta }{y}^{\beta }$	$y^{\beta -1}$
$\beta =0$	$x{y}^{\beta -1}$	$\log y$	$y^{-1}$
$1\le \beta \le 2$	$d_{\beta }\left(x|y\right)$	$0$	$0$
$\beta >2$	$\frac{1}{\beta }{y}^{\beta }$	$-\frac{1}{\beta -1}x{y}^{\beta -1}$	$-x{y}^{\beta -2}$

shows the various functions for $\check{d}\left(y|x\right)$, $\widehat{d}\left(y|x\right)$ and $\overline{d}\left(x\right)$. The problem we want to tackle is to minimize the following function with respect to $\theta =\left\{{\left(W_{i,k}^{\tau }\right)}_{I\times K\times {\tau }_{max}},{\left(H_{k,j}^{\varphi }\right)}_{K\times J\times {\varphi }_{max}}\right\}$ where $\beta$ can assume fractional number:

$$\begin{array}{ll}G\left(\theta \right)& ={\displaystyle {\displaystyle \sum }_{i,j}}{d}_{\beta }\left(V_{i,j}|\sum _{k=1}^K\sum _{\tau =0}^{{\tau }_{max}}\sum _{\varphi =0}^{{\varphi }_{max}}{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }\right)\\ & =\frac{1}{\beta \left(\beta -1\right)}{\displaystyle {\displaystyle \sum }_{i,j}}{V}_{i,j}^{\beta }+{\displaystyle {\displaystyle \sum }_{i,j}}\underset{G_1\left(\beta \right)}{\underbrace{\frac{1}{\beta }{\left({\displaystyle {\displaystyle \sum }_{k,\tau \varphi }}{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }\right)}^{\beta }}}+{\displaystyle {\displaystyle \sum }_{i,j}}{V}_{i,j}\underset{G_2\left(\beta \right)}{\underbrace{{\left(-\frac{1}{\beta -1}{\displaystyle {\displaystyle \sum }_{k,\tau \varphi }}{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }\right)}^{\beta -1}}}\end{array}$$

(16)

In Equation (16), $G_1\left(\beta \right)$ is convex for $\beta \ge 1$ and concave for $\beta <1$, and $G_2\left(\beta \right)$ is convex for $\beta \le 2$ and concave for $\beta >2$. Thus, there is a need to alleviate this problem by decomposing the above function into several terms to be either convex or concave depending on the value of $\beta$ and use the appropriate inequalities to build an auxiliary function.

Lemma 1.

For the case of $\beta \ge 1$, we have

$$\frac{1}{\beta }{\left({\displaystyle \sum }_{k,\tau \varphi }{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }\right)}^{\beta }\le \frac{1}{\beta }{\displaystyle \sum }_{k,\tau \varphi }{\omega }_{i,j,k,\tau ,\varphi }{\left(\frac{W_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }}{{\omega }_{i,j,k,\tau ,\varphi }}\right)}^{\beta }=P_{i,j}^{\left(\beta \right)}$$

(17)

where ${\omega }_{i,j,k,\tau ,\varphi }\ge 0$ for all $k,\tau ,\varphi$ and $\sum _{k,\tau \varphi }{\omega }_{i,j,k,\tau ,\varphi }=1$. The equality holds when

$${\omega }_{i,j,k,\tau ,\varphi }=\frac{W_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }}{{\sum }_{k^{\prime },{\tau }^{\prime },{\varphi }^{\prime }}{W}_{i-{\varphi }^{\prime },k^{\prime }}^{{\tau }^{\prime }}{H}_{k^{\prime },j-{\tau }^{\prime }}^{{\varphi }^{\prime }}}$$

(18)

Proof. 

Let $f:\mathbb{R}\to \mathbb{R}$ be a convex function. If ${\alpha }_k\left(k=1, 2,\dots ,K\right)$ satisfies $\forall k,{\alpha }_k>0$ and $\sum _k{\alpha }_k=1$, then for any $x_k\left(k=1, 2,\dots ,K\right)\in \mathbb{R}$,

$$f\left({\displaystyle \sum }_k{x}_k\right)\le {\displaystyle \sum }_k{\alpha }_{k}f\left(\frac{x_k}{{\alpha }_k}\right)$$

and with equality holds if and only if ${\alpha }_k=x_k/\sum _k{x}_k$. Substituting $f(·)=\frac{1}{\beta }{\left(·\right)}^{\beta }$ with $\beta \ge 1$, $x_k=W_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }$ and ${\alpha }_k={\omega }_{i,j,k,\tau ,\varphi }$ yields Equation (16). ☐

Lemma 2.

For the case of $\beta <1$, we have

$$\frac{1}{\beta }{\left({\displaystyle \sum }_{k,\tau \varphi }{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }\right)}^{\beta }\le {\mathsf{\Lambda }}_{i,j}^{\beta -1}\left({\displaystyle \sum }_{k,\tau \varphi }{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }-{\mathsf{\Lambda }}_{i,j}\right)+\frac{{\mathsf{\Lambda }}_{i,j}^{\beta }}{\beta }=Q_{i,j}^{\left(\beta \right)}$$

(19)

The equality holds when

$${\mathsf{\Lambda }}_{i,j}={\displaystyle \sum }_{k,\tau \varphi }{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }$$

(20)

Proof. 

Let $f:\mathbb{R}\to \mathbb{R}$ be a continuously differentiable and concave function. Then, for any point $z$,

$$f\left(x\right)\le f^{\prime }\left(x\right)\left(x-z\right)+f\left(z\right)$$

and with equality holds if and only if $x=z$. Substituting $f\left(·\right)=\frac{1}{\beta }{\left(·\right)}^{\beta }$ with $\beta <1$, $x=\sum _{k,\tau \varphi }{W}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }$ and $z={\mathsf{\Lambda }}_{i,j}$ yields Equation (17). ☐

Using Lemmas 1 and 2, we may proceed with the following analysis. When $\beta <1$, we use $Q_{i,j}^{\left(\beta \right)}$ instead of $G_1\left(\beta \right)$ and $V_{i,j}{P}_{i,j}^{\left(\beta -1\right)}$ instead of $G_2\left(\beta \right)$, then the cost function becomes a convex function. Let us denote $G^{+}(\theta |\widehat{\theta })$ as an auxiliary function for and $\widehat{\theta }=\left\{{\left({\omega }_{i,j,k,\tau ,\varphi }\right)}_{I\times J\times K\times {\tau }_{max}\times {\varphi }_{max}},{\left({\mathsf{\Lambda }}_{i,j}\right)}_{I\times J}\right\}$ as the auxiliary variables. For $G^{+}(\theta |\widehat{\theta })$ to qualify as auxiliary function, it must satisfy $G\left(\theta \right)=\underset{\widehat{\theta }}{\min }{G}^{+}(\theta |\widehat{\theta })$. Thus, the cost function can be shown to be bounded by the auxiliary function $G^{+}(\theta |\widehat{\theta })$:

$$G\left(\theta \right)\le G^{+}(\theta |\widehat{\theta })={\displaystyle \sum }_{i,j}\frac{V_{i,j}^{\beta }}{\beta \left(\beta -1\right)}+Q_{i,j}^{\left(\beta \right)}-V_{i,j}{P}_{i,j}^{\left(\beta -1\right)}$$

(21)

when $1\le \beta \le 2$, we use $P_{i,j}^{\left(\beta \right)}$ instead of $G_1\left(\beta \right)$ and $V_{i,j}{P}_{i,j}^{\left(\beta -1\right)}$ instead of $G_2\left(\beta \right)$, then the cost function becomes a convex function and is bounded by the auxiliary function $G^{+}(\theta |\widehat{\theta })$:

$$G\left(\theta \right)\le G^{+}(\theta |\widehat{\theta })={\displaystyle \sum }_{i,j}\frac{V_{i,j}^{\beta }}{\beta \left(\beta -1\right)}+P_{i,j}^{\left(\beta \right)}-V_{i,j}{P}_{i,j}^{\left(\beta -1\right)}$$

(22)

Finally, when $\beta >2$, we use $P_{i,j}^{\left(\beta \right)}$ instead of $G_1\left(\beta \right)$ and $V_{i,j}{Q}_{i,j}^{\left(\beta -1\right)}$ instead of $G_2\left(\beta \right)$, then the cost function is bounded by

$$G\left(\theta \right)\le G^{+}(\theta |\widehat{\theta })={\displaystyle \sum }_{i,j}\frac{V_{i,j}^{\beta }}{\beta \left(\beta -1\right)}+P_{i,j}^{\left(\beta \right)}-V_{i,j}{Q}_{i,j}^{\left(\beta -1\right)}$$

(23)

From above, we can conclude that

$$G\left(\theta \right)\le G^{+}\left(\theta |\widehat{\theta }\right)={\displaystyle \sum }_{i,j}\frac{V_{i,j}^{\beta }}{\beta \left(\beta -1\right)}+\{\begin{array}{cc}{Q}_{i,j}^{\left(\beta \right)}-V_{i,j}{P}_{i,j}^{\left(\beta -1\right)},\hfill & \hfill (\beta <1)\\ P_{i,j}^{\left(\beta \right)}-V_{i,j}{P}_{i,j}^{\left(\beta -1\right)},\hfill & \hfill \left(1\le \beta \le 2\right)\\ P_{i,j}^{\left(\beta \right)}-V_{i,j}{Q}_{i,j}^{\left(\beta -1\right)},\hfill & \hfill (\beta >2)\end{array}$$

(24)

The equality holds when $\widehat{\theta }$ satisfies Equations (18) and (20). The above function yields three different sub-functions which depend on the $\beta$ value. In different $\beta$ range, we use different cost function in the algorithm. This allows the user to choose the optimal $\beta$ value to separate the mixture and caters for more flexibility than the previous algorithms.

2.3. Auxiliary Update Function of “Fractional” β-Divergence

To minimize $G^{+}(\theta |\widehat{\theta })$, we formulate the derivative of $G^{+}(\theta |\widehat{\theta })$ with respect to $\theta$. First, we consider the derivative for $W_{i,k}^{\tau }$:

$$\frac{\partial G^{+}(\theta |\widehat{\theta })}{\partial W_{i,k}^{\tau }}={\mathcal{V}}_{\mathrm{w}}-{\mathcal{W}}_{\mathrm{w}}$$

(25)

where

$${\mathcal{V}}_{\mathrm{w}}=\{\begin{array}{ll}{\displaystyle {\displaystyle \sum }_{j,\varphi }}{\mathsf{\Lambda }}_{i,j}^{\beta -1}{H}_{k,j-\tau }^{\varphi },& (\beta <1)\\ {\left(W_{i,k}^{\tau }\right)}^{\beta -1}{\displaystyle {\displaystyle \sum }_{j,\varphi }}{\omega }_{i+\varphi ,j,k,\tau ,\varphi }^{1-\beta }{\left(H_{k,j-\tau }^{\varphi }\right)}^{\beta },& \left(\beta \ge 1\right)\end{array}$$

(26)

$${\mathcal{W}}_{\mathrm{w}}=\{\begin{array}{cc}{(W_{i,k}^{\tau })}^{\beta -2}{\displaystyle {\displaystyle \sum }_{j,\varphi }}{V}_{i+\varphi ,j}{\omega }_{i+\varphi ,j,k,\tau ,\varphi }^{2-\beta }{\left(H_{k,j-\tau }^{\varphi }\right)}^{\beta -1}\hfill & \hfill \left(\beta \le 2\right)\\ {\displaystyle {\displaystyle \sum }_{j,\varphi }}{V}_{i+\varphi ,j}{\mathsf{\Lambda }}_{i+\varphi ,j}^{\beta -2}{H}_{k,j-\tau }^{\varphi } ,\hfill & \hfill (\beta >2)\end{array}$$

(27)

The second derivative of $G^{+}(\theta |\widehat{\theta })$ with respect to $W_{i,k}^{\tau }$ in then expressed as:

$$\frac{{\partial }^2{G}^{+}(\theta |\widehat{\theta })}{\partial W_{i,k}^{\tau }\partial W_{i^{\prime },k^{\prime }}^{{\tau }^{\prime }}}=({\mathcal{V}}_w{}^{\prime }-{\mathcal{W}}_w{}^{\prime }){\delta }_{i,i^{\prime }}{\delta }_{k,k^{\prime }}{\delta }_{\tau ,{\tau }^{\prime }}$$

(28)

where ${\delta }_{i,j}=1$ if $i=j$ and ${\delta }_{i,j}=0$ if $i\ne j$, and

$${\mathcal{V}}_{\mathrm{w}}^{\prime }=\{\begin{array}{cc}0,\hfill & \hfill (\beta <1)\\ (\beta -1){\left(W_{i-\varphi ,k}^{\tau }\right)}^{\beta -2}{\displaystyle {\displaystyle \sum }_{j,\varphi }}{\omega }_{i+\varphi ,j,k,\tau ,\varphi }^{1-\beta }{(H_{k,j-\tau }^{\varphi })}^{\beta } , \left(\beta \ge 1\right)\hfill & \hfill (\beta \ge 1)\end{array}$$

(29)

$${\mathcal{W}}_{\mathrm{w}}^{\prime }=\{\begin{array}{cc}\left(\beta -2\right){(W_{i,k}^{\tau })}^{\beta -3}{\displaystyle {\displaystyle \sum }_{j,\varphi }}{V}_{i+\varphi ,j}{\omega }_{i+\varphi ,j,k,\tau ,\varphi }^{2-\beta }{(H_{k,j-\tau }^{\varphi })}^{\beta -1},\hfill & \hfill \left(\beta \le 2\right)\\ 0,\hfill & \hfill (\beta >2)\end{array}$$

(30)

We can see $G(\theta |\widehat{\theta })$ is a convex function in $W_{i,k}^{\tau }$, so by setting $\frac{\partial G\left(\theta |\widehat{\theta }\right)}{\partial W_{i,k}}$ to 0, we can then express the update function for $W_{i,k}^{\tau }$ as:

$$W_{i,k}^{\tau }=\{\begin{array}{cc}{\left(\frac{{\sum }_{j,\varphi }{V}_{i+\varphi ,j} {\omega }_{i+\varphi ,j,k,\tau ,\varphi }^{2-\beta }{\left(H_{k,j-\tau }^{\varphi }\right)}^{\beta -1}}{{\sum }_{j,\varphi }{\mathsf{\Lambda }}_{i,j}^{\beta -1}{H}_{k,j-\tau }^{\varphi }}\right)}^{\frac{1}{2-\beta }} ,\hfill & \hfill (\beta <1)\\ \frac{{\sum }_{j,\varphi }{V}_{i+\varphi ,j} {\omega }_{i+\varphi ,j,k,\tau ,\varphi }^{2-\beta }{\left(H_{k,j-\tau }^{\varphi }\right)}^{\beta -1}}{{\sum }_{j,\varphi }{\omega }_{i+\varphi ,j,k,\tau ,\varphi }^{1-\beta }{\left(H_{k,j-\tau }^{\varphi }\right)}^{\beta }} ,\hfill & \hfill \left(1\le \beta \le 2\right)\\ {\left(\frac{{\sum }_{j,\varphi }{V}_{i+\varphi ,j} {\mathsf{\Lambda }}_{i+\varphi ,j}^{\beta -2} H_{k,j-\tau }^{\varphi }}{{\sum }_{j,\varphi }{\omega }_{i+\varphi ,j,k,\tau ,\varphi }^{1-\beta }{\left(H_{k,j-\tau }^{\varphi }\right)}^{\beta }}\right)}^{\frac{1}{\beta -1}} ,\hfill & \hfill (\beta >2)\end{array}$$

(31)

We next consider the auxiliary variables $\widehat{\theta }$. Since both Equations (17) and (18) minimize $G^{+}(\theta |\widehat{\theta })$ with respect to $\widehat{\theta }$, substituting these into Equation (30) gives the following update rule:

$$W_{i,k}^{\tau }=W_{i,k}^{\tau }{\left(\frac{{\sum }_{j,\varphi }{V}_{i+\varphi ,j} {\mathsf{\Lambda }}_{i+\varphi ,j}^{\beta -2} H_{k,j-\tau }^{\varphi }}{{\sum }_{j,\varphi }{\mathsf{\Lambda }}_{i+\varphi ,j}^{\beta -1} H_{k,j-\tau }^{\varphi }}\right)}^{\delta \left(\beta \right)}$$

(32)

where

$$\delta \left(\beta \right)=\{\begin{array}{cc}\frac{1}{2-\beta },\hfill & \hfill \left(\beta <1\right)\\ 1,\hfill & \hfill \left(1\le \beta \le 2\right)\\ \frac{1}{\beta -1},\hfill & \hfill \left(\beta >2\right)\end{array}$$

(33)

The above can be written in the matrix form:

$$W^{\tau }=W^{\tau }·{\left[\frac{{\sum }_{\varphi }\left(\stackrel{\uparrow \varphi }{V}·\stackrel{\uparrow \varphi }{{\mathsf{\Lambda }}^{\left(\beta -2\right)}}\right){\stackrel{\to \tau }{H^{\varphi }}}^T}{{\sum }_{\varphi }\stackrel{\uparrow \varphi }{{\mathsf{\Lambda }}^{\left(\beta -1\right)}}{\stackrel{\to \tau }{H^{\varphi }}}^T}\right]}^{\delta \left(\beta \right)}$$

(34)

Similarly, for $H_{k,j}^{\varphi }$ update function, first we have

$$\frac{\partial G^{+}(\theta |\widehat{\theta })}{\partial H_{k,j}^{\varphi }}={\mathcal{V}}_{\mathrm{H}}-{\mathcal{W}}_{\mathrm{H}}$$

(35)

where

$${\mathcal{V}}_H=\{\begin{array}{ll}{\displaystyle {\displaystyle \sum }_{i,\tau }}{\mathsf{\Lambda }}_{i,j+\tau }^{\beta -1} W_{i-\varphi ,k}^{\tau } ,& (\beta <1)\\ {(H_{k,j}^{\varphi })}^{\beta -1}{\displaystyle {\displaystyle \sum }_{i,\tau }}{\omega }_{i,j+\tau ,k,\tau ,\varphi }^{1-\beta }{(W_{i-\varphi ,k}^{\tau })}^{\beta } ,\hfill & \left(\beta \ge 1\right)\end{array}$$

(36)

$${\mathcal{W}}_H=\{\begin{array}{cc}{\left(H_{k,j}^{\varphi }\right)}^{\beta -2}{\displaystyle {\displaystyle \sum }_{i,\tau }}{V}_{i,j+\tau } {\omega }_{i,j+\tau ,k,\tau ,\varphi }^{2-\beta }{(W_{i-\varphi ,k}^{\tau })}^{\beta -1} ,\hfill & \hfill \left(\beta \le 2\right)\\ {\displaystyle {\displaystyle \sum }_{i,\tau }}{V}_{i,j+\tau }{\mathsf{\Lambda }}_{i,j+\tau }^{\beta -2}{W}_{i-\varphi ,k}^{\tau } ,\hfill & \hfill (\beta >2)\end{array}$$

(37)

From Equations (17)–(31), we can minimize the cost-function by setting $\frac{\partial G_s\left(\theta |\widehat{\theta }\right)}{\partial H_{k,j}^{\phi }}=0$ and obtain $H_{k,j}^{\varphi }$ as:

$$H_{k,j}^{\varphi }=\{\begin{array}{cc}{\left(\frac{{\sum }_{i,\tau }{V}_{i,j+\tau } {\omega }_{i,j+\tau ,k,\tau ,\varphi }^{2-\beta }{\left(W_{i-\varphi ,k}^{\tau }\right)}^{\beta -1}}{{\sum }_{i,\tau }{\mathsf{\Lambda }}_{i,j+\tau }^{\beta -1} W_{i-\varphi ,k}^{\tau }}\right)}^{\frac{1}{2-\beta }}\hfill & \hfill (\beta <1)\\ \frac{{\sum }_{i,\tau }{V}_{i,j+\tau } {\omega }_{i,j+\tau ,k,\tau ,\varphi }^{2-\beta }{\left(W_{i-\varphi ,k}^{\tau }\right)}^{\beta -1}}{{\sum }_{i,\tau }{\omega }_{i,j+\tau ,k,\tau ,\varphi }^{1-\beta }{\left(W_{i-\varphi ,k}^{\tau }\right)}^{\beta }}\hfill & \hfill \left(1\le \beta \le 2\right)\\ {\left(\frac{{\sum }_{i,\tau }{V}_{i,j+\tau }{\mathsf{\Lambda }}_{i,j+\tau }^{\beta -2}{W}_{i-\varphi ,k}^{\tau }}{{\sum }_{i,\tau }{\omega }_{i,j+\tau ,k,\tau ,\varphi }^{1-\beta }{\left(W_{i-\varphi ,k}^{\tau }\right)}^{\beta }}\right)}^{\frac{1}{\beta -1}},\hfill & \hfill (\beta >2)\end{array}$$

(38)

Again, since both Equations (19) and (20) minimizes $G^{+}(\theta |\widehat{\theta })$ with respect to $\widehat{\theta }$, substituting these into Equation (38) gives the following update rule for $H_{k,j}^{\varphi }$:

$$H_{k,j}^{\varphi }=H_{k,j}^{\varphi }{\left(\frac{{\sum }_{i,\tau }{V}_{i,j+\tau } {\mathsf{\Lambda }}_{i,j+\tau }^{\beta -2} W_{i-\varphi ,k}^{\tau }}{{\sum }_{i,\tau }{\mathsf{\Lambda }}_{i,j+\tau }^{\beta -1} W_{i-\varphi ,k}^{\tau }}\right)}^{\delta \left(\beta \right)}$$

(39)

In matrix form, the above can be written as

$$H^{\varphi }=H^{\varphi }·{\left[\frac{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{W^{\tau }}}^T\left(\stackrel{\leftarrow \tau }{V}·\stackrel{\leftarrow \tau }{{\mathsf{\Lambda }}^{\left(\beta -2\right)}}\right)}{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{W^{\tau }}}^T\stackrel{\leftarrow \tau }{{\mathsf{\Lambda }}^{\left(\beta -1\right)}}}\right]}^{\delta \left(\beta \right)}$$

(40)

2.4. Sparsity-Aware Optimization

The cost-function in Equation (21) can be augmented with a regularization term to render sparsity to the solution. We can define a prior on $H$ as an exponential distribution with independent decay parameters, namely,

$$p\left(H|\lambda \right)={\displaystyle \prod }_{\varphi }p\left(H^{\varphi }|{\mathsf{\lambda }}^{\varphi }\right)={\displaystyle \prod }_{\varphi }{\displaystyle \prod }_k{\displaystyle \prod }_{j}p\left(H_{k,j}^{\varphi }|{\lambda }_{k,j}^{\varphi }\right)$$

(41)

where $p\left(H_{k,j}^{\varphi }|{\lambda }_{k,j}^{\varphi }\right)=\prod _{\varphi }\prod _k\prod _j{\lambda }_{k,j}^{\varphi }\exp \left(-{\lambda }_{k,j}^{\varphi }{H}_{k,j}^{\varphi }\right)$. The negative log prior on $H$ is defined as $-\log p\left(H|\mathsf{\lambda }\right)=f\left(H\right)=\sum _{\varphi ,k,j}\left({\lambda }_{k,j}^{\varphi }{H}_{k,j}^{\varphi }-\log {\lambda }_{k,j}^{\varphi }\right)$. It is worth pointing out that each individual element in $H$ is constrained to an exponential distribution with independent decay parameter ${\lambda }_{k,j}^{\varphi }$ so that each element in $H$ can be driven to be optimally sparse in the $L_1$-norm. Other forms of sparseness exist¹⁹ but the proposed $L_1$-norm is computationally favourable. First, we define $G_s\left(\theta \right)$ and $G_s(\theta |\widehat{\theta })$ as follow:

$$\begin{array}{ll}{G}_s\left(\theta \right)& \triangleq G\left(\theta \right)+\alpha f\left(H\right)\\ & \le G^{+}(\theta |\widehat{\theta })+\alpha f\left(H\right)\\ & =G_s(\theta |\widehat{\theta })\end{array}$$

(42)

where $\alpha$ is the regularization constant. To avoid the scaling misbehavior when incorporating the sparseness for $H$, we reformulate the cost function to work with normalized matrix for $W^{\tau }$ i.e.,

$${\overline{W}}_{i,k}^{\tau }=\frac{W_{i,k}^{\tau }}{\sqrt{{\sum }_{\tau ,i}{\left(W_{i,k}^{\tau }\right)}^2}}=\frac{W_{i,k}^{\tau }}{||W_k{||}_2}$$

(43)

and

$${\overline{\mathsf{\Lambda }}}_{i,j}={\displaystyle \sum }_{k,\tau ,\varphi }{\overline{W}}_{i-\varphi ,k}^{\tau } H_{k,j-\tau }^{\varphi }$$

(44)

Thus, the cost function takes the following form:

$$\begin{array}{ll}{G}_s\left(\theta \right)& \le G_s(\theta |\widehat{\theta })\\ & ={\displaystyle {\displaystyle \sum }_{i,j}}\frac{V_{i,j}^{\beta }}{\beta \left(\beta -1\right)}+\alpha {\displaystyle {\displaystyle \sum }_{\varphi ,k,j}}\left({\lambda }_{k,j}^{\varphi }{H}_{k,j}^{\varphi }-\log {\lambda }_{k,j}^{\varphi }\right)+\{\begin{array}{cc}{\overline{Q}}_{i,j}^{\left(\beta \right)}-V_{i,j}{\overline{P}}_{i,j}^{\left(\beta -1\right)} ,\hfill & \hfill (\beta <1)\\ {\overline{P}}_{i,j}^{\left(\beta \right)}-V_{i,j}{\overline{P}}_{i,j}^{\left(\beta -1\right) ,}\hfill & \hfill \left(1\le \beta \le 2\right)\\ {\overline{P}}_{i,j}^{\left(\beta \right)}-V_{i,j}{\overline{Q}}_{i,j}^{\left(\beta -1\right) ,}\hfill & \hfill (\beta >2)\end{array}\end{array}$$

(45)

where

$${\overline{Q}}_{i,j}^{\left(\beta \right)}={\overline{\mathsf{\Lambda }}}_{i,j}^{\beta -1}\left({\displaystyle \sum }_{k,\tau \varphi }{\overline{W}}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }-{\overline{\mathsf{\Lambda }}}_{i,j}\right)+\frac{{\overline{\mathsf{\Lambda }}}_{i,j}^{\beta -1}}{\beta }$$

and

$${\overline{P}}_{i,j}^{\left(\beta \right)}=\frac{1}{\beta }{\displaystyle {\displaystyle \sum }_{k,\tau ,\varphi }}{\omega }_{i,j,k,\tau ,\varphi }{\left(\frac{{\overline{W}}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }}{{\omega }_{i,j,k,\tau ,\varphi }}\right)}^{\beta }$$

To obtain ${\lambda }_{k,j}^{\varphi }$, we minimize the cost function with respect ${\lambda }_{k,j}^{\varphi }$ and set it to zero which results:

$${\lambda }_{k,j}^{\varphi }=\frac{1}{H_{k,j}^{\varphi }}$$

(46)

provided that $H_{k,j}^{\varphi }\ne 0$. However, it has been observed in many cases that optimizing the factor matrices with β-divergence and the sparseness in Equation (46) increases the likelihood for some $H_{k,j}^{\varphi }$ to converge very close to zero, thus leading to numerical divergence when dividing by zero. Other practices introduced a small constant to $H_{k,j}^{\varphi }$ to prevent direct division by zero. Unfortunately, such approach is identical to constant sparsity and no longer preserves the $L_1$-norm optimal solution. In this paper, we adopt the maximum likelihood approach to formulating the adaptive estimation of sparsity parameter ${\lambda }_{k,j}^{\varphi }$. Considering the following maximum likelihood criterion [31,36]:

$${\lambda }^{ML}=\arg \underset{\lambda }{\max }\ln p(v|\lambda ,\check{W})$$

(47)

where $\ln p(v|\lambda ,\check{W})$ is the log-likelihood conditional probability of the observations given $\check{W}$ and $\lambda$. By using the Jensen’s inequality, for any distribution $Q\left(h\right)$, the log-likelihood function satisfies the following:

$$\begin{array}{ll}{\mathsf{\lambda }}^{ML}& =\arg \underset{\mathsf{\lambda }}{\max }{{\displaystyle \int }}^{\text{}}Q\left(h\right)\ln p(v,h|\mathsf{\lambda },\check{W})dh\\ & =\arg \underset{\mathsf{\lambda }}{\max }{{\displaystyle \int }}^{\text{}}Q\left(h\right)(\ln p(v|h,\check{W})+\ln p\left(h|\mathsf{\lambda }\right))dh\\ & =\arg \underset{\mathsf{\lambda }}{\max }{{\displaystyle \int }}^{\text{}}Q\left(h\right)\ln p\left(h|\mathsf{\lambda }\right)dh\end{array}$$

(48)

Since each element of $H$ is constrained to be exponential distributed with independent decay parameters, Equation (48) becomes:

$${\lambda }_g^{ML}=\arg \underset{\lambda }{\max }{\displaystyle \int }Q\left(h\right)\left(\ln {\lambda }_g-{\lambda }_g{h}_g\right)dh$$

(49)

Thus, we have

$${\lambda }_g^{ML}=\frac{1}{\int h_{g}Q\left(h\right)dh}$$

(50)

One can easily check that the distribution that maximizes the maximum likelihood is given by $Q(h)=p(h|v,\mathsf{\lambda }, \check{W})=p(v|h,\mathsf{\lambda }, \check{W})p(h|\mathsf{\lambda })/p(v|\mathsf{\lambda }, \check{W})$ which is the posterior distribution of $h$ and $p(v|h,\mathsf{\lambda }, \check{W})$ is the log-likelihood function of the observation which is usually expressed by a Gaussian density function with mean centered at $\sum _{k,\tau ,\varphi }{\overline{W}}_{i-\varphi ,k}^{\tau } H_{k,j-\tau }^{\varphi }$. However, as $H_{k,j}^{\varphi }$ is directly acquired from the original code matrix $H_{k,j}^0$, we can simply work with ${\tau }_{max}=0$. This allows us to express the log-likelihood function of the posterior distribution of $h$ up to a constant as

$$\begin{array}{ll}\ln p(h|v,\mathsf{\lambda }, \check{W})& \dot{=}\ln p(v|h,\mathsf{\lambda }, \check{W})+\ln p\left(h|\mathsf{\lambda }\right)\\ & \dot{=}\frac{1}{2}||vec\left(V\right)-{\displaystyle {\displaystyle \sum }_{\varphi }}\left(I⨂\stackrel{\downarrow \varphi }{\overline{W}}\right)vec\left(H^{\varphi }\right){\left|\right|}_F^2+\alpha {\displaystyle {\displaystyle \sum }_{\varphi }}\left\{{\left({\lambda }^{\varphi }\right)}^{T}vec\left(H^{\varphi }\right)-{\left(\log {\lambda }^{\varphi }\right)}^{T}1\right\}\\ & =F\left(H,\mathsf{\lambda }\right)\end{array}$$

(51)

where “$\dot{=}$” denotes equality up to a constant, “$⨂$“ is the Kronecker product, 1 is vector contains unit elements, $I$ is the identity matrix, $\alpha$ assumes the role of a regularization constant to balance the cost function fit and smoothness of $H$. For ease of presentation, we simplify the above terms as $v=vec\left(V\right)$, $\check{W}=\left[\begin{array}{ccc}I⨂\stackrel{\downarrow 0 }{\overline{W}}& \dots & I⨂\stackrel{\downarrow {\varphi }_{max} }{\overline{W}}\end{array}\right]$, $h=\left\{h_g\right\}={\left[\begin{array}{ccc}vec{\left(H^0\right)}^T& \cdots & vec{\left(H^{{\varphi }_{max}}\right)}^T\end{array}\right]}^T$, $\lambda =\left\{{\lambda }_g\right\}={\left[\begin{array}{ccc}{\lambda }^0{}^T& \dots & {\lambda }^{{\varphi }_{max}}{}^T\end{array}\right]}^T$ which enables us to rewrite Equation (46) as

$$F\left(H,\mathsf{\lambda }\right)=\frac{1}{2}\left|\right|v-\check{W}h{\left|\right|}_F^2+\alpha \left({\mathsf{\lambda }}^{T}h-{(\log \lambda )}^T\mathbf{1}\right)$$

(52)

For ease of analysis, $Q\left(h\right)$ is represented using Gibbs distribution as $Q\left(h\right)=\frac{1}{Z}\exp \left(-F\left(h\right)\right)$ where $Z=\int \exp \left(-F\left(h\right)\right)dh$. Let $P$ represents the index set of inactive code i.e., $P=\left\{\varphi ,k,j|H_{k,j}^{\varphi }=0\right\}$ and $M$ the index set of active code i.e., $M=\left\{\varphi ,k,j|H_{k,j}^{\varphi }\ne 0\right\}$. Thus, $Q\left(h\right)$ can be factorized as

$$\begin{array}{ll}Q\left(h\right)& =\frac{1}{Z}\exp \left(-F\left(h,\lambda \right)\right)\\ & \approx \frac{1}{Z_P}\exp \left(-F\left(h_P,{\lambda }_P\right)\right)\frac{1}{Z_M}\exp \left(-F\left(h_M,{\lambda }_M\right)\right)\\ & =Q_P\left(h_P\right)Q_M\left(h_M\right)\end{array}$$

(53)

Since $h_M$ corresponds to the original non-zero value of $h$, it then follows that $Q_M\left(h_M\right)$ is not of interest to us. We are only interested in $h_P$ and therefore, to characterize $Q_P\left(h_P\right)$, we need to allow some positive deviation to $h_P$. A suitable distribution is to use the factorized exponential distribution given by

$${\widehat{Q}}_P\left(h_P\ge 0\right)={\displaystyle \prod }_{p\in P}\frac{1}{u_p}\exp \left(-\frac{h_p}{u_p}\right)$$

(54)

as the approximate distribution. The variational parameters $u=\left\{u_p\right\}$ are determined by minimizing the Kullback–Leibler divergence between true $Q_P$ and approximate ${\widehat{Q}}_P$:

$$\begin{array}{ll}\mathit{u}& =\arg \underset{\mathit{u}}{\min }{{\displaystyle \int }}^{\text{}}{\widehat{Q}}_P\left(h_P\right)\ln \frac{{\widehat{Q}}_P\left(h_P\right)}{Q_P\left(h_P\right)}d{h}_P\\ & =\arg \underset{\mathit{u}}{\min }{{\displaystyle \int }}^{\text{}}{\widehat{Q}}_P\left(h_P\right)\left\{\ln {\widehat{Q}}_P\left(h_P\right)-Q_P\left(h_P\right)\right\}d{h}_P\end{array}$$

(55)

which leads to the following optimization:

$$\underset{u_p}{\min }{\mathit{b}}_P^T\mathit{u}+\frac{1}{2} u^{T}C\mathit{u}-{\displaystyle \sum }_{p\in P}\ln u_p$$

(56)

where ${\mathit{b}}_P={(Ch-{\check{W}}^{T}v+\lambda )}_P$ and $C=C_P+diag\left(C_P\right)$ with $C={\check{W}}^T\check{W}$, $C_P={\check{W}}_P^T{\check{W}}_P$. Solving Equation (56) for $u_p$ leads to the following update:

$$u_p\leftarrow u_p\frac{-b_p+\sqrt{b_p^2+4\frac{{\left(C\mathit{u}\right)}_p}{u_p}}}{2{\left(C\mathit{u}\right)}_p}$$

(57)

Once $u_p$ is obtained and re-arranged to the original form $u_{k,j}^{\varphi }$, the final update for ${\lambda }_{k,j}^{\varphi }$ takes the form of:

$${\lambda }_{k,j}^{\varphi }=\frac{1}{H_{k,j}^{\varphi }+{\delta }_{k,j}^{\varphi }}$$

(58)

where

$${\delta }_{k,j}^{\varphi }=\{\begin{array}{c} 0 \mathrm{if} H_{k,j}^{\varphi }\ne 0 \\ u_{k,j}^{\varphi } \mathrm{if} H_{k,j}^{\varphi }=0\end{array}$$

(59)

Equipped with above, we obtain the multiplicative update for the normalized $W$ as

$$W^{\tau }={\overline{W}}^{\tau }·{\left[\frac{\sum _{\varphi }\left(\stackrel{\uparrow \varphi }{V}·\stackrel{\uparrow \varphi }{{\overline{{\Lambda }}}^{\left(\beta -2\right)}}\right){\stackrel{\to \tau }{H^{\varphi }}}^T+{\overline{W}}^{\tau }diag\left(\sum _{\tau }1\left(\left(\stackrel{\uparrow \varphi }{{\overline{{\Lambda }}}^{\left(\beta -1\right)}}{\stackrel{\to \tau }{H^{\varphi }}}^T\right)·{\overline{W}}^{\tau }\right)\right)}{\sum _{\varphi }\stackrel{\uparrow \varphi }{{\overline{{\Lambda }}}^{\left(\beta -1\right)}} {\stackrel{\to \tau }{H^{\varphi }}}^T+{\overline{W}}^{\tau }diag\left(\sum _{\tau }1\left(\left(\left(\stackrel{\uparrow \varphi }{V}·\stackrel{\uparrow \varphi }{{\overline{{\Lambda }}}^{\left(\beta -2\right)}}\right) {\stackrel{\to \tau }{H^{\varphi }}}^T\right)·{\overline{W}}^{\tau }\right)\right)}\right]}^{-\delta \left(\beta \right)}$$

(60)

for $\tau =0, 1, \dots , {\tau }_{max}$. By using the same approach, we can obtain the update for the sparse $H$ as follows:

$$H^{\varphi }=H^{\varphi }·{\left[\frac{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{{\overline{W}}^{\tau }}}^T\left(\stackrel{\leftarrow \tau }{V}·\stackrel{\leftarrow \tau }{{\overline{{\Lambda }}}^{\left(\beta -2\right)}}\right)}{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{{\overline{W}}^{\tau }}}^T\stackrel{\leftarrow \tau }{{\overline{{\Lambda }}}^{\left(\beta -1\right)}}+\alpha {\lambda }^{\varphi }}\right]}^{\delta \left(\beta \right)}$$

(61)

for $\varphi =0, 1, \dots , {\varphi }_{max}$. In Equation (61), $\alpha$ assumes the role of a regularization constant to balance the cost function fit and smoothness of $H$. In this work, we set $\alpha \in \left[0.5, 1\right]$ which has been found to give satisfactory results.

2.5. Optimizing the Fractional β

To determine the optimal value for β, we perform the investigation from the source separation viewpoint. Mathematically, the single-channel signal separation (SCSS) [37,38,39] problem can be treated as one mixture of $N$ unknown source signals:

$$y\left(t\right)=x_1\left(t\right)+x_2\left(t\right)+\dots +x_N\left(t\right)$$

(62)

where $t=1, 2, \dots , T$ denotes time index and the goal is to estimate the sources $x_k\left(t\right), \forall k\in N$ of length $T$ when only the observation signal $y\left(t\right)$ is available. For simplicity, we consider only $N=2$ sources in the mixture. We also use 50 different pieces of piano music, 50 different pieces of trumpet music and 50 different pieces of violin music from the RWC [40] database to generate different mixtures. The signal-to-distortion (SDR) [41] is used to measure the performance. The SDR results and its corresponding β value that produces the best performance in the separation of mixtures are shown in

Table 2. Results using different sources.

Mixtures	SDR (dB)	β
Piano + trumpet	16.11	2.11
	9.19	2.13
	9.43	1.93
	7.73	1.82
	12.21	2.09
Piano + violin	13.07	1.07
	8.15	1.23
	6.25	0.92
	9.33	1.20
	8.19	0.89
Trumpet + violin	14.63	0.68
	8.14	0.62
	7.81	0.67
	9.81	0.51
	7.55	0.52

. From these experiments, we can propose some general ideas of how to choose a suitable β value: (i) The mixtures from same type of music share similar β value, e.g., the best results of piano and trumpet mixture occur around β = 2 but the best results of piano and violin mixture occur around $\beta =1$. (ii) If the power of one source is clearly weaker than the other source, then a smaller β value should be selected. (iii) When there is a large amount of overlap between the two sources in the in the time–frequency domain, a larger β value should be selected.

Table 2 strongly suggests that the β value depends on the mixture of original sources. Generally, it depends mainly on the two factors: (i) the weight of each source in the mixture; and (ii) the frequency spread of each source which the frequency band contains most weight of the signal. Firstly, we define weight of each source in the mixture by the following function:

$${\gamma }_k=1-\frac{|x_k\left(t\right)-y\left(t\right)|}{{\sum }_{l=1}^N{|x_l\left(t\right)-y\left(t\right)|}^2}$$

(63)

for $k=1,\dots ,N$. The term ${\gamma }_k$ is nonnegative and bounded to unity. It measures the dominance of $k$-th source in the mixture. The higher value of ${\gamma }_k$, the greater the contribution from the $k$-th source is to the mixture. Secondly, we consider the separability of each source in the time–frequency domain by the following function:

$${\eta }_k=\frac{||M_k\left(i,j\right)X_k{\left(i,j\right)||}_F^2-||M_k\left(i,j\right){\sum }_{l=1,l\ne k}^N{X}_l{\left(i,j\right)||}_F^2}{||X_k{\left(i,j\right)||}_F^2}$$

(64)

where $||·{||}_F$ is the Frobenius norm, $X_k\left(i,j\right)$ is the short-time Fourier Transform (STFT) of $x_k\left(t\right)$ with $i$ representing the frequency bins and $j$ the timeslot, and $M_k\left(i,j\right)$ is the binary mask Obtained from the $k$-th source as

$$M_k\left(i,j\right)=\{\begin{array}{cc}1& \hfill \mathrm{if}{|X_k\left(i,j\right)|}^2>{|X_l\left(i,j\right)|}^2\\ 0& \hfill \mathrm{otherwise}\end{array}$$

(65)

The function ${\eta }_k$ is also nonnegative and determines the degree separability of the signal in each frequency band. Based on the experiments conducted, both ${\gamma }_k$ and ${\eta }_k$ have an inverse relation to $\beta$. Thus, one possible empirical approach to determine $\beta$ is proposed as follows:

$$\beta \left(n+1\right)=\rho \left(n\right)\beta \left(n\right)+\left(1-\rho \left(n\right)\right)\min \left[\left({{\displaystyle \sum }}_{k=1}^N\frac{{\epsilon }_1·{\eta }_k+\left(1-{\epsilon }_1\right)·{\gamma }_k}{{\gamma }_k{\eta }_k}\right), {\epsilon }_2\right]$$

(66)

where $\rho \left(n\right)$ is step size, ${\epsilon }_1$ is a constant to weight the effects of ${\eta }_k$ and ${\gamma }_k$. For example, in the experiments conducted, we have given more emphasis to ${\gamma }_k$ and set ${\epsilon }_1=1/3$. The term ${\epsilon }_2$ is a constant to control the value of $\beta \left(n+1\right)$ to ensure its value is bounded within an interval chosen by the user (for example, in the experiments conducted we have set ${\epsilon }_2=4$ as normally does not exceed 4). Equation (66) is inserted into the update funtions in Equations (60) and (61) to update $\beta$ at every iteration in conjunction with the update of $W$ and $H$. In this case, $\beta$ can be optimized based on the type of sources and the separation process. This enables the separation process to be fully automated and enables more accurate performance. In the case of SCSS, the sources are unknown and these are estimated from the mixture as:

$${\widehat{X}}_k\left(i,j\right)=M_k\left(i,j\right)Y\left(i,j\right)$$

(67)

where

$$M_k\left(i,j\right)=\{\begin{array}{c}1 if {|{\check{X}}_k\left(i,j\right)|}^2>{|{\check{X}}_l\left(i,j\right)|}^2\\ 0 otherwise \end{array}$$

(68)

and

$${|{\check{X}}_k\left(i,j\right)|}^2={\displaystyle \sum }_{\tau =0}^{{\tau }_{max}}{\displaystyle \sum }_{\varphi =0}^{{\varphi }_{max}}{\overline{W}}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }$$

(69)

The expression in (69) is computed using the time–frequency deconvolution model. The main steps of the proposed algorithm have been summarized in Algorithm 1.

Algorithm 1. Overview Proposed Algorithm

1.  Initialize $W^{\tau }$ and $H^{\varphi }$ with non-negative random values. 2.  Compute the STFT:    $Y\left(i,j\right)=\mathrm{STFT}\left(y\left(t\right)\right)$, and let $V_{i,j}={|Y\left(i,j\right)|}^2$. 3.  Compute ${\overline{\mathsf{\Lambda }}}_{i,j}=\sum _{k,\tau ,\varphi }{\overline{W}}_{i-\varphi ,k}^{\tau } H_{k,j-\tau }^{\varphi }$. 4.  Compute $u_p\leftarrow u_p\left\{\left(-b_p+\sqrt{b_p^2+4\frac{{\left(C\mathit{u}\right)}_p}{u_p}}\right)/2{\left(C\mathit{u}\right)}_p\right\}$ 5.  Assign ${\lambda }_{k,j}^{\varphi }=\frac{1}{H_{k,j}^{\varphi }+{\delta }_{k,j}^{\varphi }}$ where ${\delta }_{k,j}^{\varphi }=\{\begin{array}{c} 0 \mathrm{if} H_{k,j}^{\varphi }\ne 0 \\ u_{k,j}^{\varphi } \mathrm{if} H_{k,j}^{\varphi }=0\end{array}$ 6.  Update $H^{\varphi }=H^{\varphi }·{\left[\frac{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{{\overline{W}}^{\tau }}}^T\left(\stackrel{\leftarrow \tau }{V}·\stackrel{\leftarrow \tau }{{\overline{{\Lambda }}}^{\left(\beta -2\right)}}\right)}{{\sum }_{\tau }{\stackrel{\downarrow \varphi }{{\overline{W}}^{\tau }}}^T\stackrel{\leftarrow \tau }{{\overline{{\Lambda }}}^{\left(\beta -1\right)}}+\alpha {\lambda }^{\varphi }}\right]}^{\delta \left(\beta \right)}$ 7.  Compute ${\overline{\mathsf{\Lambda }}}_{i,j}=\sum _{k,\tau ,\varphi }{\overline{W}}_{i-\varphi ,k}^{\tau } H_{k,j-\tau }^{\varphi }$. 7.  Update the spectral bases: $$W^{\tau }={\overline{W}}^{\tau }·\left[\frac{{\sum }_{\varphi }\left(\stackrel{\uparrow \varphi }{V}·\stackrel{\uparrow \varphi }{{\overline{\Lambda }}^{\left(\beta -2\right)}}\right){\stackrel{\to \tau }{H^{\varphi }}}^T+{\overline{W}}^{\tau }diag\left(\sum _{\tau }1\left(\left(\stackrel{\uparrow \varphi }{{\overline{\Lambda }}^{\left(\beta -1\right)}}{\stackrel{\to \tau }{H^{\varphi }}}^T\right)·{\overline{W}}^{\tau }\right)\right)}{\sum _{\varphi }\stackrel{\uparrow \varphi }{{\overline{\Lambda }}^{\left(\beta -1\right)}}{\stackrel{\to \tau }{H^{\varphi }}}^T+{\overline{W}}^{\tau }diag\left(\sum _{\tau }1\left(\left(\left(\stackrel{\uparrow \varphi }{V}·\stackrel{\uparrow \varphi }{{\overline{\Lambda }}^{\left(\beta -2\right)}}\right){\stackrel{\to \tau }{H^{\varphi }}}^T\right)·{\overline{W}}^{\tau }\right)\right)}\right]$$ 8.  9.  For $k=1,\dots ,N$, compute: $$\begin{array}{l}{|{\check{X}}_k\left(i,j\right)|}^2={\displaystyle {\displaystyle \sum }_{\tau =0}^{{\tau }_{max}}}{\displaystyle {\displaystyle \sum }_{\varphi =0}^{{\varphi }_{max}}}{\overline{W}}_{i-\varphi ,k}^{\tau }{H}_{k,j-\tau }^{\varphi }\\ M_k\left(i,j\right)=\{\begin{array}{c}1 if {|{\check{X}}_k\left(i,j\right)|}^2>{|{\check{X}}_l\left(i,j\right)|}^2\\ 0 otherwise \end{array}\\ {\widehat{X}}_k\left(i,j\right)=M_k\left(i,j\right)Y\left(i,j\right)\\ {\widehat{x}}_k\left(t\right)={\mathrm{STFT}}^{-1}\left[{\widehat{X}}_k\left(i,j\right)\right]\\ {\gamma }_k=1-\frac{|{\widehat{x}}_k\left(t\right)-y\left(t\right)|}{{{\displaystyle \sum }}_{l=1}^N{|{\widehat{x}}_l\left(t\right)-y\left(t\right)|}^2}\\ {\eta }_k=\frac{M_k\left(f,t\right){\widehat{X}}_k{\left(f,t\right)}_F^2-M_k\left(f,t\right){{\displaystyle \sum }}_{l=1,l\ne k}^N{\widehat{X}}_l{\left(f,t\right)}_F^2}{{\widehat{X}}_k{\left(f,t\right)}_F^2}\\ \beta \leftarrow \rho \beta +\left(1-\rho \right)\min \left[\left({\displaystyle \sum }_{k=1}^N\frac{{\epsilon }_1�{\eta }_k+\left(1-{\epsilon }_1\right)�{\gamma }_k}{{\gamma }_k{\eta }_k}\right), {\epsilon }_2\right]\end{array}$$ 10.  Repeat Steps 3–9 until it converges or reaches the pre-defined number of iteration.

3. Experiments, Results and Analysis

In this section, we conduct in-depth investigations of the proposed algorithm to analyze the impact of fixed and adaptive sparsity, the adaptive behavior of the sparsity parameter, and the analysis of fractional β-divergence. The analysis is necessary as the issue of sparsity of the temporal codes would undermine the quality of matrix factorization when the $\beta$ value is inappropriately chosen. The selection of the $\beta$ value should consider the sparseness constraint used in the cost function. In addition, the proposed algorithm based on matrix factor time–frequency deconvolution is also compared to conventional NMF models. This allows us to quantify the impacts of fractional β-divergence and sparsity behaviors when using the time–frequency deconvolution model.

3.1. Experimental Set-Up

To investigate the proposed method, we use the algorithm to separate several pieces of mixed music signals. Several experimental simulations under different conditions have been designed to investigate the efficacy of the proposed method. All simulations were performed using MATLAB as the programming platform and performed using a PC with dual core processor @ 2.4 GHz (i7 Intel processor) 8 GB RAM and 320 GB HDD. The tested signals are generated by mixing several music sources. The polyphonic music is 4 s long and the sampling frequency is 16 kHz. In this experiment, we randomly chose 50 different pieces of piano music, 50 different pieces of trumpet music and 50 different pieces of violin music from the RWC database to produce the different mixtures. The mixed signal was then generated by adding the chosen sources. In all cases, the sources were mixed with equal average power over the duration of the signals. The time–frequency (TF) representation was obtained by first normalizing the time-domain signal to unit power and then by computing the STFT using 2048 point Hanning window FFT with a 50% overlap. We evaluated our separation performance in terms of the signal-to-distortion ratio (SDR) which is one form of perceptual measure. This is a global measure that unifies source-to-interference ratio (SIR), source-to-artifacts ratio (SAR) and source-to-noise ratio (SNR). The definition and mathematical expression and MATLAB routines for computing these criteria can be obtained online [42].

3.2. Analysis of Adaptive and Fixed Sparsity

In this implementation, we conducted several experiments to compare the performance of the proposed method using different $\beta$ values. Our aim was to investigate the impact of $\beta$ value used in the separation. Figure 1 shows the time and TF domains of the original trumpet, piano music and its mixture. The TF domain is displayed using the log-frequency spectrogram. The trumpet and the piano play a different short melodic passage each consisting of three distinct notes. However, both trumpet and piano overlap in time, and the piano notes are interspersed in frequency with the trumpet notes. Hence, this is a challenging task for single channel separation which tests the impact of flexible $\beta$ for matrix factorization.

Figure 2 shows the estimation of bases $W^{\tau }$ and temporal codes $H^{\varphi }$ when using different $\lambda$ values. In Figure 2a, we set $\lambda =0$ which renders non-sparse solution. There is obvious spreading of the estimated temporal codes, as shown in the red part of the figure. In Figure 2b, when $\lambda =0.1$, there are some improvements over the spreading but they still exist in the red parts. Here, the sparseness is not strong enough and, as a result, the estimated mixture becomes under-sparse. In Figure 2c, when $\lambda =100$, it is visibly shown in the blue parts that some information has been lost in the estimated temporal codes and the resulting estimated mixture becomes very noisy. Finally, Figure 2d shows the case where the sparseness parameters are adaptively and individually estimated using the prior information of $H$. The obtained result has shown that the estimated temporal codes are just appropriately sparse and, by visual inspection, the resulting estimated mixture retains all information, as evidenced by the musical notes, while the noise level has been kept small, which very closely resembles the original mixture.

3.3. Adaptive Behavior of Sparsity Parameter

In this section, we use the piano and trumpet mixture, and fix $\beta$ value to 1 and the $\lambda$ value ranges from 0 to 3 with each step of 0.1 to show the results. The adaptive behavior of the sparsity parameters using the proposed method is demonstrated. Figure 3 presents the convergence trajectory of four adaptive sparsity parameters, ${\lambda }_{1,1}^{\varphi =0}$, ${\lambda }_{1,5}^{\varphi =0}$, ${\lambda }_{1,10}^{\varphi =0}$ and ${\lambda }_{1,15}^{\varphi =0}$, corresponding to their respective element codes, $h_{1,1}^{\varphi =0}$, $h_{1,5}^{\varphi =0}$, $h_{1,10}^{\varphi =0}$ and $h_{1,15}^{\varphi =0}$. All sparsity parameters are initialized as $\lambda =1$. After 150 iterations, the above sparsity parameters converge to their steady-states. By examining Figure 3, it is noted that the converged steady-state values are significantly different for each sparsity parameter, e.g., ${\lambda }_{1,1}^{\varphi =0}=0.9$, ${\lambda }_{1,5}^{\varphi =0}=0.18$, ${\lambda }_{1,10}^{\varphi =0}=0.29$ and ${\lambda }_{1,15}^{\varphi =0}=0.08$, even though they started at the same initial condition. This shows that each element code has its own sparseness.

In Figure 4, we compare the SDR results of using different $\lambda$ values. In the figure, we can see the $\lambda$ value that can get the best result changes with the mixture. For each different mixture, the best $\lambda$ values are different. In Figure 4, we can see the best separation results of piano and trumpet mixture occurs near $\lambda =1$, and the SDR = 14.7 dB. However, as $\lambda$ increases, the SDR performance begins to deteriorate rapidly due to over-sparseness of the temporal code $H^{\varphi }$.

Although the SDR performance in Figure 4 seems to suggest good performance, it may not necessary refer to the optimum setting. In fact, when $\lambda$ is fixed to a constant value, the matrix factors deconvolution process may still be subjected to under- and over-fitting. In the previous sparsity algorithm, $\lambda$ is fixed for the whole process and this sparsity may not necessarily suited for the whole signal. This calls for the need to allow each temporal code to have its own sparsity parameter. In the adaptive sparsity algorithm in Equation (53), the sparsity parameter $\lambda$ is updated alongside $W^{\tau }$ and $H^{\varphi }$ in the process. Therefore, the sparsity parameter is optimized for each element of the temporal code. In addition, we plot the histogram of the converged adaptive sparsity parameters in Figure 5. The plot strongly suggests that the histogram can be represented as a bimodal distribution. We have used the Gaussian mixture model (GMM) [43] to learn the distribution of this histogram and the result produces two Gaussian distributions with mean 0.16 and 1.1. The global mean of the GMM is given by 0.92.

With the GMM analysis, we can proceed to further investigate the assignment of sparsity parameters and compare them with the adaptive approach. We considered the following sparseness assignments:

Case (1):

No sparseness $\mathsf{\lambda }=0$.

Case (2):

Uniform and constant sparseness $\lambda =0.16$ corresponding to the mean of the first Gaussian distribution of the GMM.

Case (3):

Uniform and constant sparseness $\lambda =1.1$ corresponding to the mean of the second Gaussian distribution of the GMM.

Case (4):

Uniform and constant sparseness $\lambda =0.92$ corresponding to the global mean of the converged adaptive sparsity.

Case (5):

Uniform and constant sparseness $\lambda =1$ corresponding to the global mean of the converged adaptive sparsity.

Case (6):

Maximum likelihood adaptive sparseness, i.e., Equation (55).

The SDR results are tabulated in

Table 3. Results of separation for different mixture.

Methods	SDR (dB)
Case (1)	12.77
Case (2)	13.01
Case (3)	14.60
Case (4)	14.62
Case (5)	14.70
Case (6)	15.60

where we can see the separation results of all the six cases. The obtained results readily informed that the source separation with adaptive sparsity has rendered the best separation result.

3.4. Analysis of Fractional β-Divergence

Figure 6 shows the SDR values of the separation results using different values of β values. In this implementation, we fixed the sparsity parameter $\lambda$ to 0, and the value of β ranges from 0 to 4, and each step is 0.1. In the figure, we can observe that the SDR value changes as the $\beta$ value is increased. The best result is obtained at when $\beta =2.5$, where SDR = 14.26 dB. The SDR value keeps increasing for values of $\beta$ value within the range of $0\le \beta <2.5$, and, after the best performance is attained, the performance deteriorates as β increases. In this figure, we can see that the best performance does not necessarily occur at value of $\beta$ used other algorithms, i.e., $\beta =2, 1, 0$. This means, if we choose the best β to carry out the separation, we can obtain better results than the other algorithms.

We also conducted several experiments to compare the performance of the proposed method with the non-sparse and normal sparse methods using different $\beta$ value. To investigate the impact of $\beta$ and $\lambda$ value used in the separation, we used $\beta$ that ranges from 0 to 4 (with every increment of 0.1), and $\lambda$ was set to several configurations, i.e., non-sparse, best fixed sparsity that is obtained in Figure 4 and the proposed adaptive sparsity. The results are plotted in Figure 7, which shows the mutual influence between λ and $\lambda$. It is noted that the SDR performance of non-sparse algorithm is the lowest of which its best result occurs when $\beta =2.5$ with SDR = 14.26 dB. The normal sparse algorithm using best $\lambda$ value gives better performance than the non-sparse method with its best result occuring when $\beta =0.5$ with SDR = 15.96 dB. On the other hand, the proposed algorithm with adaptive $\lambda$ delivers the best performance where $\beta$ occurs around 0.3 with SDR = 16.71 dB. It is also noted that the worst SDR performance given by $\beta =2.9$ with adaptive $\lambda$ is still higher than the highest SDR when $\beta =2.5$ without sparsity optimization $\lambda =0$.

All the above experiments used the pre-determined β that are fixed for the whole process. In this section, we present the results of experiments where $\beta$ is adaptively tuned alongside with the adaptation of $W^{\tau }$, H^ϕ and ${\lambda }^{\varphi }$. We applied this method to the source separation problem and compared the results with the situation where $\beta =1$ and $\beta =2$ corresponding to the Kullback–Leibler divergence and Least Square cost function, respectively. In the update of adaptive $\beta$, the step size $\rho \left(n\right)={0.95}^n$ is selected which represents an exponential decay update process. The SDR results of the top five best performance are tabulated in

Table 4. SDR (dB) results of adaptive $\beta$ versus fixed $\beta$.

Mixtures	SDR (dB) Using Adaptive β	SDR (dB) Using β = 1	SDR (dB) Using β = 2
Piano + Trumpet	16.85	14.11	15.93
	10.74	7.95	9.01
	9.93	8.12	9.11
	8.95	6.57	7.44
	13.64	10.26	12.03
Piano + Violin	14.17	12.12	11.67
	9.04	7.95	7.11
	8.13	6.09	5.81
	10.4	9.08	8.71
	9.59	7.85	7.19
Trumpet + Violin	15.40	12.49	12.13
	8.87	6.23	6.31
	9.14	6.87	7.17
	10.51	7.92	8.11
	9.17	7.77	7.95

. It is interesting to note from the table that the proposed algorithm with adaptive $\beta$ delivers better performance by about 2.1 dB compared to that when $\beta =1$ (Kullback–Leibler divergence) and 1.9 dB compared to that when $\beta =2$ (Least Square distance). The obtained performance improvement is attributed to the fact that the joint optimization of β with $W^{\tau }$, $H^{\varphi }$ and ${\lambda }^{\varphi }$ has enabled the current estimate of $\beta$ to better fit mixture of sources and thus rendered better source separation performance.

3.5. Comparison with Other Nonnegative Factorization Models

In this section, we compare the proposed algorithm with other signal separation algorithms, in both time-domain representation and analysis the SDR results of all algorithms. The signal chosen was the same piano and trumpet mixture music used in Section 3.4. We compared the Least Square NMF (NMF-LS) and Kullback–Leibler NMF (NMF-KLD) algorithms introduced in the earlier sections of this paper, NMF with temporal continuity and sparseness criteria (NMF-TCS) [23], and NMF with automatic relevance determination (NMF-ARD) [44]. The obtained results are summarized in

Table 5. Performance comparison of proposed method with NMF models.

Algorithm	SDR (dB)
NMF-LS	4.17
NMF-KLD	3.47
NMF-TCS	5.12
NMF-ARD	3.98
NMF using proposed method	7.63
Proposed method using matrix factor time–frequency deconvolution	12.02

.

When using the various NMF models, it is seen that the average improvement per source of about 3 dB has been gained by leveraging the fractional $\beta$ value and adaptive sparsity. In addition, a step jump of approximately 5–8 dB in performance improvement is further obtained when the model is switched to the matrix factor time–frequency deconvolution. This is attributed to the latter model which represents both temporal structure and the pitch change which occur when an instrument plays different notes. The pitch change corresponds to a displacement on the frequency axis. Where NMF methods needed one component to model each note for each instrument, the time–frequency deconvolution model represents each instrument compactly by a single time–frequency profile convolved in both time and frequency by a time–pitch weight matrix [45]. The model dramatically decreases the number of components needed to model various instruments and effectively solves the blind single channel source separation problem for certain classes of musical signals.

4. Conclusions

This paper presents an adaptive fractional β-divergence with sparsity-aware optimization for non-negative factor time–frequency deconvolution algorithm. The impetus behind this work is that the previous $\beta$-divergence algorithms are all limited to special cases of $\beta$, and the previous sparsity methods are limited to a fixed sparsity parameter which are determined manually. Thus, these algorithms may not always produce the best results. In the proposed method, $\beta$ is made adaptive and takes on fractional value. The sparsity parameter is also concurrently updated along with the estimation of $\beta$ and model parameters. The convergence is theoretically proven for any $\beta$ based on the auxiliary function method. This paper has shown that the proposed method is more general and can deliver better performance than other algorithms, as demonstrated using real audio recordings.