Efficient Noisy Sound-Event Mixture Classification Using Adaptive-Sparse Complex-Valued Matrix Factorization and OvsO SVM

Abstract

This paper proposes a solution for events classification from a sole noisy mixture that consist of two major steps: a sound-event separation and a sound-event classification. The traditional complex nonnegative matrix factorization (CMF) is extended by cooperation with the optimal adaptive L₁ sparsity to decompose a noisy single-channel mixture. The proposed adaptive L₁ sparsity CMF algorithm encodes the spectra pattern and estimates the phase of the original signals in time-frequency representation. Their features enhance the temporal decomposition process efficiently. The support vector machine (SVM) based one versus one (OvsO) strategy was applied with a mean supervector to categorize the demixed sound into the matching sound-event class. The first step of the multi-class MSVM method is to segment the separated signal into blocks by sliding demixed signals, then encoding the three features of each block. Mel frequency cepstral coefficients, short-time energy, and short-time zero-crossing rate are learned with multi sound-event classes by the SVM based OvsO method. The mean supervector is encoded from the obtained features. The proposed method has been evaluated with both separation and classification scenarios using real-world single recorded signals and compared with the state-of-the-art separation method. Experimental results confirmed that the proposed method outperformed the state-of-the-art methods.

1. Introduction

Surveillance systems have become increasingly ubiquitous in our living environment. These systems have been used in various applications including CCTV in traffic and site monitoring, and navigation. Automated surveillance is currently based on video sensory modality and machine intelligence. Recently, intelligent audio analysis has been taken into account in surveillance to improve the monitoring system via detection, classification, and recognition sound in a scenario. However, in a real-world situation, background noise has interfered in both the image and sound of a surveillance system. This will hinder the performance of a surveillance system. Hence, an automatic signal separation and event classification algorithm was proposed to improve the surveillance system by classifying the observed sound-event in noisy scenarios. The proposed noisy sound separation and event classification method is based on two approaches (i.e., blind signal separation and sound classification, which are introduced in the sections to follow, respectively).

The classical problem of blind source separation (BSS), the so-called “cocktail party problem”, is a psycho-acoustic spectacle that alludes to the significant human-auditory capability to selectively focus on and identify the sound-source speaker from the scenarios. The interference is produced by competing speech sounds or a variety of noises that are often assumed to be independent of each other. In the case of only a single microphone being available, this reduces to the single channel blind source separation (SCBSS) [1,2,3,4]. The majority of SCBSS algorithms work in time-frequency domain, for example, binary masking [5,6,7] or nonnegative matrix factorization (NMF) [8,9,10,11]. NMF has been continuously developed with great success for decomposing underlying original signals when a sole sensor is available. NMF was developed using the multiplicative update (MU) algorithm to solve its parametrical optimization based on a cost function such as the Kullback–Liebler divergence and the least square distance. Later, other families of cost functions have been continuously proposed, for example, the Beta divergence [12], Csiszar’s divergences, and Itakura–Saito divergence [13]. Additionally, iterative gradient update was presented where a sparsity constraint can be included into the optimizing function through regularization by minimizing penalized least squares [14] and using different sparsity constraints for dictionary and code [15]. The complex nonnegative matrix factorization (CMF) spreads the NMF model by combining a sparsity representation with the complex-spectrum domain to improve the audio separability. The CMF can extract the recurrent patterns of the phase estimates and magnitude spectra of constituent signals [16,17,18]. Nevertheless, the CMF lacks the generalized mechanics used for controlling the sparseness of the code. However, the sparsity parameter is manually determined for the above proposed methods. Approximate sparsity is an important consideration as they represent important information. Many sparse solutions have been proposed in the last decade [19,20,21,22,23,24,25]. Nonetheless, the optimal sparse solution remains an open issue.

Sound event classification (SEC) has vastly been exploited by many researchers. Sound can be categorized into speech, music, noise, environmental sound, or daily living sound [26]. Sound events are available in all classes, for example, car horn, traffic, walking, or knocking, etc. [27,28]. Sound-events contain significant information that can be used to describe what has happened or to predict what will happen next in the future. Most algorithms of the SEC methods are conveyed from sound classification approaches such as sparse coding, deep learning, and support vector machine (SVM). These approaches have been exploited to categorize a sound event in both indoor and outdoor scenarios. In recent years, the deep learning approach has been used to classify the sound-event. A deep learning framework can be established with two convolutional neural networks (CNNs) and a deep multi-layer perceptron (MLP) with rectified linear units (ReLU) as the activation function [29,30]. A Softmax function that is the final activation function is used to classify the sound into its corresponding class. The Softmax function is considered as the generalization of the logistic function, which aims to avoid overfitting. One of the advantages of deep learning is that it does not require feature extraction for the input sound. However, a deep neural network requires large training samples and despite a plethora of research, there is a general consensus that deep neural networks are still difficult to fine tune and generalize to test data. Moreover, it does not lend itself to the explanation as to why a certain decision is being made. Separate from the deep learning framework, another SEC approach is support vector machines [31,32], which has been practically presented to solve the classifier problem in various fields. The SVM algorithm relies on supervised learning by using the fundamental concept of statistic and risk minimization. The main process of the SVM is to draw the optimal separating hyperplane as the decision boundary located in such a way that the margin of separation between classes is maximized. The SVM approach is considered as supervised learning algorithm that is comprised of two sections: (1) a training section to model feature space and an optimal hyperplane, and (2) a testing section to use the SVM model for separating the observed data. The margin denotes the distance of the closest instance and the hyperplane. SVM has the desirable properties in that it requires only two differentiating factors to categorize two classes and a hyperplane that can be constructed to suit for an individual problem, even in the nonlinear case by selecting a kernel. Second, SVM provides a unique solution, since it is a convex optimization problem.

The rest of this paper is organized as follows. Section 2 presents the proposed noisy sound separation and event classification method, respectively. Next, Section 3 demonstrates and analyzes the performance of the proposed method. Finally, conclusions are drawn up in Section 4.

2. Background

Noisy mixed signals observed via a recording device can be stated as: $y\left(t\right)= s_1\left(t\right)+s_2\left(t\right)+n\left(t\right)$ where $s_1\left(t\right)$ and $s_2\left(t\right)$ denote the original sounds, and $n\left(t\right)$ is noise. This research is focused on two sound events in a single recorded signal. The proposed method consists of two steps: noisy sound separation and sound event classification, which is illustrated in Figure 1, where $y\left(t\right)$ and $Y\left(\omega ,t\right)$ denote a sound-event mixture in the time domain and time-frequency domain, respectively. The terms $W^k\left(\omega \right),H^k\left(t\right), {\mathit{\varphi }}^k\left(\omega ,t\right)$ are spectral basis, temporal code or weight matrix, and phase information, respectively. The term ${\lambda }^k\left(t\right)$ represents sparsity and ${\widehat{\mathrm{s}}}_{\mathrm{j}}\left(\mathrm{t}\right)$ is an estimated sound event source. The abbreviations MFCC, STE, and STZCR stand for Mel frequency cepstral coefficients, short-time energy, and short-time zero-crossing rate, respectively. The proposed method is consecutively elaborated in the following parts.

2.1. Single-Channel Sound Event Separation

The problem formulation in time-frequency (TF) representation is given by an observed complex spectrum, $Y_{f,t} \in ℂ$, to estimate the optimal parameters $\theta = \left\{W,H, \mathsf{\varphi }\right\}$ of the model. A new factorization algorithm named as the adaptive L₁-sparse complex non-negative matrix factorization (adaptive L₁-SCMF) is derived in the following section. The generative model is given by

$$Y\left(\omega ,t\right)={\displaystyle \sum }_{k=1}^K{W}^k\left(\omega \right)H^k\left(t\right)Z^k\left(\omega ,t\right)=X\left(\omega ,t\right)+\mathsf{ϵ}\left(\omega ,t\right)$$

(1)

where $Z^k\left(\omega ,t\right)=e^{j{\mathsf{\varphi }}^k\left(\mathsf{\omega },t\right)}$ and the reconstruction error $\mathsf{ϵ}\left(\omega ,t\right)~ {\mathcal{N}}_{ℂ}\left(0,{\sigma }^2\right)$ is assumed to be independently and identically distributed (i.i.d.) with white noise having zero mean and variance ${\sigma }^2$. The term $\mathsf{ϵ}\left(\omega ,t\right)$ is used to denote a modeling error for each source. The likelihood of $\theta = \left\{W,H, \mathsf{\varphi }\right\}$ is thus written as

$$P\left(Y|\theta \right)={\displaystyle \prod }_{f,t}\frac{1}{\pi {\sigma }^2}\exp (-\frac{{\left|Y\left(\omega ,t\right)- X\left(\omega ,t\right)\right|}^2}{{\sigma }^2})$$

(2)

It is assumed that the prior distributions for $W,H,$and $\mathsf{\varphi }$ are independent, which yields

$$P\left(\theta |\mathit{\lambda }\right)= P\left(\mathit{W}\right)P\left(\mathit{H}|\mathit{\lambda }\right)P\left(\mathit{\varphi }\right)$$

(3)

The prior $P\left(H|\mathit{\lambda }\right)$ corresponds to the sparsity cost, for which a natural choice is a generalized Gaussian prior. When $p=1$, $P\left(H|\mathit{\lambda }\right)$ promotes the $L_1$-norm sparsity. $L_1$-norm sparsity has been shown to be probabilistically equivalent to the pseudo-norm, $L_0$, which is the theoretically optimum sparsity [33,34]. However, $L_0$-norm is non-deterministic polynomial-time (NP) hard and is not useful in large datasets such as audio. Given Equation (3), the posterior density [35,36] is defined as the maximum a posteriori probability (MAP) estimation problem, which leads to minimizing the following optimization problem with respect to $\theta$. Equations of Gaussian prior and maximum a posteriori probability (MAP) estimation are expressed in Appendix A.

The CMF parameters have been upgraded by using an efficient auxiliary function for an iterative process. The auxiliary function for $f\left(\theta \right)$ can be expressed as the following: for any auxiliary variables with ${\displaystyle \sum }_k{\overline{Y}}^k\left(\omega ,t\right)=Y\left(\omega ,t\right)$, for any ${\beta }^k\left(\omega ,t\right)> 0$, ${\displaystyle \sum }_k{\beta }^k\left(\omega ,t\right)=1$, for any $H^k\left(t\right)\in ℛ,{\overline{H}}^k\left(t\right)\in ℛ$, and $p=1$. The term $f\left(\theta \right)\le f^{+}\left(\theta ,\overline{\theta }\right)$ with an auxiliary function was defined as:

$$f^{+}\left(\theta ,\overline{\theta }\right)\equiv {\displaystyle \sum }_{f,k,t}\frac{{\left|{\overline{Y}}^k\left(\omega ,t\right)-W^k\left(\omega \right)H^k\left(t\right)·e^{j{\mathit{\varphi }}^k\left(\omega ,t\right)}\right|}^2}{{\beta }^k\left(\omega ,t\right)}+{\displaystyle \sum }_{k,t}\left[{\left({\lambda }^k\left(t\right)\right)}^p\left(p{\left|{\overline{H}}^k\left(t\right)\right|}^{p-2}{H}^k{\left(t\right)}^2+\left(2-p\right){\left|{\overline{H}}^k\left(t\right)\right|}^p\right)-\log {\lambda }^k\left(t\right)\right]$$

(4)

where $\overline{\theta }= \left\{{\overline{Y}}^k\left(\omega ,t\right),{\overline{H}}^k\left(t\right)| 1\le f\le F, 1\le t\le T, 1\le k\le K\right\}$. The function $f^{+}\left(\theta ,\overline{\theta }\right)$ is minimized w.r.t. $\overline{\theta }$ when

$${\overline{Y}}^k\left(\omega ,t\right)= W^k\left(\omega \right){\overline{H}}^k\left(t\right)·e^{j{\mathit{\varphi }}^k\left(\omega ,t\right)}+ {\beta }^k\left(\omega ,t\right)\left(Y\left(\omega ,t\right)-X\left(\omega ,t\right)\right)$$

(5)

$${\overline{H}}^k\left(t\right)=H^k\left(t\right)$$

(6)

2.2. Formulation of Proposed CMF Based Adaptive Variable Regularization Sparsity

2.2.1. Estimation of the Spectral Basis and Temporal Code

In Equation (4), the update rule for $\theta$ is derived by differentiating $f^{+}\left(\theta ,\overline{\theta }\right)$ partially w.r.t. $W^k\left(\omega \right)$ and $H^k\left(t\right)$, and setting them to zero, which yields the following:

$$W^k\left(\omega \right) =\frac{{{\displaystyle \sum }}_t\frac{H^k\left(t\right)}{{\beta }^k\left(\omega ,t\right)}Re\left[{\overline{Y}}^k{\left(\omega ,t\right)}^{*}·e^{j{\mathit{\varphi }}^k\left(\omega ,t\right)}\right]}{{{\displaystyle \sum }}_t\frac{H^k{\left(t\right)}^2}{{\beta }^k\left(\omega ,t\right)}}$$

(7)

$$H^k\left(t\right)=\frac{{{\displaystyle \sum }}_f\frac{W^k\left(\omega \right)}{{\beta }^k\left(\omega ,t\right)}Re\left[{\overline{Y}}^k{\left(\omega ,t\right)}^{*}·e^{j{\mathit{\varphi }}^k\left(\omega ,t\right)}\right]}{{{\displaystyle \sum }}_f\frac{W^k{\left(\omega \right)}^2}{{\beta }^k\left(\omega ,t\right)}+ {\left({\lambda }^k\left(t\right)\right)}^p p{\left|{\overline{H}}^k\left(t\right)\right|}^{p-2}}$$

(8)

The update rule for the phase, ${\mathit{\varphi }}^k\left(\omega ,t\right)$, can be derived by reformulating Equation (4) as follows:

$$\begin{array}{cc}{f}^{+}\left(\theta ,\overline{\theta }\right)\hfill & ={\displaystyle \sum }_{k,f,t}\begin{array}{c}\frac{{\left|{\overline{Y}}^k\left(\omega ,t\right)\right|}^2- 2W^k\left(\omega \right)H^k\left(t\right)\mathrm{Re}\left[{\overline{Y}}^k\left(\omega ,t\right)·e^{-j{\mathit{\varphi }}^k\left(\omega ,t\right)}\right]+W^k{\left(\omega \right)}^2{H}^k{\left(t\right)}^2}{{\beta }^k\left(\omega ,t\right)}+{\displaystyle {\displaystyle \sum }_{k,t}}{\lambda }^k\left(t\right)\left({\left|{\overline{H}}^k\left(t\right)\right|}^{-1}{H}^k{\left(t\right)}^2-{\overline{H}}^k\left(t\right)\right)-{\displaystyle {\displaystyle \sum }_{k,t}}\log {\lambda }^k\left(t\right)\end{array}\hfill \\ \hfill & =A-2 {\displaystyle \sum }_{k,f,t}\left|B^k\left(\omega ,t\right)\right|\cos \left({\mathit{\varphi }}^k\left(\omega ,t\right)-{\mathit{\Omega }}^k\left(\omega ,t\right)\right)\hfill \end{array}$$

(9)

where A denotes the terms that are irrelevant with ${\mathit{\varphi }}^k\left(\omega ,t\right)$, $B^k\left(\omega ,t\right)=\frac{W^k\left(\omega \right)H^k\left(t\right){\overline{Y}}^k\left(\omega ,t\right)}{{\beta }^k\left(\omega ,t\right)}$, $\cos {\Omega }^k\left(\omega ,t\right)=\frac{Re\left[{\overline{Y}}^k\left(\omega ,t\right)\right]}{\left|{\overline{Y}}^k\left(\omega ,t\right)\right|}$, and $\sin {\mathit{\Omega }}^k\left(\omega ,t\right)= \frac{\mathrm{Im}\left[{\overline{Y}}^k\left(\omega ,t\right)\right]}{\left|{\overline{Y}}^k\left(\omega ,t\right)\right|}$. Derivation of (9) is elucidated in Appendix B. The auxiliary function, $f^{+}\left(\theta ,\overline{\theta }\right)$ in Equation (4) is minimized when $\cos \left({\mathit{\varphi }}^k\left(\omega ,t\right)-{\Omega }^k\left(\omega ,t\right)\right)=\cos {\mathit{\varphi }}^k\left(\omega ,t\right)\cos {\Omega }^k\left(\omega ,t\right)+ \sin {\mathit{\varphi }}^k\left(\omega ,t\right)\sin {\Omega }^k\left(\omega ,t\right)=1$, namely, $\cos {\mathit{\varphi }}^k\left(\omega ,t\right)=\cos {\Omega }^k\left(\omega ,t\right)$and $\sin {\mathit{\varphi }}^k\left(\omega ,t\right)=\sin {\Omega }^k\left(\omega ,t\right)$. The update formula for $e^{j{\mathit{\varphi }}^k\left(\omega ,t\right)}$ eventually leads to

$$\begin{array}{cc}{e}^{j{\mathit{\varphi }}^k\left(\omega ,t\right)}& =\cos {\mathit{\varphi }}^k\left(\omega ,t\right)+j\sin {\mathit{\varphi }}^k\left(\omega ,t\right)\hfill \\ & =\frac{{\overline{Y}}^k\left(\omega ,t\right)}{\left|{\overline{Y}}^k\left(\omega ,t\right)\right|}\hfill \end{array}$$

(10)

The update formula for ${\beta }^k\left(\omega ,t\right)$and $H^k\left(t\right)$ for projection onto the constraint space is set to

$${\beta }^k\left(\omega ,t\right)= \frac{W^k\left(\omega \right)H^k\left(t\right)}{{{\displaystyle \sum }}_k{W}^k\left(\omega \right)H^k\left(t\right)}$$

(11)

$$H^k\left(t\right) \leftarrow \frac{H^k\left(t\right)}{{{\displaystyle \sum }}_k{H}^k\left(t\right)}$$

(12)

2.2.2. Estimation of L₁-Optimal Sparsity Parameter ${\mathsf{\lambda }}^{\mathrm{k}}\left(\mathrm{t}\right)$

This section aims to facilitate spectral dictionaries with adaptive sparse coding. First, the CMF model is defined as the following terms:

$$\begin{gathered} \begin{array}{c}\overline{W}=\left[I\otimes W^1\left(\omega \right)⋮I\otimes W^2\left(\omega \right)⋮\cdots ⋮I\otimes W^K\left(\omega \right)\right],\hfill \\ e^{j\overline{\underset{\_}{\mathsf{\varphi }}}\left(t\right)}=\left[e^{j{\underset{\_}{\mathsf{\varphi }}}^1\left(t\right)}⋮\cdots ⋮e^{j{\underset{\_}{\mathsf{\varphi }}}^K\left(t\right)}\right]\hfill \end{array} \\ \underset{\_}{\mathbf{y}}=\mathrm{vec}\left(Y\right)=\left[\begin{array}{c}{\underset{\_}{\mathbf{Y}}}^1(:)\\ \dots \\ {\underset{\_}{\mathbf{Y}}}^2(:)\\ \dots \\ ⋮\\ \dots \\ {\underset{\_}{\mathbf{Y}}}^K(:)\end{array}\right], \underset{\_}{\mathbf{h}}=\left[\begin{array}{c}{H}^1\left(t\right)\\ \dots \\ H^2\left(t\right)\\ \dots \\ ⋮\\ \dots \\ H^K\left(t\right)\end{array}\right], \underset{\_}{\mathsf{\lambda }}=\left[\begin{array}{c}{\lambda }^1\left(t\right)\\ \dots \\ {\lambda }^2\left(t\right)\\ \dots \\ ⋮\\ \dots \\ {\lambda }^K\left(t\right)\end{array}\right], \underset{\_}{\mathsf{\varphi }}=\left[\begin{array}{c}{\mathsf{\varphi }}^1\left(:,t\right)\\ \dots \\ {\mathsf{\varphi }}^2\left(:,t\right)\\ \dots \\ ⋮\\ \dots \\ {\mathsf{\varphi }}^K\left(:,t\right)\end{array}\right] \\ \overline{A}=\left[\begin{array}{cccc}\overline{W}°e^{j\overline{\underset{\_}{\mathsf{\varphi }}}\left(t\right)}& \mathbf{0}& \dots & \mathbf{0}\\ \mathbf{0}& \overline{W}°e^{j\overline{\underset{\_}{\mathsf{\varphi }}}\left(t\right)}& \mathbf{0}& ⋮\\ ⋮& \mathbf{0}& \overline{W}°e^{j\overline{\underset{\_}{\mathsf{\varphi }}}\left(t\right)}& \mathbf{0}\\ \mathbf{0}& \dots & \mathbf{0}& \overline{W}°e^{j\overline{\underset{\_}{\mathsf{\varphi }}}{\left(t\right)}_t}\end{array}\right] \end{gathered}$$

(13)

where “⊗” and “$°$” are the Kronecker product and the Hadamard product, respectively. The term vec(∙) denotes the column vectorization and the term $I$ is the identity matrix. The goal is then set to compute the regularization parameter ${\lambda }^k\left(t\right)$ related to each $H^k\left(t\right)$. To achieve the goal, the parameter $p$ in Equation (4) is set to 1 to acquire a linear expression (in ${\lambda }^k\left(t\right)$). In consideration of the noise variance ${\sigma }^2$, Equation (4) can concisely be rewritten as:

$$F\left(\underset{\_}{\mathbf{h}},\underset{\_}{\mathit{\lambda }}\right)=\frac{1}{2{\sigma }^2}\parallel \underset{\_}{\mathbf{y}}- \overline{A}\underset{\_}{\mathbf{h}}{\parallel }_F^2+{\underset{\_}{\mathsf{\lambda }}}^T\underset{\_}{\mathbf{h}}-{\left(\log \underset{\_}{\mathsf{\lambda }}\right)}^T\underset{\_}{\mathbf{1}}$$

(14)

where the $\underset{\_}{\mathbf{h}}$ and $\underset{\_}{\mathsf{\lambda }}$ terms indicate vectors of dimension $R\times 1$ (i.e., $R=F\times T\times K$), and the superscript ‘$T$’ is used to denote complex Hermitian transpose (i.e., vector (or matrix) transpose followed by complex conjugate). The Expectation–Maximization (EM) algorithm will be used to determine $\underset{\_}{\mathsf{\lambda }}$ and $\underset{\_}{\mathbf{h}}$ is the hidden variable where the log-likelihood function can be optimized with respect to $\underset{\_}{\mathit{\lambda }}$. The log-likelihood function satisfies the following [12]:

$$\ln p\left(\underset{\_}{\mathbf{y}} |\underset{\_}{\mathit{\lambda }},\overline{A},{\sigma }^2\right) \ge {\displaystyle \int }Q\left(\underset{\_}{\mathbf{h}}\right)\ln \left(\frac{p\left(\underset{\_}{\mathbf{y}},\underset{\_}{\mathbf{h}}|\underset{\_}{\mathit{\lambda }},\overline{\mathbf{A}},{\sigma }^2\right)}{Q\left(\underset{\_}{\mathbf{h}}\right)}\right) d\underset{\_}{\mathbf{h}}$$

(15)

by applying the Jensen’s inequality for any distribution $Q\left(\underset{\_}{\mathbf{h}}\right)$. The distribution can simply verify the posterior distribution of $\underset{\_}{\mathbf{h}}$, which maximizes the right-hand side of Equation (15), is given by $Q\left(\underset{\_}{\mathbf{h}}\right)=p\left(\underset{\_}{\mathbf{h}}|\underset{\_}{\mathbf{y}},\underset{\_}{\mathit{\lambda }},\overline{A},{\sigma }^2\right)$. The posterior distribution in the form of the Gibbs distribution is proposed as follows:

$$Q\left(\underset{\_}{\mathbf{h}}\right)= \frac{1}{Z_h}\exp \left[-F\left(\underset{\_}{\mathbf{h}}\right)\right] \mathrm{where} Z_h={\displaystyle \int }\exp \left[-F\left(\underset{\_}{\mathbf{h}}\right)\right]d\underset{\_}{\mathbf{h}}$$

(16)

The term $F\left(\underset{\_}{\mathbf{h}}\right)$ in Equation (16) as the function of the Gibbs distribution is essential for simplifying the adaptive optimization of $\underset{\_}{\mathit{\lambda }}$. The maximum-likelihood (ML) estimation of $\underset{\_}{\mathit{\lambda }}$ can be decomposed as follows:

$${\underset{\_}{\mathit{\lambda }}}^{ML}=\underset{\underset{\_}{\mathit{\lambda }}}{\arg \max }{\displaystyle \int }Q\left(\underset{\_}{\mathbf{h}}\right)\ln p\left(\underset{\_}{\mathbf{h}}|\underset{\_}{\mathit{\lambda }}\right)d\underset{\_}{\mathbf{h}}$$

(17)

In the same way,

$${\sigma }^2{}_{ML} =\underset{{\sigma }^2}{\arg \max }{\displaystyle \int }Q\left(\underset{\_}{\mathbf{h}}\right)\ln p\left(\underset{\_}{\mathbf{y}} |\underset{\_}{\mathbf{h}},\overline{A},{\sigma }^2\right)d\underset{\_}{\mathbf{h}}$$

(18)

Individual element of $H$ is required to be exponentially distributed with independent decay parameters that delivers $p\left(\underset{\_}{\mathbf{h}}|\underset{\_}{\mathit{\lambda }}\right)= {\displaystyle \prod }_g{\lambda }_g\exp \left(-{\lambda }_g{h}_g\right)$, thus Equation (17) obtains

$${\underset{\_}{\mathit{\lambda }}}^{ML}=\underset{\underset{\_}{\mathit{\lambda }}}{\arg \max }{\displaystyle \int }Q\left(\underset{\_}{\mathbf{h}}\right)\left(\ln {\lambda }_g-{\lambda }_g{h}_g \right)d\underset{\_}{\mathbf{h}}$$

(19)

The term $\underset{\_}{\mathbf{h}}$ denotes the dependent variable of the distribution $Q\left(\underset{\_}{\mathbf{h}}\right),$ whereas other parameters are assumed to be constant. As such, the $\underset{\_}{\mathit{\lambda }}$ optimization in Equation (19) is derived by differentiating the parameters within the integral with respect to $\underset{\_}{\mathbf{h}}$. As a result, the functional optimization [37] of $\underset{\_}{\mathit{\lambda }}$ then obtains

$${\lambda }_g=\frac{1}{{\displaystyle \int }{h}_{g}Q\left(\underset{\_}{\mathbf{h}}\right)d\underset{\_}{\mathbf{h}}}$$

(20)

where $g=1,2,\dots ,R$, ${\lambda }_g$ denotes the $g^{th}$ element of $\underset{\_}{\mathit{\lambda }}$. Notice that the solution $\underset{\_}{\mathbf{h}}$ naturally splits its elements into distinct subsets ${\underset{\_}{\mathbf{h}}}_M$ and ${\underset{\_}{\mathbf{h}}}_P$, consisting of components ${\forall }_{m }\in M$ so that $h_m>0$ and components ${\forall }_{p }\in P$ so that $h_{P }=0$. The sparsity parameter is then obtained as presented in Equation (21):

$${\lambda }_{g }=\{\begin{array}{c}\frac{1}{{\displaystyle \int }{h}_g{Q}_M\left({\underset{\_}{\mathbf{h}}}_M\right)d{\underset{\_}{\mathbf{h}}}_M }=\frac{1}{h_g^{\mathrm{MAP}}} \mathrm{if} g \in M\\ \frac{1}{{\displaystyle \int }{h}_g{\widehat{Q}}_P\left({\underset{\_}{\mathbf{h}}}_P\right)d{\underset{\_}{\mathbf{h}}}_P }=\frac{1}{u_g} \mathrm{if} g \in P\end{array}$$

(21)

and its covariance X is given by

$$X_{ab }= \{\begin{array}{c}{\left({\overline{C}}_P^{-1}\right)}_{ab}, \mathrm{if} a,b \in M\\ u_p^2{\delta }_{ab}, \mathrm{Otherwise}.\end{array}$$

(22)

where ${\widehat{Q}}_P\left({\underset{\_}{\mathbf{h}}}_P\ge 0\right)= {\displaystyle \prod }_{p\in P}\frac{1}{u_p}\exp \left(\frac{-h_p}{u_p}\right)$, ${\overline{C}}_P=\frac{1}{{\sigma }^2}{\overline{A}}_P^T{\overline{A}}_P$ and $u_p\leftarrow u_p\frac{-{\widehat{b}}_p+\sqrt{{\widehat{b}}_p^2+4\frac{{\left(\widehat{\mathbf{C}}\underset{\_}{\mathbf{u}}\right)}_p}{{\tilde{u}}_p}}}{2{\left(\widehat{\mathbf{C}}\underset{\_}{\mathbf{u}}\right)}_p}$. The function $Q_M\left({\underset{\_}{\mathbf{h}}}_M\right)$ will be expressed as the unconstrained Gaussion with mean ${\underset{\_}{\mathbf{h}}}_M^{\mathrm{MAP}}$ and covariance ${\overline{C}}_M^{-1}$ based on a multivariate Gaussian distribution. Similarly, the inference for ${\sigma }^2$ can be computed as

$${\sigma }^2=\frac{1}{N_0}{\displaystyle \int }Q\left(\underset{\_}{\mathbf{h}}\right)\left(\Vert \underset{\_}{\mathbf{y}}- \overline{A}\underset{\_}{\mathbf{h}}{\Vert }^2\right)d\underset{\_}{\mathbf{h}}$$

(23)

where

$${\widehat{h}}_g=\{\begin{array}{c} h_g^{\mathrm{MAP}} \mathrm{if} g \in M\\ u_g \mathrm{if} g \in P\end{array}$$

The core procedure of the proposed CMF method is based on $L_1$-optimal sparsity parameters. The estimated sources are discovered by multiplying the respective rows of the $W^k\left(\omega \right)$ components with the corresponding columns of the $H^k\left(t\right)$ weights and time-varying phrase spectrum $e^{j{\mathit{\varphi }}^k\left(\omega ,t\right)}$. The separated source ${\widehat{s}}_j\left(t\right)$ is obtained by converting the time-frequency represented sources into the time domain. Derivation of L₁-optimal sparsity parameter, is elucidated in the Appendix C.

2.3. Sound Event Classification

Once the separated sound signal is obtained, the next step is to identify the sound event. A multiclass support vector machine (MSVM) is employed to achieve the goal. The MSVM is comprised of two phases: the learning phase and the evaluation phase. The MSVM is based on one versus one strategy (OvsO) that splits observed $c$ classes into $\frac{c\left(c-1\right)}{2}$ binary classification sub-problems. To train the ${𝓌}^{th}$ MSVM model, the MSVM will construct hyperplanes for discriminating each observed data into its corresponding class by executing the series of binary classification. Starting from the learning phase, sound signatures are extracted from the training dataset in the time-frequency domain. The sound signatures that were studied in this research were the Mel frequency cepstral coefficients (MFCC: MF), short-time energy (STE: $E\left(t\right)$), and short term zero-crossing rate (STZCR: $STZ\left(t\right)$), which can be orderly expressed as: $MF=2525\times \log \left[1+\left(f/7\right)\right]$, $E\left(t\right) ={\displaystyle \sum }_{\tau =-\infty }^{\infty }{\left[y\left(t\right)·f_w\left(t-\tau \right)\right]}^2$, $Z\left(t\right) ={\displaystyle \sum }_{\tau =-\infty }^{\infty }\left|sgn\left[s\left(\tau \right)\right]-sgn\left[s\left(\tau -1\right)\right]\right|·f_w\left(t-\tau \right)$ where $f_w\left(t\right)$ denotes the windowing function. The training signals are segmented into small blocks, then the individual block is extracted to the three signature parameters. The mean supervector is then computed as an average of individual feature of all blocks for each sound event input. Thus, the mean feature supervector ($O$) with a corresponding sound-event-label vector ($\left(w\right)$) is paired together (i.e., ($\psi \left(O,w\right)$)) and supplied to the MSVM model. The discriminant formula can be expressed as:

$$\begin{array}{cc}\left\{\widehat{𝓌},\beta \right\}& =\underset{𝓌}{\arg \max }\left\{{\alpha }_{𝓌}^T\psi \left(O,w;\beta \right)\right\}\hfill \\ & =\underset{𝓌}{\arg \max }\left\{\underset{\beta }{\max }{\displaystyle {\displaystyle \sum }_{i=1}^{\left|w\right|}}{\alpha }_{𝓌}^T\psi \left(O_{i|\beta },w_i\right)\right\}\hfill \end{array}$$

(24)

where $\left(O_{i|\beta },w_i\right),i=1,\dots ,M$ represents the $i^{th}$ separated sound signals; the weight vector ${\alpha }_{𝓌}$ is employed for individual class $𝓌$ to compute a discriminant score for the $O$; the $i$ term is the index of the block order ($\beta$); and the function ${\alpha }_{𝓌}^T\psi \left(O,w;\beta \right)$ measures a linear discriminant distance of the hyperplane with the extracted feature vector from the observed data. The MSVM based OvsO strategy for class ${𝓌}^{th}$ and other, the hyperplane, can be maximized as ${\alpha }_{𝓌}^T\psi \left(O,w;\beta \right)+b_{𝓌}$ and can then be learned via the following equation as

$$\underset{{\alpha }_{𝓌},{\xi }_{𝓌}}{\min }\frac{1}{2}\Vert {\alpha }_{𝓌}{\Vert }^2+C{\displaystyle \sum }_{i=1}^M{\xi }_i^{𝓌}$$

(25)

where ${\xi }_i^{𝓌}\ge 0$, $b_{𝓌}$ is a constant. The term ${\displaystyle \sum }_{i=1}^M{\xi }_i^{𝓌}$ denotes a penalty function for tradeoff between a large margin and a small error penalty. The optimal hyperplane can be determined by minimizing $\frac{1}{2}\Vert {\alpha }_{𝓌}{\Vert }^2$ to maximize the condition (i.e., ${\alpha }_{𝓌}^T\psi \left(O,w;\beta \right)+b_{𝓌} \ge 1-{\xi }_i^{𝓌})$. If the conditional term is greater than $1-{\xi }_i^{𝓌}$, then the estimated sound event belongs to the ${𝓌}^{th}$ class. Otherwise, the estimated sound event classifies into other classes.

The overview of the proposed algorithm is presented in the following table as Algorithm 1.

Algorithm 1 Overview of the proposed algorithm.

(1) Compute $Y\left(\omega ,t\right)=\mathrm{STFT}\left(y\left(t\right)\right)$ from the noisy single-channel mixture $y\left(t\right)$. (2) Initial values ${\mathbf{W}}^k\left(\omega \right),{\mathbf{H}}^k\left(t\right)$, ${\beta }^k\left(\omega ,t\right)$, fixing the value of ${\mathit{\varphi }}^k\left(\omega ,t\right)$ at $e^{j{\mathsf{\varphi }}^k\left(\omega ,t\right)} =\frac{Y\left(\omega ,t\right)}{\left|Y\left(\omega ,t\right)\right|}$, and calculate ${\lambda }^k\left(t\right)$ and ${\sigma }^{2 }$. (3) Update $\overline{\theta }= \left\{ \overline{\mathbf{X}},\overline{\mathbf{H}}\right\}$, $\theta = \left\{{\mathbf{W}}^k\left(\omega \right),{\mathbf{H}}^k\left(t\right), {\mathit{\varphi }}^k\left(\omega ,t\right)\right\}$, ${\beta }^k\left(\omega ,t\right)$. (4) Update parameters (21) and (23) until convergence is reached as determined by the rate of change of the parameters update falling within a pre-determined threshold. (5) Estimation of each source by multiplying the respective rows of the spectral components ${\mathit{W}}^k\left(\omega \right)$ with the corresponding columns of the mixture weights ${\mathit{H}}^k\left(t\right)$ and time-varying phrase spectrum $e^{j{\mathit{\varphi }}^k\left(\omega ,t\right)}$. (i.e., ${\left|{\widehat{S}}_i\right|}^{.2}={\displaystyle \sum }_{k=1}^{K_i}{W}_i{}_f^k{H}_i{}_t^k·e^{{j\varphi }_i{}_{f,t}^{\left(k\right)}}$ and construct the binary TF mask for the $i^{th}$ source $M_i\left(f,t_s\right):=\{\begin{array}{c}\begin{array}{cc}1,& \mathrm{if} {\left|{\widehat{S}}_i\left(f,t_s\right)\right|}^{.2}> {\left|{\widehat{S}}_j\left(f,t_s\right)\right|}^{.2}, i\ne j\end{array}\\ \begin{array}{cc}0,& otherwise \end{array}\end{array}$). (6) Convert the time-frequency represented sources into time domain to obtain the separated sources ${\widehat{s}}_j\left(t\right)$ i.e., ${\widehat{s}}_j\left(t\right)= STF{T}^{-1}\left({\left|{\widehat{S}}_i\right|}^2\right)$. (7) Classify the ${𝓌}^{th}$ sound event by computing the optimal hyperplane ${\alpha }_{𝓌}^T\psi \left(O,w;\beta \right)+b_{𝓌}$ by minimizing the following equation: $\underset{{\alpha }_{𝓌},{\xi }_{𝓌}}{\min }\frac{1}{2}\Vert {\alpha }_{𝓌}{\Vert }^2+C{\displaystyle \sum }_{i=1}^M{\xi }_i^{𝓌}$.

3. Experimental Results and Analysis

The performance was evaluated on recorded sound-event signals in a low noisy environment at 20 signal-to-noise ratios (SNRs). The sound-event database had a total of 500 recorded signals containing four event classes: speech (SP), door open (DO), door knocking (DK), and footsteps (FS). An overview of the experimental setup is given as the following: all signals had a 16-bit resolution and a sampling frequency of 44.1 KHz. A 2048 length of Hanning window with 50% overlap was used for signal processing. Nonlinear SVM with a Gaussian RBF kernel was used for constructing the MSVM learning model. Other kernels such as polynomials, sigmoid, and even linear function were tested, but the best performance was delivered by the Gaussian kernel. A 4-fold cross-validation strategy was used in the training phase for tuning the classifier parameters when using 80% of the recorded signals (n = 400) from the sound-event database.

The performance of the proposed noisy sound separation and event classification (NSSEC) method was demonstrated and presented into the following two sections: (1) the separating performance, and (2) the MSVM classifier.

3.1. Sound-Event Separation and Classification Performance

Event mixtures consist of two sound-event signals in low noisy environment at 20 dB SNRs. A hundred sound-event signals of four classes were randomly selected and then mixed to generate 120 mixtures of six types (i.e., DO + DK, DO + FS, DO + SP, DK + FS, DK + SP, and FS + SP). The separation performance measured the signal-to-distortion ratio (SDR) (i.e., $SDR=10 lo{g}_{10}\left(\Vert s_{target}{\Vert }^2/\Vert e_{interf}+e_{noise}+e_{artif}{\Vert }^2\right)$ where $e_{interf}$, $e_{noise}$, and $e_{artif}$). The SDR represents the ratio of the magnitude distortion of the original signal by the interference from other sources. The proposed separation method was compared with the state-of-the-art NMF approach (i.e., CMF [38], NMF-ISD [14,39], and SNMF [40,41,42] methods). The cost function was the least squares with 500 maximum number of iterations.

3.1.1. Variational Sparsity Versus Fixed Sparsity

In this implementation, several experiments were conducted to investigate the effect of sparsity regularization on source separation performance. The proposed separation method was evaluated by variational sparsity in the case of (1) uniform constant sparsity with low sparseness e.g.,${\lambda }_t^k= 0.01$ and (2) uniform constant sparsity with high sparseness (e.g., ${\lambda }_t^k= 100)$. The hypothesis is that the proposed variational sparsity will significantly yield improvement of the audio source separation when compared with fixed sparsity.

To investigate the impact of uniform sparsity parameter, the set of sparsity regularization values from 0 to 10 with a 0.5 interval were determined for each experiment of 60 mixtures of six types. Results of the uniform regularization given by various sparsity (i.e., ${\lambda }_t^k= 0, 0.5, \dots , 10$) is illustrated in Figure 2.

Figure 2 illustrates that the best performance of the unsupervised CMF was in a range of 1.5–3, which yielded the highest SDR of over 8dB. When the term ${\lambda }_t^k$ was set too high, the low spectral values of sound-event signals were overly sparse. This overfitting sparsity ${\mathit{H}}^k\left(t\right)$ caused the separation performance toward a tendency to degrade. Conversely, the underfitting sparsity ${\mathit{H}}^k\left(t\right)$ occurred when the term ${\lambda }_t^k$ was set too low. The coding parameter ${\mathit{H}}^k\left(t\right)$ could not distinguish between the two sound-event signals. It was also noticed that if the factorization is non-regularized, this will cause the separation results to contain a mixed sound. According to the uniform sparsity results in Figure 2, the separation performance of the proposed method varies depending on the assigned sparsity values. Thus, it is challenging to find a solution for the indistinctness among the sound-event sources in the TF representation to determine the optimal value of sparseness. Thus, this introduces the importance of determining the optimal $\lambda$ for separation.

Table 1. Comparison of average SDR performance on three types of mixtures between uniform regularization methods and the proposed method.

Mixtures	Methods	SDR
DO + DK	Proposed method	7.63
	(Best) Uniform regularization sparsity	6.59
DO + FS	Proposed method	9.06
	(Best) Uniform regularization sparsity	8.74
DO + SP	Proposed method	8.45
	(Best) Uniform regularization sparsity	6.91
DK + FS	Proposed method	7.04
	(Best) Uniform regularization sparsity	6.35
DK + SP	Proposed method	9.72
	(Best) Uniform regularization sparsity	7.78
FS + SP	Proposed method	9.81
	(Best) Uniform regularization sparsity	7.42

presents the essential sparsity value on the separation performance by comparing the proposed method given by variational sparsity against the uniform sparsity scheme. The average performance improvement of the proposed adaptive CMF method against the uniform constant sparsity was 1.32 dB SDR. The SDR results clearly indicate that the adaptive sparsity yielded the surpass separation performance over the constant sparsity scheme. Hence, the proposed variational sparsity improves the performance of the discovered original sound-event signals by adaptively selecting the appropriate sparsity parameters to be individually adapted for each element code (i.e., ${\lambda }_{g }=\{\begin{array}{c}\frac{1}{{\displaystyle \int }{h}_g{Q}_M\left({\underset{\_}{\mathbf{h}}}_M\right)d{\underset{\_}{\mathbf{h}}}_M }=\frac{1}{h_g^{\mathrm{MAP}}} \mathrm{if} g \in M\\ \frac{1}{{\displaystyle \int }{h}_g{\widehat{Q}}_P\left({\underset{\_}{\mathbf{h}}}_P\right)d{\underset{\_}{\mathbf{h}}}_P }=\frac{1}{u_g} \mathrm{if} g \in P\end{array}$ and ${\sigma }^2=\frac{1}{N_0}{\displaystyle \int }Q\left(\underset{\_}{\mathbf{h}}\right)(\Vert \underset{\_}{\mathbf{y}}- \overline{A}\underset{\_}{\mathbf{h}}{\Vert }^2)d\underset{\_}{\mathbf{h}}$ where ${\widehat{h}}_g=\{\begin{array}{c} h_g^{\mathrm{MAP}} \mathrm{if} g \in M\\ u_g \mathrm{if} g \in P\end{array}$). Consequently, the optimal sparsity facilitates the estimated spectral dictionary via the estimated temporal code. The quantitative measures of separation performance were performed to assess the proposed single-channel sound event separation method. The overall average signal-to-distortion ratio (SDR) was 8.62 dB as illustrated in Figure 3.

Each sound-event signal has its own temporal pattern that can be clearly noticed in TF representation. Examples of sound-event signals in the TF domain are illustrated in Figure 4. Through the adaptive L₁-SCMF method, the proposed single-channel separation method can generate complex temporal patterns such as speech. Thus, the separation results clearly indicate that the performances of noisy source separation perform with high SDR values.

3.1.2. Comparison of the Proposed Adaptive CMF with Other SCBSS Methods Based on NMF

This section presents the adaptive CMF separating performance against the state-of-the-art NMF methods (i.e., CMF, SNMF, and NMF-ISD). In the compared methods, the experimental variables such as the normalizing time-frequency domain were computed by using the short-time Fourier transform (i.e., 1024-point Hanning window with 50% overlap). The number of factors was two, with a sparsity weight of 1.5. One hundred random realizations of twenty second-event mixtures were executed. As a result, the average SDRs are presented in

Table 2. Comparison of average SDR and SIR performance on three types of mixtures between SCICA, NMF-ISD, SNMF, CMF, and the proposed method.

Mixtures	Methods	SDR
Door Open	Proposed method	8.38
	CMF	7.11
	SNMF	6.23
	NMF-ISD	6.17
Door Knocking	Proposed method	8.13
	CMF	7.06
	SNMF	6.52
	NMF-ISD	6.55
Footsteps	Proposed method	8.36
	CMF	7.89
	SNMF	6.62
	NMF-ISD	6.06
Speech	Proposed method	9.60
	CMF	6.73
	SNMF	5.61
	NMF-ISD	5.32

. The proposed adaptive CMF method yielded the best separating performance over the CMF, SNMF, and NMF_ISD methods with the average improvement SDR at 2.13 dB. The estimated door open signals obtained the highest SDR among the four event categories.

The sparsity parameter was carefully adapted using the proposed adaptive L₁-SCMF method exploiting the phase information and temporal code of the sources, which is inherently ignored by SNMF and NMF-ISD and has led to an improved performance of about 2 dB in SDR. On the other hand, the parts decomposed by the CMF, SNMF, and NMF-ISD methods were unable to capture the phase spectra and the temporal dependency of the frequency patterns within the audio signal.

Additionally, the CMF and NMF-ISD are unique when the signal adequately spans the positive octant. Thus, the rotation of $W$ and opposite $H$ can obtain the same results. The CMF method can easily be over or under sparse resolution of the factorization due to manually determining the sparsity value.

3.2. Performance of Event Classification Based on MSVM Algorithm

This section elucidates the features and performance of the MSVM-learning model. The MSVM-learning model was investigated to obtain the optimal size of the sliding window and then determine the significant features that led to the classification performance. Finally, the efficiency of the MSVM model was evaluated. These topics are presented in order in the following parts.

3.2.1. Determination Optimal Window Length for Feature Encoding

For the MSVM method, sound-event signals are segmented into small blocks for encoding feature parameters by using a fixed-length of the sliding window. The sets of feature vectors are computed using the mean supervector and then loaded to the MSVM model for learning and constructing the hyperplane. The size of blocks can affect the information of the feature vectors, which leads to the classifier performance. The block’s size will affect the ${\alpha }_{𝓌}$, hence modifying the block size will mark the learning efficiency of the MSVM model. Therefore, in order to obtain the optimal value of ${\alpha }_{𝓌}$, the optimal block size was exploited by training the MSVM model given various lengths of window sizes (i.e., 0.5, 1, 1.5, and 2.0 s) to learn the 400 noisy sound-event signals of four event classes with cross-validation.

The experimental results are plotted in Figure 5, where the block size varies from 0.5 to 2.0 with 0.5 increments. The MSVM model of the 1.5 s block size yielded the best sound-event classification at 100% accuracy. The sliding window function benefits from SVM to learn an unknown sound event by generating the set of blocks from the observed event, regarded as a number of observed events. As a result, a set of sound event characteristics were computed for each block (i.e., $O_{i|\beta },w_i$ in Equation (24)).

The optimal length of the window size can capture the signature of the sound event. If the window length is too short, the encoded features will then deviate from the character of the sound event. In addition, the mean supervector is computed from the set of features of all blocks, which can be regarded as the mean of the probability distribution of the features. This mean supervector advantages the MSVM to reduce misclassifications when compared to the conventional SVM. Hence, the STFT of all experiments set the window function at 1.5 s.

3.2.2. Determination of Sound-Event Features

Each sound-event signal was encoded with three features: Mel frequency cepstral coefficients (MFCCs), short-time energy (STE), and short-time zero-crossing rate (STZCR). MFCCs are represented as a frequency domain feature that is evaluated in a similar assembly to the human ear (i.e., logarithmic frequency perception). STE is the total spectrum power of an observed event. The STZCR denotes the number of times that the signal amplitude interval satisfies the condition (i.e., $STZCR= \left(1/T-1\right){\displaystyle \sum }_{t=1}^{T-1}\left[\right[\left\{s_t{s}_{t-1}<0\right\}$ where $\left[\right[\left\{s_t{s}_{t-1}<0\right\}$ is 1 if the condition is true and 0 otherwise). The STZCR features of four sound-event classes are illustrated in Figure 6.

The STZCR feature represents unique patterns of four sound-event classes. The four sound-event patterns are different in shape and data range. Similarly, the MCFFs and STE features extract distinctive patterns of all event classes, except for the patterns between door knocking and footstep, as illustrated in Figure 7.

Figure 7 aims to compare the characteristics of similar sound events such as door knocking and footsteps. Thus, MFCCs and STE features were used to illustrate the patterns of sound events. Figure 7a represents the five orders of MFCC features to compare patterns between door knocking and walking while the STE features are shown in Figure 7b.

The proposed method separated the six categories of mixtures, then classified each estimated sound event signal into its corresponding class. Classified results of the six categories are presented as confusion matrixes below:

	Actual
Predict	DO	DK		DO	FS		DO	SP
DO	19	3	DO	12	8	DO	19	5
DK	3	15	FS	4	16	SP	3	13
	DK	FS		DK	SP		FS	SP
DK	12	4	DK	16	2	FS	14	6
FS	9	15	SP	5	17	SP	3	17

The classification of the proposed method was measured by Precision = TP/(TP + FP), Recall = TP/(TP + FN), and F1-score = 2 × (Precision × Recall)/(Precision + Recall). The TP and TN terms refer to the true positive and true negative, while the FP and FN terms mean false positive and false negative. The scores of Precision, Recall, and F1-score were 0.7667, 0.7731, and 0.7699, respectively.

Each feature represents unique characteristics of an individual sound event. Thus, features were matched into seven cases for exploiting their influence on the MSVM classifiers (i.e., {(MFCC), (STE), (STZCR), (MFCC, STE), (MFCC, STZCR), (STE, STZCR), (MFCC, STE, STZCR)}).

As shown in Figure 8, the MSVM model given by MFCCs and STZCR yielded the best classified accuracy at 100%, with less deviation among the other cases. Therefore, the separated signals were then classified by the proposed MSVM method given by the MFCC and STZCR vectors and the 1.5 s window function. The computational complexity of the proposed method was analyzed by two steps. First, the adaptive L1-SCMF method was NP-hard. Big-O of the adaptive L₁-SCMF method consists of spectral basis ($m$), temporal code ($n$), and phase information that rely on components ($k$). Thus, Big-O of the separation step is ${\left(mn\right)}^{O\left(k^2\right)}$. For MSVM steps, it is a polynomial algorithm where Big-O is $O\left(n^3\right)$. Therefore, the computational complexity of the proposed method based on Big-O is ${\left(mn\right)}^{O\left(k^2\right)}$. All experiments were performed by a PC with Intel^® Core™ i7-4510U CPU2.00 GHz and 8 GB RAM. MATLAB was used as the programming platform.

3.2.3. Performance of MSVM Classifier

The MSVM-classifier performance is presented in terms of percentage of the corrected sound-event classification. The 240 separated signals of four classes from the proposed separation method were individually identified by the MSVM classifier.

Figure 9 compares the classification performance on the four classes of individual sound events. The best classification accuracy was door open, followed by footstep, door knocking, and speech. On the other hand, the classification results based on the mixed sound events are illustrated in Figure 10. The MSVM model delivered the highest performance of the door-open event with 84% accuracy.

From the above experiments, the proposed method yields an average classification accuracy of 76.67%. The MSVM method can well discriminate and classify the mixed event signals with high classification accuracy (i.e., the mixture of door open with door knocking and door knocking with speech were correctly classified with above 80% accuracy). Due to the MFCC and STZCR features in the individual event, these signals had obvious distinguishable patterns, as shown in the example of STZCR plots in Figure 6. Despite the SDR scores of the separated signals between door open and door knocking being relatively low (as given in Figure 3), the MSVM yielded the highest classification accuracy for the door open with door knocking mixture (DO + DK). This is attributed to the fact that interference remaining in the separated event signals causes the extracted MFCC and STZCR vectors to deviate from their original sound event vectors.

4. Conclusions

A novel solution for classification of the noisy mixtures using a single microphone was presented. The complex matrix factorization was proposed and extended by adaptively tuning the sparse regularization. Thus, the desired L₁-optimal sparse decomposition was obtained. In addition, the phase estimates of the CMF could extract the recurrent pattern of the magnitude spectra. The updated equation was derived through an auxiliary function. For classification, the multiclass support vector was used as the mean supervector for encoding the sound-event signatures. The proposed noisy sound separation and event classification method was demonstrated by using four sets of noisy sound-event mixtures, which were door open, door knocking, footsteps, and speech. Based on the experimental results, first, the optimal window length of STFT was found where 1.5 s of the sliding window yielded the best separation performance. The second was two significant features that were ZCR and MFCCs. These parameters were set for examining the proposed method. The proposed method achieved outstanding results in both separation and classification. In future work, the proposed method will be evaluated on a public dataset such as the DCASE 2016, alongside the comparison with other machine learning algorithms.