Review on Sound-Based Industrial Predictive Maintenance: From Feature Engineering to Deep Learning

Abstract

Sound-based predictive maintenance (PdM) is a critical enabler for ensuring operational continuity and productivity in industrial systems. Due to the diversity of equipment types and the complexity of working environments, numerous feature engineering methods and anomaly diagnosis models have been developed based on sound signals. However, existing reviews focus more on the structures and results of the detection model, while neglecting the impact of the differences in feature engineering on subsequent detection models. Therefore, this paper aims to provide a comprehensive review of the state-of-the-art feature extraction methods based on sound signals. The judgment standards in the sound detection models are analyzed from empirical thresholding to machine learning and deep learning. The advantages and limitations of sound detection algorithms in varied equipment are elucidated through detailed examples and descriptions, providing a comprehensive understanding of performance and applicability. This review also provides a guide to the selection of feature extraction and detection methods for the predictive maintenance of equipment based on sound signals.

1. Introduction

Predictive maintenance (PdM) is a critical strategy to ensure the reliable operation of industrial equipment in the manufacturing, energy, and transportation sectors [1,2,3,4]. Unscheduled equipment shutdown can cause substantial economic losses and pose risks to operator safety, prompting manufacturing companies to allocate considerable manpower and resources to equipment safety inspection [5,6,7,8]. However, unnecessary or improper maintenance activities consume up to one-third of the total maintenance budget in manufacturing firms [9,10].

Recently, sound-based PdM methods have attracted considerable interest in academic and industrial communities due to the non-invasive sensing and the ability to capture high-dimensional information [11]. As an inherent characteristic of equipment operation, sound can precisely detect subtle changes during operation, including friction, impact, and resonance [12]. These changes often reflect early-stage faults, enabling the PdM of industry equipment. Saucedo et al. [13] demonstrated that mechanical acoustic emission signals can accurately characterize critical fault modes, such as bearing wear, gear tooth breakage, and rotor imbalance [14].

The sound PdM system includes online/training and offline/testing processes [15,16]. The general flow of sound anomaly detection is shown in Figure 1. In each part, the analysis processing of the sound signal involves two steps: the acoustic feature engineering and the anomalous sound detection. On the one hand, the construction of feature engineering for sound signals forms the foundation of predictive maintenance systems, ensuring that the extracted features reflect subtle changes in the operation of the equipment. The feature extraction methods transform the one-dimensional raw sound signals into high-dimensional feature vectors, which can represent the operational state of the target equipment. Therefore, the existing feature extraction methods are classified into statistical parameters, waveform decomposition, and spectrogram analysis.

On the other hand, anomaly detection serves as the core component, responsible for accurately identifying abnormal events from the extracted features. Driven by the rapid progress of intelligent detection, equipment sound analysis has progressed from empirical threshold-based methods to feature-driven machine learning, then to data-centric deep learning, and is now approaching physics-integrated paradigms as a future direction. The development trend of sound anomaly detection for the PdM system is shown in Figure 2. The traditional experience-driven model has the highest detection efficiency, but the interference from complex external noise makes experience unable to meet the current needs of predictive maintenance for complex equipment. Thereby, feature-driven Support Vector Machine (SVM) models determine anomalies by detecting the distance between features. The data-driven convolutional neural networks (CNNs) extract latent information from raw sound signals. Feature and data-driven models have become the mainstream of current sound detection methods for improving the robustness and accuracy. The main feature extraction methods and detection models in recent sound-based industrial PdM are summarized in

Table 1. Summary of sound equipment anomaly detection methods in recent years.

Applications	Feature Engineering	Detection Methods	Accuracy	Ref.
Power System	Wavelet Transform (WT) + Histogram of Oriented Gradient	Support Vector Machine (SVM)	90%	[19]
	Statistical Features Mel-Frequency Cepstral Coefficients (MFCCs)	Experience Thresholds Hidden Markov Model (HMM)	97.83%	[20]
Motor	Statistical Features	Decision Tree (DT) + SVM + K-Nearest Neighbor (KNN)	99.75%	[21]
	Raw Waveform	Deep Convolutional Neural Networks (DCNNs)	95%	[22]
	Discrete Wavelet Transform (DWT)	SVM + KNN	92.12%	[23]
	Short-Time Fourier Transform (STFT)	AutoEncoder (AE)	96.4%	[24]
Bearing	Statistical Features DWT Time-Spectral Features	SVM Random Forest (RF) Linear Discriminant Analysis (LDA)	96.78% 96.79% 95.60%	[25]
	Wavelet Packet Transform (WPT) + Teager Energy Operator	Kurtosis Index	-	[26]
	Empirical Mode Decomposition (EMD) + Kurtosis Superposition	Appropriate Threshold	-	[27]
	Raw Waveform	Convolutional Natural Network (CNN) + Bi-directional Long Short-Term Memory (BiLSTM) + Bidirectional Gated Recurrent Unit (BiGRU)	90%	[28]
	Raw Waveform	Deep Graph Convolutional Networks (DGCN)	91%	[29]
Engine	Variational Mode Decomposition (VMD) + MFCC	Long Short-Term Memory (LSTM)	97%	[30]
	DWT + WPT	Artificial Neural Network (ANN)	90%	[31]
	MFCC	Multilayer Perceptron (MLP) + CNN	94.55%	[32]
Rotating machinery	STFT	MPNet	95.83%	[33]
	STFT + VMD	LSTM + Bi-directional Temporal Convolutional Network (BiTCN)	94.1%	[34]
Car Mirror Folding	VMD + Statistical Features	SVM	95.8%	[35]
Equipment	STFT	CNN	82.8%	[36]
Environments	Statistical Features + MFCC	KNN	85%	[37]
Railway Machines	EMD + Statistical Features	Weighted Majority Voting	99%	[38]
Home Appliance	STFT	Unsuper-Transformer and Dynamic Graph Convolution (TDGCN)	99.23%	[39]
Pipes	Principal Component Analysis	One Class-SVM	100%	[40]
	Statistical Features + MFCC	RF	95.59%	[41]
	Sparrow Search Algorithm-VMD	Wavelet Thresholding	-	[42]
DCASE 2020 Task 2	Raw Waveform + MFCC	TgramNet + WaveNet + SpecNet + MobileFaceNet	95.97%	[43]
	STFT	CNN + Gaussian Mixture Model (GMM)	96.86%	[44]
	Raw Waveform + MFCC	STgram + MFN	92.36%	[45]
MIMII	MFCC + Statistical Features	Deep Denoising Autoencoder	96.51%	[46]
	Timbral Features	SVM	97%	[47]
	STFT + MFCC + ID	LSTM-AE	84.14%	[48]

.

Nevertheless, acoustic diagnostics still face challenges such as insufficient transferability across devices [49], lack of robustness in feature extraction under extreme noise conditions [50], and the trade-off between real-time performance and accuracy for lightweight edge deployment [51]. Existing reviews focus more on the structures and results of the detection model, while neglecting the impact of differences in feature engineering on subsequent detection models. Therefore, this paper starts with the sound feature engineering for guiding the selection of sound feature extraction methods in different equipment and environments. Further, the judgment standards in the sound detection models are analyzed from the empirical threshold to machine learning and deep learning.

This study provides a comprehensive analysis of the evolution of acoustic diagnostic technologies, identifying the technical bottlenecks associated with noise suppression, feature generalization, and cross-device transferability. Additionally, it discusses next-generation solutions propelled by the fusion of artificial intelligence and physical modeling to offer theoretical support for the transition and standardization framework within this research area. Thus, this study aims to explore the theoretical foundations and application processes of sound signal analysis in the context of predictive maintenance for equipment.

The paper is divided into four sections. Section 2 focuses on the feature engineering of sound signals with statistical parameters, waveform decomposition, and spectrogram analysis. Section 3 reports on anomalous sound detection and classification methods. The last section is devoted to a discussion and conclusion.

2. Acoustic Feature Engineering

Acoustic feature engineering plays a vital role in the PdM of equipment [52]. Feature extraction methods transform the one-dimensional sound signal into a high-dimensional and interpretable feature vector [53]. The extracted feature vectors are then used to diagnose the operating state of the target equipment using the anomalous sound detection model. With the differences in the processes and routes of more feature extraction methods, this paper classifies the existing sound feature engineering methods into statistical parameters, waveform decomposition, and spectrogram analysis. The classification of acoustic feature engineering in the PdM of equipment systems is shown in Figure 3. This section explores the principles and application scenarios of these methods.

2.1. Statistical Parameters

Statistical parameter extraction is a straightforward approach to describing the characteristics of equipment sounds because changes in waveform and energy occur when a device malfunctions [54,55]. Typically, statistical features are not applied independently. All statistical parameters are calculated from a finite-length sound sequence. To improve the real-time performance of the device’s prognostic system, the acquired sound signals $X\left[n\right]$ are segmented into frame signals $x_i$ of equal length by a slip window. The framing process of a sound signal is shown below:

$$x_i\left(n\right)=Win\left(n\right)\times X[(i-1)\times s+n],1\le n\le N,$$

(1)

where n is the time variant of the sampling point, i is the position variant of the frame, $x_i\left(n\right)$ records the $nth$ amplitude value of the sound signal in the frame i, $Win(·)$ is the window function, s is the increased distance between the windows, and N is the length of the frame and the slipping window.

According to the different statistical targets and analysis methods, the statistical parameters are classified into temporal features, spectral features, and timbral metrics. The equations of statistical parameters applied in equipment sound detection are summarized in

Table 2. The summary of statistical parameters in equipment sound detection.

Classification	Feature Names	Equations
Temporal	Short-Time Energy	$E\left(i\right)={\sum }_{n=1}^N{\left[x_i\left(n\right)\right]}^2$
	Zero-Crossing Rate	$Z\left(i\right)={\displaystyle \frac{1}{N-1}}{\sum }_{n=2}^{N}f\left(x_i\left(n\right),x_i(n-1)\right)$
		$f\left(x_i\left(n\right),x_i(n-1)\right)=\left\{\begin{array}{cc}1& if{x}_i\left(n\right)\times x_i(n-1)<0\\ 0& if{x}_i\left(n\right)\times x_i(n-1)\ge 0\end{array}\right.$
	Root Mean Square	$X_{rms}\left(i\right)=\sqrt{{\displaystyle \frac{1}{N}}{\sum }_{n=0}^{N-1}{x}^2\left(n\right)}$
	The $rth$-order moment	$M_r\left(i\right)={\displaystyle \frac{1}{N}}{\sum }_{n=1}^N{(x_i\left(n\right)-{\overline{x}}_i)}^r$
	Standard Deviation	$\sigma \left(i\right)={\displaystyle \frac{1}{N}}{\sum }_{n=1}^N\sqrt{x_i\left(n\right)-{\overline{x}}_i}$
	Kurtosis	$X_{ku}\left(i\right)={\left[{\displaystyle \frac{M_r\left(i\right)}{\sigma \left(i\right)}}\right]}^4$
	Skewness	$X_{sk}\left(i\right)={\left[{\displaystyle \frac{M_r\left(i\right)}{\sigma \left(i\right)}}\right]}^3$
	Peak	$X_p\left(i\right)=max\left(x_i\right)$
	Peak-to-Peak	$X_{pp}\left(i\right)=max\left(x_i\right)-min\left(x_i\right)$
	Crest Factor	$X_{cf}\left(i\right)={\displaystyle \frac{X_p\left(i\right)}{X_{rms}\left(i\right)}}$
	Impulse Factor	$X_{If}\left(i\right)={\displaystyle \frac{max\left(\left|x_i\right|\right)}{\frac{1}{N}{\sum }_{n=1}^N\left|x_i\left(n\right)\right|}}$
	Margin Factor	$X_{mf}\left(i\right)={\displaystyle \frac{X_p\left(i\right)}{{\left(\frac{1}{N}{\sum }_{n=1}^N\sqrt{|x_i\left(n\right)|}\right)}^2}}$
	Talaf	$X_{ta}\left(i\right)=log(X_{ku}\left(i\right)+{\displaystyle \frac{X_{rms}\left(i\right)}{X_{rms}\left(1\right)}})$
	Tikhat	$X_{ti}\left(i\right)={\left(log(X_{ku}{\left(i\right)}^{X_{cf}}+{\displaystyle \frac{X_{rms}\left(i\right)}{X_{rms}\left(1\right)}})\right)}^{X_p}$
Spectral	Fast Fourier Transform (FFT)	$f_i\left(k\right)={\sum }_{n=-\infty }^{\infty }{x}_i\left(n\right)e^{-jkn}$
	Peak	$F_p=max\left(f_i\right)$
	Peak Frequency	$S_p=S\left(F_p\right)$
	Mean Frequency	$F_{mean}={\displaystyle \frac{{\sum }_{k=1}^K{f}_i\left(k\right)}{K}}$
	Frequency Center	$F_c={\displaystyle \frac{{\sum }_{k=1}^{K}f\left(k\right)k}{K}}$
	RMS Frequency	$F_{rms}=\sqrt{{\sum }_{k=1}^K{\displaystyle \frac{f_i\left(k\right)}{K}}}$
	Standard Deviation	$F_{st}=\sqrt{{\displaystyle \frac{{\sum }_{k=1^K}{(k-F_{mean})}^{2}f\left(k\right)}{{\sum }_{k=1}^K}}f\left(k\right)}$
	Frequency Power Envelop	$F_{pw}={\displaystyle \frac{1}{K}}{\sum }_{k=1}^K{f}_i^2\left(k\right)$
	Effective Value	$X_{wrms}=\sqrt{{\displaystyle \frac{{\sum }_{j=1}^{K}k{W}_s\left(k\right)}{{\sum }_{j=1}^K{W}_s\left(k\right)}}}$
	Envelop Amplitudes	$X_{pcwt}={\displaystyle \frac{{\sum }_{j=1}^k{W}_s\left(k\right)}{k-1}}$
	Median Frequency	$F_{mf}={min}_m\left({\sum }_{a=1}^m{f}_i\left(a\right)-{\sum }_{b=m}^K{f}_i\left(b\right)\right)$
	Relative Peak	$F_{rp}={max}_k{\displaystyle \frac{f\left(k\right)}{F_m}}$
Timbral Metrics	Sharpness	$S=0.11{\displaystyle \frac{{\int }_0^{24Bark}{N}^{\prime }\left(z\right)g\left(z\right)zdz}{{\int }_0^{24Bark}{N}^{\prime }dz}}$
	Roughness	$R_{frame}=X^{0.1}\times 0.5\left(Y^{3.11}\right)\times Z$
	Boominess	$Boomingindex=\mathrm{Bandsum}\times \left({\displaystyle \frac{S_l}{S_t}}\right)$
	Brightness	$S{C}_{\mathrm{br}}={\displaystyle \frac{{\sum }_{k\in 2000\mathrm{to}8000\mathrm{Hz}}f\left(k\right)m\left(k\right)}{{\sum }_{k\in 2000\mathrm{to}8000\mathrm{Hz}}m\left(k\right)}}$
	Depth	$S{C}_{\mathrm{dp}}={\displaystyle \frac{{\sum }_{k\in 20\mathrm{to}8000\mathrm{Hz}}f\left(k\right)m\left(k\right)}{{\sum }_{k\in 20\mathrm{to}8000\mathrm{Hz}}m\left(k\right)}}$

The statistical parameters are recorded and classified from [20,25,37,38,40,47].

.

Given the prolonged operation of stable transformers, acoustic signals often exhibit significant variations in amplitude and frequency compared to normal conditions. Based on this operational environment, the purpose of feature extraction for transformer equipment is to detect shock signals in the acoustic signal. Therefore, Yu et al. [20] utilized the Short-Time Energy (STE) and Zero-Crossing Rate (ZCR) as foundational metrics for the preliminary assessment of acoustic signatures in power transformers.

Cao et al. [38] proposed an ensemble classifier to diagnose railway point machines with the temporal and spectral features. The recorded sound signals are decomposed into 15-level Intrinsic Mode Functions (IMFs) by the EMD method. Then, the entropy and statistical parameters are extracted from the obtained IMFs. In the feature engineering process, time and frequency domain statistical parameters are extracted as the raw discrimination features from each IMF. The extracted features include 10-dimensional time-domain statistical parameters, 4-dimensional frequency-domain statistical parameters, and 8-dimensional energy entropy. Finally, the feature matrix $(22\times 15)$ is sent into the proposed classifier.

In [25], based on the frequency domain signal, the traditional spectral features include the spectral envelope slope, spectrum mass center, and peak amplitude crest. Meanwhile, some temporal features can also be used in the frequency domain, such as peak value, skewness, and kurtosis. In [40], six sound features are selected and sent into the unsupervised learning model. Extracted sound features include the RMS, crest factor, peak value, peak frequency, median frequency, and relative peak.

Yasuji and Masashi [47] proposed a method that employs timbral metrics to extract and analyze fault features from machine sounds. The timbral metrics include sharpness, boominess, brightness, depth, and roughness. These timbral metrics are tailored to specific machine types based on their fault-related sound impressions. The theoretical foundations and computational models of these metrics are illustrated as follows: Sharpness is a quantification of the sensation of a sound being sharp or shrill. Roughness describes the perception of a sound as buzzing, harsh, or raspy. Boominess quantifies the sensation of low-frequency vibration or resonance. Brightness is associated with the perception of high-frequency energy in a sound. Depth, as a timbral attribute, reflects the prominence of low-frequency components in a sound.

The applied statistical parameters in different detection frameworks and applications are recorded in

Table 3. The summary of statistical parameters in different detection frameworks.

Features Extraction Method				Feature Dimensions	Ref.
Statistical Parameters		Decomposition	Spectrogram
Temporal	Spectral
Short-time Energy Zero-crossing Rate	-	-	12-level MFCC	$(1\times 2),(1\times 12)$	[20]
Mean	Mean Frequency	15-level EMD	-	$(15\times 22)$	[38]
Standard Deviation	Frequency Center	8-level WPD
Skewness	RMS Frequency
Kurtosis	Standard Deviation
Peak	Energy Entropy
RMS
Crest Factor
Shape Factor
Impulse Factor
Margin Factor
RMS	RMS Frequency	8-level DWT	-	$(1\times 14)$	[25]
Kurtosis	Mean Frequency
Peak to Peak	Frequency Power Envelop
Crest Factor	Effective Value
Skewness	Envelop Amplitudes
Impulse Factor
Standard Deviation
Talaf
Tikhat
RMS	Peak	-	MFCC	$(6\times nf)$	[40]
Crest Factor	Peak Frequency
	Median Frequency
	Relative Peak
RMS	Spectral Envelope	-	mass center		[37]
Standard Deviation	Centroid		peak amplitude
Skewness	Skewness
Zero-crossing Rate
Crest Factor
Kurtosis
Spread
Decrease
Sharpness	-	-			[47]
Roughness
Boominess
Brightness
Depth

.

2.2. Waveform Decomposition

Waveform decomposition serves as a critical signal decoupling technique in equipment predictive maintenance. It effectively identifies latent failure signatures by applying standardized computational frameworks to separate single-channel acoustic signals into multi-layer components. Current industrial implementations predominantly adopt Empirical Mode Decomposition (EMD), Variational Mode Decomposition (VMD), and Wavelet Packet Transform (WPT) [56]. The pros and cons of the waveform decomposition methods are summarized in Table 4.

2.2.1. Empirical Mode Decomposition

Empirical Mode Decomposition (EMD), a non-stationary signal processing, decomposes complex signals into a set of Intrinsic Mode Functions (IMFs) [57]. The algorithm proceeds as follows:

Step 1: Extrema Identification. Detect all local maxima and minima points in the target signal.

Step 2: Envelope Construction. Generate upper and lower envelopes via cubic spline interpolation using the identified extrema.

Step 3: Mean Envelope Calculation. Compute the instantaneous mean envelope $m\left(t\right)$ by averaging the lower and upper envelopes.

Step 4: IMF Extraction. Subtract $m\left(t\right)$ from the original signal $f\left(t\right)$ to derive a candidate IMF $h_1\left(t\right)=f\left(t\right)-m\left(t\right)$. Verify if $h_1\left(t\right)$ satisfies the IMF criteria [58]. And then, the IMF component $h_n\left(t\right)$ is saved in memory $C_i\left(t\right)$ If unsatisfied, iteratively refine $h_1\left(t\right)$ by treating it as the new input for Steps 1–3 until convergence. The iteration terminates when

$$\sum _{t=0}^T{\displaystyle \frac{{[h_{n-1}\left(t\right)-h_n\left(t\right)]}^2}{h_{n-1}^2\left(t\right)}}\le SD,$$

(2)

where $SD$ is an empirical parameter $(0.2-0.3)$, which is used to control the distance between IMFs, and $n=1,\dots ,N$ records the number of the iterations.

Step 5: Residual Computation. Update the residual signal $re{s}_i\left(t\right)=f\left(t\right)-C_i\left(t\right)$, where $i=1,\dots ,M$ is the number of the IMFs. This residual component is treated as new data and is subjected to the same processes described previously to calculate the next IMF component [59].

Step 6: Termination Condition. Repeat the process until the residual is monotonic or envelope amplitudes fall below a predefined threshold.

Shi et al. [27] introduced a dual-phase fusion framework integrating acoustic and vibration signals to improve rolling bearing fault detection. This methodology suppresses ambient noise interference while augmenting system state characterization through synergistic signal fusion. EMD is employed to refine fused signals by temporal smoothing and adaptive decomposition based on intrinsic time-scale features.

Delgado et al. [60] implement Complete Ensemble Empirical Mode Decomposition (CEEMD) [61], a noise-augmented approach, to isolate fault-induced frequency components within IMFs without compromising temporal fidelity. Post-decomposition, Gabor transform frequency margins are analyzed to derive discriminative spectral signatures from IMFs.

2.2.2. Variational Mode Decomposition

Variational Mode Decomposition (VMD) [62] operates through iterative optimization to partition signals into bandwidth-limited modal components [63]. This technique effectively resolves frequency overlap while addressing inherent limitations of traditional methods, including boundary distortion and modal aliasing, establishing its prominence in mechanical fault diagnostics [64].

The objective of VMD is to determine a collection of IMFs that collectively reduce total bandwidth expenditure while maintaining signal reconstruction fidelity. Bandwidth quantification utilizes the squared $L2$ norm of gradient-modulated signals.

Step 1: Analytical Signal Derivation. Compute the analytic signal of each mode $u_k\left(t\right)$ via the Hilbert transform to obtain its single-sided spectrum:

$$u_k\left(t\right):\left[\delta \left(t\right)+{\displaystyle \frac{j}{\pi t}}\right]\ast u_k\left(t\right),$$

(3)

where $\delta \left(t\right)$ is the Dirichlet function, and ∗ is the convolution symbol.

Step 2: Frequency Demodulation. Shift the spectrum of each mode to the baseband by mixing with its pre-estimated center frequency:

$$\left[\left(\delta \left(t\right)+{\displaystyle \frac{j}{\pi t}}\right)\ast u_k\left(t\right)\right]e^{-j{\omega }_{k}t}.$$

(4)

where ${\omega }_k$ represents the center frequency, $u_k\left(t\right)$ denotes the $kth$ IMF component, $\delta \left(t\right)$ represents a impulse function and ∗ is the convolution calculation.

Step 3: Bandwidth Optimization. Estimate the bandwidth of each mode using the squared $L^2$ norm of the demodulated signal gradient. The constrained variational model is formulated as

$$\left\{\begin{array}{c}\underset{u_k,{\omega }_k}{min}\left\{\sum _{k=1}^K{∥{\partial }_t[({\delta }_t+{\displaystyle \frac{j}{\pi t}})\ast uk\left(t\right)]e^{-j{\omega }_{k}t}∥}_2^2\right\}\\ s.t.{\sum }_{k=1}^K{u}_k\left(t\right)=x\left(t\right),\end{array}\right.$$

(5)

where K is the number of modal components, $u_k:=u_1,u_2,\dots ,u_K$, and ${\omega }_k:={\omega }_1,{\omega }_2,\dots ,{\omega }_K$ are shorthand notations for the set of all modes and their center frequencies.

Step 4: Introducing the quadratic penalty factor and Lagrange multiplication operator. To transform the constrained problem into an unconstrained problem, the augmented Lagrange function $L(u_k,w_k,\lambda )$ is expressed as follows:

$$L(u_k,w_k,\lambda )=\alpha \sum _{k=1}^K{∥{\partial }_t[({\delta }_t+{\displaystyle \frac{j}{\pi t}})\ast uk\left(t\right)]e^{-j{\omega }_{k}t}∥}_2^2+{∥x\left(t\right)-\sum _{k=1}^K{u}_k\left(t\right)∥}_2^2+〈\lambda \left(t\right),x\left(t\right)-\sum _{k=1}^K{u}_k\left(t\right)〉$$

(6)

where $\alpha$ is the penalty factor, $\lambda \left(t\right)$ is the multiplication operator, ${\left|\right|\left|\right|}_2^2$ represents the $L^2$-norm, and 〈〉 means the operation of the inner product.

The augmented Lagrangian method converts the constrained problem into an unconstrained form. However, the algorithm exhibits high computational complexity $O\left(n^4\right)$, limiting its applicability to real-time scenarios.

Lou et al. [34] introduce a hybrid feature extraction framework combining FFT and VMD. Spectral analysis and multi-scale temporal decomposition enable the capture of complementary frequency–energy distributions and transient characteristics. The integrated preprocessing strategy attenuates noise interference while preserving critical time–frequency signatures, enhancing neural networks’ ability to discern subtle fault patterns.

Yin et al. [35] devise a non-invasive fault identification system utilizing VMD and Support Vector Machines (SVMs). To overcome the degradation of fault diagnosis accuracy caused by IMFs with multiple feature sets. Six scale-invariant statistical metrics (including kurtosis and impulse factor) are computed from these IMFs, serving as discriminative inputs for SVM-based classification.

Xiao et al. [65] engineer a sealing strip acoustic assessment device coupled with an enhancement algorithm. The combination of Wiener filtering and VMD (WF-VMD) is proposed to enhance weak acoustic signals. The sound signal is divided into four IMFs ($K=4$) with the penalty constraint $\alpha =1000$.

Lei et al. [66] present a novel denoising method integrating metaheuristic-optimized VMD with wavelet thresholding to address these challenges. The core innovation lies in the coati optimization algorithm (COA)-driven parameter selection for VMD, which adaptively determines the optimal decomposition mode count K and penalty factor $\alpha$, thereby overcoming the empirical parameterization limitations of conventional VMD. Subsequent cross-correlation analysis filters out low-frequency drift and high-frequency artifacts, achieving multi-stage noise suppression.

Li et al. [42] develop a Sparrow Search Algorithm-optimized VMD (SSA-VMD) model paired with enhanced wavelet thresholding. Leveraging the Sparrow Search Algorithm [67], the framework minimizes envelope entropy to optimize VMD parameters, preserving high-frequency transient features often lost in conventional methods. Experimental validation confirms superior noise attenuation and signal fidelity, particularly in scenarios involving intermittent fault impulses.

2.2.3. Wavelet Packet Transform

The Wavelet Packet Transform (WPT) extends signal decomposition capabilities by recursively partitioning high-frequency sub-bands [68], enabling refined frequency-band segmentation throughout the analyzed spectrum [69]. This hierarchical decomposition mechanism improves the spectral resolution, particularly in localized time–frequency regions [70]. In the context of sound detection, Discrete Wavelet Transform (DWT) generalizes the decomposition process by applying the wavelet function $\psi \left(n\right)$, mathematically expressed as

$$DWT(j,k)={\displaystyle \frac{1}{\sqrt{2^j}}}{\int }_{-\infty }^{\infty }x\left(n\right)\psi \left({\displaystyle \frac{n-k\times 2^j}{2^j}}\right)dt,$$

(7)

where $2^j$ is the scale factor, and $k\times 2^j$ is the shift factor.

In [23], 1D-DWT with 4 levels and 4 low-pass filter subbands is proposed to decompose the engine sounds. Proposed multileveled 1D-DWT applies wavelet filters to both low- and high-frequency components, mathematically expressed as

$$\left\{\begin{array}{c}{b}_{i,j}=\sqrt{2}{\sum }_{n=0}^K{H}_n{b}_{i-1,j+n}\\ b_{i,j+l}=\sqrt{2}{\sum }_{n=0}^K{G}_n{b}_{i-1,j+n}\end{array}\right.,$$

(8)

where $b_{i,j}$ represents the $jth$ frequency-band signal at the i level. $j=1,2,\dots ,J$. J indicates the total number of decomposed frequency-band signals and equals $2^l$. $i=1,2,\dots ,I$, with I denoting the decomposition depth. $H_n$ is the low-pass filter, and $G_n$ is the high-pass filter.

For recognizing power system equipment based on sound signals, Bai et al. [19] carry out the WT to obtain the wavelet coefficient–time diagram, where the mean square integrable space $L^2$ is applied as the basic wavelet and mother wavelet. The extracted diagram is transformed into an image by the structural similarity image processing method and then sent to the machine learning classifier.

In [31], the multi-resolution wavelet transform is used to decompose the engine’s sound signal based on the microphone. Both a 5-level DWT and a 4-level Wavelet Packet Analysis (WPA) are used to measure noise signals for extracting fault features of the engine using the Daubechies wavelet function. The Daubechies wavelet function is defined by a set of finite-length filter coefficients [71].

For detecting the rolling bearing of the belt conveyor, the sound signals are divided into four levels of the IFMs by the WPT method [26]. The Teager Energy Operator (TEO) is used to strengthen the weak shock component of the original signal [72]. For a digital signal $x_i$, the operator ${\Psi }_d$ can be approximated in the time domain, which is expressed as follows:

$${\Psi }_d\left[x_i\left(n\right)\right]={\left[x_i\left(n\right)\right]}^2-x_i(n+1)x_i(n-1).$$

(9)

Table 4. The pros and cons of the waveform decomposition methods.

Methods	Pros	Cons	Ref.
EMD	Adaptive decomposition	Severe mode mixing issues.	[27,59]
	without basis functions.	Highly sensitive to noise.
	Suitable for non-stationary.
CEEMD	Suppresses mode mixing	Doubled computational complexity	[60]
	Better detail preservation	Requires manual noise amplitude selection
EEMD	Reduces mode mixing through	Ensemble size requires empirical selection.	[73]
	noise-assisted decomposition.	Influenced by the type of noise introduced.
	Anti-noise Interference.
	More reproducible and consistent.
VMD	Variational framework	Requires preset mode number K	[34,35,65]
		ensures unique decomposition
WF-VMD	Wiener-filtered enhancement	Latency Per Frame	[65]
COA-VMD	Auto-optimizes K and $\alpha$	Increased computation time for optimization	[66]
SSA-VMD	Effective for multi-fault coexistence detection	Requires large iterations	[42]
WT	Flexible basis function selection	Energy leakage	[19,68]
DWT	Efficient implementation with filter banks	Limited shift-invariance	[31]
WPT + TEO	Quickly capture the resonant frequency	Manual threshold setting	[26]
	bands of fault characteristics.

Table 4. The pros and cons of the waveform decomposition methods.

Methods	Pros	Cons	Ref.
EMD	Adaptive decomposition	Severe mode mixing issues.	[27,59]
	without basis functions.	Highly sensitive to noise.
	Suitable for non-stationary.
CEEMD	Suppresses mode mixing	Doubled computational complexity	[60]
	Better detail preservation	Requires manual noise amplitude selection
EEMD	Reduces mode mixing through	Ensemble size requires empirical selection.	[73]
	noise-assisted decomposition.	Influenced by the type of noise introduced.
	Anti-noise Interference.
	More reproducible and consistent.
VMD	Variational framework	Requires preset mode number K	[34,35,65]
		ensures unique decomposition
WF-VMD	Wiener-filtered enhancement	Latency Per Frame	[65]
COA-VMD	Auto-optimizes K and $\alpha$	Increased computation time for optimization	[66]
SSA-VMD	Effective for multi-fault coexistence detection	Requires large iterations	[42]
WT	Flexible basis function selection	Energy leakage	[19,68]
DWT	Efficient implementation with filter banks	Limited shift-invariance	[31]
WPT + TEO	Quickly capture the resonant frequency	Manual threshold setting	[26]
	bands of fault characteristics.

2.3. Spectrogram Analysis

As a time–frequency analysis method, spectrogram analysis transforms the one-dimensional sound sequence into a two-dimensional spectrogram. The existing spectrogram analysis methods include Short-time Fourier Transform (STFT) and Mel-Frequency Cepstral Coefficients (MFCCs) [74,75]. The spectrogram is spliced by multiple frequencies transformed from the framed sound signals along the time axis. The framing process of the spectrogram is expressed in Figure 4. Therefore, the spectrogram can effectively represent the operational state of the target equipment. The frame parameters of the spectrogram analysis are summarized in Table 5.

2.3.1. Short-Time Fourier Transform

STFT is one of the basic sound analysis methods that extracts potential feature information by framing, windowing, and frequency transforming. Furthermore, sound frequency signals are widely used as inputs to the model. STFT is constrained by the time–frequency resolution of the window function, and the time–frequency resolution is fixed [64].

In [76], the STFT transforms a 1D sound signal into a 2D frequency matrix. The extracted matrix is sent into the deep learning model to diagnose bearing faults. Based on a single-channel off-the-shelf microphone, Behnam et al. [36] convert the real-time collected time-domain signals of construction site operation sounds into the frequency domain by the STFT. The one-dimensional sound data are transformed into two-dimensional image data, and then the image data are sent into the convolutional neural network.

2.3.2. Mel-Frequency Cepstral Coefficient

Mel-Frequency Cepstral Coefficients (MFCCs) constitute a perceptually motivated feature set derived from cepstral analysis in the Mel-scaled frequency domain. These coefficients quantitatively characterize the energy distribution of acoustic signals across critical frequency bands, encapsulating both low-frequency spectral envelopes and high-frequency fine structures within the cepstral domain [77]. The Mel scale operates as a psychoacoustic frequency scaling system that emulates human auditory frequency sensitivity, nonlinearly compressing higher frequencies to align with biological perception mechanisms [41]. The workflow of the MFCC is shown in Figure 5. Meanwhile, the frame parameters of the FFT process in MFCC are recorded in Table 5.

Firstly, the calculation process for MFCC is based on the frequency signal $F\left(k\right)$. The spectral energy $E\left(k\right)$ of the framed frequency signal is shown as follows:

$$E\left(k\right)=\left|F\right(k\left)\right|,k=0,\dots ,K-1$$

(10)

where K is the number of the frequency sample rate in the FFT, and k denotes the frequency-position variable.

Secondly, the energy spectrum of a frame in the frequency domain is filtered by the Mel filter bank [45]. The parameters $m_n$ of the MFCC are expressed as follows:

$$m_n=\sum _{k=0}^{K}E\left(k\right)D_n\left(k\right),n=1,\dots ,N,$$

(11)

where $D_n\left(k\right)$ represents the $nth$ Mel filters bank, and N is the number of Mel filters. The calculation process of the Mel filters [78] is illustrated as follows:

$$M\left(f\right)=2595\times lg(1+{\displaystyle \frac{f}{700}})$$

(12)

where M is the Mel frequency and f is the frequency:

$$D_n\left(k\right)=\left\{\begin{array}{cc}0& k<f(n-1)\\ {\displaystyle \frac{2(k-f(n-1\left)\right)}{\left(f\right(n+1)-f(n-1\left)\right)\left(f\right(n)-f(n-1\left)\right)}}& f(n-1)\le k\le f\left(n\right)\\ {\displaystyle \frac{2(k-f(n-1\left)\right)}{\left(f\right(n+1)-f(n-1\left)\right)\left(f\right(n+1)-f(n\left)\right)}}& f\left(n\right)\le k\le f(n+1)\\ 0& k>f(n-1)\end{array}\right.$$

(13)

where $f\left(n\right),n=1,\dots ,N$ is the scale value in the linear frequency domain, which is obtained through the inverse operation of the Mel frequency.

Finally, the logarithms of Mel filter energy are calculated and processed by the discrete cosine transform for MFCC, which is expressed as follows:

$$MFCC\left(n\right)=\sqrt{{\displaystyle \frac{2}{K}}}\sum _{i=1}^{N}ln\left[m_i\right]cos\left({\displaystyle \frac{\pi n(2i-1)}{2N}}\right).$$

(14)

To eliminate the effect of noise on sound anomaly detection, Yu et al. [20] use the MFCC method to extract the sound feature information.

For detecting the click sound event of the automobile car engine, Espinosa et al. [32] apply five extraction methods to represent the feature vector. The MFCC method is used to identify and track timbre fluctuations. The chromogram is calculated to represent pitch classes and harmony based on the STFT method. Spectral Centroid (SC) provides complementary rich information with MFCC for sound discrimination tasks [79]. Tonnetz representation (TR) measures the tonal centroid of a sound based on harmonic changes [80]. The extracted 1D vector has 213 dimensions, which include MFCCs (60 dimensions), chromogram (12 dimensions), Mel-scaled spectrogram (128 dimensions), SC (7 dimensions), and TR (6 dimensions).

Table 5. The summary of the frequency frame parameters and applications.

Sample Rate	Window Function	Window Length	Increase Length	Applications	Ref.
8820	Hanning	819	205	Multiple-equipment Recognition	[36]
1920	Hamming	1920	960	Underground Pipeline Damage	[41]
256	Hanning	4096	2048	Excavation Equipment	[81]
1024	-	1024	512	DCASE 2020 Task 2	[45]
2048	Hamming	2048	512	Rotating Machinery	[33]

Table 5. The summary of the frequency frame parameters and applications.

Sample Rate	Window Function	Window Length	Increase Length	Applications	Ref.
8820	Hanning	819	205	Multiple-equipment Recognition	[36]
1920	Hamming	1920	960	Underground Pipeline Damage	[41]
256	Hanning	4096	2048	Excavation Equipment	[81]
1024	-	1024	512	DCASE 2020 Task 2	[45]
2048	Hamming	2048	512	Rotating Machinery	[33]

2.4. Signal Subspace

Different from the feature engineering of the single-channel sound signal, the signal subspace methods focus on analyzing the interactions between array signals [82]. By transforming multi-channel array signals into a matrix, this method aims to achieve sound source localization and feature dimensionality reduction. The signal subspace methods are applied to analyze the microphone array signal [83].

Mahyub et al. [84] propose the Signal Latent Subspace method to classify environmental sound. In [85], the multichannel signals recorded are first preprocessed using the Principal Component Analysis (PCA) method to monitor the liquid–gas pipe. The PCA is used to identify the subspace in the sound signal that has the maximum variance. The multi-channel sound signal matrix $\widehat{Y}\in {\mathbb{R}}^{\overline{M}\times N}$ is inputted, where $\overline{M}$ denotes the number of the sound vector, and N is the number of the microphones. The calculation process of the transformed matrix Y is expressed as follows:

$$\overline{Y}=\widehat{Y}-\mu ,$$

(15)

where $\mu =E\left[\widehat{Y}\right]$ is the mean value of the sound signal matrix:

$$C_y=E\left[\overline{Y}{\overline{Y}}^T\right]=P{\Sigma }_p{P}^H,$$

(16)

where $P\in {\mathbb{R}}^{\overline{M}\times \overline{M}}$ is a matrix that consists of orthonormal eigenvectors, and ${\Sigma }_p$ denotes a diagonal matrix:

$$Y=\Phi \overline{Y}$$

(17)

where $\Phi =p^H$ represents an orthonormal transformation matrix, and $\Phi \in {\mathbb{R}}^{M\times N},M<\overline{M}$. Therefore, the applied PCA method reduces the vector dimension of the array sound signals, thus improving the computational efficiency.

3. Anomalous Sound Detection

The identification of anomalous acoustic signatures is emerging as an increasingly complex challenge across diverse industrial applications. The timely recognition of such deviations enables preventive measures to mitigate potential equipment failures, catastrophic breakdowns, or safety hazards. Contemporary methodologies leverage advanced feature extraction techniques and machine learning architectures to develop intelligent diagnostic frameworks. According to the scale and complexity of the model, the current state-of-the-art equipment sound anomaly detection models can be classified into experience-based thresholding analysis, feature-driven machine learning, and data-driven deep learning. These systems facilitate the evolution of predictive maintenance strategies by correlating sound feature engineering with incipient fault conditions, thereby transforming reactive maintenance paradigms into data-driven prognostic approaches. The classification of anomalous sound detection methods is shown in Figure 6. These approaches enhance system adaptability across diverse operating scenarios and requirements, effectively meeting the needs of predictive maintenance while improving operational efficiency and safety. Therefore, the model structures and anomaly judgments, such as empirical thresholds, feature distances, and loss functions, are illustrated in this section.

3.1. Experience-Driven Thresholding Analysis

Experience-based thresholding analysis is the most fundamental and quickest method for detecting sound anomalies. The distinction between normal and abnormal sounds is derived from manual empirical judgment. The process of setting thresholds uses statistical methods without neural networks. Common threshold analysis methods include the Hidden Markov Model (HMM) and adaptive thresholding.

In [20], the sound signals of the power transformer are recorded by a single-channel microphone. To detect the impulse events of the power transform, the HMM is trained and applied based on the temporal features (Short-Time Energy and zero-crossing rate) and spectrogram features (MFCCs). Temporal thresholds are set by multiple statistical experiments, which are expressed as follows:

$$Outpu{t}^{\left(1\right)}=\left\{\begin{array}{c}Normal,k_1\ge T_{P1}\&k_2\ge T_{P2}\hfill \\ Fauly,others.\hfill \end{array}\right.$$

(18)

where $T_{P1}=0.01$ and $T_{P2}=0.17$ are set through multiple statistical experiments, $k_1$ represents the ratio of abnormal Short-Time Energy to total frames, and $k_2$ represents the ratio of abnormal zero-crossing rate to total frames.

Yu et al. [20] use the HMM to set thresholds [86] based on the results of the MFCC method. The diagnosis process is illustrated as follows:

$$P\left(H\right|\lambda )=\sum _{i=1}^G{\alpha }_T\left(i\right)$$

(19)

where $H=h_1,\dots ,h_T$ denotes the results of the MFCC method, T is the number of the IMF components, G is the state number, and the ${\alpha }_T\left(i\right)$ is calculated as follows:

$${\alpha }_1={\pi }_i·b_i\left(h_1\right)$$

(20)

$${\alpha }_{t+1}=\left[\sum _{i=1}^G{\alpha }_t\left(i\right)a_{ij}\right]b_j\left(h_{t+1}\right)$$

(21)

where $a_{ij}$ denotes the transition probability from state i to state j at time t, and $b_j\left(h_{t+1}\right)$ is the probability of observing $h_{t+1}$ in state j:

$$M=ln\left(\prod P\left(H\right|\lambda )\right)$$

(22)

where M refers to the matching degree.

The fault diagnosis methodology employs kurtosis-driven component selection, where Intrinsic Mode Functions (IMFs) are quantitatively assessed through a kurtosis thresholding scheme to identify components containing critical fault signatures [27]. These optimized IMF subsets undergo selective reconstruction, effectively amplifying rolling bearing defect characteristics in the composite signal.

A novel Teager Energy Spectral Kurtosis (TESK) index is developed through a four-stage process [26]. Firstly, the sound signal is decomposed via WPT. Secondly, sub-band energy is demodulated using the Teager operator [72]. Thirdly, time–frequency is mapped into two-dimensional spectral information. Finally, comprehensive TESK distribution is generated via spectral kurtosis computation across time–frequency planes. This hybrid approach enables simultaneous resolution in both temporal and spectral domains, effective for detecting transient bearing faults under variable operating conditions.

3.2. Feature-Driven Machine Learning

Unlike conventional threshold analysis, machine learning utilizes neural network architectures to fit normal data patterns, allowing for the detection of anomalies and outliers. To increase the speed of detection and to enable the running of sound detection models with limited storage space, the machine learning methods based on feature engineering have widely been applied to detect equipment sounds in recent years [87]. The pipeline frameworks include the Artificial Neural Network (ANN), K-Nearest Neighbor (KNN) [23], Multilayer Perceptron (MLP) [88], Support Vector Machine (SVM) [89], and Random Forest (RF) [90]. These methods have been successfully applied in various applications.

For diagnosis engine faults based on array microphone and accelerometer, Wang et al. [31] propose a three-layer artificial neural network (ANN). The recorded sounds are preprocessed into 4 parts, in the wavelet packet analysis illustrated in Section 2.3. The input feature matrix is defined by 21 blocks in 9 signals. The calculation process of the proposed ANN-based detection model is illustrated as follows:

$$y_i^{\left(2\right)}=f_1\left(\sum _{j=1}^M{\omega }_{ij}^{\left(2\right)}{f}_2\left(\sum _{k=1}^N{\omega }_{jk}^{\left(1\right)}{x}_k\right)\right),$$

(23)

where i, j, and k represent the number of dimensions in the output layer, the hidden layer, and the input layer. $y_i^{\left(2\right)}$ is the output result. $f_1\left(x\right)=x$ is a pure linear function. $f_2\left(x\right)={\displaystyle \frac{1}{1+e^{-x}}}$ represents the log-sigmoid function.

The mean squared error is used as the loss function for updating the trained parameters ${\omega }_{jk}^{\left(1\right)}$ and ${\omega }_{ij}^{\left(2\right)}$, which is shown as follows:

$$E={\displaystyle \frac{1}{2\left|L\right|}}\sum _{i=1}^{\left|L\right|}{\sum }_{x\in L}\left[d_x\right]-y^{\left(2\right)}{]}^2,$$

(24)

where $d_x$ is the desired output, and $\left|L\right|$ is the cardinality of the learning dataset.

This approach demonstrates robust capabilities in processing both stationary and non-stationary signals, making it highly suitable for engine fault diagnosis. The model’s versatility extends beyond vehicle engine fault diagnosis, offering potential applications in various sound-related engineering fault detection domains.

To recognize the damaged bearing, Pacheco et al. [25] design and compare three supervised machine learning methods.

In method 1, the multi-domain feature extraction with SVM classifier (MFE-SVM) is designed. Time, frequency, and time–frequency parameters (14 dimensions) are extracted as a vector. Then, the vectors are categorized by a binary classifier using the SVM, which is expressed as follows:

$$f^{ij}\left(x\right)=sign\left(\sum _{t=1}^l{y}_t{\alpha }^{ij}.K(x,x_t)+b^{ij}\right),$$

(25)

where $x_t$ is the input vector, l is the training sample number, ${\alpha }^{ij}$ is the Lagrange multiplier, $K(x,x_t)$ refers to the kernel function, and $b^{ij}$ represents the bias value.

Method 2 calculates the relative energy share based on the 8-level DWT method, where the Daubechies 8 and Biorthogonal 3.7 are used as the mother wavelet functions, and then the decomposed sound signals are classified by the Random Forest (RF) classifier with cross-validation.

In method 3, time-spectral features extraction and reduction with linear discriminant analysis (TSFDR-LDA) combine temporal and spectral features to perform classification. The LDA method obtains classified vectors by the KNN and GMM models.

Experiments show that TSFDR-LDA performs optimally in combining acoustic and vibration signals, which is suitable for scenarios with high accuracy requirements; MFE-SVM and RWE outperform in single signals (especially acoustic) and are suitable for non-invasive or resource-constrained scenarios; and acoustic signals outperform vibration signals in most methods, validating their potential as a low-cost, non-contact monitoring solution.

For pipeline leakage localization in municipal water systems based on a microphone unit, Notani et al. [40] develop a clustering framework employing One-Class Support Vector Machines (OC-SVMs) to process hydroacoustic signatures. This unsupervised technique constructs an optimal hyperplane in kernel space that maximizes the margin between normalized acoustic data points and the coordinate origin, effectively isolating leak-induced acoustic patterns from background noise.

Yin et al. [91] introduce an ensemble diagnostic architecture combining EMD with multi-feature SVM classification for automotive component monitoring. Based on the decomposed IMFs, the kurtosis and skewness are selected as the fusion features of fault signals according to the Pearson correlation coefficient. Then, the extracted features are sent into the SVM model to detect the sound of the automobile rearview mirror folding. The proposed method significantly improves the accuracy of fault diagnosis compared to single-feature detection methods.

Liu et al. [41] present a sound monitoring system for the prevention of underground pipeline damage caused by construction. The temporal features (including STE and ZCR) and spectrogram features based on the MFCC are extracted and then sent into the designed Random Forest (RF) model.

In [32], this study builds and trains five classical machine learning classifiers and a CNN classifier to detect the sound event. The 213-dimensional feature vector is extracted in the feature engineering process, which includes five extraction methods illustrated in Section 2.3.2. Compared with SVM, KNN, MLP, and RF classifiers, the MLP classifier is selected and optimized for the sound detection of click-event assembled harnesses. The MLP contains three parts: an input layer, hidden layers, and an output layer. Each layer is connected by neurons with a simple activation function. Meanwhile, the structure of the proposed CNN contains sparse interaction, parameter sharing, and quasi-variant representation [92].

3.3. Data-Driven Deep Learning

The data-driven methodology has emerged as a pivotal factor in predictive maintenance paradigms. By enabling the proactive identification of potential failures, this approach significantly mitigates operational downtime, reduces costs associated with redundant maintenance activities, and optimizes resource allocation in maintenance scheduling. Consequently, extensive research efforts in recent years have been dedicated to the development and refinement of advanced anomaly detection systems, with notable advancements in algorithmic robustness and operational efficiency [93]. According to the different input data types, the existing depth-based methods for detecting abnormal sounds in devices are categorized into image-based methods [94], sequence-based methods [95], and graph-based methods [96]. The data-driven deep learning methods with different input data types are summarized in

Table 6. The summary of the data-driven deep learning methods with different input data types.

Types	Model	Loss Function	Pros	Cons	Ref.
Image	MPNet	Cross-Entropy	Lightweight	Weak Generalizability	[33]
			Embedded Deployment	Speed Needs Improving
	CNN + GMM	Arcface	High stability	Unsteady signals inefficient	[44]
	STgram + MFN	Arcface	spectral temporal fusion	-	[45]
	SE-MFN	Arcface	Multi-feature fusion	Limitations real time	[43]
		Classification + ID
	CNN	Cross-Entropy	Better resource allocation	Scene limitations	[36]
	DNN	-	Noise robustness	Dependent noise samples	[22]
Sequence	DDAE	Squared Error	Rapid processing speed	Noise Interference	[46]
	FARED	Euclidean distance	fast adaptation	huge storage resources	[97]
				long training time
	LSTM	cross-entropy	Quickly Judge	Dependent trained samples	[30]
	ID-CN + LSTM + AE	ID loss	noise reduction	Scenario limitations	[48]
	BiLSTM + BiGRUs	Binary cross-entropy	high accuracy	Dataset limitations	[28]
			Computational efficiency	High model complexity
	LSTM + BiTCN	Binary Cross-Entropy	Efficient feature integration	generalization Limitations	[34]
Graph	DGCN	Cross-Entropy	Information Richness	Graph Construction	[39]
				Dependencies
	Unsuper-TDGCN	Fusion Loss	Domain Adaptation	Parameter Sensitivity	[29]
			Efficiency and Robustness	Real-World Validation

.

3.3.1. Image-Based Deep Learning

Image-based deep learning methods aim to detect the anomalous sound signals using image analysis technologies. The 1D sound signals are transformed into a 2D image, which includes temporal and spectral information.

To detect bearing failures in rotating machinery based on the array microphone, Liu et al. [33] propose the lightweight fault diagnosis network called MPNet. The recorded sound signals are transformed into a frequency image using the STFT method. The extracted frequency image includes temporal information and frequency information. The proposed MPNet model contains three parts: the multibranch feature fusion module, the residual attention pyramid module, and the classifier. The multibranch feature fusion module is used to obtain multi-scale information by multi-scale convolutional branches. Then, the residual attention pyramid module, containing 4-level attention networks, is designed to extract deep fault features and enhance network representation ability. The cross-entropy loss is applied to optimize the detection framework, which is expressed as follows:

$$Loss(x,y)=-\sum _{i=1}^C{x}_{i}log{y}_i,$$

(26)

where $x_i$ represents the $ith$ element of the true fault category, and $y_i$ represents the probability.

The MPNet can accurately, and in real-time, diagnose typical faults of bearings and belt conveyor idlers with fewer network parameters and faster computational efficiency. Compared with other detection models, the proposed model has higher accuracy and robustness under noise.

Due to the overlapping feature distributions and limited separability of subtle anomalies, unsupervised anomalous sound detection faces inherent challenges related to ambiguous decision boundaries. To address these limitations, Wu et al. [44] integrate the ArcFace classifier and GMM to enhance feature discriminability and model complex decision boundaries in unsupervised settings. The proposed framework includes two parts: the pre-training process and the detection process.

The CNN is trained using the ArcFace loss function [98] to extract the hidden features in the pre-training process. The extracted features are mapped into a high-dimensional arc space to maximize inter-class distances while ensuring intra-class compactness. The Arcface loss function is expressed as follows:

$$L_{Arc}(z,{\Theta }_{\mathcal{A}},y)=-{\displaystyle \frac{1}{M}}\sum _{i=1}^{M}log{\displaystyle \frac{e^{scos({\theta }_{y_i}+m)}}{e^{scos({\theta }_{y_i}+m)}+{\sum }_{k=1,k\ne y_i}^K{e}^{scos{\theta }_k}}},$$

(27)

where M is the batch size, $k=1,\dots ,K$ is the number of vector elements, ${\theta }_{y_i}$ denotes the angle of the label, and ${\theta }_k$ is the angle between the hidden feature vector and the weight vector.

The process of the ArcFace loss-based classification neural network is shown in Figure 7.

In the detection process, the two-dimensional spectrogram transformed from the recorded sound signal is sent into the trained deep classification network. Then, the designed GMM model is classified as a binary output. The anomaly score is expressed as follows:

$$A_{GMM}\left(z_t\right)=-ln\left(p\left(z_t\right|r,z^k)\right),$$

(28)

where $p\left(z_t\right|r)$ represents the GMM function, and $z_t$ is the output result from the deep classification network.

The framework is validated by experiments on the ToyADMOS datasets [93] and MIMII datasets. The method effectively trains the distribution boundaries of different classes of data, and the extracted hidden features have high separability, thus improving the detection performance [44].

Liu et al. [45] adopt a self-supervised classification strategy based on the spectral–temporal fusion feature (STgram) and MobileFaceNet (MFN) [99]. The STgram is a fusion of the Tgram $F_T$ and the MFCC spectrogram $F_S$. The Tgram is obtained by the TgramNet, which contains a 1D convolution with a smaller kernel size. The proposed framework applies the ArcFace loss function [98] rather than the cross-entropy error as the loss function.

To obtain more information from the raw sound signal and MFCCs, Kong et al. [43] fuse the Tgram, Wgram, MFCC spectrogram, and Sgram into the multi-features. Then, the obtained sound feature is sent into the Squeeze-and-Excitation MobileFaceNet (SE-MFN). The designed framework is shown in Figure 8. The Tgram is calculated from the raw sound signal using the TgramNet [45]. The Wgram is extracted from the raw signal based on the WaveNet [100] utilizing dilated causal convolutional networks. The Sgram is extracted from the STFT spectrogram using SpecNet as a convolutional neural network.

In this study, loss functions are designed with the ArcFace loss function ($L_1$) [44] and the combined loss function ($L_2$). The combined loss function

$$L_2=\beta L_{type}+(1-\beta )L_{id},$$

(29)

where $\beta$ is a balancing weight, $L_{type}$ represents the classification loss function based on the machine type, and $L_{id}$ denotes the loss function of the machine ID.

In addressing the challenge of simultaneously recognizing activities of multiple heavy construction equipment based on the microphone, Sherafat et al. [36] propose a multi-label and multilevel sound classification method based on STFT and CNN, utilizing single-channel microphones as input. The model architecture leverages STFT for feature extraction and CNN for classification, with the output being the identification of activities from multiple equipment. To enhance robustness, a data augmentation technique is developed to simulate real-world sound mixtures. The method is tested on both synthetic and real-world equipment sound mixtures, demonstrating its effectiveness in accurately identifying activities of multiple pieces of equipment on construction sites without the need for prior sound signal separation.

To address the challenge of sensor data noise interference in predictive maintenance for industrial equipment, Priscile et al. [22] propose a noise-robust predictive maintenance framework. Focusing on milling machines and motors, the research collects multi-modal sensor data, including sound, vibration, and ultrasonic signals, and constructs two novel datasets. Three models are proposed: a deep neural network (DNN), a deep convolutional neural network (DCNN), and an ensemble model (RFC + KNN + SVC). To enhance robustness, a Gaussian noise layer is introduced into the DCNN, and a training strategy with progressively reduced noise levels is implemented. Experimental results demonstrate that the noise-convolutional deep learning method significantly improves model robustness, offering a new solution for industrial equipment condition monitoring.

Cheng et al. [101] propose a hybrid model combining Generative Adversarial Networks (GANs) and the unsupervised CNN model for acoustic anomaly detection. The proposed GAN model aims to reduce the difference between normal samples and anomalous samples. The extracted features, as abnormal samples, are sent into the training process of the sound detection model.

3.3.2. Sequence-Based Deep Learning

Different from the image discrete detection method, the sequence-based deep learning methods focus on the causality of the input signal in time. The common sequence-based deep learning structures include the Recurrent Neural Network (RNN) [30], Temporal Convolutional Networks (TCNs), and AutoEncoder (AE) [24].

In [46], both MFCC and statistical features are sent into the Deep Diagnosis AutoEncoder (DDAE) to improve the accuracy of equipment anomaly diagnosis in high-noise environments. The proposed DDAE model contains two types of architectures. On the one hand, dimensionality reduction coding is performed for MFCC features to map the 128-dimensional MFCC features to 8 dimensions; on the other hand, dimensionality enhancement coding is performed for dimensional statistical features to map the 8-dimensional statistical features to 128 dimensions. The calculation processes of the encoder and decoder layers in the proposed DDAE model are expressed as follows:

$$h=\sigma (Wx+b),$$

(30)

where h is the result of the encoder, x is the input data, $\sigma$ means the activation function, W is the trainable parameter to control the output dimension of the encoder layer, and b is the biases:

$$x^{\prime }=\sigma (w^{\prime }h+b^{\prime }),$$

(31)

where $x^{\prime }$ is the result of the decoder, $W^{\prime }$ is the trainable parameter to control the output dimension of the decoder layer, and $b^{\prime }$ is the biases in the decoder.

Further, the mean squared error $L(x,x^{\prime })$ is applied as the loss function when assessing the outcomes, which is shown as follows:

$$L(x,x^{\prime })={∥x-x^{\prime }∥}^2.$$

(32)

The experimental results show that the proposed methodology affords an average area under the curve performance improvement of approximately 18% based on the MIMII dataset [102].

Park et al. [97] propose a sound anomaly detection model named Fast Adaptive RNN Encoder–Decoder (FARED) for improving the fast adaptive capability of sound detection. The designed model is composed of a recurrent neural network (RNN) architecture, where each cell employs a Long Short-Term Memory (LSTM). To handle variations in longitudinal feature dimensions, the model adopts an autoencoder framework, and its architecture is illustrated in Figure 9. The real-time equipment sound signal is framed into a sequential input spectrum. Each LSTM cell has three types of gates and two types of states as follows:

$$i_t=\sigma (W_i{x}_t+U_i{h}_{t-1}+b_i),$$

(33)

$$o_t=\sigma (W_o{x}_t+U_o{h}_{t-1}+b_o),$$

(34)

$$f_t=\sigma (W_f{x}_t+U_f{h}_{t-1}+b_f),$$

(35)

$$c_t=f_t\odot c_{t-1}+i_t\odot tanh(W_c{x}_t+U_c{h}_{t-1}+b_c),$$

(36)

$$h_t=o_t\odot tanh\left(c_t\right),$$

(37)

where $x_t$ is the framed sound signal at time t, $i_t$ is the input gate, $o_t$ means the output gate, $f_t$ is the forget gate, $c_t$ is the cell state vector, $h_t$ is the output result, and W, U, and b are trainable parameters in the LSTM cells.

Further, the Euclidean distance is used to determine anomalies. An experimental evaluation is conducted on a set of datasets from the SMD assembly machine. Results show cutting-edge performance with fast adaptation.

To ensure the functionality of the sound anomaly detection model on various devices, Emanuele et al. [48] propose a joint framework using the ID Conditioned Network (ID-CN), LSTM, encoder, and decoder. The output of the joint framework is shown below:

$$Output=D\left(H\right(E\left(X\right)\left)\right),$$

(38)

where $E:X\to Z$ is the domain related to the latent representation of input X produced by the encoder. $D:Z\to \chi$ represents the domain of latent samples reconstructed by the decoder. H is the condition function.

Considering the input ID number, the classical autoencoder loss is not enough in the training process. To address this problem, an arbitrary value C is added to the distance loss, which is shown below:

$$Loss=∥Output-C∥,$$

(39)

In the offline detection part, the threshold for judging anomalies is generated from the built error of the training samples or based on the ROC curves.

Validated on the MIMII dataset, which simulates real-world industrial acoustic scenarios, the framework achieves a 17.61% improvement in partial AUC (pAUC) over non-conditioned counterparts and baseline methods.

To achieve early fault detection in belt conveyor systems, Alharbi et al. [28] propose an Intelligent fault detection method using the Yet Another Audio Mobilenet Network (YAMNet) using Bidirectional Long Short Term Memory (BiLSTM) and Bidirectional Gated Recurrent Units (BiGRUs). The designed framework enhances detection capabilities by extracting temporal features from generated embeddings with a soft attention mechanism. Further, the categorical hinge loss is employed to compute the classification outcomes for distinct sound categories.

In [34], spectral and temporal features are extracted from the sound signals of the rotating machine using the FFT and VMD methods. Further, a parallel network model with the LSTM and Bidirectional Temporal Convolutional Network (BiTCN) is employed to obtain multi-dimensional information. To extract the latent sound information based on the feature engineering, the frequency sequence signals are inputted into the LSTM model. Meanwhile, the results of the VMD method are sent into the BiTCN model:

$$y_t^{(j,k)}=\sum _{i=0}^{K-1}\left(f\left(i\right)·y_{t-1}^{(i-1,k)}\right)+y_t^{(j,k)},$$

(40)

where $y_t^{(j,k)}$ represents the output of the j layer in the $kth$ block, and $f\left(i\right)$ is the convolutional function.

Further, Lou et al. [34] propose a detection model using the cross-attention mechanism to fuse the latent features ($F_s$ and $F_T$) extracted by the BiTCN and LSTM, which embeds the multi-information into Binary classification. The cross-attention mechanism is expressed as follows:

$$Atten(Q_{F_T},K_{F_s},V_{F_s})=softmax\left({\displaystyle \frac{Q_{F_T}{K}_{F_s}^T}{\sqrt{d_k}}}\right)V_{F_s},$$

(41)

where $Q_{F_T}=F_T{W}_Q$ is the query vectors, $K_{F_s}=F_s{W}_K$ is the key vectors, and $V_{F_s}=F_s{W}_V$ is the value vectors. $W_Q$, $W_K$, and $W_V$ are trainable parameters. $d_k$ is the dimension of $K_{F_s}$.

The proposed framework enriches the feature representation of rotating machine sound signals. The proposed framework performs well in processing rotating machine sound signals with a low signal-to-noise ratio (SNR), yielding evaluation metrics that surpass those of traditional methods and providing an effective approach for abnormal sound detection in rotating machines.

However, sequence-based methods may not be optimal for every machine type and working condition, particularly when the sound characteristics of machines exhibit high similarity and when the training and test sets are relatively unbalanced. Future research should focus on exploring multimodal information fusion methods with stronger generalization capabilities for machine abnormal sound detection (ASD) under a broader range of working conditions.

3.3.3. Graph-Based Deep Learning

In recent years, Graph Convolutional Networks (GCNs) have offered a promising alternative for detecting dynamic sequences, excelling in contextual relationship analysis within discrete sequences independently of feature numbers [103]. The extracted sound features are transformed into a graph G. A graph structure is defined as follows:

$$G=(V,E,A),$$

(42)

where V represents a set of vertices for recording the result of the feature engineering, E denotes a set of edges connecting two nodes, and A is the adjacency matrix.

Compared with the traditional CNN, the GCN model focuses on the relationship between the target features. The aggregation operation of GCNs inherently has a smoothing effect on noise, making it particularly suitable for low signal-to-noise ratio environments. The message passing mechanism of GCNs allows nodes to aggregate information from multi-hop neighbors while preserving local details and global patterns.

The Deep Graph Convolutional Network (DGCN) proposed by Zhang et al. [29] leverages graph theory and spectral graph convolution to enable the acoustic-based fault diagnosis of roller bearings. The architecture begins by transforming raw acoustic signals into graph-structured data, where each 3600-point signal is mapped into a 60 × 60 matrix. Nodes represent signal points, and edges encode Euclidean distances between neighboring vertices.

The DGCN employs multiple graph convolutional blocks, each comprising a spectral convolution layer using Chebyshev polynomial approximations of graph Laplacian filters, a graph coarsening layer (via multilevel clustering to group similar nodes), and a graph pooling layer (via balanced binary tree rearrangement for 1D pooling). The final network structure (depth = 3, layer widths = 64/256/32) feeds extracted features into a Fully Connected layer and SoftMax classifier. The Chebyshev calculation of the GCN convolutional layer is shown as follows:

$$H^{l+1}=\delta ({\widehat{D}}^{-{\displaystyle \frac{1}{2}}}\widehat{A}{\widehat{D}}^{-{\displaystyle \frac{1}{2}}}{H}^l{W}^l),$$

(43)

where $H^{l+1}$ and $H^l$ mean the output results at the $l+1$ and l layers of the GCN. $\widehat{A}=A+I$ is the sum of the adjacency matrix A and self-loop matrix I. $\widehat{D}$ is the degree of $\widehat{A}$. $W^l$ is the trainable parameter.

While the loss function is not explicitly stated, cross-entropy loss is inferred due to the classification task and backpropagation-based optimization. Key advantages include superior classification accuracy over conventional methods, robustness to noise and variable speeds, and end-to-end learning without manual feature engineering. However, limitations include reliance on laboratory data (untested in real-world industrial environments), sensitivity to graph construction methods (fixed Euclidean edge weights), computational overhead from graph coarsening, and empirical parameter tuning (e.g., optimal Chebyshev kernel length = 7). The DGCN demonstrates promising potential for intelligent fault diagnosis but requires further validation under operational conditions and the exploration of adaptive graph topologies for broader applicability.

Yan et al. [39] extend this paradigm with Unsuper-TDGCN, combining Transformer-based time–frequency feature extraction, Dynamic GCN (DyGCN) for domain-shift-aware dependency modeling, and a Domain Self-adaptive Network (DSN) for covariance alignment. This framework achieves state-of-the-art performance in unsupervised machine anomalous sound detection under domain shifts, demonstrating superior generalization and efficiency. Both works underscore the efficacy of GNNs in capturing geometric relationships and domain-invariant features, yet face limitations in real-world validation and computational overhead. To achieve unsupervised anomalous sound detection, a fusion loss $L_{total}$ is designed as follows:

$$L_{total}=l_{IDT-GCN}+l_{VDT-GCN}+l_{arc}+l_{soc}+l_{poly},$$

(44)

where $l_{IDT-GCN}$ is the loss function of the IDT-GCN model, $l_{VDT-GCN}$ is the loss function of the VDT-GCN model, $l_{soc}$ is the second-order covariance distance, $l_{arc}$ is the Additive Angular Margin (AAM) loss function, and $l_{poly}$ is the poly loss function. The loss function of the IDT-GCN model is shown as follows:

$$l_{IDT-GCN}=\sum _{r=1}^R{e}^{r}log\left(FC\left(S_g\right)\right)+(1-e^r)log(1-FC\left(S_g\right)),$$

(45)

where $FC(·)$ means the Fully Connected (FC) layer and $S_g$ is the features computed by the IDT-GCN model.

The loss function of the IDT-GCN model is shown as follows:

$$l_{VDT-GCN}=\sum _{n=1}^N{y}^{n}log\left(Y^{\prime }\right)+(1-y^n)log(1-Y^{\prime }),$$

(46)

where $y^n$ is the features computed by the VDT-GCN model, and $Y^{\prime }$ represents category labels.

While DGCN excels in structured fault diagnosis, Unsuper-TDGCN pioneers adaptive graph learning for broader industrial applications, though the scalability to other modalities remains unexplored. Collectively, these studies advance intelligent fault detection by harmonizing graph theory, spectral analysis, and domain adaptation, yet emphasize the need for lightweight architectures and cross-domain validation.

4. Opportunities

The evolution of acoustic-based predictive maintenance will be propelled by the synergistic integration of generative networks and physics-informed learning paradigms. Generative adversarial networks [104] and diffusion models are poised to address data scarcity through synthetic fault sound generation with domain-adversarial training, enabling robust anomaly detection under low-fault-sample regimes [105]. Concurrently, physics-informed neural networks (PINNs) [106] will bridge data-driven models with first-principle constraints, enhancing feature interpretability and extrapolation capability for novel failure modes [107]. Emerging hybrid architectures, such as physics-embedded GANs, will unify synthetic data augmentation with mechanism-driven feature learning, achieving cross-domain fault diagnosis. Edge deployment will benefit from neuromorphic computing for real-time acoustic processing. Future breakthroughs demand the co-design of generative physics-aware models and standardized validation protocols across industrial ecosystems.

5. Conclusions

Sound-based predictive maintenance has emerged as a pivotal technology for industrial equipment health monitoring, offering non-invasive sensing, cost-effectiveness, and high-dimensional information capture. This review systematically outlines the evolution of acoustic diagnostic methodologies, spanning foundational signal processing techniques to advanced deep learning frameworks. The suggestions of the feature engineering methods in different equipment and environments are recorded in

Table 7. The suggestions of the feature engineering methods in different equipment and environments.

Equipment	Environments	Feature Engineering	Applications
Constant Power	Low Noise	Statistical Parameters	Power System
	High Noise	EMD, MFCC	Motor, bearing, Railway
Variable Power	Low Noise	VMD, DWT	Engine
	High Noise	STFT, Signal Subspace	Rotating Machines

. In fields with high real-time requirements, it is crucial to analyze the potential features within sound signals. When dealing with constant power devices, statistical features can effectively detect abnormal fluctuations and impacts. In complex noise environments, signal decomposition can significantly reduce noise and improve the signal-to-noise ratio. Furthermore, when facing variable power devices, spectral analysis can quickly obtain the operating status of the equipment. Subspace analysis can achieve sound source localization and signal enhancement.

By integrating physical propagation models with data-driven approaches, these methods address critical challenges such as non-stationarity, noise interference, and sparse fault samples. Key advancements include adaptive feature engineering for interpretable fault signatures, noise-robust architectures, and edge-compatible models for real-time deployment. However, persistent limitations of SNR in harsh environments, cross-device generalization gaps, and the trade-off between computational efficiency and accuracy hinder large-scale industrial adoption. Future research should prioritize interdisciplinary innovations, such as physics-informed neural networks for mechanism-aligned learning, lightweight architectures for edge intelligence, and multimodal fusion with vibration or thermal data. Standardized evaluation benchmarks and self-supervised learning paradigms will further enhance robustness in data-scarce scenarios. Ultimately, the convergence of domain knowledge, advanced algorithms, and industrial digitization will drive the next generation of intelligent maintenance systems, enabling proactive fault mitigation and sustainable operational excellence in Industry 4.0 ecosystems.