An Intelligent Voice-Based Authentication and Anomaly Detection Framework for Secure Smart-Home Environments

Abstract

Smart-home environments require secure and reliable user authentication mechanisms to prevent unauthorized access and spoofing attacks. Traditional password- and PIN-based methods remain vulnerable to theft, replay attacks, and credential compromise. To address these challenges, this study proposes an intelligent voice-based authentication and anomaly detection framework for secure smart-home environments. The framework utilizes benchmark ASVspoof 2019 and ASVspoof 2021 datasets containing bona fide and spoofed speech samples. After preprocessing, discriminative acoustic features, including Mel-Frequency Cepstral Coefficients (MFCC) and Constant-Q Cepstral Coefficients (CQCC), are extracted and provided to a Hybrid CNN-LSTM model for speaker verification. An integrated anomaly detection module further enhances security by identifying replay, spoofing, and synthetic speech attacks. Access is granted only when the input voice is authenticated and classified as non-anomalous. Experimental results demonstrate the effectiveness of the proposed framework, achieving an overall accuracy of 97.2% and a macro-AUC of 0.972. The model also achieves low Equal Error Rates of 3.8%, 2.9%, and 2.1% across the evaluated classes, indicating robust spoof detection and anomaly generalization capabilities. These results highlight the suitability of the proposed framework for secure and intelligent smart-home access control applications.

1. Introduction

The rapid development of Internet of Things (IoT) technology has significantly changed the modern living space, resulting in the development of smart-home systems. Users can easily manage their homes through voice commands, including smart speakers, smart lighting, and surveillance cameras. However, the increasing use of voice interfaces to control and access smart homes has raised a significant concern about secure and reliable authentication [1]. Unauthenticated access, spoofing attacks, and voice replay attacks are serious threats to privacy, security, and safety in smart homes. Anomaly detection is necessary for locating critical or significant actionable information in a range of applications, including unusual computer network traffic patterns. This abnormal traffic shows that malicious activities are sending data from the system to the outside world. Methods of anomaly detection depend on a number of factors, including the nature of the data set, the type of anomaly, data set labeling, output pattern of data, and data set output for various application domains and research areas, including information theory, statistics, machine learning, and data mining [2]. Anomaly detection (AD) systems are designed to detect anomalies in the data, and privacy preservation methods are designed using various cryptographic techniques and optimization methods to maintain data securely in the cloud environment. Anomalies are anomalies in data or the activities of the user while accessing the resources, and the detection of these anomalies is called anomaly detection.

The performance of anomaly detection systems and authentication systems has been significantly improved by recent advances in deep learning technology. Autoencoders have been extensively used for unsupervised anomaly detection by learning to reproduce typical speech patterns and detecting anomalies based on the errors exceeding certain criteria [3]. The process of anomaly detection can further be improved by simulating the random distribution of typical speech samples and detecting anomalies based on the deviation from the expected representation by using Variational Autoencoders (VAEs). Recurrent Neural Networks (RNNs), particularly Long Short-Term Memory (LSTM), are efficient for detecting abnormal speech patterns over time by capturing the temporal dependency of speech signals. GANs have shown promise for speech authentication by simulating the distribution of real speech samples and detecting spoofing by comparing the similarity between the real and generated speech features [4]. Convolutional Neural Networks (CNNs) have emerged as efficient tools for speech authentication systems by analyzing the spectrograms of speech signals. CNNs can efficiently extract spatial patterns from the time-frequency representation of speech signals, such as Mel Frequency Cepstral Coefficients (MFCC) and CQCC spectrograms. Traditional smart-home authentication mechanisms rely on passwords, PINs, or mobile applications, which remain vulnerable to replay attacks, credential theft, and spoofing. Although voice biometrics provide a convenient authentication mechanism, many existing systems focus solely on speaker verification and ignore anomalous voice patterns generated through replay or synthetic speech attacks. Therefore, an integrated framework combining voice authentication and anomaly detection is required to improve smart-home security. Figure 1 illustrates the overall architecture of an anomaly detection framework developed to enhance security in smart-home access control systems. The framework continuously monitors data generated from connected smart devices, sensors, user activities, and access requests within the home environment. Initially, data acquisition modules collect real-time information such as user credentials, device status, behavioral patterns, network traffic, and environmental conditions. The primary novelty of this work lies in the integration of voice authentication and voice spoofing anomaly detection within a unified IoT-enabled smart-home security framework. Unlike conventional speaker verification systems, the proposed approach combines MFCC and CQCC acoustic representations with a Hybrid CNN-LSTM architecture to simultaneously verify authorized users and detect replay, synthetic, and spoofed speech attacks. The framework is validated using benchmark ASVspoof datasets, demonstrating its suitability for secure smart-home access control applications.

2. Review of Literature

SH is one of the popular IoT applications that has gained much attention in recent times. The communication of the components is facilitated by the use of SH (Wu et al., 2017) [5]. Automatic central control was only possible for large commercial buildings and luxurious mansions. In addition to basic control, automation provides scheduling and observation capabilities that may be accessed from specific control locations within the building (Akbarzadeh et al., 2019) [6]. Demand is high for home automation systems that can monitor and control a number of smart devices. Technical systems, automated processes, and networked, remotely controlled equipment in houses are all part of SH. Numerous household appliances make life more equitable by facilitating rapid growth, which reduces the amount of time spent working. In the absence of SHs, individuals must physically start processes and set off each machine at the proper moment to motivate it to function [6]. Improving home convenience and quality of life is the primary goal of SH design. Ensuring security and energy efficiency are further objectives. Homeowners can take control of their houses and improve their quality of life with home automation. It makes it possible to lower electricity costs and energy usage. Devices like security cameras, doorbells that are connected, home lights, and heating and conditioning systems are all included in home automation systems. Devices may be controlled from any location thanks to these solutions. For a home to be deemed smart, gadget compatibility is essential [7]. This compatibility enhances the user experience overall while lowering operating costs.

SHs offer enhanced comfort, safety, and security at a lower cost. For remote monitoring, every object in the house has an IP address. Ethernet or Wi-Fi can be used to connect appliances and smart devices to a local area network (LAN). These services entail sending vast amounts of private and sensitive information via unprotected networks. Confidentiality and security are crucial for consumer technologies. Maintaining the security and reliability of the content in the SH environment is essential. It is important to safeguard data produced by sensors, appliance controls, and surveillance systems [8]. In IoT-enabled smart homes, secure communication between devices, gateways, and cloud servers is critical. Unauthorized interception of voice data may compromise user privacy and system security. Therefore, encrypted communication protocols and secure alert transmission mechanisms are required. When anomalies are detected, the system immediately generates alerts and triggers predefined security actions such as access denial, administrator notification, or device lockdown. When the thermostat determines that no one is home, it can also switch off the heating system (Jiani et al., 2023) [9]. SH is a developing Internet of Things concept that enables people to remotely operate SH items via the internet. However, since people and devices interact over unreliable channels, malicious adversaries may readily intercept and alter sensitive data sent by devices. To guarantee secure connections, there is a high demand for an efficient and secure authentication system. Although a great deal of study has recently been done on security challenges in SH, many of the current solutions are insufficiently secure. Although SH environments are designed to make it easy for consumers to communicate with their home appliances, if appropriate security measures are not included, communication between BS and distant devices can be readily exploited.

In Internet of Things applications, protecting user-device communication is essential (Fan et al., 2023) [10]. Although there are numerous cryptographic algorithms for protecting device-to-device communication, putting these processes into practice on devices with limited processing power is difficult. It is crucial to have a safe and effective system that can safeguard connections in devices with limited capabilities. Numerous symmetric and asymmetric cryptographic techniques have been put forth to get around these restrictions. Choi and colleagues (2021) [11]. Nevertheless, not every cryptographic technique now in use is able to successfully satisfy the security requirements. A technique known as signcryption can accomplish the intended security goals with less memory usage and less overhead in computation and communication. Compared to independent encryption and signature systems, signcryption provides better authenticity, integrity, and privacy. Joshi et al. (2023) [12] suggested an anomaly detection approach to identify anomalies in non-stationary streaming time series data. The sliding window strategy is used to identify the anomalous data, and extreme value theory is used to determine the boundary for the system behavior. The time series characteristics are utilized as input, and the time series data is detected using a density-based comparison method. Even when noisy data is present, it has a decent detection rate and has demonstrated good performance on univariate data. However, the window size chosen affects the classification performance. Since there is no system in place to choose the ideal window size automatically, the user must choose the size [13]. Additionally, the model has not detected anomalies in multivariate data. By integrating speech recognition techniques with deep learning-based anomaly detection, a solution was developed to enhance the security of smart-home environments. Their ability to accurately identify users and instantly report any irregularities makes their security system successful. In order to identify unauthorized users and discover abnormalities in smart-home environments, the author of the research proposed a hybrid strategy that combines speech recognition and machine learning approaches. The outcomes showed how smart-home security systems may be more accurate and effective. Research has been done on the application of adaptive learning strategies to increase user safety with IoT devices at home. The present study proposes an Intelligent Voice-Based Authentication and Anomaly Detection Framework designed specifically for secure smart-home environments [14]. Although several voice authentication approaches have been proposed in the literature, many existing systems focus primarily on speaker verification and do not adequately address sophisticated spoofing attacks, replay attacks, or synthetic speech generation. To overcome these limitations, this study proposes an Intelligent Voice-Based Authentication and Anomaly Detection Framework for Secure Smart-Home Environments. The framework employs Mel-Frequency Cepstral Coefficients (MFCC) and Constant-Q Cepstral Coefficients (CQCC) to extract discriminative speech characteristics and utilizes a Hybrid CNN-LSTM architecture to capture both spatial and temporal voice features. Experimental evaluation on ASVspoof 2019 and ASVspoof 2021 datasets demonstrates that the proposed framework achieves an overall accuracy of 97.2%, highlighting its effectiveness for secure smart-home access control.

Objectives of the Study

The main objective of this project is to create a voice-based authentication framework for SH settings that is intelligent, lightweight, and secure in order to improve defenses against spoofing and unwanted access. The following are the study’s precise goals:

• To design a trusted authentication model integrated with anomaly prediction, combining deep learning-based speaker verification with behavioral transaction history to enhance secure remote access in IoT-supported smart homes.
• To implement an efficient and privacy-preserving framework that reduces communication and computational overhead in smart-home environments, while maintaining user privacy, device confidentiality, and secure key agreement within the Home Area Network (HAN).
• To strengthen system resilience against security threats, including spoofing attacks, replay attacks, impersonation attempts, stolen smart device misuse, and session key leakage, through intelligent anomaly detection and adaptive security mechanisms.

3. Methodology

The proposed voice authentication and anomaly detection system, with the use of voice commands, is intended to reduce the delay between the detection of the unauthorized voice and the production of the alert signal. The response time of the system is dependent upon the deployment architecture, processing power, and the location of the processing in the smart-home IoT environment. To make the system adaptable, the framework allows for various deployment options based on the system requirements [15,16]. The proposed framework follows a layered IoT architecture consisting of sensing, data acquisition, feature extraction, authentication, anomaly detection, and decision-making layers. Voice commands are captured through smart-home microphones and transmitted to the processing unit. The Hybrid CNN-LSTM model performs user authentication, while the anomaly detection module identifies spoofing and replay attacks. The decision layer grants or denies access and generates alerts when suspicious activities are detected.

3.1. Dataset Collection

The ASVspoof 2019 and 2021 datasets are benchmark datasets for anti-spoofing, which can be utilized for testing the proposed Intelligent Voice-Based Authentication and Anomaly Detection Framework. The research on the automatic speaker verification technique often utilizes these datasets for testing the strength of the proposed systems against various attacks like voice conversion, replay, speech synthesis, and deep fakes. These datasets are appropriate for legitimate modeling as well as illegitimate attempts made to access the smart-home systems. The datasets contain speech samples that are manipulated or replayed, as well as bona fide speech samples. The performance and robustness of the proposed anomaly detection framework were evaluated using benchmark speech spoofing datasets.

Table 1. Dataset Details.

Dataset	Tracks Included	Attack Types	Total Samples (Approx.)	Sampling Rate	Data Splits	Labels
ASVspoof 2019 (https://www.kaggle.com/datasets/awsaf49/asvpoof-2019-dataset) [17]	LA, PA	Speech Synthesis, Voice Conversion, Replay Attacks	~117,000+	16 kHz	Train/Dev/Eval	Bona fide/Spoof
ASVspoof 2021 (https://www.kaggle.com/datasets/mohammedabdeldayem/avsspoof-2021) [18]	LA, PA, DF	Advanced Synthesis, Replay, Deepfake Attacks	~600,000+	16 kHz	Train/Dev/Eval	Bona fide/Spoof

presents the details of the datasets employed in this study, including the included tracks, attack categories, sampling rates, data splits, and class labels. The ASVspoof 2019 and ASVspoof 2021 datasets provide diverse spoofing scenarios for assessing system effectiveness under various speech manipulation and replay attack conditions.

3.2. Preprocessing

Preprocessing is an integral part of the proposed Intelligent Voice-Based Authentication and Anomaly Detection Framework, as the raw voice signals collected from the ASVspoof 2019 and ASVspoof 2021 databases may include background noise, silent periods, channel distortion, and amplitude variations. Poor preprocessing of the voice signal can lead to inconsistent input quality, incorrect feature extraction, and poor performance of the deep learning model [18].

To ensure uniform resolution, normalization of the sampling frequency of all the voice signals to 16 kHz is performed. Amplitude normalization is performed to ensure that the waveform amplitude is within a fixed range, thus eliminating variations based on recording intensity. This is to ensure that the model concentrates on speech properties rather than differences in signal magnitude. The next step involves silence removal using Voice Activity Detection (VAD) algorithms to remove non-speech parts at the start and end of the audio files. The removal of silent frames helps to reduce computational complexity and also ensures that the model is not affected by patterns that are not important. Noise reduction algorithms, including spectral subtraction or filtering, are used to reduce background noise while retaining important speech information.

After the cleaning process, the audio signals are segmented into overlapping frames, typically 20–25 ms in duration with a 10 ms overlap. The ability to assess speech signals as quasi-stationary signals over a short period of time is made possible by the process of framing. Before the transformation of the audio signals into the frequency domain, a windowing process, such as the Hamming window, is employed on each frame to reduce spectral leakage [19]. Finally, the audio signals are transformed from the time domain to the time-frequency domain for feature extraction. The preprocessed audio frames are then used to extract the discriminative acoustic features, including MFCC, CQCC, and spectral features, which are then used as input features for the deep learning models used in the authentication and anomaly detection processes of the proposed framework.

The overall workflow of the proposed IoT-enabled intelligent voice authentication and anomaly detection framework is illustrated in Figure 2. The framework integrates voice-based user authentication, IoT communication, feature extraction, deep learning-based anomaly detection, and access control mechanisms to enable secure smart-home access management. Incoming voice inputs are processed and authenticated, while anomalous behaviors are identified in real time to prevent unauthorized access.

3.3. Feature Extraction

In the context of the suggested Intelligent Voice-Based Authentication and Anomaly Detection Framework, feature extraction is a critical component, as the speech signal cannot be directly fed into the deep learning classification model for efficient classification purposes. Following the preprocessing of the speech signal, the unique features are obtained to detect the presence of spoofing attacks in the voice signals. In the context of this research, Mel Frequency Cepstral Coefficients (MFCCs), Constant-Q Cepstral Coefficients (CQCCs), and spectral features, as obtained in the datasets of anti-spoofing attacks, are considered for feature extraction purposes [20]. The features, including MFCCs and CQCCs, are presented in the form of time-frequency matrices, called feature maps, to be fed into the deep learning model for efficient classification purposes. Normalization of the features is carried out to maintain uniformity in terms of the dimensions of the features of all the samples. By considering the features of anti-spoofing attacks, including Mel Frequency Cepstral Coefficients, Constant-Q Cepstral Coefficients, and spectral features, the suggested system is capable of offering robust feature representation, thus enhancing the efficiency of the authentication system in terms of resistance to replay attacks, voice synthesis attacks, and deepfake attacks in smart-home environments. Constant-Q Cepstral Coefficients (CQCCs) are widely used feature extraction methods in audio signal processing and are very effective in tasks such as spoofing detection, speaker recognition, and audio classification [21]. Constant-Q Cepstral Coefficients (CQCCs) are extracted from the Constant-Q Transform (CQT), which provides a nonlinear time-frequency representation that very closely resembles human auditory perception. By employing a logarithmic frequency axis, the CQT provides minute spectral details and provides better temporal resolution at high frequencies and higher resolution at low frequencies. Due to this unique property, CQCCs are particularly effective in audio signals with subtle variations and complex harmonic structures [22].

The CQT, which is specified mathematically in Equation (1), is applied to the audio stream in order to extract the CQCC features

$$\mathit{C}\mathit{Q}\mathit{T}(\mathit{f},\mathit{t})=\sum _{\mathit{n}=\mathbf{0}}^{\mathit{N}-\mathbf{1}}\mathit{x}[\mathit{n}]\cdot \mathit{w}[\mathit{n}-\mathit{t}]\cdot {\mathit{e}}^{-\mathit{j}{\displaystyle \frac{\mathbf{2}\mathit{\pi }}{\mathit{Q}}}\mathit{f}\mathit{n}}$$

(1)

In this equation, $\mathit{x}\left[\mathit{n}\right]$ represents the audio signal, $\mathit{w}[\mathit{n}-\mathit{t}]$ denotes the window function, and ${\mathit{e}}^{-\mathit{j}{\displaystyle \frac{\mathbf{2}\mathit{\pi }}{\mathit{Q}}}\mathit{f}\mathit{n}}$ applies the frequency-based transformation. After computing the CQT, logarithmic scaling is applied to the magnitude of the CQT, as shown in Equation (2), to compress the dynamic range of the spectrum:

$$\mathit{S}(\mathit{f},\mathit{t})=\mathbf{l}\mathbf{o}\mathbf{g}\left|\mathit{C}\mathit{Q}\mathit{T}\right(\mathit{f},\mathit{t}\left)\right|$$

(2)

Equation (3) then describes how the Discrete Cosine Transform (DCT) is used to convert the log-scaled spectrum into the cepstral domain [23]:

$$\mathit{C}\mathit{Q}\mathit{C}\mathit{C}(\mathit{k})=\sum _{\mathit{n}=\mathbf{0}}^{\mathit{N}-\mathbf{1}}\mathit{S}(\mathit{n})\cdot \mathbf{c}\mathbf{o}\mathbf{s}\left({\displaystyle \frac{\mathit{\pi }}{\mathit{N}}}\left(\mathit{n}+{\displaystyle \frac{\mathbf{1}}{\mathbf{2}}}\right)\mathit{k}\right) \mathrm{for} \mathit{k}=\mathbf{0},\mathbf{1},\dots ,\mathit{K}-\mathbf{1}$$

(3)

The resulting CQCC features, represented by the vector in Equation (4), capture complex spectral and temporal details, making them highly effective for differentiating between genuine and spoofed audio samples:

$$\mathit{C}\mathit{Q}\mathit{C}\mathit{C}=\left[\mathit{C}\mathit{Q}\mathit{C}\mathit{C}\right(\mathbf{0}),\mathit{C}\mathit{Q}\mathit{C}\mathit{C}(\mathbf{1}),\dots ,\mathit{C}\mathit{Q}\mathit{C}\mathit{C}(\mathit{K}-\mathbf{1}\left)\right]$$

(4)

CQCCs are particularly advantageous in spoof detection because they offer effectiveness against variations in pitch, speed, and recording conditions, which are often exploited in spoofing attacks such as voice conversion, synthesis, or manipulation. Their ability to capture fine-grained spectral features enhances the discriminative power of machine learning models, resulting in improved classification performance in real-world spoof detection applications.

3.4. Anomaly Detection Module

The anomaly detection module operates in a supervised learning framework. During training, labeled bona fide and spoofed speech samples obtained from the ASVspoof datasets are used to learn discriminative patterns associated with legitimate and malicious voice inputs. During inference, the module identifies anomalous speech samples corresponding to replay attacks, synthetic speech, and voice spoofing attempts. For the identification of unusual vocal activity, a Hybrid CNN-LSTM Voice Authentication module is added to the system, aside from the direct classification process. The generation of false samples is treated as unusual patterns. Through the identification of differences from the usual voice embeddings, the module enables the detection of spoofing attacks or unusual acoustic patterns that were not identified before. The incoming audio is processed and converted to feature embeddings during the authentication process [24]. The approved voice patterns learned from the embeddings are used for comparison. Enhanced security is provided aside from the classification process when unusual deviations exceed a certain threshold and are identified as anomalies. The proposed model employs a hybrid CNN–LSTM architecture to improve feature extraction and sequential pattern learning for intelligent voice authentication and anomaly detection. As illustrated in Figure 3, the convolutional layers capture spatial and local feature representations from the input data, while the LSTM layers model temporal dependencies to enhance classification performance and anomaly identification accuracy.

The CNN’s feature extraction process can be mathematically described as follows: the CNN applies a number of convolutional filters to an input facial picture σ, ${\mathit{W}}_{\mathit{i}}$ and bias terms ${\mathit{b}}_{\mathit{i}}$ to generate feature maps ${\mathit{H}}_{\mathit{i}}$. The numerical representation of this process is as follows:

$${\mathit{H}}_{\mathit{i}}=\mathit{\sigma }({\mathit{W}}_{\mathit{i}}\ast {\mathit{H}}_{\{\mathit{i}-\mathbf{1}\}}+{\mathit{b}}_{\mathit{i}})$$

(5)

where ∗ denotes the convolution operation, σ represents the activation function, and ${\mathit{H}}_{\{\mathit{i}-\mathbf{1}\}}$ represents the feature maps from the previous layer. CNN eventually derives higher-level characteristics from the input image by cascading numerous convolutional layers and pooling procedures. We generate a feature vector that encodes the acquired representations of the facial picture following the feature extraction stage. The input for the next round of categorization is this vector. CNN, GBC, and LR are the classifiers used in our experiments. The LR and GBC Detection processes are shown mathematically in the following:

$$\mathit{y}=\mathit{\sigma }({\mathit{\beta }}_{\mathbf{0}}+{\mathit{\beta }}_{\mathbf{1}}\ast {\mathit{x}}_{\mathbf{1}}+{\mathit{\beta }}_{\mathbf{2}}\ast {\mathit{x}}_{\mathbf{2}}+\dots +{\mathit{\beta }}_{\mathit{n}}\ast {\mathit{x}}_{\mathit{n}})$$

(6)

where ${\mathit{\beta }}_{\mathbf{0}},{\mathit{\beta }}_{\mathbf{1}},{\mathit{\beta }}_{\mathbf{2}}$,…,${\mathit{\beta }}_{\mathit{n}}$ are the learned weights, and ${\mathit{x}}_{\mathbf{1}},{\mathit{x}}_{\mathbf{2}}$,…,${\mathit{x}}_{\mathit{n}}$ are the input features. Stochastic gradient descent and backpropagation are used to optimize the network parameters (weights and biases) in order to train the CNN classifier. By adjusting the parameters according to the gradients found during the backward run, the CNN learns to reduce the classification error. Here is an illustration of CNN’s Detection methodology:

$$\mathit{y}=\mathit{a}\mathit{r}\mathit{g}\mathit{m}\mathit{a}\mathit{x} {\mathit{p}}_{\mathit{i}}$$

(7)

where ${\mathit{p}}_{\mathit{i}}$ shows the classes’ likelihood distribution, and argmax chooses the class most likely to have the anticipated label.

The model must construct terminal areas once the first regression tree fits the residual ${\mathit{R}}_{\mathit{i}\mathit{j}}$, where the sample and tree leaf indices are denoted by i and j, whereas the index of the tree under construction is denoted by m. The residuals’ leaves in created trees contain information (rjm) in place of the predicted value, which must be mapped back in the testing step using Equation (8).

$$\mathit{r}\mathit{j}\mathit{m}=\mathit{a}\mathit{r}\mathit{g}\mathit{m}\mathit{i}\mathit{n}(\mathit{r}){\sum }_{{\mathit{x}}_{\mathit{i}\in {\mathit{R}}_{\mathit{i}\mathit{j}}}}\mathit{L}({\mathit{y}}_{\mathit{i}},{\mathit{F}}_{\mathit{m}-\mathbf{1}}({\mathit{x}}_{\mathit{i}})+\mathit{r})$$

(8)

The little difference is that models are now accounting for the prior forecast (from a produced model), as opposed to the initiation stage, when there was no prior prediction. The other difference is that certain samples (I) from the terminal region are incorporated into the sample-based summation at this step ${\mathit{R}}_{\mathit{i}\mathit{j}}$ instead of every sample (x). As demonstrated in the first phase, the value of r is selected to minimize the summation in each step.

$${\mathit{F}}_{\mathit{t}}\left(\mathit{x}\right)={\mathit{F}}_{\mathit{t}-\mathbf{1}}\left(\mathit{x}\right)+{\mathit{p}}_{\mathit{t}}{\mathit{h}}_{\mathit{t}}\left(\mathit{x}\right),$$

(9)

where ${\mathit{p}}_{\mathit{t}}$ is the weight of the ${\mathit{t}}^{\mathit{t}\mathit{h}}$ function, ${\mathit{h}}_{\mathit{t}}\left(\mathit{x}\right)$. The approximation is built step-by-step, meaning that a new model is created at each iteration ${\mathit{h}}_{\mathit{t}}$ is constructed without altering any of the earlier models that are part of ${\mathit{F}}_{\mathit{t}-\mathbf{1}}$(x). First, the additive expansion is initialized using constant approximation.

$${\mathit{F}}_{\mathbf{0}}(\mathit{x})=\mathit{a}\mathit{r}\mathit{g}\mathit{m}\mathit{i}\mathit{n}{\displaystyle \sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}}\mathit{L}({\mathit{y}}_{\mathit{i}},\mathit{\alpha })$$

(10)

And the models that follow are constructed to minimize RNNs.

$$({\mathit{p}}_{\mathit{t}}{\mathit{h}}_{\mathit{t}}(\mathit{x}))={\mathit{a}\mathit{r}\mathit{g}\mathit{m}\mathit{i}\mathit{n}}_{\mathit{p},{\mathit{h}}_{\mathit{t}}}{\displaystyle \sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}}\mathit{L}({\mathit{y}}_{\mathit{i}},{\mathit{F}}_{\mathit{t}-\mathbf{1}}({\mathit{x}}_{\mathit{i}})+\mathit{p}{\mathit{h}}_{\mathit{t}}({\mathit{x}}_{\mathit{i}}))$$

(11)

Nevertheless, rather than solving for the optimum of ρ and ${\mathit{h}}_{\mathit{t}}$, the problem is divided into two steps. First, each model ${\mathit{h}}_{\mathit{t}}$ is trained to identify the loss function’s gradient vector based on the data.

To do it, each model is trained using a new dataset, ${\mathit{h}}_{\mathit{t}}$

$$\mathit{D}={\{\mathit{x}}_{\mathit{i}},{\mathit{r}}_{\mathit{t}\mathit{i}}\}{\displaystyle \genfrac{}{}{0pt}{}{\mathit{N}}{\mathit{i}=\mathbf{1}}}$$

where the loss function’s negative gradient at rti is the pseudo-residuals ${\mathit{F}}_{\mathit{t}-\mathbf{1}}\left(\mathit{X}\right)$

$${\mathit{r}}_{\mathit{t}\mathit{i}}={\displaystyle \frac{\mathbf{\partial }\mathit{L}({\mathit{y}}_{\mathit{i}},\mathit{F}({\mathit{x}}_{\mathit{i}}\left)\right)}{\mathbf{\partial }\mathit{F}\left({\mathit{x}}_{\mathit{i}}\right)}}{|}_{\mathit{F}\left(\mathit{x}\right)=}{\mathit{F}}_{\mathit{t}-\mathbf{1}}(\mathit{X})$$

(12)

The function, ${\mathit{h}}_{\mathit{t}}$, is anticipated to produce values that are near the pseudo residuals at the specified data points, which run parallel to the gradient of L at ${\mathit{F}}_{\mathit{t}-\mathbf{1}}\left(\mathit{x}\right)$. However, keep in mind that the square-error loss, which typically directs the h training process, may diverge from the specified objective loss function. However, the worth of ${\mathit{p}}_{\mathit{t}}$ is then calculated using the provided loss function and a line search optimization problem [25,26,27].

$$\mathit{p}\mathit{t}={\displaystyle \genfrac{}{}{0pt}{}{\mathit{a}\mathit{r}\mathit{g}\mathit{m}\mathit{i}\mathit{n}}{\mathit{p}}}{\displaystyle \sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}}\mathit{L}\left({\mathit{y}}_{\mathit{i}},{\mathit{F}}_{\mathit{t}-\mathbf{1}}\left({\mathit{x}}_{\mathit{i}}\right)+\mathit{p}{\mathit{h}}_{\mathit{t}}\left({\mathit{x}}_{\mathit{i}}\right)\right)$$

(13)

The initial concept of gradient boosting is restricted to some final derivations. The primary goal of this article is to describe how to integrate several methods to train a single neural network. Additionally, we provide a gradient boosting formulation modification that allows the base model to be any potential regressor.

4. Result and Discussion

The proposed Intelligent Voice-Based Authentication and Anomaly Detection Framework was developed using Python 3.9 in a high-performance computing setup. The proposed framework can be deployed using low-cost edge devices such as Raspberry Pi 5, NVIDIA Jetson Nano, or ESP32-based gateways. Since MFCC and CQCC extraction are computationally lightweight, the system can operate efficiently on resource-constrained hardware. Compared with cloud-only architectures, edge deployment reduces communication overhead, latency, and energy consumption. Therefore, the proposed solution is suitable for low-cost and low-power smart-home environments. The experiments were carried out on a computer setup with an Intel Core i7 processor, 16 GB RAM, and an NVIDIA RTX-series GPU for accelerating deep learning computations via CUDA support. The deep learning model was designed using TensorFlow 2.x with Keras as the high-level API for constructing the hybrid CNN-LSTM model.

Audio processing and feature extraction were carried out using the Librosa and SciPy libraries, which facilitate the extraction of discriminative features such as Mel-Frequency Cepstral Coefficients (MFCC), Constant-Q Cepstral Coefficients (CQCC), and spectral features. The model was trained and tested using benchmark spoofing datasets from the ASVspoof 2019 and ASVspoof 2021 challenges, which comprise both bona fide and spoofed speech samples. An 80:20 train-validation split was employed to ensure appropriate generalization evaluation. The Adam optimizer with a learning rate of 0.001 was employed for training the model for 50–100 epochs with batch sizes of 32 or 64. Performance analysis was done using the following metrics: accuracy, precision, recall, F1-score, Area Under the Curve (AUC), Equal Error Rate (EER), and normalized minimum Tandem Detection Cost Function (tDCF). The simulation environment facilitated stable convergence, minimized computational time using GPU acceleration, and ensured the validity of performance analysis for multi-class spoof detection tasks. In the proposed model, Constant Q Cepstral Coefficients (CQCC) are integrated with a DL to enhance the accuracy of spoof detection in audio signals. To enhance feature extraction, hybrid models are developed. Each approach aims at maximizing accuracy and minimizing training time per epoch.

The CNN–LSTM model was developed and evaluated using a defined computational and training configuration to ensure consistent performance assessment across the ASVspoof datasets.

Table 2. Training and optimization parameters used for CNN-LSTM model development and evaluation on ASVspoof datasets.

Category	Specification
Operating System	Windows 10 (64-bit)/Ubuntu 20.04 LTS
Processor	Intel Core i7 (10th Gen) or equivalent
RAM	16 GB DDR4
GPU	NVIDIA RTX 2060/3060 (6–8 GB VRAM)
Programming Language	Python 3.9
Deep Learning Framework	TensorFlow 2.x/Keras
Machine Learning Libraries	Scikit-learn, NumPy, Pandas
Audio Processing Library	Librosa, SciPy
Feature Extraction Techniques	MFCC, CQCC, Spectral Features
Dataset Used	ASVspoof 2019 &ASVspoof 2021
Model Architecture	Hybrid CNN-LSTM
Batch Size	32/64
Learning Rate	0.001 (Adam Optimizer)
Epochs	50–100
Evaluation Metrics	Accuracy, Precision, Recall, F1-score, AUC, EER, tDCF
Validation Strategy	80:20 Train-Validation Split
Hardware Acceleration	CUDA-enabled GPU Support

summarizes the implementation environment, model configuration, feature extraction methods, training parameters, optimization settings, evaluation metrics, and hardware specifications adopted in this study.The spectrogram of the ASVspoof 2019 synthetic signal, as shown in Figure 4, indicates the time-frequency distribution of the energy contained within the 1 s sample. There are clear bands of horizontal lines that can be observed at the lower frequency ranges, around 200 to 500 Hz, corresponding to the fundamental harmonic components present within the spoofed logical access signal. However, the level of energy concentration appears to remain consistent and smooth, reflecting the harmonic pattern that is characteristic of the older generation of spoofing attacks, such as voice conversion or text-to-speech synthesis. In addition, the level of background noise appears to remain relatively low, thereby providing clearer boundaries between the different frequency ranges.

In contrast, the spectrogram of the ASVspoof 2021 deepfake-style synthetic signal, as shown above, indicates a more complex time-frequency pattern. Although the fundamental frequency ranges can still be observed, there appears to be a greater level of spectral spread and background noise, particularly at the mid to higher frequency ranges. This reflects the presence of stronger artifacts and noise components that are characteristic of the newer generation of neural-based voice synthesis attacks. In comparison to the 2019 sample, the 2021 sample appears to exhibit a greater level of spectral. To enable effective feature extraction and classification, spectrogram representations were generated from the input speech signals and used as model inputs. Figure 5 presents the generated spectrograms, where subfigure (a) illustrates the spectrogram representation of a sample input signal and subfigure (b) shows the processed spectrogram after transformation and preparation for CNN–LSTM-based analysis. These visual representations capture the time–frequency characteristics required for intelligent voice authentication and anomaly detection.

The MFCC visualization for the ASVspoof 2019 sample shows, in Figure 6, consistent cepstral patterns across frames, particularly in lower-order coefficients, which capture the overall spectral envelope. The smooth distribution of energy across frames suggests stable vocal tract modeling characteristics. Lower variation in higher-order coefficients indicates fewer abrupt spectral fluctuations, aligning with structured synthetic speech generation techniques. These characteristics are often leveraged by spoof detection systems to identify predictable spectral envelopes in logical access attacks. The MFCC representation of the ASVspoof 2021 deepfake-style signal displays comparatively higher variability across frames, especially in mid-level cepstral coefficients. This increased fluctuation reflects subtle spectral distortions and nonlinear artifacts introduced by advanced neural speech synthesis methods. The wider dispersion in MFCC intensity values suggests a more complex spectral structure, which may resemble natural speech more closely while still containing detectable inconsistencies. Such variations are critical features for deep learning-based spoof detection systems.

The anomaly detection graph in Figure 7 presents a variance-based scoring comparison between the two synthetic signals. The ASVspoof 2019 sample shows a slightly higher variance value compared to the ASVspoof 2021 signal. Variance in this context represents overall signal energy dispersion and fluctuation intensity. Although this simple statistical metric provides preliminary differentiation, it is insufficient for real-world spoof detection, where more sophisticated feature-based or deep learning anomaly detection models (e.g., autoencoders, CNNs, or transformer-based architectures) are typically required. The graph primarily demonstrates how statistical indicators can serve as baseline anomaly measures before applying advanced classifiers.

The confusion matrix in Figure 7 of the baseline CNN-LSTM model without optimization indicates relatively higher values of false positives (FP) and false negatives (FN). This implies that the baseline model is not able to effectively distinguish between bona fide, spoof, and anomaly classes. The presence of errors in various classes indicates that the model has not been able to optimize the features and weights effectively. Therefore, the baseline model indicates lower classification accuracy and discriminative ability. The classification performance of the proposed CNN–LSTM model was further evaluated using a confusion matrix, as illustrated in Figure 8. The confusion matrix provides a detailed representation of prediction outcomes by comparing actual class labels with predicted classifications. It highlights the distribution of true positives, true negatives, false positives, and false negatives, thereby enabling assessment of the model’s accuracy and its effectiveness in distinguishing bona fide and spoofed voice samples.

The training and validation accuracy for the proposed model in Figure 9 converges slowly with small variations during the training process. The validation accuracy increases slowly but is slightly lower than the training accuracy, which shows moderate generalization ability. The loss function also decreases slowly, which shows inefficient learning with possible instability during processing.

After the training, the process becomes more stable. The training and validation accuracy curves in Figure 10 show smoother convergence and reduced fluctuation compared to the baseline model. The loss decreases more rapidly, indicating improved weight optimization. The reduced gap between training and validation accuracy demonstrates enhanced generalization and minimized overfitting.

The multi-class ROC curve in Figure 11 represents the discriminative power of the proposed model for bona fide, spoof, and anomaly classes. In all classes, the model has a high Area Under the Curve (AUC) value, which represents high sensitivity and specificity. The curves tend towards the top-left corner, which again verifies the high classification power of the proposed model for all classes.

Figure 12 shows the overall performance, which indicates the improvements of the intermediate results over the baseline model. The above comparison clearly shows that the proposed techniques have greatly improved classification performance and optimization.

The training performance curve of the CNN-LSTM model in Figure 13 indicates a steady drop in loss and improvement in accuracy with each epoch. The small difference between the training and validation curves ensures that the model converges properly. This is an indication that the CNN-LSTM model is capable of extracting the spatial and temporal features of the data without any overfitting.

The loss comparison curve in Figure 14 emphasizes the faster convergence and that this model has the lowest final loss value. The lower loss value and the smooth progress of the curve validate the efficient hyperparameter adjustment and weight update.

The training and validation loss curves of Figure 14 of the proposed CNN-LSTM model show convergence without divergence. The steady reduction in loss and improvement in validation accuracy ensure good generalization ability. The Detection Error Tradeoff (DET) curve represents the trade-off between the false acceptance rate (FAR) and the false rejection rate (FRR). The Equal Error Rate (EER) in Figure 15 is the point where FAR and FRR are equal. The model with the lowest EER value has the best spoofing detection ability. The graph near the origin ensures better detection reliability and zero classification errors.

In

Table 3. Overall performance table.

Class	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)	AUC	EER (%)
Bona fide	96.0	95.4	96.2	95.8	0.960	3.8
Spoof	97.3	97.0	97.5	97.2	0.973	2.9
Anomaly	98.3	98.1	98.5	98.3	0.983	2.1
Macro Average	97.2	96.8	97.4	97.1	0.972	2.9
Weighted Average	97.3	97.0	97.3	97.2	0.974	—

the proposed Hybrid CNN-LSTM model has been able to provide strong multi-class discrimination capabilities for the bona fide, spoof, and anomaly classes. The model is able to provide a macro-AUC of 0.972 and an overall accuracy of 97.2%. The Equal Error Rates for all classes are low (3.8%, 2.9%, and 2.1%), which is a clear indication of the strong spoofing detection and anomaly generalization abilities of the model. The balanced classification performance without any preference for any class is indicated by the small difference between precision and recall values.

The small difference (approximately 1–1.5%) between training and validation accuracy values indicates that the hybrid model is able to overcome the overfitting problem effectively using the temporal feature regularization capabilities of the LSTM layer. Additionally, the small EER values (<4%) and smooth DET curves are a clear indication that the model is able to provide balanced discrimination for the bona fide, spoof, and anomaly classes without any preference for any class. The proposed model is able to provide stable convergence during training with no significant difference between training and validation performance. The EER and AUC values further confirm the robust spoofing detection and anomaly recognition capability of the model. The system is thus highly applicable in real-world scenarios for secure voice authentication and anti-spoofing applications in smart homes and voice-controlled assistants. Each classifier learns to predict the class labels for new examples given training using a specific training method.

From

Table 4. Comparison with existing methods.

Method	Dataset	Accuracy (%)
CNN	ASVspoof 2019	92.4
LSTM	ASVspoof 2019	94.1
ResNet	ASVspoof 2021	95.6
Transformer-based	ASVspoof 2021	96.5
Proposed CNN-LSTM	ASVspoof 2019 + 2021	97.2

the proposed CNN-LSTM framework achieves superior performance compared with existing deep learning approaches due to its ability to jointly learn spatial and temporal speech characteristics. In training, a loss function is optimized using gradient descent techniques. The goal of the classifier is to find the best weights and biases to minimize the loss function and enhance classification performance. With the integration of these deep learning methods, the proposed framework improves smart-home security by providing reliable voice authentication and smart anomaly prediction, thus ensuring protection against spoofing attacks, replay attacks, and unauthorized access attempts. The proposed framework was evaluated using benchmark datasets rather than real-world smart-home deployments. Environmental noise, device heterogeneity, network latency, and privacy constraints may affect performance in practical implementations. Future work will focus on real-time IoT deployment and federated learning-based privacy preservation.

5. Conclusions

In this paper, a novel Intelligent Voice-Based Authentication and Anomaly Detection Framework has been introduced to enhance the security of smart-home devices. In the context of increasing voice-based devices, the need to develop highly sophisticated security systems, other than password or PIN, has become more important than ever, as existing voice-based authentication systems are found to be ineffective in dealing with modern-day spoofing and replay attacks. To address these issues, a strong two-layered security system has been introduced, incorporating a speaker verification system based on deep learning and anomaly prediction, thereby creating a robust voice-based authentication system. By incorporating existing benchmarking datasets from ASVspoof 2019 and ASVspoof 2021, the system has been trained to identify bona fide and spoofed voice signals accurately. By applying highly sophisticated acoustic feature extraction techniques, including MFCC, CQCC, and spectral features, the system has been enabled to identify unique voice features necessary to perform accurate voice-based authentication. The architecture has demonstrated robust discriminative power, and the anomaly detection module has further improved the reliability of the system by identifying anomalies in the learned voice patterns in real-time scenarios. The models proposed in this paper perform better than the existing state-of-the-art models, which require larger datasets for training, when applied to a smaller dataset. According to the proposed model, the accuracy of the home security system is 97.3%.