DriNet: Dynamic Backdoor Attack against Automatic Speech Recognization Models

Abstract

Automatic speech recognition (ASR) is popular in our daily lives (e.g., via voice assistants or voice input). Once its security attributes are destroyed, it poses as a severe threat to a user’s life and ‘property safety’. Prior research has demonstrated that ASR systems are vulnerable to backdoor attacks. A model embedded with a backdoor behaves normally on clean samples yet misclassifies malicious samples that contain triggers. Existing backdoor attacks have mostly been conducted in the image domain. However, they can not be applied in the audio domain because of poor transferability. This paper proposes a dynamic backdoor attack method against ASR models, named DriNet. Significantly, we designed a dynamic trigger generation network to craft a variety of audio triggers. It is trained jointly with the discriminative model incorporated with an attack success rate on poisoned samples and accuracy on clean samples. We demonstrate that DriNet achieves an attack success rate of 86.4% when infecting only 0.5% of the training set without reducing its accuracy. DriNet can still achieve comparable attack performance to backdoor attacks using static triggers, further enjoying richer attack patterns. We further evaluated DriNet’s resistance to a current state-of-the-art defense mechanism. The anomaly index of DriNet is more than 37.4% smaller than that of BadNets method. The triggers generated by DriNet are hard reverse, keeping DriNet from the detectors.

1. Introduction

Automatic speech recognition is being widely used with the development of deep learning and big data technologies, such as Google Assistant (https://assistant.google.com, accessed on 23 May 2022), Microsoft Cortana (https://www.microsoft.com/en-us/cortana, accessed on 23 May 2022), Xiaomi AI speaker (https://www.mi.com/aispeaker/, accessed on 23 May 2022), and Deep Speech (http://research.baidu.com/Blog/index-view?id=90, accessed on 23 May 2022). It plays an essential role in using voice as a carrier and medium to interconnect people and devices. Automatic speech recognition is so wildly used that, once it suffers from an adversary, it may cause severe security and privacy consequences in real-world applications.

Despite a broad application of neural networks, the literature shows that neural networks are vulnerable to various attacks [1,2,3,4]. Backdoor attacks—different from the adversarial attacks launched during the testing phase—pose a more serious threat, i.e., by adding trigger patterns to the target model at training time. In general, the adversary intends to embed hidden backdoors into deep neural networks during training so that the victim models normally perform on benign samples, while their predictions will be maliciously altered if the hidden backdoor is activated by the trigger patterns specified by the adversary [5]. While the backdoor attack is a well-studied phenomenon and has been demonstrated to work in the vision field [6,7,8,9], research on automatic speech recognition is rare. The structures and characteristics of audio data make attacks against them very different from those in image domains. Backdoor attacks in audio domains deserve further study.

The triggers used by backdoor attacks can be divided into two categories: (1) static trigger, which is a fixed pattern placed in a fixed location; and (2) dynamic trigger, which has flexible styles and positions and appears in multiple areas. Current state-of-the-art backdoor attack schemes [6,10,11] rely excessively on fixed static triggers (in terms of patterns and locations), which lack flexibility and are easily detected by existing defense mechanisms [12,13]. They generate a trigger with a static value and position corresponding to a certain target label to obtain this simple one-to-one relationship learned by the neural network model. Due to the monotonous attack style, once the static backdoor attack method is interfered with and restricted in practical applications, its attack performance will be significantly reduced. Compared with static triggers, a dynamic trigger backdoor attack is more powerful and more applicable to the physical world [14]. However, backdoor attacks using dynamic triggers have not been well explored in the audio domain yet. It emphasizes the necessity of designing dynamic backdoor attacks under more realistic conditions.

In this paper, we present the design and implementation of DriNet, a targeted dynamic backdoor attack method for automatic speech recognition models that rely on neural networks to generate different triggers. Triggers of different frequencies and amplitudes can be embedded into the audio samples as required. In contrast to static trigger backdoor attacks, DriNet exploits flexible trigger patterns for more effective backdoor attacks. We evaluated the effectiveness of the proposed method on two audio classification model architectures over two benchmark datasets. The results indicate that DriNet achieves an over 98% targeted attack success rate with only 1% of poisoned training samples (without compromising accuracy). To further discuss the performance of triggers of different sizes, we designed three types of triggers to launch the attack. Moreover, we utilized the state-of-the-art backdoor defense technique to evaluate our method. Our results demonstrate that our dynamic triggers can not be reversed by the defense algorithm. The main contributions of this paper are the following:

• We explored a novel dynamic backdoor attack paradigm in the audio domain (a challenge rarely explored). In our threat model, dynamic triggers are used to attack automatic speech recognition systems, enjoying rich attack styles and greater stealth.
• We introduce a dynamic trigger generation method via the generative adversarial network (GAN) jointly trained with a discriminative model. The effectiveness of backdoor attacks with dynamic triggers is successfully demonstrated.
• Experimental results on two benchmark datasets show the superiority of our method compared to the baseline BadNets. Moreover, an extensive analysis verifies the resistance of the proposed method to the state-of-the-art defense.

The remainder of the paper is structured as follows. Related works on automatic speech recognition and backdoor attacks are provided in Section 2. We elaborate on the detailed design of our dynamic backdoor attack in Section 3. In Section 4, We demonstrate the analysis and evaluation results. We conclude this paper in Section 5.

2. Related Work

2.1. Automatic Speech Recognition

An automatic speech recognition model can map the raw audio waveforms into the content. For the digit recognition task, an audio speech recognition system consists of the following three critical steps.

Preprocessing. The speech signal may be muted part of the time, which brings an extra cost to the speech signal processing. Therefore, voice activity detection (VAD) first needs to be performed on the audio waveform to extract the part with voice activity and ignore the silent part.

Feature Extraction. The most popular speech features in previous speech processing systems are the Mel frequency cepstrum coefficient (MFCC) [15] and Mel spectrogram features [16]. Mel spectrogram feature extraction requires pre-emphasis, framing, windowing, fast Fourier transform, applying Mel filter banks, and logarithmic. After discrete cosine transform (DCT), the Mel spectrogram is converted to MFCC. In the context of the previous defective machine learning algorithms, the Mel spectrogram features are intercorrelated, causing some algorithm trouble. Nevertheless, the DCT operation itself is a lossy compression, which can discard some information and cause damage to the data. Nowadays, neural networks are internally complex, and the problem of intercorrelation can be weakened during the training process. Therefore, in this paper, we adopted the Mel spectrogram as the feature.

Model Prediction. The extracted features are fed into the constructed model and matched with a model to generate the output prediction results. Finally, the mapping from the audio waveform to the digit is completed.

2.2. Backdoor Attack

Backdoor attacks are emerging forms of attacks against deep learning models that can be triggered via a crafted input by the adversary. Poisoning the training samples is the most straightforward and widely adopted method for adding functionality to the training process. The adversary modifies some training samples by adding specified triggers. These poisoned samples with specified target labels and the remaining benign training samples are fed into the model for training, embedding backdoors in the victim model. Existing backdoor attacks have wide applications in the visual and audio fields.

Backdoor attacks in the vision domain. BadNets [6] introduced the first backdoor attack in deep learning by poisoning some training samples. They use white pixel blocks of specific sizes as triggers and always place them in the upper left corners of all inputs. BadNets is a typical static backdoor attack technique that has gained much popularity in backdoor attacks. Currently, most backdoor attacks are sample-independent, i.e., all poisoned samples contain the same trigger pattern. This property is widely used in the design of backdoor defenses. Nguyen and Tran [17] proposed the first sample-specific backdoor attack, where different poisoned samples contain different trigger patterns. This attack can bypass many existing backdoor defenses because it breaks the basic assumptions. Salem et al. [8] proposed the first class of dynamic backdoor attacks, namely BaN and c-BaN. They dynamically determine the location as well as the style of triggers through a network, enhancing the effectiveness of the attack. They achieve strong performances while bypassing the current state-of-the-art backdoor defense techniques.

Backdoor attacks in the audio domain. Kong and Zhang [18] proved that hidden information can illegally be used by an adversarial to activate the backdoor of a DNN-based ASR model that is embedded into an intelligent speaker. Zhai et al. [4] designed a clustering-based attack scheme where poisoned samples from different clusters contain different triggers to infect speaker verification models by poisoning the training data. Koffas et al. [19] designed ultrasonic triggers to perform backdoor attacks on automatic speech recognition systems by poisoning the data. However, the ultrasonic waves fail in the low-pass filter and lack practicality. VenoMave [20] proposed the first targeted poisoning attack against automatic speech recognition. This method causes the automatic speech recognition system to transcribe $x_t$ into the target word W by altering the specific sequence of frames (poison frames) in the original audio input $x_t$. Nevertheless, it takes hours to complete the attack. Li et al. [21] introduced a new backdoor attack method—the trojan neural network (TNN)—against deep learning models. The TNN scans the target neural network, selecting some neurons. Then, it iteratively modifies the input according to the differences between the current value and the expected value of the selected neuron.

2.3. Defense for Backdoor Attack

STRIP [13] can evaluate a model after it has been deployed rather than during the validation phase. STRIP relies on the assumption that the trigger is sample–agnostic. This method adds clean images from the validation set as watermarks to the samples and obtains multiple perturbed inputs with watermarks added. After that, it makes the model predict the entropy of the perturbed inputs. If the calculated entropy value is less than a certain threshold, the sample is considered poisoned.

In contrast, the research results of a neural cleanse are based on an important assumption: the triggers corresponding to backdoor models are much smaller than the minimum perturbation that modifies the input to make the normal model misclassify [12]. It is the first approach used to detect and identify backdoors in deep neural networks, which can mitigate backdoor attacks.

3. Design of Dynamic Backdoor Attacks

3.1. Treat Model

In this paper, we assume that the adversary has full access to the model and the data it needs for training, whereas the adversary has no information about (or modifies other) training configurations. The threats discussed in this paper may occur in many real-world scenarios, such as users adopting third-party training data to train their models or transferring models to training platforms.

Generally, the adversary intends to induce the models trained on it by end users by adding triggers to the training data to contaminate the dataset. The goal of the adversary is to mislead the network when the backdoor trigger is presented but perform well in predicting clean data.

3.2. Problem Formulation

We denote the backdoor discriminative model as $M:X\to Y$, which maps an input $x_i$ from the instance space X to $y_i$ from label space $Y=\{y_1,y_2,\cdots ,y_n\}$ as follows:

$$M\left(x_i\right)=y_i.$$

(1)

The adversary adds the crafted trigger $t\in T$ to the input $x_i$ via a backdoor adding function as,

$$x_i^{\prime }=\mathcal{A}(x_i,t),$$

(2)

where $\mathcal{A}(·)$ denotes a mixing of two pieces of audio. T is the set of triggers generated by the adversary. Then the poisoned input $x_i^{\prime }$ is to be predicted as $y_t$ by the M, i.e.,

$$M\left(x_i^{\prime }\right)=y_t,$$

(3)

where $y_t\ne y_i$. It is a perfect model that the adversary prefers when any clean input is predicted to be the correct label and inputs with the trigger are incorrectly determined to be other labels.

3.3. Overview of Dynamic Backdoor Attack

The details of our dynamic backdoor attack method are illustrated in Figure 1. It is divided into three steps: (a) dynamic trigger generation. Given a random noise, this step generates an adversarial dynamic trigger using gradient information through iterative optimization. (b) Joint training. We choose clean data (a certain percentage) and inject the generated trigger (by step (a)) into them. At the training stage, the discriminative is trained on all clean data and the infected data produced by the trigger generation model. (c) Poisoning. We use the generated dynamic triggers to infect audio with a certain poisoning rate as a training set for end users.

3.4. Dynamic Trigger Generation

We use a dynamic technique to optimize the trigger dynamic generation algorithm by building a neural network to adjust the patterns and frequency of audio triggers adaptively. The details of the architecture and parameters of the trigger generation model constructed in this paper are shown in

Table 1. Architecture of the trigger generation model.

Layers Type	Input	Filter	Stride	Activation
Conv1d	$freq\times n$	$freq\times 1$	1	ReLU
FC	n	$n\times 64$	-	ReLU
FC	64	$64\times 128$	-	ReLU
Dropout	-	-	-	-
FC	128	$128\times 128$	-	ReLU
Dropout	-	-	-	-
FC	128	$128\times (freq\ast n)$	-	ReLU
Dropout	-	-	-	-
Reshape	$freq\times n$	$(freq\ast n)\times freq\times n$	-	sigmoid

. We use the conv1d layer (Conv1d) and the fully connected layer (FC) to build the trigger generation model. The input of the model is a two-dimensional array ($freq\times n$) of noise, where the freq and n denote the frequency of the noise and the number of frames it has, respectively. The noise vector is sampled from the uniform distribution ($noise\sim U(0,1)$). Alternatively, the adversarial can utilize different distributions to sample noise, such as Gaussian or a custom distribution function. Moreover, dropout layers are applied after the last three fully connected layers to avoid the model producing the same output, increasing the diversity of the generated triggers. After the reshape layer, the model can generate an adversarial audio trigger with the same shape as the random noise. The audio trigger is converted back to the time domain via an inverse Fourier transform to facilitate it to be mixed with the original sound waves and be played in the physical world.

3.5. DriNet Algorithm

For the training phase, inspired by the generative adversarial network (GAN) technique [22], we consider the target victim model as the discriminative model D and the trigger generation model as the generative model G. We jointly train the D and the G using Algorithm 1.

Algorithm 1 Joint training of the target victim model and the trigger generation model

Input: The training dataset Q, the epochs, the loss function $\mathcal{L}$, the uniform distribution function (random), the target label l, the backdoor add function $\mathcal{A}$, the weight $\alpha$ Output: D, G   1: Initialize $epoch=0$   2: while $epoch<epochs$  do   3:    $x,y\leftarrow Q$   4:    $\widehat{y}=D\left(x\right)$   5:    $Los{s}_1=\mathcal{L}(y,\widehat{y})$   6:    $\tilde{x}\leftarrow$ Select the samples from x whose $y\ne l$   7:    $noise\leftarrow Random(0,1)$   8:    $trigger=G\left(noise\right)$   9:    $x^{\prime }=\mathcal{A}(\tilde{x},trigger)$   10:   ${\widehat{y}}^{\prime }=D\left(x^{\prime }\right)$   11:  $Los{s}_2=\mathcal{L}(l,{\widehat{y}}^{\prime })$   12:  Update G with $Los{s}_2$   13:  $Loss=Los{s}_1+\alpha \ast Los{s}_2$  14:   Update D with Loss  15:   $epoch=epoch+1$   16: end while   17: return D, G

During training, we first fetch a batch of samples from the training dataset Q. Then, we calculate the cross-entropy loss $Los{s}_1$ with the loss function $\mathcal{L}$ between the prediction y of the D for this batch of samples and the true label $\widehat{y}$, as in (4),

$$\mathcal{L}(y,\widehat{y})=-\sum _{i=1}^{\left|x\right|}{y}_{i}log{\widehat{y}}_i,$$

(4)

where $\left|x\right|$ is the dimension of the input. After that, we select different audios from all non-target classes, which are relabeled with the target label. In the following, we craft poisoned samples by mixing the clean audios selected with the triggers generated by the trigger generation model. The backdoor add function used in this work is to embed the trigger at a random frequency into the beginning of the clean audio. Likewise, we calculate the second cross-entropy loss $Los{s}_2$ with the $\mathcal{L}$. Finally, we update the trigger generation model G with $Los{s}_2$ and update the target discriminative model D with $Los{s}_1+\alpha \ast Los{s}_2$, where $\alpha$ is a hyperparameter to trade-off between accuracy and attack success rate.

3.6. Poisoning

To complete the backdoor attack, we use the trained trigger generation model to make a poisoned data subset $Q_{poison}$, i.e., $Q_{poison}=\left\{(x^{\prime },l)\right|x^{\prime }=\mathcal{A}(x,t),(x,y)\in Q_{clean}\}$, where $x^{\prime }$ indicates the poisoned samples, l denotes the target label, $Q_{clean}$ is the original clean dataset. Then, we fuse it with $Q_{clean}$ to form a new dataset $Q_{train}$, i.e.,

$$Q_{train}=Q_{poison}\cup Q_{clean}.$$

(5)

We define $\gamma$ as the poisoning rate as in (6),

$$\gamma =\frac{|Q_{poison}|}{|Q_{train}|}.$$

(6)

4. Experiments

4.1. Experimental Settings

Datasets and Target Models. To evaluate the effectiveness of our proposed method in this experiment, we take the raw audios from the AudioMNIST dataset [23], and the speech commands dataset [24], described in

Table 2. A summary of datasets.

Dataset	Zero	One	Two	Three	Four	Five	Six	Seven	Eight	Nine	Total
AudioMNIST	3000	3000	3000	3000	3000	3000	3000	3000	3000	3000	30,000
Speech Commands	4052	3890	3880	3727	3728	4052	3860	3998	3787	3934	38,908

. The AudioMNIST dataset consists of 30,000 audio samples of spoken digits (0–9), in English, from 60 speakers of multiple age groups, different genders, and different accents. The speech commands dataset contains 105,829 audio files, constituted by 35 different kinds of spoken single-word commands, including ten digits, command words, and other common words. Among them, we choose 10 classes of digit words for evaluation, a total of 38,908 samples. These two datasets are split into the 80% training set and 20% test set.

We use two target victim models with different architectures in this paper, namely 1D CNN [25] and X-vector [26]. The CNN models are well known for their excellent performance in the computer vision domain. Specifically, 1D CNN demonstrates its good performance in speaker recognition systems. The X-vector is the current baseline model framework in speech recognition. The network consists of a deep neural network that maps the speech feature sequences to a fixed dimensional embedding. We modify them to adapt to the Mel spectrogram features in this paper.

Baseline and attack setup. We adopt the adapted BadNets method as the baseline for comparison. Similar to how it works in images, BadNets poisons the training set with a fixed static audio trigger to form a backdoor. The amplitude of the fixed static audio trigger used by BadNets is about the same as the average amplitude of the dynamic trigger we generate. Moreover, the frequency of the fixed static is permanently fixed at the lowest three frequencies on the Mel scale [27].

All models are implemented in PyTorch [28] 1.11.0 and CUDA 11.4. We set $epoch=100$ and $\alpha =0.1$ for the 1D CNN model, while $epoch=100$ and $\alpha =0.07$ are set for the X-vector. We set the target label to zero for demonstration. In the training phase, we use an Adam [29] optimizer with a learning rate of 0.001. As for the size of the triggers, we discuss the performance impact of three kinds of shapes of triggers, (1 × 5), (2 × 5), and (3 × 5).

4.2. Evaluation Metrics

The dynamic backdoor attack aims to make the victim model misclassify samples with triggers while maintaining good performance for clean samples. Thus, we employ two metrics to evaluate DriNet: clean samples accuracy and attack success rate.

Clean Samples Accuracy. We use the equation, $Accuracy=\frac{N_c}{N_a}$, to obtain the accuracy of the model predicting the clean samples, where $N_c$ is the number of clean samples that are correctly predicted, and $N_a$ denotes the number of all clean samples that we test.

Attack Success Rate (ASR). We calculate the attack success rate as $ASR=\frac{N_t}{N_p}$, where $N_t$ is the number of poisoned samples that are misclassified as the target label and $N_p$ represents the number of samples with triggers. In particular, a targeted attack is considered successful only if the poisoned sample is predicted to be the specific label.

4.3. Results

We evaluate the performance of our dynamic backdoor attack for the 1D CNN model and the X-vector model on the AudioMNIST dataset and the speech commands dataset, respectively. For the AudioMNIST dataset,

Table 3. The results of the accuracy and attack success rate for the AudioMNIST dataset.

AudioMNIST
Model	Poisoning Rate	Trigger Size
		(1 × 5)		(2 × 5)		(3 × 5)
		ACC	ASR	ACC	ASR	ACC	ASR
1D CNN	0.005	0.998	0.697	0.998	0.758	1	0.864
	0.01	0.999	0.87	0.998	0.848	1	0.984
	0.03	0.999	0.983	0.999	0.988	0.999	0.989
	0.05	0.998	0.984	0.999	0.994	0.999	0.999
	0.1	0.998	0.987	0.999	0.997	0.999	0.995
	0.2	0.998	0.996	0.998	0.997	0.998	0.997
X-vector	0.005	0.997	0.676	0.997	0.516	0.997	0.633
	0.01	0.998	0.857	0.999	0.968	0.999	0.926
	0.03	0.998	0.983	0.998	0.957	0.998	0.977
	0.05	0.998	0.987	0.998	0.984	0.998	0.994
	0.1	0.997	0.985	0.999	0.999	0.998	0.997
	0.2	0.998	0.994	0.999	0.999	0.999	0.997

demonstrates that successively increasing the poisoning rate from 0.005 to 0.2 leads to a higher attack success rate while maintaining excellent clean sample accuracy(at least 0.997). Moreover, with the trigger size of (3 × 5), we can achieve up to an 86.4% attack success rate by poisoning only 0.5% of the training set for the 1D CNN model and achieve over 98% targeted ASR with 1% of poisoned samples. Moreover, for the speech commands dataset shown in

Table 4. The results of the accuracy and attack success rate for the speech commands dataset.

Speech Commands
Model	Poisoning Rate	Trigger Size
		(1 × 5)		(2 × 5)		(3 × 5)
		ACC	ASR	ACC	ASR	ACC	ASR
1D CNN	0.03	0.927	0.211	0.936	0.341	0.946	0.455
	0.05	0.917	0.411	0.926	0.467	0.924	0.541
	0.1	0.877	0.484	0.893	0.559	0.906	0.629
	0.2	0.834	0.617	0.852	0.662	0.871	0.722
	0.3	0.8	0.68	0.825	0.753	0.847	0.778
	0.4	0.79	0.762	0.815	0.81	0.845	0.824
	0.5	0.782	0.812	0.822	0.842	0.844	0.866
X-vector	0.03	0.931	0.105	0.941	0.342	0.949	0.522
	0.05	0.915	0.287	0.932	0.52	0.939	0.574
	0.1	0.891	0.466	0.905	0.627	0.92	0.705
	0.2	0.835	0.636	0.875	0.755	0.892	0.756
	0.3	0.812	0.727	0.851	0.797	0.863	0.807
	0.4	0.797	0.795	0.841	0.847	0.862	0.868
	0.5	0.811	0.861	0.84	0.888	0.861	0.898

, the attack success rate gradually goes up as the poisoning rate increases from 0.03 to 0.5. However, this improvement has side effects that the accuracy reduces to varying degrees, the lowest is 8.8%, and the highest is 17.1%.

We also discuss the effect of the trigger size on dynamic backdoor attack effectiveness. In general, a larger trigger size achieves a more vigorous attack. For example, triggers of 3 × 5 shapes significantly surpass triggers of 1 × 5 shapes, in terms of both accuracy and attack success rate with the same poisoning rate. However, this will inevitably make the trigger more conspicuous. A trade-off should be made between the attack success rate and stealthiness in a real-world application.

The clean sample accuracy and attack success rate concerning BadNets and DriNet are shown in

Table 5. The performance comparison between BadNet and DriNet. We let the poisoning rate be 0.01 for the AudioMNIST dataset and 0.1 for the speech commands dataset. “×” means cannot bypass the defense. “√” means can bypass the defense.

Attack Method	Victim Model	AudioMNIST		Speech Commands		Neural Cleanse
		ACC	ASR	ACC	ASR
BadNets	1D CNN	0.999	0.956	0.912	0.677	×
	X-vector	0.999	0.949	0.915	0.658
DriNet	1D CNN	0.999	0.984	0.906	0.629	√
	X-vector	0.999	0.945	0.920	0.705

. We find that our dynamic backdoor attack method can still achieve comparable attack performance to the static backdoor attack method. Intuitively, a dynamic trigger enjoys a richer attack pattern than a static trigger due to its multiple styles. In a complex environment, for the dynamic backdoor model, the adversary can use a variety of attack samples, but for the static backdoor model, the attack is limited. Furthermore, dynamic backdoor attacks enjoy more resistance and stealth than the static method. Thus, DriNet has more tremendous advantages than a static backdoor attack.

Defense. We use both datasets to evaluate our technique by the state-of-the-art defense, namely neural cleanse. Neural cleanse is a model-based defense mechanism in the model validation testing phase.

Neural cleanse iteratively optimizes a trigger for each given target label to find the minimum one required to misclassify the samples from other labels. In this way, it can reverse N potential triggers for each label. Then, neural cleanse measures the sizes of the triggers and detects if any trigger is significantly smaller than the others based on the median absolute deviation (MAD) [30].

As shown in Figure 2, the models constructed by DriNet have a lower anomaly index than the baseline models and perform even better than the clean models in most circumstances. The anomaly index of DriNet is 37.4% smaller than that of the BadNets method for the 1D CNN models on the AudioMNIST dataset. Moreover for X-vector models on the speech commands dataset, this ratio is as high as 79.4%. Moreover, the anomaly index of our models is less than the threshold, which means that neural cleanse fails to detect the backdoor models attacked by DriNet. The potential triggers reversed by neural cleanse contain information in multiple locations and styles, resulting in large enough masks to make the anomaly index more minor than the threshold. In other words, dynamic triggers generated by DriNet are more challenging to reverse than static triggers. We believe that the neural cleanse mechanism fails to detect our hidden backdoor because DriNet breaks the fundamental assumption about backdoor attacks—that triggers are invariant in position and style.

4.4. Visualization

Figure 3 shows three examples of poisoned audio files from three digits. They are all marked with target tags after adding triggers, shown right below themselves. Triggers are mixed with the original audio instead of appended to it, which does not change the audio file’s size and improves the stealth of the attack. The frequencies and patterns of triggers are different for the same target tag, providing more adversarial attack samples to the adversary.

Figure 4 shows three kinds of triggers generated by our trigger generation model. For example, Figure 4a presents an audio trigger with the size of 1 × 5, meaning a sound that lasts five frames at one certain frequency. Similarly, we craft triggers at two or three adjacent frequencies on the Mel scale, as in Figure 4b,c.

5. Conclusions

In this paper, we explored the problem of backdoor attacks against automatic speech recognition. We propose DriNet, a dynamic backdoor attack method to generate different patterns and frequency audio triggers against automatic speech recognition models. Introducing the GAN technique, we jointly designed a trigger generation model and trained it and the target discriminative model. We conducted extensive experiments using two benchmark datasets. The evaluation verifies the effectiveness of our proposed dynamic backdoor attack by poisoning a small proportion of the training set. We show that DriNet can successfully bypass a state-of-the-art defense against backdoor attacks while achieving strong performance as the static backdoor attack method.

Although we took some steps in dynamic backdoor attack technology against speech recognition models, improvements could be made to our method. The poisoned audio does not sound the same as normal audio if listeners pay attention to the difference. Furthermore, we aim to make the triggers imperceptible by introducing principles of psychoacoustics.