Generating Hard-Label Black-Box Adversarial Examples for Video Recognition Models

Abstract

In recent years, video recognition models have witnessed the rapid development of Deep Neural Networks (DNNs). However, these models remain not robust to adversarial examples that are created by adding imperceptible perturbations to clean samples. Recent studies indicate that generating adversarial examples in the hard-label black-box setting is particularly challenging yet highly practical. Compared to image recognition models, there are few hard-label black-box adversarial example generation algorithms for video recognition models. To this end, we propose a hard-label black-box video adversarial example generation algorithm, referred to as Dynamic Black-box Algorithm (DBA). First, DBA uses the binary search algorithm to find the boundary video between two original videos; then, the sampling-based algorithm is used to estimate the gradient on the boundary video; finally, with a dynamic step size adjustment strategy, DBA moves the boundary video towards the direction of the estimated gradient to generate the adversarial video. Additionally, we designed another strategy to skip invalid samples generated during the adversarial example generation process. Experiments demonstrate that DBA attains a superior trade-off between the magnitude of perturbations and query efficiency. Specifically, DBA outperforms state-of-the-art algorithms, achieving an average reduction in Mean Squared Error (MSE) of over 50%.

1. Introduction

As shown in Figure 1, Deep Neural Networks (DNNs) consist of the input layer, the hidden layer, and the output layer [1]. The input layer collects the input data, which is then processed in the hidden layer where most of the calculations occur. The output layer generates the final outputs of DNNs, which can be predictions, classifications, or probabilities. To prevent overfitting, an effective technique called dropout is used in DNNs [2], which randomly drops some nodes of DNNs during training. DNNs have advanced rapidly in recent years [3], yet they still face the challenges from adversarial examples, which are initially discovered in image recognition and crafted by injecting imperceptible perturbations into clean samples [3,4]. When processed by DNNs, these adversarial examples can trigger erroneous classification results that diverge significantly from the original predictions. As depicted in Figure 1, the classification label of the clean sample is “smile”; after introducing imperceptible perturbations, the clean sample becomes the adversarial example. The adversarial example is indistinguishable from the clean sample, yet its classification label is “smoke”—completely different from the clean sample’s classification label. Essentially, adversarial examples expose potential security vulnerabilities of DNNs [5], which restricts the application prospects of DNNs in the security-critical fields [6]. Therefore, an in-depth study of adversarial example generation algorithms can help researchers find the vulnerabilities of DNNs proactively so that they can take some measures (such as adversarial training) to enhance the security of DNNs [7], which is significantly important for improving the robustness of DNNs [8]. Hence, an increasing number of researchers are devoted to the field of adversarial example generation algorithms [9,10,11].

To generate high-quality adversarial examples, the process of adding perturbations to clean samples usually needs to test the DNN-based model (also called the victim model or the target model) iteratively so that the perturbation sampling strategy can be adjusted dynamically according to the target model’s output [12,13]. As illustrated in Figure 2, depending on how much information can be obtained from the target model, adversarial example generation can be categorized into three types: the white-box setting, the score-based black-box setting, and the hard-label black-box setting [14,15]. In the white-box setting, the researcher has full access to the target model’s information, including the network structure, parameters, probabilities, and labels [16]. In the score-based black-box setting, the researcher can only query the target model for probabilities and labels [17]. In the hard-label black-box setting, the researcher can only query the target model for labels (also called hard labels) [18]. Among these categories, the hard-label black-box setting is widely regarded as the most practical yet challenging [18]. In real-world scenarios, commercial models (e.g., MEGVII Face++ and Microsoft Azure) typically only return labels, withholding internal information such as the network architecture, parameters, and probabilities.

Since adversarial examples originate from image recognition, there are more studies in this field [19,20,21]. However, with the rapid development of short-video applications like TikTok and Kuaishou, videos have become the primary information transmission medium in the internet era, and more and more video recognition models based on DNNs are proposed [22,23,24]. Yet, compared with image recognition models, there are few adversarial example generation algorithms for video recognition models [25,26]. Most current video adversarial example (also called adversarial video) generation algorithms focus on the white-box setting and the score-based black-box setting [27,28], and the hard-label black-box setting is rarely explored. In the white-box setting, SVASTIN [28] is a novel sparse video adversarial example generation algorithm, which consists of the GTVL [28] module and the STIN [28] module. SVASTIN generates adversarial videos by exchanging the spatio-temporal feature space information. In the score-based black-box setting, VBAD [29] is the first score-based black-box video adversarial example generation algorithm. VBAD takes advantage of the migration of the perturbations from image models to video models and initializes the video perturbations with the image perturbations. In order to correct the perturbation deviation between image models and video models, VBAD divides the frames into many small parts and then uses the NES [29] algorithm to estimate the gradient of these small parts. EARL [30] uses the agent of reinforcement learning to select the keyframes of videos, and only adds perturbations to these keyframes, which reduces the magnitude of perturbations and improves the efficiency of adversarial example generation. In the hard-label black-box setting, STDE [31] is the state-of-the-art hard-label black-box video adversarial example generation algorithm, which has the best performance in this field. STDE introduces target videos as initial perturbations and only adds perturbations on keyframes that are adaptively selected by the temporal difference algorithm. However, one drawback of STDE is that its adversarial examples contain too many perturbations that can even be easily recognized by human vision.

In summary, current adversarial example generation algorithms for video recognition models still face the challenge of perturbation optimization. Therefore, we introduce Dynamic Black-box Algorithm (DBA), a novel hard-label black-box video adversarial example generation algorithm designed to address this gap. In some security-critical video recognition scenarios, video adversarial examples may pose significant security risks. For example, surveillance models could be illegally bypassed, or autonomous driving models might cause traffic accidents due to misclassifications [29]. The purpose of our study is to generate high-quality adversarial examples for video recognition models in the hard-label black-box setting, thereby helping researchers find the vulnerabilities of these models proactively so that they can take some measures (such as adversarial training with adversarial videos generated by DBA) to enhance the security of these models. This is crucial for improving the robustness of video recognition models. DBA focuses on the hard-label black-box setting, which is widely regarded as the most practical yet challenging [18]. In this setting, commercial models typically only return labels, withholding internal information such as the network architecture, parameters, and probabilities. Our main challenges are as follows:

(1) Efficient gradient estimation in the hard-label black-box setting. While gradient estimation is effective for optimizing perturbations, the high dimensionality of videos poses a severe “challenges of dimensionalities” [30]. Existing algorithms primarily target the white-box setting or the score-based black-box setting [29,30]; efficiently estimating gradients using only hard labels remains an open challenge.

(2) Efficient step size adjustment for video data. When generating adversarial examples, it is necessary to move boundary samples along the estimated gradient direction with specific step sizes. In the hard-label black-box setting, several step size adjustment strategies have been developed for image models [32,33]. But these strategies are only applicable to low-dimensional image data and perform poorly when handling high-dimensional video data.

(3) Robustness against invalid samples. The vast search space in videos frequently leads to “invalid samples” during the iterative process, which can stall or terminate gradient estimation. Designing a robust strategy to bypass these invalid samples is essential for stable optimization.

To address the above three challenges, we designed the following algorithms and strategies for DBA, which are also our main contributions:

(1) We used a sampling-based algorithm to estimate the gradient on the boundary video in the hard-label black-box setting. By introducing two lemmas and one theorem, we ultimately proved the effectiveness of this gradient estimation algorithm.

(2) We designed a new step size adjustment strategy to generate adversarial videos, which can dynamically adjust the step size as the model queries change during the adversarial example generation process. Experiments show that our strategy can help to generate adversarial videos with fewer model queries.

(3) We designed a strategy to skip invalid samples generated during the adversarial example generation process. By this strategy, DBA can find valid samples quickly with very few model queries. Extensive experiments demonstrate that when generating adversarial videos of the same quality, the model queries of DBA are 40% lower than those of other comparison algorithms on average.

Table 1. Explanation of symbols.

Serial Number	Symbol	Description
1	$x_{bod}^t$	The boundary video generated in the t-th iteration
2	$x_{adv}^t$	The adversarial video generated in the t-th iteration
3	$x_{tgt}$	The target video
4	$CS$	The probability difference function
5	$\varphi$	The sign function
6	u	The sampled perturbation
7	$\delta$	A small constant
8	L	The Lipschitz continuity coefficient of $CS$
9	$\nabla CS$	The true gradient of $CS$
10	$\widetilde{\nabla CS}$	The estimated gradient of $CS$
11	$\mathbb{E}$	The expectation function
12	d	The number of pixels of videos
13	$\mathbb{P}$	The probability function
14	$\mathcal{B}$	The Beta distribution
15	$x^{\prime }$	The video between $x_{bod}^t$ and $x_{bod}^t+\delta u$
16	$\beta$	The uniform distribution
17	$E_i(i=1,2,3)$	The three subregions of the spherical region
18	$v_1,\dots ,v_d$	The orthogonal basis

provides an explanation of the main symbols used in our paper.

In the field of image recognition, several hard-label black-box adversarial example generation algorithms have been proposed [32,33]. DBA differs from these algorithms. First, these algorithms are only applicable to low-dimensional image data and perform poorly when handling high-dimensional video data. Related research [34,35] shows that an efficient step size adjustment strategy can improve algorithm performance. Inspired by these studies, we designed a new step size adjustment strategy. This strategy enables the generation algorithm to dynamically adjust the step size based on the stage of the adversarial example generation process, thereby improving the efficiency of generating adversarial videos. Additionally, the low dimensions of images eliminate the problem of invalid samples. But due to the high dimensionality of video data, it is inevitable to generate invalid samples, and these algorithms designed for image models cannot address this problem. To this end, we designed a strategy to skip invalid samples, which can find valid samples quickly with very few model queries.

2. Proposed Framework

We denote the video recognition model as $f\left(x\right):X\to Y$; $x\in X\subset R^{N\times W\times H\times C}$ and $y\in Y=\{1,2.\dots .K\}$, where x and y represent the video and the model’s output, respectively. $N,W,H,C$ denote the number of frames, frame width, frame height, and the number of channels, respectively; K represents the number of classes. The goal of generating the adversarial example for the video recognition model is to produce an adversarial video $x_{adv}$ that can fool f into making misclassifications, that is, $f\left(x_{adv}\right)=y_{adv}$, where $y_{adv}\ne y$. In order to ensure that x and $x_{adv}$ are undistinguishable for human vision, the researcher will keep $x_{adv}$ within a ball centered at the clean video x (${∥x_{adv}-x∥}_p\le {ϵ}_{adv}$).

In the hard-label black-box setting, the researcher can only get the hard label from f. Other information of f is not accessible, such as parameters, the network architecture, probabilities, etc. First, the researcher chooses a source video $x_{src}$ and a target video $x_{tgt}$, and their labels are $y_{adv}$ and y, respectively. The researcher’s purpose is to get an adversarial video that is “visually close” with $x_{tgt}$ but misclassified as $y_{adv}$ (also called the adversarial label). So the researcher will move $x_{src}$ towards $x_{tgt}$ so that they are indistinguishable for human vision while keeping $x_{src}$ classified as $y_{adv}$ by f. During this process, the videos that are classified as $y_{adv}$ and located at the decision boundary between the two classes (e.g., $y_{adv}$ and y) are called boundary videos.

The general framework of the DBA is shown in Figure 3. The videos shown in Figure 3 are collected from the HMDB-51 [36] dataset. We sampled some frames from the original videos at even frame intervals and preprocessed these sampled frames using the resizing function and the cropping function. After preprocessing the original videos according to the above steps, we finally get the video samples that can be directly input into the target model. Our purpose is to get an adversarial video that is “visually close” with “punch” (target video) but misclassified as “smile” (source video). First of all, we begin with two videos (the source video and the target video) together with a sampling subspace that obeys the Gaussian distribution, and then iteratively conduct the following three steps:

(1) Finding a boundary video at the decision boundary between the two classes (e.g., “smile” and “punch”), which is used to provide a boundary video to estimate the gradient.

(2) Calculating the next movement direction by estimating the gradient, which is used to estimate the gradient on the boundary video.

(3) Moving along the estimated gradient direction, which is used to move the boundary video along the estimated gradient to generate the adversarial video.

Next, we will describe the three steps in detail.

2.1. Finding a Boundary Video at the Decision Boundary

For DNNs, there is a decision boundary between any two samples belonging to different labels [32]. As shown in Figure 4, $x_{src}$ and $x_{tgt}$ are two videos whose labels are $y_{adv}$ and y, respectively. There is a decision boundary between $x_{src}$ and $x_{tgt}$. The boundary video $x_{bod}$ lying on the decision boundary has the features of both $x_{src}$ and $x_{tgt}$. If we add some small random perturbations to $x_{bod}$, its classification label will randomly oscillate between $y_{adv}$ and y, which indicates that $x_{bod}$ is highly sensitive to the random perturbations. In contrast, if we add the same perturbations to $x_{src}$ or $x_{tgt}$, their classification labels are likely to remain unchanged because they are farther from the decision boundary. When estimating the gradient, the more sensitive the samples are to the perturbations, the more accurate the estimated gradient is [33]. Therefore, if $x_{bod}$ is used to estimate the gradient, we will get more accurate gradients. We use the binary search algorithm [32] to find the new boundary video between the target video and the previous adversarial video:

$$x_{bod}^t=\alpha ·x_{adv}^{t-1}+(1-\alpha )·x_{tgt},$$

(1)

where $x_{bod}^t$ is the new boundary video; $x_{adv}^{t-1}$ is the previous adversarial video generated in the $(t-1)$-th iteration; $x_{tgt}$ is the target video; and $\alpha$ is the parameter of the binary search algorithm. At the beginning, we set $x_{adv}^0=x_{src}$. With Equation (1), we can adjust $\alpha$ so that $x_{bod}^t$ simultaneously has the features of both $x_{adv}^{t-1}$ and $x_{tgt}$, which is the essence of the binary search algorithm.

2.2. Calculating the Next Movement Direction by Estimating the Gradient

Above all, we define the probability difference function $CS$ and the sign function $\varphi$ as follows:

$$CS\left(x\right)={H\left(x\right)}_{y_{adv}}-max\left[{H\left(x\right)}_{y\ne y_{adv}}\right],$$

(2)

$$\varphi \left(x\right)=sign\left(CS\left(x\right)\right)=\{\begin{array}{}\mathrm{(3)}\hfill & -1,\hspace{1em}\mathrm{if}CS\left(x\right)<0,\hfill \mathrm{(4)}\hfill & 1,\hspace{1em}\mathrm{otherwise},\hfill \end{array}$$

where $H{\left(x\right)}_y$ represents the probability of the label y. In the hard-label black-box setting, the researcher can only obtain the value of $\varphi$ and cannot get the value of $CS$.

We next try to estimate the gradient on the boundary video. Inspired by relevant research [32,33], we employ the sampling-based algorithm [33] to estimate the gradient:

$$\widetilde{\nabla CS}\left(x_{bod}^t\right)={\displaystyle \frac{1}{N}}\sum _{k=1}^N\varphi (x_{bod}^t+\delta u)u,$$

(5)

where $x_{bod}^t$ is the boundary video produced in the t-th iteration; N is the number of the sampled perturbations; $\delta$ is a small constant; u is the perturbation sampled from the sampling subspace that obeys the Gaussian distribution; and $\widetilde{\nabla CS}$ is the estimated gradient.

The function $CS$ represents the difference between two probability values, and the value of $CS$ lies in the range [−1, 1]. Related research [32,33] indicates that $CS$ can be considered Lipschitz continuous in the field of hard-label black-box adversarial example generation. If the value of $CS$ can be obtained, its gradient can be estimated by calculating the sensitivity of $CS$ to perturbations. However, the value of $CS$ cannot be directly obtained in the hard-label black-box setting; therefore, as shown in Equation (5), we estimate the gradient of $CS$ by evaluating the sensitivity of $\varphi$ to perturbations.

Inspired by related research [18,32,33], we next prove the validity of gradient estimation. We first give Lemmas 1 and 2, which are used to prove Theorem 1:

Lemma 1.

Suppose that $\nabla CS$ is the true gradient and $CS\left(x_{bod}^t\right)=0$, then $CS(x_{bod}^t+\delta u)$ can be rewritten as

$$\begin{array}{c}CS(x_{bod}^t+\delta u)=\delta \nabla CS{\left(x_{bod}^t\right)}^{\top }u+{\displaystyle \frac{1}{2}}{\delta }^2{u}^{\top }{\nabla }^{2}CS\left(x^{\prime }\right)u,\end{array}$$

(6)

where $x^{\prime }$ is a video between $x_{bod}^t$ and $x_{bod}^t+\delta u$.

Lemma 2.

Suppose that we have a function as follows:

$$\begin{array}{c}f\left(r\right)={\displaystyle \frac{1-r}{\sqrt{1-2r+2r^2}}}.\end{array}$$

(7)

Then the following inequality holds:

$$\begin{array}{c}f\left(r\right)\ge 1-{\displaystyle \frac{1}{2}}{r}^2.\end{array}$$

(8)

Based on Lemmas 1 and 2, we formally present Theorem 1.

Theorem 1.

Suppose that $\nabla CS$ is the true gradient and $CS\left(x_{bod}^t\right)=0$, we have

$$\begin{array}{c}cos\angle \left(\mathbb{E}\left[\varphi (\delta u+x_{bod}^t)u\right],\nabla CS\left(x_{bod}^t\right)\right)\ge 1-{\displaystyle \frac{9L^2{\delta }^2{(d-1)}^2}{8\parallel \nabla CS\left(x_{bod}^t\right){\parallel }_2^2}}.\end{array}$$

(9)

When $\delta \to 0$, we also have

$$\begin{array}{c}\underset{\delta \to 0}{\lim }cos\angle \left(\mathbb{E}\left[\widetilde{\nabla CS}\left(x_{bod}^t\right)\right],\nabla CS\left(x_{bod}^t\right)\right)=1,\end{array}$$

(10)

where $\widetilde{\nabla CS}$ is the estimated gradient; d is the number of pixels of videos; and L is the Lipschitz continuity coefficient of $CS$.

Theorem 1 indicates that the direction of the estimated gradient aligns with the direction of the true gradient direction.

In summary, the gradient estimation method described by Equation (5) is essentially a mean-based algorithm that evaluates the significance of perturbations. First, sampled perturbations are assigned different weights based on their impact on the target model: perturbations that cause the target model to produce the adversarial label are assigned a weight of 1, while others receive a weight of −1. Next, perturbations are multiplied by their respective weights to form temporary results. Finally, the mean of all these temporary results is regarded as the final estimated gradients. Because perturbations are introduced during the gradient estimation process, this method inevitably yields errors. Theorem 1 indicates that these errors can be controlled within a certain range, and the fewer perturbations added, the higher the accuracy of the estimated gradient.

See Appendix A, Appendix B and Appendix C for the proof of Lemmas 1 and 2, and Theorem 1.

2.3. Moving Along the Estimated Gradient Direction

The estimated gradient provides the moving direction for the boundary video. Inspired by research on image recognition models [34,37,38,39], we try to update the boundary video towards the direction of the estimated gradient to generate the new adversarial video:

$$x_{adv}^{t+1}=x_{bod}^t+\eta {\displaystyle \frac{\widetilde{\nabla CS}\left(x_{bod}^t\right)}{{\parallel \widetilde{\nabla CS}\left(x_{bod}^t\right)\parallel }_2}},$$

(11)

where $x_{adv}^{t+1}$ is the new generated adversarial video, and $\eta$ is the step size. In practice, we observe that the original step size adjustment strategy for image recognition models is inefficient in video recognition scenarios, because this strategy fails to take into consideration the characteristics at different stages of the adversarial example generation process. Generally speaking, at the initial stages, the distance between the target video and the adversarial video is large, and the gradient estimation algorithms can easily yield relatively accurate gradients, so we should increase the step size. As the adversarial example generation process goes on, the distance between the target video and the adversarial video becomes small, and the estimated gradient also becomes inaccurate; therefore, the step size should be reduced. The original step size adjustment strategy fails to take into account the variation process of the gradient accuracy. To address this limitation, we propose a novel step size adjustment strategy based on the function $\psi$:

$$\psi \left(q\right)=max(v_{min},v_{ini}-{\displaystyle \frac{q}{d}}),$$

(12)

$${\eta }_{new}=\psi \left(q\right)\eta ,$$

(13)

$$x_{adv}^{t+1}=x_{bod}^t+{\eta }_{new}{\displaystyle \frac{\widetilde{\nabla CS}\left(x_{bod}^t\right)}{{\parallel \widetilde{\nabla CS}\left(x_{bod}^t\right)\parallel }_2}},$$

(14)

where q denotes the current model queries; $v_{ini}$ is the initial parameter; $v_{min}$ is the minimum value of $v_{ini}$; and d is the decay factor of $v_{ini}$. With q increasing, $\psi \left(q\right)$ gradually becomes small, so if we set $v_{min}>1$, our improved step size adjustment strategy can dynamically adjust the step size as the model queries change during the adversarial example generation process.

Relevant research [32,33] indicates that in the field of hard-label black-box adversarial example generation, as the number of iterations increases, adversarial examples become increasingly similar to target samples and the estimated gradients are also more and more inaccurate. Therefore, the step size should be reduced. However, previous step size adjustment strategies [32,33] did not take into account the stage of the adversarial example generation process, so they are relatively inefficient. To address this limitation, we introduce three parameters—$v_{ini}$, $v_{min}$, and d—into our step size adjustment strategy. $v_{ini}$ represents the maximum step size. d is used to adjust the decay rate of the step size using $v_{min}$ and q, and $v_{min}$ ensures that the step size does not decrease to an excessively small value (since the dimensions of videos are higher than those of images, too small step sizes are inefficient in video scenarios).

Because of the high dimensionalities of the videos, it is inevitable to generate invalid samples during the adversarial example generation process, which will terminate the process of the gradient estimation. After analyzing, we find that the reason for this phenomenon is that a large ${\eta }_{new}$ is used when generating adversarial examples. But if we explore the new ${\eta }_{new}$ in the vast search space, it will consume a large number of model queries. So we design a strategy that explores the new ${\eta }_{new}$ within a specific small range randomly, that is, when we get invalid samples, ${\eta }_{new}$ will be recalculated:

$${\eta }_{new}={\eta }_{new}·b·v,$$

(15)

where v is a random number sampled from the Gaussian distribution, and b is a small number. This strategy confines the exploration space of the new ${\eta }_{new}$ within a small ball of radius ${\eta }_{new}·b$, and b is used to control the radius of the ball. Our strategy aims to help DBA find valid samples quickly; when the algorithm starts its next iteration, ${\eta }_{new}$ should be set to its original value according to Equations (12) and (13). b is highly sensitive to the number of iterations. Fewer iterations imply a larger difference between the adversarial video and the target video, making it easier to escape the invalid sample space quickly. Experiments show that this strategy has relatively stable performance, which can find valid samples within 20 query attempts. After detailing the three steps of DBA, we formally present the complete description of DBA in Algorithm 1.

Algorithm 1 DBA

Input: The source video $x_{src}$, the target video $x_{tgt}$, the target model f, and the adversarial label $y_{adv}$. Output: The adversarial video $x_{adv}^t$.   1: Initialize the maximum iteration number T, the iteration variable t, the binary search threshold h, the step size $\eta$, the constant $\delta$, the constant b, the variable v, and the number of perturbations N   2: $t\leftarrow 1$   3: $x_{adv}^{t-1}\leftarrow x_{src}$   4: $m\leftarrow 0$   5: while  $t\le T$  do   6:     $x_{bod}^t\leftarrow FindBodVideo(x_{adv}^{t-1},x_{tgt},h,f,y_{adv})$   7:     $u\leftarrow Gaussian\left(\right)$   8:     $\widetilde{\nabla CS}\left(x_{bod}^t\right)\leftarrow {\displaystyle \frac{1}{N}}{\sum }_{k=1}^N\varphi (x_{bod}^t+\delta u)u$   9:     ${\eta }_{new}\leftarrow \psi \left(q\right)\eta$ 10:     $x_{adv}^t\leftarrow x_{bod}^t+{\eta }_{new}{\displaystyle \frac{\widetilde{\nabla CS}\left(x_{bod}^t\right)}{{\parallel \widetilde{\nabla CS}\left(x_{bod}^t\right)\parallel }_2}}$ 11:     while $f\left(x_{adv}^t\right)\ne y_{adv}$ do 12:        $v\leftarrow Gaussian\left(\right)$ 13:        ${\eta }_{new}\leftarrow {\eta }_{new}·b·v$ 14:        $x_{adv}^t\leftarrow x_{bod}^t+{\eta }_{new}{\displaystyle \frac{\widetilde{\nabla CS}\left(x_{bod}^t\right)}{{\parallel \widetilde{\nabla CS}\left(x_{bod}^t\right)\parallel }_2}}$ 15:     end while 16:     $t\leftarrow t+1$ 17: end while 18: return  $x_{adv}^t$

In Algorithm 1, line 5 to line 17 represent the main loop of DBA; T denotes the maximum number of iterations. Specifically, the function FindBodVideo on line 6 is used to compute the boundary video according to Equation (1); the function Gaussian on line 7 is used to sample random perturbations; line 8 performs gradient estimation according to Equation (5); line 9 to line 10 represent our improved dynamic step size adjustment strategy; line 11 to line 15 denote the invalid sample skipping strategy.

3. Experiments

3.1. Algorithm Competitors

We design the algorithm comparison as follows:

(1) Hard-label black-box algorithm comparison. In order to evaluate the performance of our algorithm, we compare DBA with STDE [31], which is the state-of-the-art hard-label black-box video adversarial example generation algorithm.

(2) Score-based black-box algorithm comparison. Currently, there exist some state-of-the-art score-based black-box adversarial example generation algorithms on video recognition models: VBAD [29] and EARL [30]. We also compare DBA with these two algorithms. This comparison is particularly challenging as score-based algorithms leverage significantly more information (i.e., probabilities) than hard-label algorithms. Consequently, evaluating DBA with these stronger algorithms underscores the robustness and efficiency of DBA under the most restrictive information settings.

Currently, there are some state-of-the-art image-to-video transfer adversarial example generation algorithms [40,41]. We do not compare DBA with them, because these algorithms focus on the white-box setting, while DBA targets the hard-label black-box setting. Therefore, DBA and image-to-video transfer adversarial example generation algorithms represent two entirely different research fields. However, the transferability of adversarial examples provides the improvement direction for us. In the future, we will explore the transferability of hard-label black-box adversarial examples to improve the performance of DBA.

3.2. Experimental Settings

In our experiments, the human motion recognition dataset HMDB-51 [36] and UCF-101 [42] are used. HMDB-51 is a human motion recognition dataset which includes 7000 videos belonging to 51 categories. Of these, 70% of the videos are divided into training sets, and 30% of the videos are testing sets. HMDB-51 is one of the most commonly used datasets in the field of video recognition. UCF-101 is an action recognition dataset from realistic action videos, which comes from 13,320 trimmed YouTube videos belonging to 101 action categories. Of these, 80% of the videos are divided into training sets, and 20% of the videos are testing sets. UCF-101 is also one of the most widely used benchmark datasets by the state-of-the-art video recognition models.

As for video recognition models, C3D [43] and TSN [44] are used in our experiments. C3D studies spatio-temporal features of the videos by 3D convolution layers, and it is also recognized as a popular benchmark in the field of video recognition. TSN is a novel framework for the video-based action recognition, which is based on the idea of long-range temporal structure modeling. TSN combines the sparse temporal sampling strategy and the video-level supervision strategy to improve its performance.

We use the following metrics to evaluate algorithm performance:

(1) Mean Square Error (MSE), which is the square error distance between the target video and the adversarial video in every iteration. The MSE indicates the magnitude of perturbations. The lower the MSE is, the more similar the adversarial video is to the target video.

(2) Success Rate (SRT), which is the ratio of successful adversarial videos to all test videos. The MSE and model queries of these successful adversarial videos are below specific thresholds, respectively. The higher the SRT is, the more efficient the algorithm is.

(3) Mean Query Numbers (MQNs), which are the average model queries of all successful adversarial videos. The lower the MQN is, the better the algorithm is.

(4) Peak Signal to Noise Ratio (PSNR), which quantifies the ratio of the maximum pixel intensity to the global perturbations. A higher PSNR indicates better performance.

(5) Average Occluded Area (AOA), which is the ratio of the added perturbations to the distance between the source video and the target video. A lower AOA indicates better performance.

(6) Query Attempts (QA). When invalid samples are found, DBA will consume some model queries to escape the invalid sample space. We refer to these model queries as query attempts. A lower QA indicates better performance.

PSNR focuses on analyzing the impact of perturbations on samples from the perspective of perturbation intensity [45]. A higher value indicates fewer perturbations and better visual quality. AOA emphasizes analyzing the influence of perturbations from the perspective of their impact range. A higher value means a larger affected area and poorer visual quality [31].

In order to better highlight the performance of all comparison algorithms, we set the maximum model queries to 30,000, which is a challenge for all algorithms, because in the field of black-box video adversarial example generation, the maximum model queries of most algorithms are much higher than 30,000. For example, EARL and VBAD set this value to 300,000 and 600,000, respectively, which is 10 times and 20 times as much as ours. Relevant studies [32,33] show that when setting large model queries budgets, all comparison algorithms can succeed, which makes it hard to distinguish the performance of different algorithms. Therefore, the maximum model queries should be set as small as possible. This way of setting parameters is widely used in the hard-label black-box adversarial example generation [31,32,33]. Generating adversarial examples with fewer model queries also indicates that the algorithm has the capability to rapidly identify the vulnerabilities of the target model, which holds practical significance. Additionally, in real-world scenarios, the target model may incorporate an attack detection framework. Therefore, reducing maximum model queries can help algorithms avoid detection by the attack detection framework, which is of practical significance. Researchers can also set higher query budgets for a more in-depth and detailed assessment of the target model’s security. Moreover, we set $v_{ini}$ = 20, $v_{min}$ = 5, b = 1, and d = 200. When setting these experimental parameters, we drew upon previous relevant studies [29,30,31], and these parameters remained unchanged through the entire process of the experiment.

3.3. Results and Analysis

3.3.1. Magnitude of Perturbations

In this section, we will evaluate the magnitude of perturbations contained in adversarial videos generated by different algorithms using three metrics: MSE, PSNR, and AOA.

First, we show the average MSE of all successful adversarial videos of two target models during the adversarial example generation process with different model queries in Figure 5. As illustrated in Figure 5, DBA achieves a reduction in the magnitude of perturbations by 50–70% compared to STDE.

Figure 6 shows the PSNR of comparison algorithms at different model queries. It can be observed that the PSNR of DBA is 20% higher than that of STDE on average. Figure 7 shows the AOA of comparison algorithms at different model queries. It can be concluded that the AOA of DBA is 50–70% lower than that of STDE on average.

We do not draw the curves of EARL and VBAD in Figure 5, Figure 6 and Figure 7 because they are score-based algorithms and produce only one successful adversarial video if they succeed. In other words, EARL and VBAD do not generate other successful adversarial videos during the adversarial example generation process. In order to compare with them fairly, we analyze all common successful adversarial videos of the four algorithms, and the results are shown in

Table 2. Metrics on C3D within 30,000 model queries (the best results are in bold).

Algorithm	MQN	MSE	PSNR	AOA
DBA	29,910	20.8	35.0	0.08
STDE	29,954	90.1	28.6	0.33
EARL	-	-	-	-
VBAD	-	-	-	-

and

Table 3. Metrics on TSN within 30,000 model queries (the best results are in bold).

Algorithm	MQN	MSE	PSNR	AOA
DBA	9870	14.6	36.5	0.15
STDE	29,875	39.2	32.2	0.40
EARL	-	-	-	-
VBAD	11,750	14.6	36.5	0.15

(“-” means that the corresponding algorithm cannot generate adversarial videos successfully within 30,000 model queries). It can be seen that DBA gets the best performance with the fewest model queries. From Figure 5, Figure 6 and Figure 7, we can also conclude that STDE can generate low-quality adversarial videos rapidly but cannot produce high-quality adversarial videos.

We can observe that DBA has better performance than STDE, which is caused by the differences of the algorithm architecture. Specifically, STDE is a patch-based algorithm, and DBA is a pixel-based algorithm, so DBA can add perturbations to pixels more accurately, while STDE uses the target video as the initial perturbations and adds it to the source video, and then gradually optimizes and reduces the magnitude of perturbations. However, one flaw of STDE is that it can only optimize perturbations within a specific threshold and cannot optimize them continuously.

In practical applications, adversarial videos with fewer perturbations tend to be of higher quality and less detectable by human vision. The experimental results of this section clearly show that the adversarial videos generated by DBA contain significantly fewer perturbations than those produced by other comparison algorithms. In other words, DBA can generate higher-quality adversarial videos, which is particularly important for evaluating the security of target models.

3.3.2. Success Rate

In this section, we will evaluate the success rate of all algorithms under different MSE and model queries.

When the MSE threshold is 25, comparison results for the success rate of two target models with different model queries are reported in Figure 8. We do not draw the curve of EARL in two target models experiments and the curve of VBAD in C3D experiments because the two algorithms cannot generate adversarial videos successfully on the corresponding model within 30,000 model queries. As shown in Figure 8, STDE cannot optimize adversarial videos during the entire adversarial example generation process. By contrast, DBA can continuously optimize adversarial videos with the model queries increasing, and its final success rate is 1.6–4.3 times higher than that of STDE. Although VBAD can also continuously optimize adversarial videos during the adversarial example generation process, its success rate is significantly lower than that of DBA.

In order to further evaluate the performance of all algorithms under different model queries and MSE thresholds, we get Figure 9 and Figure 10. As illustrated in Figure 9 and Figure 10, DBA gets the best performance in all test cases. Additionally, we can also conclude that there are no experimental results for EARL or VBAD in Figure 9, but these results appear in Figure 10. This phenomenon is caused by the following two reasons:

(1) EARL and VBAD have bad performance. The two algorithms are based on logits (probabilities). Compared with hard labels, logits have a wider value range, and if these algorithms want to get more accurate gradients, they must visit the target model frequently [29,30]. So the algorithms based on logits usually set higher maximum model queries. For example, VBAD and EARL set their maximum model queries to 300,000 and 600,000, respectively. In order to highlight the performance of DBA, in our experiments, we set the maximum model queries to 30,000, which means that VBAD and EARL cannot successfully generate adversarial videos sometimes. Experimental results similar to ours can also be obtained in STDE [31].

(2) VBAD behaves differently when testing different models. From the above experimental results, it can be seen that, when testing TSN, VBAD has better performance than that in C3D. This is caused by the difference between TSN and C3D. Specifically, TSN utilizes sparse frames for predictions; the perturbations of each frame have a greater impact [44]. Different from TSN, C3D mainly uses more frames for predictions, and the impact of each frame’s perturbations is relatively small [43]. VBAD uses image perturbations to initialize frame perturbations, and relevant research has proven that initializing perturbations in this way can improve efficiency [46,47,48]. When testing TSN, VBAD only needs to add image perturbations to sparse frames, which reduces the interference of adding image perturbations to a great many frames, so VBAD has better performance in TSN experiments than in C3D experiments.

In practical applications, a higher success rate indicates that the algorithm can generate adversarial videos rapidly. The experimental results of this section clearly denote that the success rate of DBA is significantly higher than that of other comparison algorithms. This means that DBA can generate adversarial videos with fewer query budgets, which is critically important for quickly discovering the target model’s security vulnerabilities.

3.3.3. Ablation Studies

DBA aligns N and $\delta$ with the standard hard-label black-box image adversarial example generation algorithm [32,33]. Next, we will perform ablation studies on the dynamic step size adjustment strategy and the invalid-sample skipping strategy. We set the maximum model queries to 400,000, and the MSE threshold is 25.

Effect of dynamic step size adjustment strategy. As shown in Figure 11a, when the dynamic adjustment strategy is removed, the success rate of the DBA drops from 96% to 6%. The primary reason is that traditional step size strategies are only applicable to low-dimensional image data and perform poorly when handling high-dimensional video data. Our designed strategy enables the generation algorithm to dynamically adjust the step size based on the stage of the adversarial example generation process, thereby improving the efficiency of generating adversarial videos. Figure 11b illustrates the success rate of DBA when $v_{ini}$ is set to different values. It can be observed that as $v_{ini}$ gradually increases, the success rate exhibits an initial increase followed by a decrease. $v_{ini}$ primarily determines the magnitude of the step size. Relevant research [34] indicates that excessively large or small step sizes can degrade the efficiency of adversarial example generation. Therefore, we set $v_{ini}$ = 20.

Effect of invalid-sample skipping strategy. As shown in Figure 12a, after removing the invalid-sample skipping strategy, the DBA’s query attempts increase from 12 to 125. This implies that our designed strategy can reduce query attempts by over 90%, which helps DBA find valid samples quickly. Figure 12b illustrates how query attempts vary when b is set to different values. b primarily determines the radius of the exploration sphere. It can be observed that as the radius increases, DBA tends to require more query attempts to escape the invalid sample space. Therefore, in real-world applications, b should be set to a small value.

The experimental results of this section clearly denote the effect of the invalid-sample skipping strategy and the dynamic step size adjustment strategy. These two strategies can help improve the efficiency of the adversarial video generation in practical applications.

3.4. Failure Cases and Future Directions

DBA is a pixel-based algorithm that does not select keyframes. It regards all video frames as a whole and dynamically optimizes the perturbations based on the estimated gradient. As shown in Figure 5, while DBA significantly outperforms existing algorithms in the long run, it exhibits a relatively slower optimization speed during the initial phase compared to keyframe-based algorithms like STDE. This phenomenon is primarily attributed to the fundamental difference in search space dimensionality: DBA operates on a pixel-level dense representation, whereas STDE constrains the perturbation to sparse keyframes. In the future, we will study how to apply keyframe selection methods in adversarial example generation algorithms and attempt to calculate keyframes of videos for DBA in the initial process. Additionally, we will also explore the transferability of hard-label black-box image-to-video adversarial examples to improve the efficiency of DBA.

4. Conclusions

This paper presents DBA, a novel hard-label black-box algorithm designed for generating adversarial examples on video recognition models. First, DBA uses a sampling-based algorithm to estimate the gradient on the boundary video. Second, by updating the boundary video with an adaptive dynamic adjustment strategy, DBA effectively addresses the challenges of high dimensionality and query inefficiency inherent in video-based adversarial example generation. Furthermore, a specialized invalid sample evasion strategy is introduced to ensure optimization stability under limited query budgets. Extensive experiments on benchmark datasets (HMDB-51 and UCF-101) demonstrate that DBA significantly outperforms state-of-the-art algorithms, achieving up to a 50% reduction in MSE while maintaining superior visual imperceptibility. DBA’s pixel-level optimization exhibits a slower initial optimization speed compared to heuristic keyframe-based algorithms. Future research will focus on integrating spatio-temporal keyframe selection algorithms and transferable perturbations into the DBA framework to further enhance query efficiency in the early optimization stages.