Efficient Adversarial Attack Based on Moment Estimation and Lookahead Gradient

Abstract

Adversarial example generation is a technique that involves perturbing inputs with imperceptible noise to induce misclassifications in neural networks, serving as a means to assess the robustness of such models. Among the adversarial attack algorithms, momentum iterative fast gradient sign Method (MI-FGSM) and its variants constitute a class of highly effective offensive strategies, achieving near-perfect attack success rates in white-box settings. However, these methods’ use of sign activation functions severely compromises gradient information, which leads to low success rates in black-box attacks and results in large adversarial perturbations. In this paper, we introduce a novel adversarial attack algorithm, NA-FGTM. Our method employs the Tanh activation function instead of the sign which can accurately preserve gradient information. In addition, it utilizes the Adam optimization algorithm as well as the Nesterov acceleration, which is able to stabilize gradient update directions and expedite gradient convergence. Above all, the transferability of adversarial examples can be enhanced. Through integration with data augmentation techniques such as DIM, TIM, and SIM, NA-FGTM can further improve the efficacy of black-box attacks. Extensive experiments on the ImageNet dataset demonstrate that our method outperforms the state-of-the-art approaches in terms of black-box attack success rate and generates adversarial examples with smaller perturbations.

1. Introduction

Currently, neural networks are ubiquitously employed across intelligent software domains such as object detection [1], security monitoring [2], autonomous vehicles [3], speech recognition [4] and image classification [5]; however, despite significant advancements in these areas, they remain plagued with security and robustness concerns. Adversarial examples refer to intentionally crafted input modifications [6] imperceptible to the human, designed to cause misclassification in neural networks. The existence of such adversarial examples presents profound challenges and dilemmas for contemporary neural network systems. For instance, when an autonomous vehicle encounters an adversarial target during operation, it may be misled into altering its driving state, thereby engendering traffic safety issues. In the context of security, adversarial examples can potentially deceive security system assessments, affording opportunities for malicious actors. In military applications, where unmanned aerial vehicles are utilized for reconnaissance in uncharted territories, adversarial examples could lead to erroneous target identification, compromising battlefield awareness. Consequently, adversarial examples pose a severe threat to the application security within computer vision domains reliant on deep neural networks. Hence, it is critical to delve into the generation mechanisms and underlying principles of adversarial examples, and to develop either defensive strategies [7,8,9,10] against them or more secure and robust deep neural network architectures.

The research on adversarial examples has matured over time, giving rise to a multitude of attack methodologies. In 2018, Dong et al. [11] introduced the momentum iterative fast gradient sign method (MI-FGSM), which combined momentum-based optimization with the iterative fast gradient sign approach. This integration expedited gradient convergence and rendered the update direction of the objective function as more consistent, mitigating the influence of local noise. MI-FGSM significantly enhanced the efficacy of white-box attacks and demonstrated a marginal improvement in black-box scenarios. Subsequently, several variants of MI-FGSM were successively proposed [12,13,14], all of which achieved nearly 100% attack success rates under white-box conditions; however, their performance in black-box attacks was less satisfactory.

In gradient-based adversarial example generation methods, the gradient of the objective function is computed and then added to the original example to generate adversarial examples. If the gradient of the objective function is too large, the generated adversarial perturbation is too large and can be easily recognized by the human. In order to make the adversarial perturbation imperceptible, the gradient normalization and activation function are used to process the gradient of the objective function and limit the gradient to a range of values from −1 to 1, thus limiting the size of the adversarial perturbation. MI-FGSM and its variant methods all use the sign function as the activation function. The sign function turns the value of the gradient after normalization to −1, 1, or zero, which not only loses the gradient information that reduces the efficiency of the black-box attack, but also increases the generated adversarial perturbation, making the adversarial examples easier to be detected.

In this paper, we focus on investigating gradient-based black-box attacks and propose a novel adversarial attack method termed NA-FGTM. Unlike MI-FGSM and its variants that employ the sign function for gradient manipulation, NA-FGTM utilizes the hyperbolic tangent (Tanh) function [15] to preserve gradient information more effectively, thereby reducing the magnitude of induced adversarial perturbations. The NA-FGTM approach further incorporates the Adam [16] optimization algorithm to expedite gradient convergence and employs Nesterov acceleration [17] to enhance the transferability [18] of adversarial examples, thereby improving the success rate of black-box attacks. Moreover, NA-FGTM can be integrated with data augmentation techniques such as DIM [19], TIM [20], and SIM [12] to further boost the success rates of black-box attacks.

This study was conducted in a system environment running Python 3.9 and TensorFlow 2.6.0, employing a dataset consisting of 1000 clean images drawn from the ILSVRC 2012 validation set [21]. Several prevalent models were utilized, including Inception-v3 [22], Inception-v4, Inception-Resnet-v2 [23], and Resnet-v2-101 [24], along with three adversarial-trained models, namely Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens [25]. The experiments adopted a comparative approach to validate the performance of four distinct methods: MI-FGSM [11], NI-FGSM [12], VMI-FGSM [13], and NA-FGTM, with all experimental parameters and configurations kept consistent across the compared methods. The experimental results demonstrated that our proposed NA-FGTM outperforms other algorithms in terms of black-box attack success rates, indicating higher efficiency of our algorithm. Furthermore, under equivalent attack success rates, our method generates adversarial examples with smaller perturbation values, suggesting a superior balance between effectiveness and imperceptibility in crafting adversarial examples.

The remainder of this paper is structured as follows: Section 2 provides related work on adversarial example generation. Our novel attack algorithm, NA-FGTM, is detailed in Section 3. The experimental results and corresponding analyses are presented in Section 4. Finally, the conclusions of this paper are provided in the last section.

2. Related Work

2.1. Problem Definition

The process of an adversarial attack can be generally defined as follows: Consider a pair of pre-trained neural network models {M_w, M_b}, where M_w denotes a white-box model with transparent internal information, and M_b represents a black-box model with unknown internal information. Given a clean input sample x, when fed into the neural network M, it outputs the true label y^true, such that y^true = M(x). Let J(x, y) denote the loss function within M, and η represent the adversarial perturbation. The objective of an adversarial attack is to craft an adversarial example x^adv = x + η, which maximizes the loss function J(x^adv, y^true), thereby causing the predicted label y^adv ≠ y^true. To ensure the imperceptibility of the adversarial perturbation, η is constrained within a certain range, i.e., ||η||_p ≤ ε, where ε signifies the maximum allowable adversarial perturbation under the L_p norm, which can take values p = 0, 1, 2, or ∞. The adversarial example x^adv can potentially fool both the white-box model M_w and the black-box model M_b; however, this study primarily focuses on black-box attacks.

2.2. Fast Gradient Sign Method

The fast gradient sign method (FGSM), proposed by Goodfellow [26], is an algorithm for generating adversarial examples based on gradients, which belongs to the category of untargeted attacks in adversarial machine learning—attacks that do not mandate the perturbed sample to be misclassified into a specific class but merely require disagreement with the original prediction. In the context of training neural networks where minimization of the loss function entails movement against the gradient direction via gradient descent, FGSM can be interpreted as a gradient ascent strategy that maximizes the loss function. The FGSM algorithm modifies the image once after calculating the gradient and is a very fast and efficient single-step attack algorithm. Assuming the original data are represented by x, the adversarial example is x^adv, with the classification output denoted as y, the adversarial example x^adv is obtained by adding a small perturbation η to the original image: x^adv = x + η. Throughout this process, FGSM operates under an L_∞ norm constraint, i.e., ||x^adv−x||_p ≤ ε, ensuring that the difference between the original and adversarial examples remains within a predefined bound.

2.3. Basic Iterative Method

In 2016, Kurakin [27] proposed the basic iterative method (BIM, also known as I-FGSM), which builds upon the fast gradient sign method to efficiently generate adversarial examples. BIM essentially constitutes an iterative variant of FGSM, also employing the L_∞ norm as a constraint. It divides the one-step process of FGSM into ten smaller steps. The author suggested that this gradual approach ensured a more optimal gradient ascent trajectory, at worst matching the performance of FGSM. For untargeted attacks, FGSM is iteratively applied within each step, augmented by a clipping function to normalize the images. During the iterative update procedure, some pixel values might exceed their valid range (e.g., beyond [0, 1]), necessitating their replacement with either 0 or 1 to produce a valid image. This preserves the integrity of the new sample by keeping its pixels within a neighborhood of the original image’s pixel values, avoiding significant distortion.

Empirically, the author recommended setting the learning rate to 1. Intuitively, BIM shares FGSM’s ease of comprehension while being both concise and efficient, demonstrating superior attack effectiveness compared to FGSM. However, subsequent research has shown that BIM exhibits relatively lower transferability, leading to less effective black-box attacks.

2.4. Momentum Iterative Fast Gradient Sign Method

Due to the fact that the basic iterative method based on FGSM computes and accumulates gradients in each step during the iterative process, adding them to the adversarial example such that it can effectively deceive only the white-box model used to generate it, but not the unknown black-box models, this method is significantly constrained in practical applications. In conventional optimization algorithms, momentum terms are known to expedite convergence by mitigating the likelihood of becoming stuck in local optima and imparting greater stability to the update direction. Inspired by this concept, Dong et al. [10] incorporated a momentum term into the iterative method for generating adversarial examples, aiming to circumvent noisy data and poorly effected local extremes encountered during iterations, thereby enhancing the overall attack efficacy.

Dong et al. proposed the MI-FGSM algorithm, which integrates momentum into the I-FGSM algorithm to generate noise by the following equations:

$$g_{t+1}=\mu g_t+\frac{{\nabla }_{x}J(x_t^{*},y)}{||{\nabla }_{x}J(x_t^{*},y)|{|}_1}$$

(1)

$$x_{t+1}^{*}=x_t^{*}+\alpha \ast sign(g_{t+1})$$

(2)

where g_t+1 cumulatively integrates the velocity vector along the gradient direction, thereby transcending local optima during optimization to enhance the probability of reaching global optima. However, this momentum-based approach can lead to an accelerated growth in gradients, resulting in excessive perturbations in the generated adversarial examples.

2.5. Nesterov Iterative Fast Gradient Sign Method

Lin et al. [11] employed the Nesterov accelerated gradient (NAG) [15] and proposed the NI-FGSM algorithm to enhance the transferability of adversarial examples. In NI-FGSM, prior to each gradient iteration, a leap is made along the accumulated gradient direction. It replaces $x_t^{*}$ in Equation (1) with $x_t^{adv}+\alpha ·\mu ·g_t$, leveraging the lookahead feature of NAG to construct a robust adversarial attack. This lookahead property of NAG aids in more efficiently and rapidly escaping the shallow local maxima, thereby improving transferability.

2.6. Data Enhancement Methods

The scale invariance method (SIM) [11] generates a large number of training examples to enhance the transferability of the generated adversarial examples through the loss invariant property of the image scaled and fed into the neural network:

$$\underset{x^{adv}}{\arg \max }\frac{1}{m}{\displaystyle \sum _{i=0}^{m}J(S_i(x^{adv}),y^{true})},s.t.||x_t^{adv}-x|{|}_{\infty }\le \epsilon$$

(3)

where S_i(x) = x/2ⁱ denotes a scaled copy of the input x when the scaling factor is 1/2ⁱ, and m denotes the number of scaled copies. The scale invariant method effectively enhances the transferability of adversarial examples; however, it demands substantial computational time and resources.

The diversity input method (DIM) [16] randomly adjusts and pads input images with a fixed probability p at each iteration before feeding the transformed images into the classifier for gradient calculation. DIM can be readily integrated with other gradient-based approaches to further bolster the transferability of adversarial examples. The transformation function is represented as follows:

$$T(x_t^{adv},p)=\left\{\begin{array}{c}T(x_t^{adv}),p\\ x_t^{adv},(1-p)\end{array}\right..$$

(4)

The translation invariance method (TIM) [17] optimizes the adversarial perturbations across a set of translated images rather than on a single image, rendering it more effective against defended black-box models. To alleviate the computational demands on gradients, Dong et al. introduced small image translations and approximate gradient computations by convolving the gradients of the untranslated image with a kernel matrix. The generation of adversarial examples under this framework can be expressed as follows:

$$x_{t+1}^{adv}={\displaystyle {\sum }_{i,j}{T}_{ij}(x_t^{adv})},s.t.||x_t^{adv}-x^{real}|{|}_{\infty }\le \epsilon$$

(5)

where $T_{ij}(x_t^{adv})$ denotes the translation function that shifts the input x^adv by i and j pixels, respectively, along the two-dimensional direction.

3. Our Method

In this section, we delve into the detailed workings of the NA-FGTM method. To address the gradient information loss in sign activation function, NA-FGTM employs the Tanh function to augment gradient preservation. Subsequently, it utilizes the Adam optimization algorithm as well as the Nesterov acceleration, which is able to stabilize gradient update directions and expedite gradient convergence. The detailed principles underlying these methodologies are presented subsequently.

3.1. Activation Function

Activation functions play a pivotal role in gradient-based adversarial example generation methods. In the process of computing adversarial perturbations, once the gradient of the objective function with respect to the current iteration’s input is calculated, this gradient is fed into the activation function. The output of this function then determines the direction of the function update for the next iteration and the magnitude of the perturbation that is computed.

Most gradient-based methods for generating adversarial examples are built upon or improved from the fast gradient sign method (FGSM), hence commonly employing the sign activation function. However, the sign function in these gradient-oriented approaches has two significant drawbacks. Firstly, it normalizes all gradient values to either −1, +1, or zero, leading to the loss of gradient information. Secondly, the sign function maps gradient values within the interval (−1, 1) to binary extremes of −1 or +1, which inherently amplifies the magnitude of the perturbation introduced.

As Figure 1 illustrates, for the sign function, when x > 0, y = 1; when x < 0, y = −1; and when x = 0, y = 0. In contrast, for the hyperbolic tangent (Tanh) function, when −1 < x < 1, y = Tanh(x). Upon computation of the gradient values of the objective function, these values are fed into an activation function. When the gradient values fall within the range of −1 to 1, the sign function converts all gradient values within the range of −1 to 1 into either −1 or 1, which results in larger perturbations due to amplified gradient values. In contrast, the Tanh function preserves the original gradient values, leading to smaller perturbations as it maintains the original gradient magnitude. Consequently, the Tanh function can be employed as an alternative to the sign function to reduce the size of the generated perturbations.

3.2. Adaptive Learning Rate Adjustment Strategy Based on Moment Estimation

The Adam optimization algorithm integrates the strengths of both the AdaGrad and RMSprop methods, inheriting advantageous properties from each. Moreover, Adam leverages first-order moment estimates and second-order moment estimates to handle gradient information, where the first moment reflects the mean of gradients, guiding the direction of gradient updates, and the second moment represents the variance of gradients, used to modulate the learning rate during the update process. The procedure of the Adam algorithm can be outlined as follows:

$$m_t={\mu }_1\cdot m_{t-1}+(1-{\mu }_1)\cdot g_t$$

(6)

$$v_t={\mu }_2\cdot v_{t-1}+(1-{\mu }_2)\cdot g_t^2$$

(7)

$$m_t^{*}=m_t/(1-{\mu }_1^t)$$

(8)

$$v_t^{*}=v_t/(1-{\mu }_2^t)$$

(9)

$${\theta }_t={\theta }_{t-1}-\alpha \cdot m_t^{*}/(\sqrt{v_t^{*}}+\sigma )$$

(10)

where m_t denotes the estimate of the first moment (mean) of the gradients and v_t represents the estimate of the second moment (uncentered variance) of the gradients. Here, μ₁ and μ₂ are exponential decay rates, typically set to 0.9 and 0.999, respectively. g_t signifies the gradient information at a given time step. A constant σ is introduced to prevent division by zero in the denominator.

Here, Adam employs exponential weighted averages to accumulate the first and second moments, mitigating the impact of drastic fluctuations in the data on gradient updates. Additionally, to address the cold start problem inherent in exponential weighted averaging—where m₀ and v₀ are initialized to zero causing m₁ to be biased towards zero—Adam corrects the bias by dividing m_t by 1 − μ₂. Lastly, $m_t^{*}/(\sqrt{v_t^{*}}+\sigma )$ is used to normalize the gradients. In our method, Equations (8) and (9) are combined into Equation (10), with further refinement made to the learning rate α:

$${\alpha }_t=\frac{\epsilon }{\sqrt{v_{t+1}}+\sigma }$$

(11)

$$x_{t+1}^{adv}=Cli{p}_{\varphi }^x\{x_t^{adv}+{\alpha }_t\cdot \tanh (\frac{\sqrt{(1-{\mu }_2^t)}}{1-{\mu }_1^t}\cdot \frac{m_{t+1}}{\sqrt{v_{t+1}}+\sigma })\}$$

(12)

where ε denotes the initial learning rate and α_t represents the adaptive learning rate, which decreases when the second moment v_t increases in Equation (11), and vice versa. In Equation (12), we employ the Tanh function as the activation function, following the bias correction and normalization of gradient information. After processing the gradient through the Tanh activation function, it preserves gradient information while generating adversarial perturbations of smaller magnitude.

3.3. Lookahead Gradient Prediction

The momentum algorithm incorporates an accumulator for past gradients, thereby dominating the update steps with a substantial influence from historical directions while attenuating the sway of immediate gradient variations. This approach can be likened to a physical system where a ball descending along a gradient path maintains its cumulative momentum, rendering it less susceptible to transient perturbations and enabling a rapid, stable descent towards the global optimum. The momentum method proceeds as follows:

$$m_t=\mu \cdot m_{t-1}+g({\theta }_{t-1})$$

(13)

$${\theta }_t={\theta }_{t-1}-\alpha \cdot m_t$$

(14)

In Equation (13), m_t and m_t−1 denote the accumulated historical gradients at the current and previous iterations, respectively, whereas g represents the gradient of the objective function. μ is a hyperparameter signifying the weight assigned to the previous accumulated history gradient in the current accumulation, taking a value between 0 and 1, and α refers to the learning rate. The momentum technique constrains the current gradient update using the gradient from the previous iteration, thereby promoting a more stable convergence of the objective function toward the global optimum.

Building upon the momentum method, Nesterov proposes an enhancement where instead of computing the next update direction based solely on the current gradient, the method preemptively uses the gradient at the next iteration to calculate the update direction within the current iteration itself. Nesterov thus improves upon Equation (13) as follows:

$$m_t=\mu \cdot m_{t-1}+g({\theta }_{\mathrm{t}-1}-\alpha \cdot \mu \cdot m_{t-1})$$

(15)

From Equation (15), it can be observed that the gradient computation does not rely solely on the current position θ_t−1; rather, it takes into account the gradient a step ahead $({\theta }_{t-1}-\alpha \cdot \mu \cdot m_{t-1})$ in the future. By modifying the current gradient direction according to the anticipated gradient at the next step, this approach accelerates the convergence of gradients and hastens the objective function’s progression towards the global optimum.

Figure 2 demonstrates that although the momentum method steadily converges towards the optimal point, Nesterov achieves faster convergence, settling into a stable state shortly after the initial few updates and proceeding smoothly towards the global minimum. Analogously, our NA-FGTM method draws inspiration from Nesterov’s acceleration strategy, as follows:

$$x_t^{nes}=x_t^{adv}+\alpha \cdot \frac{m_t}{\sqrt{v_t}+\delta }$$

(16)

$$g_t={\nabla }_{x_t^{nes}}L(f(x_t^{nes}),y^{true})$$

(17)

As depicted in Equation (16), the objective function is anticipated and virtually advanced by one step before its gradient is computed. Following this, the gradient of the virtually progressed objective function is determined according to Equation (17). Thereafter, the gradient update is computed utilizing the Adam method as per Equations (6) through (12), culminating in the generation of the adversarial example x^adv and the completion of the current iteration cycle.

3.4. NA-FGTM Algorithm

In this section, we outline the general flow of the NA-FGTM method. In each iteration, the process begins with leveraging Nesterov acceleration to estimate the gradient of the function at a projected next-step point x^nes. Subsequently, the Adam optimization algorithm is employed to calculate the first-moment estimate m and second-moment estimate v. Building on these estimates, an adaptive learning rate α is derived using the second-moment estimate v. Following bias correction, gradient normalization and the Tanh activation function are used to compute the gradient update, which is then added to the original input sample to generate the adversarial example for the current iteration, thereby concluding the iteration. Once all iterative steps have been executed, the final adversarial example is obtained. We encapsulate the step-by-step process of the NA-FGTM method in Algorithm 1.

Algorithm 1: NA-FGTM

Input: classifier f with loss function L, original image x, true label ytrue, number of iterations T, first moment estimate m, second moment estimate v, exponential decay factor μ1, μ2, initial learning rate ε, constant value σ, maximum perturbation φ; Output: Adversarial example xadv; 1: x0adv = 0, m0 = 0, v0 = 0 2:    for t = 0 to T − 1 do: 3:    $x_t^{nes}=x_t^{adv}+{\alpha }_t\cdot \frac{m_t}{\sqrt{v_t}+\delta }$ 4:    $g_t={\nabla }_{x_t^{nes}}L(f(x_t^{nes}),y^{true})$ 5:    $m_{t+1}={\mu }_1\cdot m_t+(1-{\mu }_1)\cdot g_t$ 6:    $v_{t+1}={\mu }_2\cdot v_t+(1-{\mu }_2)\cdot g_t^2$ 7:    ${\alpha }_t=\frac{\epsilon }{\sqrt{v_{t+1}}+\sigma }$ 8:    $x_{t+1}^{adv}=Cli{p}_{\varphi }^x\{x_t^{adv}+{\alpha }_t\cdot \tanh (\frac{\sqrt{(1-{\mu }_2^t)}}{1-{\mu }_1^t}\cdot \frac{m_{t+1}}{\sqrt{v_{t+1}}+\sigma })\}$ 9:    end for 10: return xadv

4. Experiments

4.1. Experimental Setup

Datasets. In this study, a dataset comprising 1000 clean images selected from the ILSVRC 2012 validation set was employed. Each of these images belonged to a different class among a total of 1000 classes, and nearly all of these images were correctly classified by the experimental models.

Models. In this study, several common models were employed, including Inception-v3, Inception-v4, Inception-Resnet-v2, and Resnet-v2-101, along with three adversarial-trained models, namely Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens.

Hyperparameters. In order to ensure the effectiveness of the proposed method, we set the experimental parameters to be consistent with the MI-FGSM method, the maximum perturbation magnitude φ was set to 16, the number of iterations T was set to 10, and the initial learning rate ε was configured to 1.6. Within our methodology, the first-moment estimate m₀ and the second-moment estimate v₀ were initialized to zero, while the exponential decay factors μ₁ and μ₂ were set to 0.9 and 0.999, respectively. A constant value σ was established at 10⁻⁶.

4.2. Experimental Method

To validate the superiority of the proposed approach, we devised four experiments to compare the attack success rates and average adversarial perturbation magnitudes among MI-FGSM, NI-FGSM, VMI-FGSM, and NA-FGTM methods.

(1)

Adversarial example generation on individual models: Using MI-FGSM, NI-FGSM, VMI-FGSM, and NA-FGTM techniques, we generated adversarial examples separately on Inception-v3, Inception-v4, Inception-Resnet-v2, and Resnet-v2-101 models. These generated adversarial examples were then used to attack these four base models as well as the three adversarial-trained models—Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens. Ultimately, we contrasted the attack success rates and the sizes of the induced adversarial perturbations across these different methods.

(2)

Adversarial example generation with data augmentation for individual models: We integrated three data augmentation techniques (DIM, TIM, SIM) into the four attack methods—MI-FGSM, NI-FGSM, VMI-FGSM, and NA-FGTM—and employed these augmented variants to generate adversarial examples on Inception-v3, Inception-v4, Inception-Resnet-v2, and Resnet-v2-101 models. The generated adversarial examples were then utilized to attack both the four base models and the three robust models, namely Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens. Finally, we compared the attack success rates across these distinct integrated methods.

(3)

Adversarial example generation for a model ensemble: We formed an ensemble using Inception-v3, Inception-v4, Inception-Resnet-v2, and Resnet-v2-101 models, and individually apply MI-FGSM, NI-FGSM, VMI-FGSM, and NA-FGTM to generate adversarial examples on this ensemble model. Subsequently, these adversarial examples were used to attack the same set of robust models: Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens. Lastly, we compared the attack success rates and the sizes of the produced adversarial perturbations among these various methods.

(4)

Adversarial example generation with data augmentation for a model ensemble: We utilized the four augmented attack methods from Experiment (2) to generate adversarial examples on the ensemble model established in Experiment (3). The resulting adversarial examples were then employed to attack the same three robust models: Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens. Finally, we compared the attack success rates among these different augmented methods when applied to the ensemble setting.

4.3. Adversarial Example Generation on Individual Models

The experimental results of adversarial examples generated using individual models are presented in

Table 1. Attack success rates (%) of adversarial examples generated by individual models.

	Attack	Inc-v3	Inc-v4	IncRes-v2	Res-101	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens
Inc-v3	MI-FGSM	100.0 *	43.0	41.7	34.8	12.7	12.7	6.5
	NI-FGSM	100.0 *	50.1	49.1	39.9	13.1	13.4	6.2
	VMI-FGSM	100.0 *	51.6	48.3	42.9	25.4	26.1	17.6
	NA-FGTM	100.0 *	55.4	49.3	45.0	29.1	28.5	18.3
Inc-v4	MI-FGSM	53.9	99.9 *	43.9	39.9	16.2	14.2	7.8
	NI-FGSM	63.4	100.0 *	50.9	44.1	15.1	14.2	7.1
	VMI-FGSM	61.3	100.0 *	55.2	46.5	30.4	28.6	21.0
	NA-FGTM	64.8	100.0 *	54.3	47.6	32.4	30.0	23.7
IncRes-v2	MI-FGSM	57.1	48.6	98.5 *	42.5	21.5	15.8	11.8
	NI-FGSM	61.1	52.3	99.0 *	44.1	18.7	14.8	10.4
	VMI-FGSM	68.9	63.2	100.0 *	52.5	35.9	32.4	28.1
	NA-FGTM	73.1	66.5	99.8 *	58.3	41.6	36.1	30.8
Res-101	MI-FGSM	55.8	50.1	45.6	99.3 *	23.3	20.3	12.2
	NI-FGSM	63.8	57.7	56.4	99.4 *	23.7	21.6	11.7
	VMI-FGSM	60.2	55.4	53.6	99.8 *	31.7	28.0	26.3
	NA-FGTM	58.1	51.2	51.6	99.2 *	36.6	32.8	27.1

* Results of white-box setting.

, where entries marked with an asterisk (*) denote white-box attacks while the rest represent black-box attacks. From the table, it is evident that for models such as Inc-v3, Inc-v4, IncRes-v2, and Res-101, all four attack methods—MI-FGSM, NI-FGSM, VMI-FGSM, and NA-FGTM—attain either 100% or near-100% success rates in white-box attacks, indicating high efficiency in this setting. Under black-box scenarios, NA-FGTM exhibits a slightly higher attack success rate compared to the other methods. Specifically, when adversaries crafted on Inc-v3 were used to attack Inc-v4, IncRes-v2, and Res-101, NA-FGTM achieved black-box success rates of 55.4%, 49.3%, and 45.0%, respectively, surpassing the other three techniques. Conversely, for adversarial examples generated from Res-101, NI-FGSM demonstrated the highest black-box attack success rates at 63.8%, 57.7%, and 56.4%, suggesting that NI-FGSM is more effective when generating adversarial examples against Res-101. Moreover, across the adversarial-defended models Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens, NA-FGTM consistently displayed the highest black-box attack success rates. For instance, when adversarial examples were produced using IncRes-v2, NA-FGTM achieved a black-box success rate of 41.6% on Inc-v3_ens3, over two times higher than NI-FGSM’s 18.7%. This highlights that our proposed NA-FGTM method outperforms the other three methods significantly in attacking defensively enhanced models.

Additionally, we computed the L₂ norm values of adversarial perturbations generated by the four methods, as shown in

Table 2. Mean perturbation values (L₂) of adversarial examples produced by individual models.

Attack	Inc-v3	Inc-v4	IncRes-v2	Res-101
MI-FGSM	15.97	16.1	15.98	15.90
NI-FGSM	17.49	17.59	17.41	17.35
VMI-FGSM	21.58	22.12	21.65	21.50
NA-FGTM	11.81	11.96	11.74	11.76

. The table reveals that the average magnitude of adversarial perturbations produced by MI-FGSM is around 16, while that of NI-FGSM hovers around 17. The VMI-FGSM method yields the largest average perturbation size, consistently staying around 22, which indicates that VMI-FGSM introduces more adversarial noise to enhance its attack success rate. By contrast, our NA-FGTM approach generates the smallest average perturbation sizes, maintaining them approximately at 11. This suggests that NA-FGTM achieves a higher attack success rate while also minimizing the size of the generated adversarial perturbations, making the adversarial examples less perceptible.

4.4. Adversarial Example Generation with Data Augmentation for Individual Models

In the second experiment, we incorporated three data augmentation techniques, DIM, TIM, and SIM, into the adversarial example generation methods. These data augmentation approaches effectively enhanced the black-box attack efficacy of adversarial examples, as demonstrated in

Table 3. Attack success rates (%) of adversarial examples generated using data augmentation on individual models.

	Attack	Inc-v3	Inc-v4	IncRes-v2	Res-101	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens
Inc-v3	MI-FGSM	99.3 *	64.2	59.6	52.0	18.5	18.4	9.4
	NI-FGSM	99.6 *	60.6	58.8	47.6	16.1	14.4	7.8
	VMI-FGSM	99.8 *	61.9	59.3	53.1	31.6	34.7	21.9
	NA-FGTM	99.4 *	65.8	60.2	56.8	39.8	36.8	26.4
Inc-v4	MI-FGSM	72.0	98.5 *	63.3	54.8	21.3	22.0	11.7
	NI-FGSM	70.7	99.2 *	58.6	49.7	17.2	16.5	8.7
	VMI-FGSM	71.2	97.6 *	61.9	54.1	32.9	34.6	25.7
	NA-FGTM	73.5	98.8 *	63.8	56.8	41.4	39.9	30.3
IncRes-v2	MI-FGSM	69.5	65.7	94.2 *	58.6	30.6	23.5	18.3
	NI-FGSM	68.0	63.2	97.2 *	51.0	22.1	18.1	11.4
	VMI-FGSM	70.3	65.2	96.5 *	59.6	35.3	31.6	34.9
	NA-FGTM	73.8	69.1	94.0 *	62.2	47.9	42.3	40.4
Res-101	MI-FGSM	76.4	68.2	69.9	98.4 *	37.3	33.6	20.8
	NI-FGSM	75.2	68.2	68.4	99.0 *	27.9	26.2	15.6
	VMI-FGSM	78.1	71.4	74.6	99.1 *	48.7	45.2	32.1
	NA-FGTM	81.0	74.1	75.1	98.9 *	63.5	57.6	48.0

* Results of white-box setting.

. Comparing these results with those from the previous experiment, it is evident that data augmentation has indeed led to a significant increase in black-box attack efficiency for adversarial examples. For instance, when adversarial examples were generated using Res-101 and subsequently inputted to Inc-v3 in Table 1, MI-FGSM, NI-FGSM, VMI-FGSM, and NA-FGTM achieved attack success rates of 55.8%, 63.8%, 60.2%, and 58.1%, respectively. However, after incorporating data augmentation strategies in Table 3, these four methods recorded improved attack success rates of 76.4%, 75.2%, 78.1%, and 81.0%, respectively, representing an upsurge of up to 22.9%. This underscores the effectiveness of integrating data augmentation methods with adversarial example generation, thereby significantly boosting the success rate of black-box attacks.

Within Table 3, under white-box attacks, the attack success rates of the four methods exhibit marginal differences, all hovering close to 100%. However, in black-box scenarios, our NA-FGTM approach outperforms other methods significantly. Specifically, in Table 3, adversarial examples generated by the NA-FGTM method augmented with data augmentation techniques from the IncRes-v2 model attain success rates of 73.8%, 69.1%, and 62.2% against the other three models, manifestly higher than those of other methods. Furthermore, on the adversarial-defended models Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens, the attack success rates stand at 47.9%, 42.3%, and 40.4%, respectively, indicating that our NA-FGTM method, when coupled with data augmentation, can more effectively conduct black-box attacks against such defended models. In summary, data augmentation techniques effectively boost the black-box attack efficiency of adversarial example generation methods, and in conjunction with these techniques, our NA-FGTM method demonstrates superior black-box attack efficiency, particularly against adversarial-trained deep neural network models.

4.5. Adversarial Example Generation for a Model Ensemble

In the third experiment, we ensembled four deep neural network models, namely Inc-v3, Inc-v4, IncRes-v2, and Res-101, to form a large ensemble model collection. Subsequently, this ensemble was employed to generate adversarial examples using four distinct methods. The experimental results in

Table 4. Attack success rates (%) of adversarial examples generated by ensemble model.

Attack	Inc-v3	Inc-v4	IncRes-v2	Res-101	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens
MI-FGSM	99.9	99.1	96.2	99.9	40.8	35.5	23.4
NI-FGSM	99.8	99.8	99.0	99.8	41.8	34.9	25.5
VMI-FGSM	99.9	99.9	100.0	99.9	59.6	61.8	53.4
NA-FGTM	99.9	99.9	99.8	100.0	75.0	70.6	62.3

demonstrate that upon ensembling these models, the attack success rates on the original models for all generated adversarial examples approach nearly 100%, thereby indicating that ensemble-based approaches can significantly enhance black-box attack success rates against adversarial examples. Moreover, adversarial examples produced by the ensemble models also exhibit increased attack success rates against defense-oriented adversarial models. For instance, in Table 4, when employing MI-FGSM-generated adversarial examples to attack the defended models Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens, the respective attack success rates are 40.8%, 35.5%, and 23.4%. By contrast, in Table 1, the highest attack rates against these same defended models using MI-FGSM were 23.3%, 20.3%, and 12.2%, respectively, on the Res-101 model, thus evidencing that ensembling enhances the black-box attack efficacy. Crucially, Table 4 reveals that our proposed NA-FGTM method, after ensembling, continues to yield adversarial examples with the highest attack success rates, particularly against defense models. With attack success rates of 75.0%, 70.6%, and 62.3% on these defended models, NA-FGTM exhibits a notably superior efficiency in black-box attacks compared to the other three methods under consideration.

Additionally,

Table 5. Mean perturbation values (L₂) of adversarial examples produced by ensemble model.

Attack	Avg Per
MI-FGSM	15.95
NI-FGSM	16.71
VMI-FGSM	20.82
NA-FGTM	11.48

presents the mean perturbation sizes of adversarial examples generated by the ensemble through the four different methods. From the table, it is evident that our proposed NA-FGTM method produces adversarial examples with the smallest mean perturbation size of 11.48, while among the other methods, MI-FGSM yields the smallest perturbations at 15.95. This indicates that even after incorporating the ensemble strategy, our NA-FGTM approach consistently generates adversarial examples with the minimal average amount of distortion.

4.6. Adversarial Example Generation with Data Augmentation for a Model Ensemble

In Experiment Four, we employed both model ensembles and data augmentation techniques in the generation of adversarial examples, with the results presented in

Table 6. Attack success rates (%) of adversarial examples generated by model ensemble and data augmentation.

Attack	Inc-v3	Inc-v4	IncRes-v2	Res-101	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens
MI-FGSM	99.5	98.2	95.7	99.7	58.9	52.9	38.5
NI-FGSM	100.0	99.8	99.5	100.0	47.5	42.7	28.2
VMI-FGSM	99.9	99.7	99.9	99.8	76.4	71.8	65.3
NA-FGTM	99.8	99.7	99.8	99.3	89.5	86.2	81.0

. The table reveals that combining model ensembles and data augmentation significantly enhances the black-box attack success rates of adversarial examples compared to those generated by individual models alone. For instance, in Table 1, our NA-FGTM method achieves black-box attack success rates of 41.6%, 36.1%, and 30.8% against Inc-v3_ens3, Inc-v3_ens4, and IncRes-v2_ens adversarial defense models when tested on IncRes-v2. However, upon integrating model ensembles and data augmentation in Table 6, these rates escalate to 89.5%, 86.2%, and 81.0%, respectively, substantially outperforming three other comparative methods. This substantiates the effectiveness of combining model ensembles and data augmentation with our NA-FGTM approach, thereby greatly improving the efficiency of black-box attacks using adversarial examples.

Moreover, we randomly selected a few distinct images from the dataset and showcase the adversarial examples produced by four different methods within the Inc-v3 model in Figure 3. The visual inspection demonstrates that the adversarial perturbations in these images are imperceptible to the human eye.

5. Conclusions

This paper primarily focuses on gradient-based black-box attack methodologies, specifically enhancing an iterative fast gradient sign method (IFGSM) for black-box attacks. IFGSM-based approaches are known for their efficiency and representativeness among attack methods, generally requiring fewer resources and time compared to alternative methods. Nevertheless, conventional attack algorithms such as MI-FGSM and its variants rely on a sign function to process gradient information, which not only discards gradient details but also decreases the efficiency of black-box attacks and amplifies the generated adversarial perturbations.

Herein, we propose a novel adversarial attack method termed NA-FGTM. Unlike MI-FGSM and its relatives that utilize a sign function, NA-FGTM adopts the Tanh function to preserve gradient information more effectively, thereby reducing the magnitude of the generated adversarial perturbations. Moreover, NA-FGTM integrates Adam optimization to expedite gradient convergence and employs Nesterov acceleration to enhance the transferability of adversarial examples, thus boosting the black-box attack success rate.

Furthermore, NA-FGTM can be coupled with data augmentation techniques like DIM, TIM, and SIM to further elevate the black-box attack success rates. The experimental results on 1000 images from the ILSVRC 2012 validation set substantiate the superior performance of NA-FGTM in generating adversarial examples compared to other methods. Notably, under the same attack success rates, NA-FGTM yields adversarial examples with smaller average perturbation magnitudes. Additionally, NA-FGTM demonstrates promising black-box attack success rates against adversarial defense models, indicating its efficacy in increasing the attack efficiency against such models. Certainly, the NA-FGTM method has numerous areas for improvement, such as generating a large number of adversarial samples in short periods or applying it against large-scale deep neural networks, both of which present significant challenges. We will consider further exploration and enhancement in these aspects in future research. Additionally, future work will involve contemplating the practical application of our method in real-world scenarios, contributing to the advancement of adversarial defense mechanisms.