AggMo-Enhanced Momentum Attack: A Plug-and-Play Framework for Boosting Adversarial Transferability

Abstract

Most adversarial attack methods achieve high success rates under the white-box setting. However, these methods often lack transferability when targeting other deep neural network (DNN) models. Momentum-based attacks have emerged as an effective strategy to enhance transferability by incorporating a momentum term to stabilize update directions. While simple constant-momentum methods (e.g., MI-FGSM) or advanced variants (e.g., NI-FGSM, VMI-FGSM) have shown promise, they either use a single momentum decay factor or introduce significant computational overhead. To address this, we propose a novel plug-and-play momentum aggregation framework named AggMo-Attack. Our key insight is that a single momentum term with a fixed decay factor cannot optimally capture the multi-scale temporal correlations in gradients during adversarial optimization. Inspired by the Aggregated Momentum (AggMo) optimizer, we designed a multi-momentum aggregation module that maintains and weightedly combines multiple velocity vectors with different decay factors. This framework can be seamlessly integrated into existing momentum-based attack methods (e.g., MI-FGSM, NI-FGSM, VMI-FGSM) as a drop-in replacement for their standard momentum update step. Extensive experiments demonstrate that integrating our AggMo module significantly improves adversarial transferability. Our work provides a versatile and effective tool for enhancing momentum-based adversarial attacks and opens a new direction for designing adaptive attack strategies.

1. Introduction

Existing studies have proven that deep neural networks (DNNs) are vulnerable to adversarial example attacks [1,2,3]. Adversarial examples, which are close to the original ones, are generated by adding deliberate and imperceptible perturbations to a clean example. These perturbations cause models to produce incorrect predictions as intended by attackers, as shown in Figure 1. When DNNs are applied in safety-sensitive fields, such as autonomous driving [4,5], face recognition [6,7], video surveillance [8], camouflaged object detection [9], etc., it raises concerns about application security.

Figure 1. Taking two images from the ImageNet dataset [10], a demonstration of network attack based on AggMo-MI-FGSM on the Inception v3 [11] model. In the left column, the clean examples are displayed. The middle column showcases the adversarial noises obtained by applying AggMo-MI-FGSM. The right column exhibits the adversarial example and it is hard to tell the difference with the naked eye after the disturbance is added to the clean example. Additionally, we provide the predicted labels and probabilities assigned by the Inception v3 model for these images.

In the white-box setting, where the attacker has knowledge of the model’s structure and parameter information, a number of approaches have been proposed, including the Fast Gradient Sign Method (FGSM) [1], Iterative Fast Gradient Sign Method (I-FGSM) [12], and the Projected Gradient Descent Method (PGD) [13], among others. Additionally, some studies have demonstrated the transferability of adversarial examples [14,15,16], indicating that adversarial examples crafted for one DNN model can effectively attack other DNN models to some extent. In other words, even if the attacker does not know the relevant information of the model, it can still successfully attack the deep neural network, causing it to misclassify.

Nevertheless, most white-box attack methods tend to exhibit limited transferability. Recently, various approaches have been put forth to tackle this challenge. Existing methods to improve the capability of adversarial example migration attacks are mainly divided into five categories: momentum-based iterative attacks [17,18], variance-tuning iterative gradient-based methods [19], diverse inputs [20,21,22], translation-invariant techniques [23], and multi-model ensemble attacks [15]. Despite these efforts, these methods have not considered attack efficiency, resulting in low efficiency in generating adversarial examples. Therefore, it is necessary to propose a low-complexity method to improve the transferability of adversarial examples, enabling them to achieve higher attack success rates and efficiency when attacking other unknown DNN models.

Among the aforementioned methods aimed at enhancing transferability, momentum-based methods [17] demonstrate notable effectiveness. This approach introduces a momentum term for stabilizing update directions. However, existing momentum attacks primarily rely on a single, fixed momentum decay factor (e.g., $\mathsf{\mu }$ = 1.0 in MI-FGSM). We argue that this design is suboptimal because the gradient dynamics during adversarial optimization exhibit multi-scale temporal characteristics. A smaller decay factor causes the momentum to decay rapidly, focusing on recent gradients, which is suitable for quick directional adjustments. Conversely, a larger decay factor results in slow momentum decay, accumulating long-term gradient trends, which is conducive to maintaining a stable direction. Using only one factor fails to exploit this rich information.

Based on this insight, we draw inspiration from the Aggregated Momentum (AggMo) optimizer [24] used in neural network training. We propose AggMo-Attack, a plug-and-play momentum aggregation framework specifically designed for adversarial attacks. Instead of tuning a single decay factor, our framework maintains $K$ independent momentum buffers, each with a distinct decay factor ${\beta }_{\mathrm{k}}$. The final update direction is a weighted aggregation of these buffers. This design provides several advantages: (1) it captures gradient information at multiple time scales; (2) it is highly modular and can enhance any momentum-based attack.

Our main contributions are as follows:

(1)

We propose AggMo-Attack, the first plug-and-play, generalized momentum aggregation framework for adversarial attacks. It can be seamlessly integrated into methods like MI-FGSM, NI-FGSM, etc., by simply replacing their momentum update step.

(2)

We present three instantiations of the framework, AggMo-MI-FGSM, AggMo-NI-FGSM, and AggMo-VMI-FGSM, demonstrating its versatility and effectiveness.

2. Related Work

In this section, we provide the background knowledge of adversarial attacks as well as review the related works about the transferability of adversarial examples.

2.1. Adversarial Attack

Define a classifier $f\left(x\right):x\in \chi \to y\in \mathcal{Y}$, where $x$ represents the input image and $y$ represents the output label. The objective of the adversarial example attack is to add a small amount of noise to the clean example $x$, resulting in an adversarial example $x^{\mathrm{*}}$ that causes the classifier to misclassify. It is a common requirement that the $l_p$ norm of the adversarial noise be constrained to be less than a specified value $ϵ$ as ${\parallel x^{*}-x\parallel }_p\le ϵ$, where $p$ can take values of 0, 1, 2, ∞. This constraint can be formalized as

$$f_{\theta }\left(x^{*}\right)\ne y,s.t{\parallel x^{*}-x\parallel }_p\le ϵ$$

(1)

Adversarial attack methods for solving Equation (1) are classified into eight categories: constrained optimization [2,25,26], gradient optimization [1,27,28], sensitivity analysis [10], estimated decision boundary [11], migration mechanism [29], generative model [30,31,32,33], evolution strategies [34], and gradient approximation [35,36]. We focus on improving the transferability of gradient optimization adversarial attack methods, thus introducing two representative gradient optimization attack methods.

One-step gradient-based approaches, such as the fast gradient sign method (FGSM) [1], generate an adversarial example $x^{*}$ by performing one-step update to maximize the loss function:

$$x^{*}=x+ϵ\cdot sign\left({\nabla }_{x}J\right(x,y\left)\right)$$

(2)

where ${\nabla }_{x}J$ is the gradient of the loss function $J(\cdot )$ with respect to $x$ and cross entropy loss is often used. $sign(\cdot )$ is the sign function to limit perturbations that conform to the $l_{\infty }$ norm bound.

Iterative methods, such as the iterative version of FGSM(I-FGSM) [12], generate an adversarial example $x^{\mathrm{*}}$ by multiple iterations with a small step size α, which can be formalized into

$$x_0^{\mathrm{*}}=x,\hspace{1em}{x}_{t+1}^{\mathrm{*}}=x_t^{\mathrm{*}}+\alpha \cdot sign\left({\nabla }_{x}J\right(x_t^{\mathrm{*}},y\left)\right)$$

(3)

The variable “t” represents the number of iterations. The Projected Gradient Descent (PGD) [13] algorithm builds upon the basis of I-FGSM by adding random perturbations to the original example within its neighborhood range as the initial input, thereby reducing the number of iterations and enhancing the attack performance.

2.2. The Transferability of Adversarial Examples

One of the simplest and most widely used techniques to improve the transferability of adversarial examples is integrating the momentum term into the input gradient.

The Momentum Iterative Fast Gradient Sign Method (MI-FGSM) [17] incorporates a temporal momentum term into the iterative attack, thereby stabilizing the update directions.

$${\mathcal{G}}_{t+1}=\mu {\mathcal{G}}_t+{\displaystyle \frac{{\nabla }_{x}J(x_t^{*},y)}{{{\parallel \nabla }_{x}J(x_t^{*},y)\parallel }_1}}$$

(4)

$$x_{t+1}^{*}=x_t^{*}+\alpha \cdot sign\left({\mathcal{G}}_{t+1}\right)$$

(5)

where ${\mathcal{G}}_t$ is the accumulated gradient and $\mu$ is the decay factor which is often set to 1.0. MI-FGSM updates the moment gradient ${\mathcal{G}}_{t+1}$ by Equation (4) and then updates $x_{t+1}^{\mathrm{*}}$ by Equation (5). Unlike I-FGSM, MI-FGSM introduces a momentum term $\mathsf{\mu }{\mathcal{G}}_{\mathrm{t}}$ in ${\mathcal{G}}_{t+1}$.

The Nesterov Iterative Fast Gradient Sign Method (NI) [18] integrates the Nesterov Accelerated Gradient (NAG) [37] in the iterative gradient-based attack to leverage its looking ahead property to help escape from poor local optima. At each iteration, NI-FGSM first moves the data point based on the accumulated update ${\mathcal{G}}_t$.

$$x_t^{net}=x_t^{*}+\alpha \cdot \mu \cdot {\mathcal{G}}_t$$

(6)

$${\mathcal{G}}_{t+1}=\mathsf{\mu }{\cdot \mathcal{G}}_t+{\displaystyle \frac{{\nabla }_{x}J(x_t^{net},y)}{{{\parallel \nabla }_{x}J(x_t^{net},y)\parallel }_1}}$$

(7)

Unlike MI-FGSM, NI-FGSM uses ${\mathrm{x}}_{\mathrm{t}}^{\mathrm{*}}+\mathsf{\alpha }\cdot \mathsf{\mu }\cdot {\mathcal{G}}_{\mathrm{t}}$ instead of ${\mathrm{x}}_{\mathrm{t}}^{\mathrm{*}}$ in Equation (4).

The Variance Tuning Momentum-based Iterative Method (VMI-FGSM) [19] further considers the gradient variance of the previous iteration to tune the current gradient so as to stabilize the update direction. It substitutes Equation (4) by

$${\mathcal{G}}_{t+1}=\mu {\mathcal{G}}_t+{\displaystyle \frac{{\nabla }_{x}J(x_t^{*},y)+v_t}{{{\parallel \nabla }_{x}J(x_t^{*},y)+v_t\parallel }_1}}$$

(8)

where $v_{t+1}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{\nabla }_{x}J(x_t,y)-{\nabla }_{x}J(x_t^{\mathrm{*}},y)$, $x_t=x_t^{\mathrm{*}}+r_i$ and $r_i$ is the random noise within a certain range.

In the field of neural network optimization, the Aggregated Momentum (AggMo) optimizer [24,38] effectively captures the dynamic characteristics of gradients across multiple temporal scales by aggregating momentum terms with different decay factors, thereby achieving smoother and more stable convergence. Inspired by this core concept, we introduce the mechanism of AggMo into the domain of adversarial attacks and propose the versatile AggMo-Attack framework. This framework features a concise structure and can be seamlessly integrated into existing momentum-based attack methods such as MI-FGSM, NI-FGSM, and VMI-FGSM in a “plug-and-play” manner—simply by replacing their original momentum update step, it enhances the transferability of adversarial examples across different models without introducing significant computational overhead. Thus, it provides an efficient and general-purpose solution for improving the transferability of adversarial attacks.

3. Methodology

We propose a novel method termed AggMo-Attack for generating adversarial examples that are effective against both white-box and black-box models. In this section, we provide a detailed explanation of the proposed algorithm. We first introduce the concept of aggregated momentum with multiple decay factors and its integration into the I-FGSM method, generating adversarial examples that adhere to the ${\mathrm{l}}_{\mathrm{\infty }}$ norm constraint. Subsequently, we extend AggMo-Attack to ${\mathrm{l}}_2$ norm bounds and targeted attack scenarios.

3.1. AggMo-Attack

To achieve more effective attacks, I-FGSM employs a greedy strategy by iteratively moving the adversarial example in the direction of the gradient sign with a small step size (as shown in Equation (3)). However, this approach tends to cause the adversarial example to converge quickly to suboptimal local maxima, limiting its transferability across different models. Building upon the observation that momentum can stabilize gradient updates and help navigate through complex loss landscapes [5], we propose to enhance the momentum mechanism by aggregating multiple momentum buffers with different decay factors. This design, inspired by the Aggregated Momentum (AggMo) optimizer [24], allows the attack to capture gradient information at multiple temporal scales, leading to more stable and effective update directions.

Specifically, instead of maintaining a single momentum buffer as in MI-FGSM, we maintain $K$ independent momentum buffers, each with a distinct decay factor. The final update direction is computed as the weighted average of all momentum buffers. This approach provides several advantages: (1) it captures both short-term and long-term gradient trends, (2) it offers more flexible control over the update dynamics, and (3) it can be seamlessly integrated into existing momentum-based methods.

The proposed AggMo-Attack method is summarized in Algorithm 1. Let ${\mathcal{G}}_{t+1}^{\left(k\right)}$ denote the momentum buffer for the k-th decay factor ${\beta }_k$ at iteration t + 1. Each buffer is updated independently, and the final aggregated momentum ${\mathcal{G}}_{t+1}$ is computed as the average of all buffers. The adversarial example is then updated using the sign of the aggregated momentum.

Algorithm 1 AggMo-Attack
Input: A classifier $f$ with loss function $J;$ a clean example $x$ and ground-truth label $y;$
Input: The size of perturbation $ϵ;$ iterations $T$ and decay factor $\{{\beta }_1,{\beta }_2\dots {\beta }_k\}.$
Output: An adversarial example $x^{*}$ with ${\parallel x^{*}-x\parallel }_{\infty }\le ϵ.$
1: $\alpha =2\times \mathsf{ϵ}/\mathrm{T}$
2: Initialize $K$ momentum buffers: ${\mathcal{G}}_0^{\left(k\right)}=0,$for k = 1, 2, …, K
3: Initialize $x_0^{*}=x$
4: for $t=0$ to $T-1$ do
5:    Input $x_t^{*}$ to $f$ and obtain the gradient ${\nabla }_{x}J(x_t^{*},y)$
6:    for $k=0$ to $K$ do
7:     Update momentum buffer:
${\mathcal{G}}_{t+1}^{\left(k\right)}={{\beta }_k·\mathcal{G}}_t^{\left(k\right)}+{\displaystyle \frac{{\nabla }_{x}J(x_t^{*},y)}{{{\parallel \nabla }_{x}J(x_t^{*},y)\parallel }_1}}$	(9)
8:   End for
9:    Compute aggregated momentum:
${\mathcal{G}}_{t+1}={\displaystyle \frac{1}{K}}{\sum }_{k=1}^K{\mathcal{G}}_{t+1}^{\left(k\right)}$	(10)
10:   Update $x_{t+1}^{*}$ by applying the sign gradient as
$x_{t+1}^{*}=x_t^{*}+\alpha \cdot sign\left({\mathcal{G}}_{t+1}\right)$	(11)
11:    Ensures the $l_{\infty }$ constraint is met:
$x_{t+1}^{*}=clip(x_{t+1}^{*},x-\mathcal{E},x+\mathcal{E})$	(12)
12:    Ensures the box constraint is met:
$x_{t+1}^{*}=clip(x_{t+1}^{*},0,1)$	(13)
13:end for
14:return $x^{*}=x_T^{*}$

Connection to existing methods: Our AggMo-Attack framework generalizes existing momentum-based methods. When K = 1, AggMo-Attack reduces to MI-FGSM with $\mu ={\beta }_1$. When K = 2 with ${\beta }_1$ = 0.99 and ${\beta }_2$ = 0.9, AggMo-Attack aggregates two momentum buffers with different decay rates. This design provides a more principled way to capture gradient information at multiple temporal scales compared to methods that manually tune a single decay factor.

Hyper-parameter design: The use of multiple independent decay factors provides clearer physical interpretations compared to tuning a single parameter. Each decay factor ${\beta }_k$ explicitly controls the weight of historical gradient information in the k-th momentum buffer, with larger values emphasizing longer-term trends and smaller values focusing on recent gradients. This decoupled design makes the hyper-parameter space more interpretable and easier to navigate, as each factor has a distinct role in controlling the temporal characteristics of gradient accumulation.

3.2. Extensions to Other Settings

The proposed AggMo-Attack framework exhibits strong flexibility and generality and can be readily extended to various practical adversarial attack scenarios. By embedding the aggregated momentum mechanism as a plug-and-play module into mainstream adversarial attack pipelines, we can derive enhanced variants that retain the intrinsic benefits of multi-scale gradient aggregation, leading to further performance improvements in different attack settings.

3.2.1. ${\mathrm{l}}_2$ Norm-Constrained Attack

For adversarial attacks under the ${\mathrm{l}}_2$ perturbation constraint ${\parallel {\mathrm{x}}^{\mathrm{*}}-\mathrm{x}\parallel }_2\le \mathsf{ϵ}$, the pixel-wise sign update (originally designed for ${\mathrm{l}}_{\mathrm{\infty }}$ constraints) is no longer optimal. Instead, we revise the sample update step (Line 11 in Algorithm 1) to a gradient-direction normalization strategy, which strictly satisfies the ${\mathrm{l}}_2$ budget constraint:

$${\mathrm{x}}_{\mathrm{t}+1}^{\mathrm{*}}={\mathrm{x}}_{\mathrm{t}}^{\mathrm{*}}+\mathsf{\alpha }\cdot {\displaystyle \frac{{\mathcal{G}}_{\mathrm{t}+1}}{{{\parallel \mathcal{G}}_{\mathrm{t}+1}\parallel }_2}}$$

(14)

This modification adapts the aggregated momentum to Euclidean-space perturbation constraints while preserving the multi-scale momentum aggregation property.

3.2.2. Targeted Adversarial Attack

In the targeted attack scenario, the objective is to misclassify the clean example into a pre-specified target class ${\mathrm{y}}^{\mathrm{*}}$. To achieve this goal, two critical adjustments are made to the original algorithm. First, the loss function is redefined to penalize the discrepancy between the model output and the target label, i.e., $\mathrm{J}({\mathrm{x}}^{\mathrm{*}},{\mathrm{y}}^{\mathrm{*}})$. Second, the gradient update direction is inverted to minimize the loss of the target class. Concretely, the normalized gradient remains:

$${\mathcal{G}}_{\mathrm{t}}={\displaystyle \frac{{\nabla }_{\mathrm{x}}\mathrm{J}({\mathrm{x}}_{\mathrm{t}}^{\mathrm{*}},{\mathrm{y}}^{\mathrm{*}})}{{{\parallel \nabla }_{\mathrm{x}}\mathrm{J}({\mathrm{x}}_{\mathrm{t}}^{\mathrm{*}},{\mathrm{y}}^{\mathrm{*}})\parallel }_1}}$$

(15)

while the sample update rule is reversed. For ${\mathrm{l}}_{\mathrm{\infty }}$ norm constraints,

$${\mathrm{x}}_{\mathrm{t}+1}^{\mathrm{*}}={\mathrm{x}}_{\mathrm{t}}^{\mathrm{*}}-\mathsf{\alpha }\cdot \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\left({\mathcal{G}}_{\mathrm{t}+1}\right)$$

(16)

For $l_2$ norm constraints,

$${\mathrm{x}}_{\mathrm{t}+1}^{\mathrm{*}}={\mathrm{x}}_{\mathrm{t}}^{\mathrm{*}}-\mathsf{\alpha }\cdot {\displaystyle \frac{{\mathcal{G}}_{\mathrm{t}+1}}{{{\parallel \mathcal{G}}_{\mathrm{t}+1}\parallel }_2}}$$

(17)

3.2.3. Integration with Advanced Momentum-Based Attacks

The core advantage of the AggMo mechanism lies in its seamless compatibility with existing momentum-driven adversarial attack methods, as the conventional single momentum update in mainstream approaches can be directly replaced with our proposed multi-scale aggregated momentum strategy. For instance, substituting the momentum component in NI-FGSM and VMI-FGSM with AggMo yields the enhanced variants AggMo-NI-FGSM and AggMo-VMI-FGSM, respectively. This plug-and-play design demonstrates the versatility of the AggMo-Attack framework, which not only systematically improves adversarial transferability through multi-scale momentum aggregation but can also be naturally extended to various scenarios such as ${\mathrm{l}}_2$-constrained attacks and targeted attacks, and flexibly integrated into advanced attack pipelines. Consequently, it serves as a general and effective enhancement module for strengthening the robustness and efficacy of adversarial attacks across diverse settings.

4. Experiments

We conducted numerous experiments on the ILSVRC 2012 dataset [39] to validate and demonstrate the effectiveness of our method. Experimental settings are detailed in Section 4.1, and the results are presented in other sections.

4.1. Setup

Datasets: For a fair comparison, we adopt the same dataset as used in prior work [15,17], which consists of 1000 images randomly selected from the ILSVRC 2012 validation set, covering 1000 categories.

Models: To comprehensively evaluate the transferability of the attack methods, we employ six representative, ImageNet-pretrained models with diverse architectures as our evaluation benchmarks. These include four classic convolutional neural networks (CNNs), Inception v3 (Inc-v3) [40], VGG-16 [41], DenseNet-121 [42], and ResNet-152 [43], as well as two mainstream Vision Transformer models: Vision Transformer (ViT) [38] and Swin Transformer [44].

Baselines: To validate the effectiveness of the proposed framework, we select three mainstream momentum-based adversarial attack methods as baselines for comparison: MI-FGSM [17], NI-FGSM [18], and VMI-FGSM [19]. This selection is based on the rationale that our AggMo-Attack is a plug-and-play module designed to integrate with momentum mechanisms.

Hyper-parameters: To ensure a fair comparison, the maximum perturbation budget ϵ is set to 16/255, and the number of iterations $T$ is fixed at 10 for all methods. The step size is set to α = 2ϵ/ T. For our method, the decay factors are set to ${\beta }_1$ = 0.99, ${\beta }_2$ = 0.9 and ${\beta }_3=0$. This design follows the core concept of the AggMo optimizer [24], which aggregates momentum terms with different decay coefficients to integrate both short-term and long-term gradient trends, thereby enhancing the stability and efficiency of the optimization process. All experiments are conducted on a computing platform equipped with an NVIDIA RTX 3090 GPU.

4.2. Attacking Models

To evaluate the proposed AggMo-Attack framework, we integrate it into three momentum-based methods—MI-FGSM, NI-FGSM, and VMI-FGSM—yielding three enhanced variants: AggMo-MI-FGSM, AggMo-NI-FGSM, and AggMo-VMI-FGSM. We assess their performance on six models, as shown in Table 1. Within each row (adversarial examples crafted on the same white-box model), the highest black-box attack success rates are bolded in black. Columns marked with * denote white-box attacks.

Table 1. The success rates of adversarial attacks. * indicates the white-box attacks.

	Attack	Inc-v3	VGG-16	DenseNet-121	Res-152	Vit	Swin
Inc-v3	MI-FGSM	99.9% *	57.4%	43.7%	35.8%	12.5%	9.4%
	AggMo-MI-FGSM	99.9% *	57.5%	45.0%	37.6%	12.8%	8.7%
	NI-FGSM	99.9% *	56.7%	45.3%	36.4%	11.6%	8.5%
	AggMo-NI-FGSM	100% *	61.9%	44.0%	35.9%	12.5%	8.6%
	VMI-FGSM	100% *	69.8%	64.3%	54.5%	18.9%	17.2%
	AggMo-VMI-FGSM	100% *	66.9%	63.9%	54.7%	20.7%	17.3%
VGG-16	MI-FGSM	39.6%	99.9% *	67.6%	49.0%	19.0%	18.3%
	AggMo-MI-FGSM	41.8%	99.9% *	68.1%	50.2%	18.3%	20.1%
	NI-FGSM	37.2%	99.9% *	61.0%	44.4%	19.3%	18.0%
	AggMo-NI-FGSM	37.8%	100% *	60.8%	45.9%	18.8%	19.0%
	VMI-FGSM	57.7%	100% *	82.6%	67.5%	25.2%	28.3%
	AggMo-VMI-FGSM	59.0%	100% *	83.1%	67.0%	25.4%	27.2%
DenseNet-121	MI-FGSM	50.1%	83.9%	100% *	76.4%	15.7%	13.8%
	AggMo-MI-FGSM	52.6%	84.6%	100% *	78.1%	16.3%	14.1%
	NI-FGSM	50%	86.9%	100% *	76.8%	15.1%	12.8%
	AggMo-NI-FGSM	52.5%	87.9%	100% *	79.0%	15.3%	13.0%
	VMI-FGSM	75.7%	93.0%	100% *	92.8%	29.4%	28.1%
	AggMo-VMI-FGSM	77.0%	92.2%	100% *	92.9%	30.7%	28.4%
Res-152	MI-FGSM	46.5%	77.2%	79.4%	100% *	15.4%	13.9%
	AggMo-MI-FGSM	47.8%	76.8%	80.3%	100% *	15.4%	13.6%
	NI-FGSM	45.7%	78.1%	75.1%	100% *	15.2%	13.0%
	AggMo-NI-FGSM	44.5%	78.2%	77.6%	100% *	15.5%	13.2%
	VMI-FGSM	76.5%	88.6%	94.0%	100% *	36.8%	37.6%
	AggMo-VMI-FGSM	76.7%	88.0%	94.1%	100% *	34.5%	38.3%
Vit	MI-FGSM	30.0%	64.5%	40.9%	33.3%	99.6% *	76.9%
	AggMo-MI-FGSM	30.2%	61.7%	41.3%	34.5%	99.8% *	77.4%
	NI-FGSM	30.5%	65.0%	40.9%	36.8%	99.5% *	76.3%
	AggMo-NI-FGSM	31.6%	65.1%	42.3%	34.3%	99.8% *	76.4%
	VMI-FGSM	32.9%	61.3%	41.2%	37.4%	100% *	77.8%
	AggMo-VMI-FGSM	31.4%	64.5%	42.8%	35.5%	100% *	77.9%
Swin	MI-FGSM	9.3%	34.6%	15.2%	12.2%	22.0%	99.8% *
	AggMo-MI-FGSM	9.6%	34.8%	16.1%	12.8%	21.0%	99.9% *
	NI-FGSM	9.3%	32.4%	16.4%	15.1%	23.4%	99.4% *
	AggMo-NI-FGSM	9.7%	37.1%	15.7%	14.4%	23.1%	100% *
	VMI-FGSM	9.1%	31.8%	13.8%	11.2%	20.6%	99.7% *
	AggMo-VMI-FGSM	7.9%	31.0%	13.8%	11.5%	22.4%	99.9% *

The main findings are drawn as follows. First, AggMo-Attack provides stable and consistent improvements over the baseline method. By leveraging multi-scale momentum aggregation, it effectively strengthens the transferability of adversarial examples for MI-FGSM across various scenarios. For instance, when using Inception-v3 as the source model, AggMo-MI-FGSM achieves higher black-box success rates than MI-FGSM on VGG-16 (57.5% vs. 57.4%), DenseNet-121 (45.0% vs. 43.7%), ResNet-152 (37.6% vs. 35.8%), and ViT (12.8% vs. 12.5%). Similar improvements are observed when attacking from VGG-16, DenseNet-121, ResNet-152, ViT, and Swin, demonstrating the effectiveness and robustness of the proposed method.

Second, the AggMo framework maintains strong compatibility and flexibility when combined with advanced momentum-based attacks. For NI-FGSM, AggMo-NI-FGSM yields gains in numerous transfer settings, such as from Inception-v3 to VGG-16 (61.9% vs. 56.7%), from VGG-16 to ResNet-152 (45.9% vs. 44.4%), from DenseNet-121 to ResNet-152 (79.0% vs. 76.8%), and from Swin to VGG-16 (37.1% vs. 32.4%). For the powerful baseline VMI-FGSM, AggMo-VMI-FGSM sustains competitive performance and further improves transferability in key cases, including from Inception-v3 to ViT (20.7% vs. 18.9%), from DenseNet-121 to ResNet-152 (92.9% vs. 92.8%), and from ResNet-152 to Swin (38.3% vs. 37.6%). These results validate that AggMo-Attack can serve as a general and effective plug-and-play enhancement module.

Third, in terms of cross-architecture transferability, attacks targeting Vision Transformers (ViT, Swin) generally achieve lower success rates compared with CNN-based models, revealing the significant structural divergence between CNNs and Transformers. Nevertheless, the AggMo-enhanced versions still achieve comparable or superior performance against their baseline methods in these challenging cross-architecture transfer tasks. Typical examples include transferring from DenseNet-121 to ViT (16.3% vs. 15.7%), from Inception-v3 to ViT (20.7% vs. 18.9%), and from Swin to VGG-16 (37.1% vs. 32.4%), illustrating that the perturbations crafted by AggMo-Attack preserve strong cross-architecture transferability.

Overall, the AggMo-Attack framework serves as an effective and adaptable plug-and-play component for momentum-based adversarial attacks. By incorporating multi-scale aggregated momentum, it consistently improves the transferability of adversarial examples, leading to higher black-box attack success rates across various network architectures, including both CNNs and Vision Transformers.

4.3. Computational Efficiency Evaluation

To validate the efficiency of the proposed AggMo-Attack, we evaluate the average time required to generate adversarial examples for all methods. Specifically, we calculate the average processing time per sample (in milliseconds per sample) for each baseline method and its AggMo-enhanced variant, with detailed results reported in Table 2.

Table 2. Average time for generating adversarial examples using different attack methods across various models (unit: ms per image).

	MI-FGSM	AggMo-MI-FGSM	NI-FGSM	AggMo-NI-FGSM	VMI-FGSM	AggMo-VMI-FGSM
Inc-v3	387.4	408.2	330.1	353.9	2356.7	2428.4
VGG-16	133.0	133.3	80.0	80.4	479.1	489.2
DenseNet-121	482.4	493.1	452.4	457.6	3006.5	3024.3
Res-152	516.1	519.1	457.1	461.4	3040.8	3072.3
Vit	271.3	285.0	279.6	225.1	1814.2	1834.6
Swin	769.9	775.1	645.4	661.4	4041.4	4043.0

Experimental results show that integrating the aggregated momentum mechanism introduces only negligible additional time overhead. Across all benchmark models, the generation speed of AggMo-MI-FGSM, AggMo-NI-FGSM, and AggMo-VMI-FGSM remains highly comparable to that of their original baseline counterparts. For relatively lightweight methods such as MI-FGSM and NI-FGSM, the speed difference is minimal. For instance, on VGG-16, MI-FGSM and AggMo-MI-FGSM require 133.0 ms and 133.3 ms per sample, respectively, while NI-FGSM and AggMo-NI-FGSM take 80.0 ms and 80.4 ms per sample, respectively. Even for computationally intensive methods like VMI-FGSM, the additional time cost introduced by AggMo remains very small. For example, on Swin Transformer, VMI-FGSM requires 4041.4 ms per sample, whereas AggMo-VMI-FGSM requires only 4043.0 ms per sample.

In summary, the proposed AggMo-Attack improves the transferability of adversarial examples with almost no increase in time cost. This excellent balance between performance and efficiency confirms that it is a practical, plug-and-play module that can be easily deployed in real-world attack scenarios.

4.4. Attack Performance Under $l_2$ Norm Constraint

To further validate the generality of the AggMo module, supplementary experiments are conducted under the $l_2$ norm constraint, comparing the performance of MI-FGSM and its AggMo-enhanced counterpart, AggMo-MI-FGSM. In this setting, the maximum perturbation is set to ϵ = 0.3, the number of iterations is T = 40, and the step size is $\mathsf{\alpha }$ = 0.01. The results are summarized in Table 3 and Table 4.

Table 3. Attack success rates (%) of MI-FGSM under the $l_2$ norm constraint.

Source Model/Target Model	Inc-v3	VGG-16	DenseNet-121	Res-152	Vit	Swin
Inc-v3	74.9%	9.4%	5.9%	4.4%	5.4%	2.5%
VGG-16	1.5%	89.8%	6.4%	4.3%	4.7%	1.7%
DenseNet-121	3%	11.4%	87%	5.9	6.1%	3.5%
Res-152	3.1%	10.4%	8.8%	76.5%	5.7%	3.0%
Vit	1.7%	8.4%	4.0%	3.9%	75%	17.9%
Swin	2.1%	8.4%	5.0%	4.0%	12.3%	84.9%

Table 4. Attack success rates (%) of AggMo-MI-FGSM under the $l_2$ norm constraint.

Source Model/Target Model	Inc-v3	VGG-16	DenseNet-121	Res-152	Vit	Swin
Inc-v3	76.3%	9.5%	6.1%	4.6%	5.8%	3.1%
VGG-16	1.8%	90.8%	6.5%	4.5%	5.1%	2.2%
DenseNet-121	3.2%	11.4%	89.1%	6.2%	6.6%	3.5%
Res-152	4.9%	11.2%	9.5%	78.7%	5.7%	3.0%
Vit	1.9%	8.9%	5.5%	4.2%	75.1%	18.5%
Swin	2.3%	9%	5.3%	4.3%	12.7%	84.2

Under the $l_2$ norm constraint, AggMo-MI-FGSM consistently demonstrates improved performance over the baseline MI-FGSM. Taking Inception v3 as an example, the white-box attack success rate increases from 74.9% to 76.3%. Improvements are also observed in black-box transferability to other models, such as VGG-16 and DenseNet-121, albeit with relatively small margins. These results confirm that the proposed AggMo framework is not limited to a specific norm constraint—it remains effective under both $l_{\infty }$ and $l_2$ settings—and serves as a generalizable plug-and-play module for enhancing adversarial transferability.

4.5. Impact of Iteration Counts

Based on the comprehensive experimental data, we further investigate the impact of iteration counts on adversarial attack performance. Adversarial examples are generated using the Inception-v3 (Inc-v3) model as the source (white-box) model, with iteration numbers ranging from 1 to 10. These adversarial examples are then transferred to attack five distinct black-box models, namely VGG-16, DenseNet-121, ResNet-152, Vision Transformer (ViT), and Swin Transformer, in addition to evaluating white-box performance on Inc-v3 itself. All hyper-parameters are held constant (ϵ = 16/255, α = 2ϵ/T, ${\beta }_1$ = 0.99, ${\beta }_2$ = 0.9 and ${\beta }_3$ = 0) to ensure a controlled comparison focused on the effect of iteration number variation.

The experimental results demonstrate that AggMo-enhanced methods consistently outperform or match their respective baseline counterparts across most iteration steps and target models, as shown in Figure 2. This performance advantage is particularly evident in black-box scenarios. For instance, AggMo-MI-FGSM achieves higher success rates than standard MI-FGSM when attacking DenseNet-121, with the performance gap tending to widen as iterations increase. Similar trends are observed for AggMo-NI-FGSM and AggMo-VMI-FGSM, which frequently surpass or equal the performance of their original versions, especially on challenging targets like ResNet-152 and ViT.

Figure 2. Attack success rates versus iteration numbers (1 to 10) for MI-FGSM, NI-FGSM, VMI-FGSM and their AggMo-enhanced versions. Adversarial examples are crafted on the Inc-v3 source model and evaluated on (a) Inc-v3 (white-box), (b) VGG-16, (c) DenseNet-121, (d) ResNet-152, (e) ViT, and (f) Swin Transformer (black-box).

5. Discussion

The process of discovering adversarial examples shares conceptual similarities with model training, where the transferability of adversarial examples is analogous to the generalization ability of models. It is therefore reasonable to enhance transferability by incorporating momentum terms that regulate the relative contributions of past and current gradients to the update direction. This rationale aligns with principles in deep learning optimization, where momentum-based methods are known to improve convergence and generalization. Our proposed AggMo framework extends this insight by maintaining multiple momentum buffers with distinct decay factors, effectively capturing gradient dynamics across different temporal scales, thus generating more transferable adversarial examples.

6. Conclusions

In this paper, we propose AggMo-Attack, a versatile and plug-and-play momentum aggregation framework designed to boost the transferability of adversarial attacks. The method employs a multi-momentum aggregation mechanism that combines multiple momentum buffers with different decay factors, enabling the attack to effectively capture both short-term and long-term gradient trends. It can be seamlessly integrated into existing momentum-based attacks—such as MI-FGSM, NI-FGSM, and VMI-FGSM—by simply replacing their standard momentum update step, resulting in AggMo-MI-FGSM, AggMo-NI-FGSM, and AggMo-VMI-FGSM, respectively.

Comprehensive experiments demonstrate that AggMo-Attack consistently enhances adversarial transferability across a diverse range of network architectures, including both CNNs (e.g., Inception-v3, VGG-16, DenseNet-121, ResNet-152) and Vision Transformers (ViT, Swin). The framework maintains high computational efficiency, introducing only negligible overhead compared to baseline methods. Beyond the standard $l_{\infty }$ norm attacks, we also validate its effectiveness under $l_2$ norm constraints, further confirming its adaptability and general applicability.