FP-ZOO: Fast Patch-Based Zeroth Order Optimization for Black-Box Adversarial Attacks on Vision Models

Deep neural networks have outperformed conventional methods in various fields such as image recognition, natural language processing, and speech recognition. In particular, vision models are widely applied to real-world domains including medical image analysis, autonomous driving, smart factories, and security surveillance. However, these models are vulnerable to adversarial attacks, which pose serious threats to safety and reliability. Among different attack types, this study focuses on evasion attacks that perturb the inputs of deployed models, with an emphasis on black-box settings. The zeroth order optimization (ZOO) attack can approximate gradients and execute attacks without access to internal model information, but it becomes inefficient and exhibits low success rates on high-resolution images due to its dependence on image resizing and its high memory complexity. To address these limitations, this study proposes a patch-based fast zeroth order optimization attack, FP-ZOO. FP-ZOO partitions images into patches and generates perturbations effectively by employing probability-based sampling and an $ϵ$-greedy scheduling strategy. We conducted a large-scale evaluation of the FP-ZOO attack on the CIFAR-10, CIFAR-100, and ImageNet datasets. In this evaluation, we adopted attack success rate, ${\mathcal{L}}_2$ distance, and adversarial example generation time as performance metrics. The evaluation results showed that the FP-ZOO attack not only achieved an attack success rate of 97–100% against ImageNet in untargeted attacks, but also demonstrated performance up to 10 s faster compared to the ZOO attack. However, in targeted attacks, it showed relatively lower performance compared to baseline attacks, leaving it as a future research topic.