Exploring Synergy of Denoising and Distillation: Novel Method for Efficient Adversarial Defense

Escalating advancements in artificial intelligence (AI) has prompted significant security concerns, especially with its increasing commercialization. This necessitates research on safety measures to securely utilize AI models. Existing AI models are vulnerable to adversarial attacks, which are a specific form of assault methodology. Although various countermeasures have been explored, practical defense models are scarce. Current adversarial defense methods suffer from reduced accuracy, increased training time, and incomplete defense against adversarial attacks, indicating performance limitations and a lack of robustness. To address these limitations, we propose a composite defense model, the knowledge Distillation and deNoising Network (DiNo-Net), which integrates knowledge distillation and feature denoising techniques. Furthermore, we analyzed a correlation between the loss surface of adversarial perturbations and denoising techniques. Using DiNo-Net, we confirmed that increasing the temperature during the knowledge distillation process effectively amplifies the loss surface around the ground truth. Consequently, this enables more efficient denoising of the adversarial perturbations. It achieved a defense success rate of 72.7%, which is a remarkable improvement over the 41.0% success rate of models with only denoising defense mechanisms. Furthermore, DiNo-Net reduced the training time and maintained higher accuracy, confirming its efficient defense performance. We hope that this relationship will spur the development of fundamental defense strategies.