Boosting Adversarial Transferability Through Adaptive-Learning-Rate with Data Augmentation Mechanism

BAO Lei; TAO Wei; TAO Qing

doi:10.12263/DZXB.20220737

您当前的位置：

首页 >

文章列表页 >

Boosting Adversarial Transferability Through Adaptive-Learning-Rate with Data Augmentation Mechanism

PAPERS | 更新时间：2026-04-10

- Boosting Adversarial Transferability Through Adaptive-Learning-Rate with Data Augmentation Mechanism
- ACTA ELECTRONICA SINICA Vol. 52, Issue 1, Pages: 157-169(2024)
- 作者机构：
  
  1.中国人民解放军陆军炮兵防空兵学院信息工程系，安徽合肥 230031
  2.中国人民解放军军事科学院，北京 100091
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20220737
  CLC： TP391;
- Received：27 June 2022，
  
  Revised：2022-09-21，
  
  Published：25 January 2024
- 稿件说明：
移动端阅览
鲍蕾,陶蔚,陶卿.结合自适应步长策略和数据增强机制提升对抗攻击迁移性[J].电子学报,2024,52(01):157-169.

BAO Lei,TAO Wei,TAO Qing.Boosting Adversarial Transferability Through Adaptive-Learning-Rate with Data Augmentation Mechanism[J].ACTA ELECTRONICA SINICA,2024,52(01):157-169.
鲍蕾,陶蔚,陶卿.结合自适应步长策略和数据增强机制提升对抗攻击迁移性[J].电子学报,2024,52(01):157-169. DOI： 10.12263/DZXB.20220737.

BAO Lei,TAO Wei,TAO Qing.Boosting Adversarial Transferability Through Adaptive-Learning-Rate with Data Augmentation Mechanism[J].ACTA ELECTRONICA SINICA,2024,52(01):157-169. DOI： 10.12263/DZXB.20220737.

摘要

深度神经网络具有脆弱性，容易被精心设计的对抗样本攻击.梯度攻击方法在白盒模型上攻击成功率较高，但在黑盒模型上的迁移性较弱.基于Heavy-ball型动量和Nesterov型动量的梯度攻击方法由于在更新方向上考虑了历史梯度信息，提升了对抗样本的迁移性.为了进一步使用历史梯度信息，本文针对收敛性更好的Nesterov型动量方法，使用自适应步长策略代替目前广泛使用的固定步长，提出了一种方向和步长均使用历史梯度信息的迭代快速梯度方法（Nesterov and Adaptive-learning-rate based Iterative Fast Gradient Method，NAI-FGM）.此外，本文还提出了一种线性变换不变性（Linear-transformation Invariant Method，LIM）的数据增强方法.实验结果证实了NAI-FGM攻击方法和LIM数据增强策略相对于同类型方法均具有更高的黑盒攻击成功率.组合NAI-FGM方法和LIM策略生成对抗样本，在常规训练模型上的平均黑盒攻击成功率达到87.8%，在对抗训练模型上的平均黑盒攻击成功率达到57.5%，在防御模型上的平均黑盒攻击成功率达到67.2%，均超过现有最高水平.

Abstract

Deep neural networks are vulnerable to adversarial examples. Gradient based attacks exhibit weak transferability in the black-box setting

though perform well in the white-box situation. The Heavy-ball momentum and Nesterov momentum based attacks boost the transferability for the consideration of gradient history. To further take advantage of the gradient history information

we propose an iterative fast gradient method (NAI-FGM) on Nesterov momentum for its faster convergence property. As the commonly used constant step size is replaced by adaptive step size

NAI-FGM makes use of gradient history information both in step size and gradient direction. Additionally

we propose a new input transformation mechanism named linear-transformation invariant method (LIM). Experimental results demonstrate that NAI-FGM and LIM perform better than the same kind attacks. Besides

the integrated method LI-NAI-FGM could achieve an average rate of 87.8% on commonly trained models

57.5% on adversarial trained models

67.2% on defense models

which are higher than the state-of-the-art results.

关键词

Keywords

references

LANG C B , CHENG G , TU B F , et al . Learning what not to segment: A new perspective on few-shot segmentation [C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2022 : 8047 - 8057 .

TIAN Z , SHEN C H , CHEN H , et al . FCOS: Fully convolutional one-stage object detection [C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2020 : 9626 - 9635 .

GOODFELLOW I J , SHLENS J , SZEGEDY C . Explaining and harnessing adversarial examples [C]// International Conference on Learning Representations . San Diego : ICLR , 2015 : 1 - 11 .

LI Y D , LI L J , WANG L Q , et al . NATTACK: Learning the distributions of adversarial examples for an improved black-box attack on deep neural networks [C]// 2019 International Conference on Machine Learning . Long Beach : ICML , 2019 : 3866 - 3876 .

HUANG Y , KONG A W . Transferable adversarial attack based on integrated gradients [C]// International Conference on Learning Representations . Virtual Conference : Ithaca , 2022 : 1 - 25 .

纪守领 , 杜天宇 , 邓水光 , 等 . 深度学习模型鲁棒性研究综述 [J]. 计算机学报 , 2022 , 45 ( 1 ): 190 - 206 .

JI S L , DU T Y , DENG S G , et al . Robustness certification research on deep learning models: A survey [J]. Chinese Journal of Computers , 2022 , 45 ( 1 ): 190 - 206 . (in Chinese)

WANG X S , HE K . Enhancing the transferability of adversarial attacks through variance tuning [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 1924 - 1933 .

KURAKIN A , GOODFELLOW I J , BENGIO S . Adversarial examples in the physical world [C]// International Conference on Learning Representations , Workshop Track Proceedings . Toulon : Ithaca , 2017 : 1 - 14 .

RUDER S . An overview of gradient descent optimization algorithms [EB/OL]. ( 2016-09-15 )[ 2022-06-27 ]. https://arxiv.org/abs/1609.04747 https://arxiv.org/abs/1609.04747 .

DEFAZIO A , JELASSI S . Adaptivity without compromise: A momentumized, adaptive, dual averaged gradient method for stochastic optimization [EB/OL]. ( 2021-01-26 )[ 2022-06-27 ]. https://arxiv.org/abs/2101.11075 https://arxiv.org/abs/2101.11075 .

陇盛 , 陶蔚 , 张泽东 , 等 . 基于AdaGrad的自适应NAG方法及其最优个体收敛性 [J]. 软件学报 , 2022 , 33 ( 4 ): 1231 - 1243 .

LONG S , TAO W , ZHANG Z D , et al . Adaptive NAG methods based on AdaGrad and its optimal individual convergence [J]. Journal of Software , 2022 , 33 ( 4 ): 1231 - 1243 . (in Chinese)

DONG Y P , LIAO F Z , PANG T Y , et al . Boosting adversarial attacks with momentum [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 9185 - 9193 .

LIN J D , SONG C B , HE K , et al . Nesterov accelerated gradient and scale invariance for adversarial attacks [C]// International Conference on Learning Representations . Virtual Conference : Ithaca , 2020 : 1 - 12 .

POLYAK B T . Some methods of speeding up the convergence of iteration methods [J]. USSR Computational Mathematics and Mathematical Physics , 1964 , 4 ( 5 ): 1 - 17 .

NESTEROV Y E . A method for solving the convex programming problem with convergence rate O (1/ k 2 ) [J]. Proceedings of the USSR Academy of Sciences , 1983 , 269 : 543 - 547 .

NEMIROVSKIĬ A S , I͡UDIN D B . Problem Complexity and Method Efficiency in Optimization [M]. New York : Wiley , 1983 .

SZEGEDY C , VANHOUCKE V , IOFFE S , et al . Rethinking the inception architecture for computer vision [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 2818 - 2826 .

PANG R , SHEN H , ZHANG X Y , et al . A tale of evil twins: Adversarial inputs versus poisoned models [C]// 2020 ACM SIGSAC Conference on Computer and Communications Security . New York : ACM , 2020 : 85 - 99 .

ZHOU W , HOU X , CHEN Y J , et al . Transferable adversarial perturbations [C]// European Conference on Computer Vision . Cham : Springer , 2018 : 471 - 486 .

WU W B , SU Y X , CHEN X X , et al . Boosting the transferability of adversarial samples via attention [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 1158 - 1167 .

GONG C Y , WANG D L , LI M , et al . KeepAugment: A simple information-preserving data augmentation approach [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 1055 - 1064 .

Raphael G , Sylvia S , Ekin D C , et al . Tradeoffs in data augmentation: An empirical study [C]// International Conference for Learning Representations . Virtual Conference : Ithaca , 2021 : 1 - 27 .

XIE S F , LV A , XIA Y C , et al . Target-side input augmentation for sequence to sequence generation [C]// International Conference for Learning Representations . Virtual Conference : Ithaca , 2022 : 1 - 18 .

邹军华 , 段晔鑫 , 任传伦 , 等 . 基于噪声初始化、Adam-Nesterov方法和准双曲动量方法的对抗样本生成方法 [J]. 电子学报 , 2022 , 50 ( 1 ): 207 - 216 .

ZOU J H , DUAN Y X , REN C L , et al . Perturbation initialization, Adam-Nesterov and quasi-hyperbolic momentum for adversarial examples [J]. Acta Electronica Sinica , 2022 , 50 ( 1 ): 207 - 216 . (in Chinese)

XIE C H , ZHANG Z S , ZHOU Y Y , et al . Improving transferability of adversarial examples with input diversity [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 2725 - 2734 .

DONG Y P , PANG T Y , SU H , et al . Evading defenses to transferable adversarial examples by translation-invariant attacks [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 4307 - 4316 .

RUSSAKOVSKY O , DENG J , SU H , et al . ImageNet large scale visual recognition challenge [J]. International Journal of Computer Vision , 2015 , 115 ( 3 ): 211 - 252 .

SZEGEDY C , IOFFE S , VANHOUCKE V , et al . Inception-v4, inception-ResNet and the impact of residual connections on learning [C]// 2017 AAAI Conference on Artificial Intelligence . New York : ACM , 2017 : 4278 - 4284 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 770 - 778 .

TRAMÈR F , KURAKIN A , PAPERNOT N , et al . Ensemble adversarial training: Attacks and defenses [C]// International Conference on Learning Representations . Vancouver : Ithaca , 2018 : 1 - 14 .

JIA X J , WEI X X , CAO X C , et al . ComDefend: An efficient image compression model to defend adversarial examples [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 6077 - 6085 .

XIE C H , ZHANG Z SH , YUILLE A L . Mitigating adversarial effects through randomization [C]// International Conference on Learning Representations . Vancouver : Ithaca , 2018 : 1 - 16 .

LIU Z H , LIU Q , LIU T , et al . Feature distillation: DNN-oriented JPEG compression against adversarial examples [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 860 - 868 .

XU X L , EVANS D , QI Y J . Feature squeezing: Detecting adversarial examples in deep neural networks [C]// Network and Distributed System Security Symposium . San Diego : Internet Society , 2017 : 1 - 15 .

GUO C , RANA M , CISSE M , et al . Countering adversarial images using input transformations [C]// International Conference on Learning Representations . Vancouver : Ithaca , 2018 : 1 - 12 .

陇盛 , 陶蔚 , 张泽东 , 等 . 非光滑强凸情形Adam型算法的最优收敛速率 [J]. 电子学报 , 2022 , 50 ( 9 ): 2049 - 2059 .

LONG S , TAO W , ZHANG Z D , et al . The optimal convergence rate of Adam-type algorithms for non-smooth strongly convex cases [J]. Acta Electronica Sinica , 2022 , 50 ( 9 ): 2049 - 2059 . (in Chinese)

REDDI S J , KALE S , KUMAR S . On the convergence of adam and beyond [C]// International Conference on Learning Representations . Vancouver : Ithaca , 2018 : 1 - 23 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on Category Similarity-Driven Dynamic Fake Target Adversarial Attack Methods

A Highly Transferable Adversarial Example Generation Method via Spatial-Frequency Dual-Domain Feature Fusion

Anti-Customization Adversarial Examples Against Text-to-Image Diffusion Models with Multi-Element Collaboration

A Multi-Layer Feature Importance Attack Method Based on Iterative Accumulated Gradients

Related Author

YU Hong-xia

LU Lei-ji

BAO Lei

CHEN Jun

ZHANG Lin-jun

HAN Shaojie

ZHANG Shihui

ZHAO Pengyu

Related Institution

Department of Information Engineering, PLA Army Services University

Hebei Key laboratory of computer virtual technology and system integration

School of Artificial Intelligence, Yanshan University

School of Cyber Science and Engineering, Wuhan University

School of Cyberspace Security, Guangzhou University

⁰