基于噪声初始化、Adam-Nesterov方法和准双曲动量方法的对抗样本生成方法

doi:10.12263/DZXB.20200839

电子学报 ›› 2022, Vol. 50 ›› Issue (1): 207-216.DOI: 10.12263/DZXB.20200839

基于噪声初始化、Adam-Nesterov方法和准双曲动量方法的对抗样本生成方法

邹军华¹, 段晔鑫¹^,², 任传伦³, 邱俊洋⁴, 周星宇¹, 潘志松¹

^1.陆军工程大学指挥控制工程学院，江苏南京 210007
^2.陆军军事交通学院镇江校区，江苏镇江 212003
^3.华北计算技术研究所，北京 100083
^4.江南计算所数字工程与先进计算国家重点实验室，江苏无锡 214083

收稿日期:2020-08-04 修回日期:2021-01-22 出版日期:2022-01-25
- 作者简介:
- 邹军华男，1991年12月出生，广东河源人.2017年获解放军理工大学硕士学位.现为陆军工程大学在读博士.研究方向为对抗学习.E-mail：278287847@qq.com
  潘志松（通信作者）男，1973年3月出生，江苏南京人.2003年获南京航空航天大学博士学位.现为陆军工程大学教授，博士生导师.研究方向为人工智能、模式识别.E-mail：panzs@nuaa.edu.cn
- 基金资助:
- 国家自然科学基金 (62076251)

Perturbation Initialization, Adam-Nesterov and Quasi-Hyperbolic Momentum for Adversarial Examples

ZOU Jun-hua¹, DUAN Ye-xin¹^,², REN Chuan-lun³, QIU Jun-yang⁴, ZHOU Xing-yu¹, PAN Zhi-song¹

^1.Command and Control Engineering College, Army Engineering University of PLA, Nanjing, Jiangsu 210007, China
^2.Zhenjiang Campus, Army Military Transportation University of PLA, Zhenjiang, Jiangsu 212003, China
^3.North China Institute of Computer Technology, Beijing 100083, China
^4.Mathematical Engineering and Advanced Computing, Jiangnan Institute of Computing Technology, Wuxi, Jiangsu 214083, China

Received:2020-08-04 Revised:2021-01-22 Online:2022-01-25 Published:2022-01-25
- Supported by:
- National Natural Science Foundation of China (62076251)

摘要/Abstract

摘要：

深度神经网络在多种模式识别任务上都取得了巨大突破，但相关研究表明深度神经网络存在脆弱性，容易被精心设计的对抗样本攻击.本文以分类任务为着手点，研究对抗样本的迁移性，提出基于噪声初始化、Adam-Nesterov方法和准双曲动量方法的对抗样本生成方法.本文提出一种对抗噪声的初始化方法，通过像素偏移方法来预先增强干净样本的攻击性能.同时，本文使用Adam-Nesterov方法和准双曲动量方法来改进现有方法中的Nesterov方法和动量方法，实现更高的黑盒攻击成功率.在不需要额外运行时间和运算资源的情况下，本文方法可以和其他的攻击方法组合，并显著提高了对抗样本的黑盒攻击成功率.实验表明，本文的最强攻击组合为ANI-TI-DIQHM*（其中*代表噪声初始化），其对经典防御方法的平均黑盒攻击成功率达到88.68%，对较为先进的防御方法的平均黑盒攻击成功率达到82.77%，均超过现有最高水平.

关键词: 对抗样本, Adam-Nesterov方法, 准双曲动量方法, 噪声初始化, 迁移性能

Abstract:

Deep neural networks(DNNs) have made great breakthrough in many pattern recognition tasks. However, relevant research shows that the DNNs are vulnerable to adversarial examples. In this paper, we study the transferability of adversarial examples in the classification task, and propose perturbation initialization, the quasi-hyperbolic momentum iterative fast gradient sign method(QHMI-FGSM) and the adam-nesterov iterative fast gradient sign method(ANI-FGSM). We propose perturbation initialization method called pixel shift in adversarial attack. Furthermore, QHMI-FGSM and ANI-FGSM proposed in this paper are the improvements on the existing momentum iterative fast gradient sign method(MI-FGSM) and nesterov iterative fast gradient sign method(NI-FGSM). Additionally, perturbation initialization, QHMI-FGSM and ANI-FGSM are easily integrated into other existing methods, which can significantly improve the success rates of black-box attacks without additional running time and computing resources. Experimental results show that our best attack ANI-TI-DIQHM* can fool six classic black-box defense models with an average success rate of 88.68%, and fool four advance black-box defense models with an average success rate of 82.77%, which are higher than the state-of-the-art results.

Key words: adversarial examples, Adam-Nesterov method, quasi-hyperbolic momentum method, perturbation initialization, transferability

中图分类号:

TP391

邹军华, 段晔鑫, 任传伦, 邱俊洋, 周星宇, 潘志松. 基于噪声初始化、Adam-Nesterov方法和准双曲动量方法的对抗样本生成方法[J]. 电子学报, 2022, 50(1): 207-216.

ZOU Jun-hua, DUAN Ye-xin, REN Chuan-lun, QIU Jun-yang, ZHOU Xing-yu, PAN Zhi-song. Perturbation Initialization, Adam-Nesterov and Quasi-Hyperbolic Momentum for Adversarial Examples[J]. Acta Electronica Sinica, 2022, 50(1): 207-216.

图/表 10

参考文献 26

1	HEK M， ZHANGX G， RENS Q， et al. Identity mappings in deep residual networks［C］//2016 14th European Conference on Computer Vision. Amsterdam，NEP： Springer， 2016： 630-645.
2	HEK M， GKIOXARIG， DOLLÁRP， et al. Mask R-CNN［C］// 2017 IEEE International Conference on Computer Vision. Venice，Italy： IEEE Computer Society， 2017： 2980-2988.
3	GOODFELLOWI J， SHLENSJ， SZEGEDYC. Explaining and harnessing adversarial examples［C］//2015 3rd International Conference on Learning Representations. San Diego，USA： Conference Track Proceedings， 2015： 1-11.
4	LIUY P， CHENX Y， LIUC， et al. Delving into transferable adversarial examples and black-box attacks［C］//2017 5th International Conference on Learning Representations. Toulon，France： Conference Track Proceedings， 2017： 1-24.
5	ATHALYEA， ENGSTROML， ILYASA， et al. Synthesizing robust adversarial examples［C］//2018 35th International Conference on Machine Learning. Stockholmsmässan，SWE： Proceedings of Machine Learning Research， 2018： 284-293.
6	TRAMÈRF， KURAKINA， PAPERNOTN， et al. Ensemble adversarial training： Attacks and defenses［C］//2018 6th International Conference on Learning Representations. Vancouver，CAN： Conference Track Proceedings， 2018： 1-22.
7	LIAOF Z， LIANGM， DONGY P， et al. Defense against adversarial attacks using high-level representation guided denoiser［C］//2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City，USA： IEEE Computer Society， 2018： 1778-1787.
8	XIEC H， WANGJ Y， ZHANGZ S， et al. Mitigating adversarial effects through randomization［C］//2018 6th International Conference on Learning Representations. Vancouver，CAN： Conference Track Proceedings， 2018： 1-16.
9	RAGHUNATHANA， STEINHARDTJ， LIANGP. Certified defenses against adversarial examples［C］//2018 6th International Conference on Learning Representations. Vancouver，CAN： Conference Track Proceedings， 2018： 1-15.
10	RAUBERJ， BRENDELW， BETHGEM. Foolbox v0.8.0： A python toolbox to benchmark the robustness of machine learning models［J/OL］. ［2020］. .
11	DONGY P， LIAOF Z， PANGT Y， et al. Boosting adversarial attacks with momentum［C］//2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City，USA： IEEE Computer Society， 2018： 9185-9193.
12	NARODYTSKAN， KASIVISWANATHANS P. Simple black-box adversarial perturbations for deep networks［J/OL］. （2016-12-19）［2020］..
13	CHENJ B， JORDANM I. Boundary attack++： Query-efficient decision-based adversarial attack［J/OL］. ［2020］. .
14	LINJ D， SONGC B， HEK， et al. Nesterov accelerated gradient and scale invariance for improving transferability of adversarial examples［J/OL］. ［2020］. .
15	XIEC H， ZHANGZ S， ZHOUY Y， et al. Improving transferability of adversarial examples with input diversity［C］//2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach，USA： Computer Vision Foundation， 2019： 2730-2739.
16	DONGY P， PANGT Y， SUH， et al. Evading defenses to transferable adversarial examples by translation-invariant attacks［C］//2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach，USA： Computer Vision Foundation， 2019： 4312-4321.
17	KURAKINA， GOODFELLOWIAN J.， BENGIOSAMY. Adversarial examples in the physical world［C］//2017 5th International Conference on Learning Representations. Toulon，France： Conference Track Proceedings， 2017： 1-14.
18	MAJ， YARATSD. Quasi-hyperbolic momentum and adam for deep learning［C］//2019 7th International Conference on Learning Representations. New Orleans，USA： Conference Track Proceedings， 2019： 1-38.
19	贾熹滨，史佳帅. Ada_Nesterov动量方法——种具有自适应学习率的Nesterov动量方法［J］. 计算机科学与应用，2019，9：351-358.
	JIAX B， SHIJ S. Ada_Nesterov momentum algorithm—the nesterov momentum algorithm with adaptive learning rate［J］. Computer Science and Application， 2019， 9： 351-358. （in Chinese）
20	KINGMAD P， BAJ. Adam： A method for stochastic optimization［C］//2015 3rd International Conference on Learning Representations. San Diego，USA： Conference Track Proceedings， 2015： 1-15.
21	DUCHI， JOHN， HAZAN， et al. Adaptive subgradient methods for online learning and stochastic optimization［J］. The Journal of Machine Learning Research， 2011， 12： 2121-2159.
22	SZEGEDYC， VANHOUCKEV， IOFFES， et al. Rethinking the inception architecture for computer vision［C］//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas，USA： IEEE Computer Society， 2016： 2818-2826.
23	SZEGEDYC， IOFFES， VANHOUCKEV， et al. Alemi. Inception-v4， inception-resnet and the impact of residual connections on learning［C］//2017 31st AAAI Conference on Artificial Intelligence. San Francisco，USA： AAAI Press， 2017： 4278-4284.
24	LIUZ H， LIUQ， LIUT， et al. Feature distillation： DNN-Oriented JPEG compression against adversarial examples［C］//2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach，USA： Computer Vision Foundation， 2019： 860-868.
25	JIAX J， WEIX X， CAOX C， et al. ComDefend： An efficient image compression model to defend adversarial examples［C］//2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach，USA： Computer Vision Foundation， 2019： 6084-6092.
26	COHENJ M， ROSENFELDE， KOLTERJ Z. Certified adversarial robustness via randomized smoothing［C］//2019 36th International Conference on Machine Learning ICML. Long Beach，USA： Proceedings of Machine Learning Research， 2019： 1310-1320.

攻击组合简称	定义
TI-DIM（基准方法）	TIM，DIM和MI-FGSM的组合
TI-DIQHM	TIM，DIM和QHMI-FGSM的组合
TI-DIQHM^*	TIM，DIM，QHMI-FGSM和噪声初始化的组合
NI-TI-DIM（基准方法）	TIM，DIM，NI-FGSM和MI-FGSM的组合
ADNI-TI-DIQHM	TIM，DIM，ADNI-FGSM和QHMI-FGSM的组合
ANI-TI-DIQHM	TIM，DIM，ANI-FGSM和QHMI-FGSM的组合
ANI-TI-DIQHM^*	TIM，DIM，ANI-FGSM，QHMI-FGSM和噪声初始化的组合
SI-NI-TI-DIM	SIM，TIM，DIM，NI-FGSM和MI-FGSM的组合

攻击组合简称	定义
TI-DIM（基准方法）	TIM，DIM和MI-FGSM的组合
TI-DIQHM	TIM，DIM和QHMI-FGSM的组合
TI-DIQHM^*	TIM，DIM，QHMI-FGSM和噪声初始化的组合
NI-TI-DIM（基准方法）	TIM，DIM，NI-FGSM和MI-FGSM的组合
ADNI-TI-DIQHM	TIM，DIM，ADNI-FGSM和QHMI-FGSM的组合
ANI-TI-DIQHM	TIM，DIM，ANI-FGSM和QHMI-FGSM的组合
ANI-TI-DIQHM^*	TIM，DIM，ANI-FGSM，QHMI-FGSM和噪声初始化的组合
SI-NI-TI-DIM	SIM，TIM，DIM，NI-FGSM和MI-FGSM的组合

攻击组合	Inc-v3	Inc-v4	IncRes-v2	Res-v2-101	多模型集成
TI-DIM	171.3	266.7	275.1	237.9	766.9
TI-DIQHM	169.2	259.3	279.5	239.5	761.4
TI-DIQHM^*	180.9	273.4	290.7	231.7	789.3
NI-TI-DIM	192.5	239.7	289.1	239.5	813.4
ADNI-TI-DIQHM	181.6	231.0	285.9	241.5	805.9
ANI-TI-DIQHM	189.3	224.8	273.2	251.6	823.7
ANI-TI-DIQHM^*	194.2	231.5	285.3	263.8	832.6
SI-NI-TI-DIM	602.7	1083.5	1158.3	1091.1	3491.4

攻击组合	Inc-v3	Inc-v4	IncRes-v2	Res-v2-101	多模型集成
TI-DIM	171.3	266.7	275.1	237.9	766.9
TI-DIQHM	169.2	259.3	279.5	239.5	761.4
TI-DIQHM^*	180.9	273.4	290.7	231.7	789.3
NI-TI-DIM	192.5	239.7	289.1	239.5	813.4
ADNI-TI-DIQHM	181.6	231.0	285.9	241.5	805.9
ANI-TI-DIQHM	189.3	224.8	273.2	251.6	823.7
ANI-TI-DIQHM^*	194.2	231.5	285.3	263.8	832.6
SI-NI-TI-DIM	602.7	1083.5	1158.3	1091.1	3491.4

QHMI-FGSM	ANI-FGSM	初始化	Inc-v3ens3	Inc-v3ens4	IncResv2ens
			85.5	84.9	79.9
√			88.0	86.0	81.4
√	√		89.6	87.7	84.3
√	√	√	91.1	88.7	84.5

基于噪声初始化、Adam-Nesterov方法和准双曲动量方法的对抗样本生成方法

Perturbation Initialization, Adam-Nesterov and Quasi-Hyperbolic Momentum for Adversarial Examples

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 26

相关文章 15

编辑推荐

Metrics

	攻击组合	Inc-v3ens3	Inc-v3ens4	IncResv2ens	HGD	R&P	NIPS-r3
Inc-v3	TI-DIM	46.7	47.1	38.6	38.3	36.2	41.5
	TI-DIQHM	50.3	50.7	38.9	38.5	37.2	43.6
	TI-DIQHM^*	54.4	54.0	39.6	40.1	39.5	45.6
Inc-v4	TI-DIM	48.3	47.7	39.4	40.7	39.1	41.3
	TI-DIQHM	52.9	52.2	40.8	42.3	41.9	43.1
	TI-DIQHM^*	56.2	57.1	45.5	46.8	45.7	48.4
IncRes-v2	TI-DIM	60.5	59.3	59.3	58.4	57.5	61.4
	TI-DIQHM	66.0	62.4	62.4	61.9	59.3	63.9
	TI-DIQHM^*	70.6	69.2	66.6	65.4	63.8	69.3
Res-v2-101	TI-DIM	56.3	55.5	49.1	51.3	50.6	52.1
	TI-DIQHM	59.8	58.6	51.1	52.9	51.5	54.4
	TI-DIQHM^*	64.0	62.4	55.4	55.9	54.1	59.5

	攻击组合	Inc-v3ens3	Inc-v3ens4	IncResv2ens	HGD	R&P	NIPS-r3
Inc-v3	NI-TI-DIM	49.2	49.1	37.1	37.9	35.6	41.3
	ADNI-TI-DIQHM	50.1	49.8	37.6	38.5	36.9	42.1
	ANI-TI-DIQHM	53.0	52.2	37.1	39.2	37.9	42.8
	ANI-TI-DIQHM^*	53.5	51.4	37.7	39.5	38.1	43.7
Inc-v4	NI-TI-DIM	49.6	51.0	37.2	37.1	36.5	41.3
	ADNI-TI-DIQHM	50.3	51.5	38.4	38.8	37.2	42.9
	ANI-TI-DIQHM	53.8	51.9	42.1	41.6	40.3	43.5
	ANI-TI-DIQHM^*	54.2	54.2	42.4	42.5	41.9	44.2
IncRes-v2	NI-TI-DIM	64.7	63.9	61.7	62.1	60.9	64.5
	ADNI-TI-DIQHM	66.5	64.9	62.5	63.4	62.9	65.2
	ANI-TI-DIQHM	68.7	66.5	66.7	65.1	64.0	67.5
	ANI-TI-DIQHM^*	68.9	67.6	67.0	66.9	65.8	68.3
Res-v2-101	NI-TI-DIM	59.2	58.7	50.0	51.2	49.7	57.6
	ADNI-TI-DIQHM	60.0	60.5	52.9	53.9	51.3	58.7
	ANI-TI-DIQHM	65.1	62.1	54.0	55.9	52.1	59.6
	ANI-TI-DIQHM^*	64.0	63.7	55.6	56.6	53.2	60.4

攻击组合	Inc-v3ens3	Inc-v3ens4	IncRes-v2ens	HGD	R&P	NIPS-r3	平均
TI-DIM	83.9	83.2	78.4	81.9	81.2	83.6	82.03
TI-DIQHM	86.5	84.8	80.9	83.1	82.5	84.1	83.65
TI-DIQHM^*	89.0	86.8	83.5	87.1	85.4	88.9	86.78
NI-TI-DIM	86.4	84.9	81.5	83.7	82.9	84.1	83.95
ADNI-TI-DIQHM	89.2	86.6	83.8	86.9	85.7	88.8	87.17
ANI-TI-DIQHM	89.6	87.7	84.3	87.3	86.3	89.8	87.50
ANI-TI-DIQHM^*	91.1	88.7	84.5	88.9	87.7	91.2	88.68

攻击组合	Feature Distillation	Comdefend	Randomized Smoothing	平均
TI-DIM	83.1	78.2	49.9	70.40
TI-DIQHM	84.3	86.9	59.2	76.80
TI-DIQHM^*	89.9	88.1	63.1	80.37
NI-TI-DIM	82.1	84.7	58.6	75.13
ADNI-TI-DIQHM	88.9	85.8	62.9	79.20
ANI-TI-DIQHM	90.3	88.5	64.5	81.10
ANI-TI-DIQHM^*	91.2	89.7	67.4	82.77

[1]	王子为, 鲁继文, 周杰. 基于自适应梯度优化的二值神经网络[J]. 电子学报, 2023, (): 1-10.
[2]	刘金平, 吴娟娟, 张荣, 徐鹏飞. 基于结构重参数化与多尺度深度监督的COVID-19胸部CT图像自动分割[J]. 电子学报, 2023, (): 1-9.
[3]	张笑宇, 沈超, 蔺琛皓, 李前, 王骞, 李琦, 管晓宏. 面向机器学习模型安全的测试与修复[J]. 电子学报, 2023, (): 1-35.
[4]	王炼红, 罗志辉, 林飞鹏, 李潇瑶. 采用多头注意力机制的C&RM-MAKT预测算法[J]. 电子学报, 2022, (): 1-9.
[5]	苏田田, 王慧敏, 张小凤. 基于多分支瓶颈结构的轻量型图像分类算法研究[J]. 电子学报, 2022, (): 1-9.
[6]	刘芳, 朱天贺, 苏卫星, 刘阳. 基于高斯隐马尔可夫模型的人机共享控制区域化决策算法[J]. 电子学报, 2022, 50(11): 2659-2667.
[7]	桑海峰, 陈旺兴, 王海峰, 王金玉. 基于多模式时空交互的行人轨迹预测模型[J]. 电子学报, 2022, 50(11): 2806-2812.
[8]	刘耿耿, 李泽鹏, 郭文忠, 陈国龙, 徐宁. 面向超大规模集成电路物理设计的通孔感知的并行层分配算法[J]. 电子学报, 2022, 50(11): 2575-2583.
[9]	魏博文, 全红艳. 基于语义与形态特征融合的语义分割网络[J]. 电子学报, 2022, 50(11): 2688-2697.
[10]	姚睿, 朱享彬, 周勇, 王鹏, 张艳宁, 赵佳琦. 基于重要特征的视觉目标跟踪可迁移黑盒攻击方法[J]. 电子学报, 2022, (): 1-10.
[11]	金紫凤, 潘思聪, 危辉. 可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法[J]. 电子学报, 2022, 50(10): 2318-2328.
[12]	魏钰轩, 陈莹. 基于自适应层信息熵的卷积神经网络压缩[J]. 电子学报, 2022, 50(10): 2398-2408.
[13]	马百腾, 张士伟, 高常鑫, 桑农. 面向行为边界框生成的端到端时间全局相关网络[J]. 电子学报, 2022, 50(10): 2452-2461.
[14]	肖斌, 陈嘉博, 毕秀丽, 张俊辉, 李伟生, 王国胤, 马旭. 基于一维卷积神经网络与循环神经网络串联的心音分析方法[J]. 电子学报, 2022, 50(10): 2425-2432.
[15]	周登文, 李文斌, 李金新, 黄志勇. 一种轻量级的多尺度通道注意图像超分辨率重建网络[J]. 电子学报, 2022, 50(10): 2336-2346.