Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning

LI Teng; TANG Zhi-liang; MA Zhuo; MA Jian-feng

doi:10.12263/DZXB.20230369

您当前的位置：

首页 >

文章列表页 >

Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning

PAPERS | 更新时间：2025-12-08

- Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning
- ACTA ELECTRONICA SINICA Vol. 51, Issue 11, Pages: 3033-3041(2023)
- 作者机构：
  
  西安电子科技大学网络与信息安全学院，陕西西安 710071
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62272370);Young Elite Scientists Sponsorship Program by CAST(2022QNRC001);Shaanxi University Science and Technology Association Youth Talent Promotion Project(20210120)
- DOI：10.12263/DZXB.20230369
  CLC： TP309.1
- Received：21 April 2023，
  
  Revised：2023-09-27，
  
  Published：25 November 2023
- 稿件说明：
移动端阅览
李腾,唐智亮,马卓等.基于强化学习的自免疫动态攻击生成方法[J].电子学报,2023,51(11):3033-3041.

LI Teng,TANG Zhi-liang,MA Zhuo,et al.Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning[J].ACTA ELECTRONICA SINICA,2023,51(11):3033-3041.
李腾,唐智亮,马卓等.基于强化学习的自免疫动态攻击生成方法[J].电子学报,2023,51(11):3033-3041. DOI： 10.12263/DZXB.20230369.

LI Teng,TANG Zhi-liang,MA Zhuo,et al.Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning[J].ACTA ELECTRONICA SINICA,2023,51(11):3033-3041. DOI： 10.12263/DZXB.20230369.

摘要

通过最优路径发动网络攻击的

方式已成为影响各企业、机构内部网络安全的重要因素.现有的针对内部网络探寻最优攻击路径大多是采用攻击图的方式实现，未考虑攻击代价和攻击利益的关系，已有的利用

-learning算法机制解决攻击路径的方法存在网络脆弱性信息利用率低的问题.为解决这些问题，本文借鉴生物免疫机制提出了一种基于强化学习的自免疫动态攻击生成方法，模拟攻击者对内网的网络攻击，从而高效地发现内部网络中存在的脆弱点，实现自免疫防御.方案首先对内部网络信息进行窃取并加以处理，在攻击图的有向边上附加权值，然后通过改进的

-learning算法寻找最优攻击路径，实现全部最优攻击路径的获取，并返回最优攻击路径的攻击图和内部网络主机脆弱性分析结果.通过理论分析和实验结果表明，该方法兼顾寻找最优攻击路径的高效性、准确性的同时，还解决了最优攻击路径中存在环型回路、多条最优攻击路径的问题，充分利用内部网络脆弱性，提升自免疫安全防护能力.

Abstract

The approach of launching network attacks through optimal pathways has become a significant factor affecting the internal network security of various enterprises and organizations. Existing methods for exploring optimal attack pathways within internal networks mostly rely on attack graphs and often neglect the relationship between attack costs and benefits. Methods that utilize the

-learning algorithm to address attack pathways suffer from low utilization of network vulnerability information. To address these issues

this paper draws inspiration from the biological immune system and proposes a reinforcement learning-based dynamic self-immune attack generation method. This method simulates network attacks by intruders on an internal network

efficiently uncovering vulnerabilities within the internal network

thereby achieving self-immune defense. The proposed approach first acquires and processes internal network information

attaches weights to directed edges in the attack graph

and then employs an improved

-learning algorithm to discover optimal attack pathways. It successfully identifies all optimal attack pathways

providing attack graphs and an analysis of host vulnerabilities within these pathways. Theoretical analysis and experimental results demonstrate that this method not only efficiently and accurately identifies optimal attack pathways but also resolves issues such as ring loops and multiple optimal attack pathways. By making full use of internal network vulnerabilitie

it enhances self-immune security defenses.

关键词

Keywords

references

胡浩 , 叶润国 , 张红旗 , 等 . 基于攻击预测的网络安全态势量化方法 [J ] . 通信学报 , 2017 , 38 ( 10 ): 122 - 134 .

HU H , YE R G , ZHANG H Q , et al . Quantitative method for network security situation based on attack prediction [J ] . Journal on Communications , 2017 , 38 ( 10 ): 122 - 134 . (in Chinese)

闫峰 , 刘淑芬 , 冷煌 . 基于转换的攻击图分析方法研究 [J ] . 电子学报 , 2014 , 42 ( 12 ): 2477 - 2480 .

YAN F , LIU S F , LENG H . Study on analysis of attack graphs based on conversion [J ] . Acta Electronica Sinica , 2014 , 42 ( 12 ): 2477 - 2480 . (in Chinese)

叶子维 , 郭渊博 , 王宸东 , 等 . 攻击图技术应用研究综述 [J ] . 通信学报 , 2017 , 38 ( 11 ): 121 - 132 .

YE Z W , GUO Y B , WANG C D , et al . Survey on application of attack graph technology [J ] . Journal on Communications , 2017 , 38 ( 11 ): 121 - 132 . (in Chinese)

DACIER M , DESWARTE Y . Privilege Graph: An extension to the typed access matrix model [M ] // Computer Security — ESORICS 94 . Berlin : Springer , 1994 : 319 - 334 .

WANG L Y , YAO C , SINGHAL A , et al . Interactive analysis of attack graphs using relational queries [M ] // Data and Applications Security XX . Berlin : Springer , 2006 : 119 - 132 .

RAMAKRISHNAN C R , SEKAR R . Model-based analysis of configuration vulnerabilities1 [J ] . Journal of Computer Security , 2002 , 10 ( 1/2 ): 189 - 209 .

RITCHEY R W , AMMANN P . Using model checking to analyze network vulnerabilities [C ] // Proceeding 2000 IEEE Symposium on Security and Privacy . Piscataway : IEEE , 2002 : 156 - 165 .

臧艺超 , 周天阳 , 朱俊虎 , 等 . 领域独立智能规划技术及其面向自动化渗透测试的攻击路径发现研究进展 [J ] . 电子与信息学报 , 2020 , 42 ( 9 ): 2095 - 2107 .

ZANG Y C , ZHOU T Y , ZHU J H , et al . Domain-independent intelligent planning technology and its application to automated penetration testing oriented attack path discovery [J ] . Journal of Electronics & Information Technology , 2020 , 42 ( 9 ): 2095 - 2107 . (in Chinese)

李庆朋 , 王布宏 , 王晓东 , 等 . 基于最优攻击路径的网络安全增强策略研究 [J ] . 计算机科学 , 2013 , 40 ( 4 ): 152 - 154 .

LI Q P , WANG B H , WANG X D , et al . Approach on network security enhancement strategies based on optimal attack path [J ] . Computer Science , 2013 , 40 ( 4 ): 152 - 154 . (in Chinese)

杨本毅 . 基于攻击图的渗透测试方法 [J ] . 电子科技 , 2019 , 32 ( 10 ): 75 - 78 .

YANG B Y . Research on corrosion detection technology of power system grounding grid [J ] . Electronic Science and Technology , 2019 , 32 ( 10 ): 75 - 78 . (in Chinese)

NGUYEN T T , REDDI V J . Deep reinforcement learning for cyber security [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 8 ): 3779 - 3795 .

KAELBLING L P , LITTMAN M L , MOORE A W . Reinforcement learning: A survey [J ] . Journal of Artificial Intelligence Research , 1996 , 4 : 237 - 285 .

CODY T , RAHMAN A , REDINO C , et al . Discovering exfiltration paths using reinforcement learning with attack graphs [C ] // 2022 IEEE Conference on Dependable and Secure Computing (DSC) . Piscataway : IEEE , 2022 : 1 - 8 .

曾庆伟 , 张国敏 , 邢长友 , 等 . 基于分层强化学习的智能化攻击路径发现方法 [J ] . 计算机科学 , 2023 , 50 ( 7 ): 308 - 316 .

ZENG Q W , ZHANG G M , XING C Y , et al . Intelligent attack path discovery based on hierarchical reinforcement learning [J ] . Computer Science , 2023 , 50 ( 7 ): 308 - 316 . (in Chinese)

CLIFTON J , LABER E . Q -learning: Theory and applications [J ] . Annual Review of Statistics and Its Application , 2020 , 7 : 279 - 301 .

李腾 , 曹世杰 , 尹思薇 , 等 . 应用 Q 学习决策的最优攻击路径生成方法 [J ] . 西安电子科技大学学报 , 2021 , 48 ( 1 ): 160 - 167 .

LI T , CAO S J , YIN S W , et al . Optimal method for the generation of the attack path based on the Q -learning decision [J ] . Journal of Xidian University , 2021 , 48 ( 1 ): 160 - 167 . (in Chinese)

胡昌振 , 陈韵 , 吕坤 . 一种基于 Q 学习的最佳攻击路径规划方法 : CN107317756A [P ] . 2017-11-03 .

YOUSEFI M , MTETWA N , ZHANG Y , et al . A reinforcement learning approach for attack graph analysis [C ] // 2018 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications / 12th IEEE International Conference on Big Data Science and Engineering (TrustCom/BigDataSE) . Piscataway : IEEE , 2018 : 212 - 217 .

JANG B , KIM M , HARERIMANA G , et al . Q -learning algorithms: A comprehensive classification and applications [J ] . IEEE Access , 2019 , 7 : 133653 - 133667 .

胡玮 . 网络拓扑自动发现 [D ] . 成都 : 电子科技大学 , 2012 .

HU W . Automatic Discovery of Network Topology [D ] . Chengdu : University of Electronic Science and Technology of China , 2012 . (in Chinese)

刘杰 , 王清贤 , 罗军勇 . 一种基于ICMP的逻辑层网络拓扑发现与分析方法 [J ] . 计算机应用 , 2008 , 28 ( 6 ): 1498 - 1500 .

LIU J , WANG Q X , LUO J Y . ICMP-based method for logical network topology discovery and analysis [J ] . Journal of Computer Applications , 2008 , 28 ( 6 ): 1498 - 1500 . (in Chinese)

PANSIOT J J , GRAD D . On routes and multicast trees in the Internet [J ] . ACM SIGCOMM Computer Communication Review , 1998 , 28 ( 1 ): 41 - 50 .

DONNET B , FRIEDMAN T . Internet topology discovery: A survey [J ] . IEEE Communications Surveys & Tutorials , 2007 , 9 ( 4 ): 56 - 69 .

陈晋音 , 胡书隆 , 邢长友 , 等 . 面向智能渗透攻击的欺骗防御方法 [J ] . 通信学报 , 2022 , 43 ( 10 ): 106 - 120 .

CHEN J Y , HU S L , XING C Y , et al . Deception defense method against intelligent penetration attack [J ] . Journal on Communications , 2022 , 43 ( 10 ): 106 - 120 . (in Chinese)

张涛 , 张文涛 , 代凌 , 等 . 基于序贯博弈多智能体强化学习的综合模块化航空电子系统重构方法 [J ] . 电子学报 , 2022 , 50 ( 4 ): 954 - 966 .

ZHANG T , ZHANG W T , DAI L , et al . Integrated modular avionics system reconstruction method based on sequential game multi-agent reinforcement learning [J ] . Acta Electronica Sinica , 2022 , 50 ( 4 ): 954 - 966 . (in Chinese)

WU Z N , TIAN L Q , WANG Y , et al . Network security defense decision-making method based on stochastic game and deep reinforcement learning [J ] . Security and Communication Networks , 2021 , 2021 : 1 - 13 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Improved PBS Multi-Agent Path Planning Algorithm Based on Conflict Cost Bayesian Weighting

CS-ROMF:Efficient Community Search Based on Graph Combinatorial Optimization

Knowledge-Based and Data-Driven Integrating Design Methodology for Air Combat Strategy in Multi-Opponent Adversarial Game

Reinforcement Learning Based Tuning-free Plug-and-Play Image Reconstruction Method for Single Photon Imaging

Related Author

LI Teng

TANG Zhi-liang

MA Zhuo

MA Jian-feng

QIAN Cheng-ze

MAO Jian-lin

LI Rui-qi

ZHOU Wen-na

Related Institution

School of Cyber Engineering， Xidian University， Xi’ an

Faculty of Information Engineering and Automation, Kunming University of Science and Technology

Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology

Beijing Information Science and Technology University

Hong Kong Baptist University, Hongkong

⁰