基于强化学习的自免疫动态攻击生成方法

李腾; 唐智亮; 马卓; 马建峰

doi:10.12263/DZXB.20230369

您当前的位置：

首页 >

文章列表页 >

基于强化学习的自免疫动态攻击生成方法

学术论文 | 更新时间：2025-12-08

- 基于强化学习的自免疫动态攻击生成方法
- Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning
- 电子学报 2023年51卷第11期页码：3033-3041
- 作者机构：
  
  西安电子科技大学网络与信息安全学院，陕西西安 710071
- 作者简介：
  
  [ "李腾男，1991年1月出生于江苏省徐州市.现为西安电子科技大学网络与信息安全学院副教授、博士生导师.主要研究领域包括攻击检测与溯源、无人机系统和网络安全、漏洞挖掘与修复等.中国电子学会会员编号：E190029987M.E-mail: tengli@xidian.edu.cn" ]
  [ "唐智亮男，2001年2月出生于安徽省铜陵市.现为西安电子科技大学网络与信息安全学院硕士研究生，研究方向为网络与信息安全.E-mail: 1719468161@qq.com" ]
  [ "马卓（通讯作者）男，1980年12月出生于陕西省延安市，现为西安电子科技大学网络与信息安全学院华山学者特聘教授、博士生导师，主要从事人工智能、无人系统安全、隐私计算、无线网络安全等方面的研究.中国电子学会会员编号：E190029397M." ]
  [ "马建峰男，1963年出生于陕西省西安市，现为西安电子科技大学网络与信息安全学院教授，博士生导师，研究方向为网络安全、系统安全、数据安全和无人机安全.中国电子学会会员编号：E190004733F. E-mail: jfma@mail.xidian.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(62272370);中国科协青年人才托举工程(2022QNRC001);陕西省科学技术协会青年人才托举计划(20210120)
- DOI：10.12263/DZXB.20230369
  中图分类号： TP309.1
- 收稿：2023-04-21，
  
  修回：2023-09-27，
  
  纸质出版：2023-11-25
- 稿件说明：
移动端阅览
李腾,唐智亮,马卓等.基于强化学习的自免疫动态攻击生成方法[J].电子学报,2023,51(11):3033-3041.

LI Teng,TANG Zhi-liang,MA Zhuo,et al.Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning[J].ACTA ELECTRONICA SINICA,2023,51(11):3033-3041.
李腾,唐智亮,马卓等.基于强化学习的自免疫动态攻击生成方法[J].电子学报,2023,51(11):3033-3041. DOI： 10.12263/DZXB.20230369.

LI Teng,TANG Zhi-liang,MA Zhuo,et al.Autoimmune Dynamic Attack Generation Method Based on Reinforcement Learning[J].ACTA ELECTRONICA SINICA,2023,51(11):3033-3041. DOI： 10.12263/DZXB.20230369.

摘要

通过最优路径发动网络攻击的

方式已成为影响各企业、机构内部网络安全的重要因素.现有的针对内部网络探寻最优攻击路径大多是采用攻击图的方式实现，未考虑攻击代价和攻击利益的关系，已有的利用

-learning算法机制解决攻击路径的方法存在网络脆弱性信息利用率低的问题.为解决这些问题，本文借鉴生物免疫机制提出了一种基于强化学习的自免疫动态攻击生成方法，模拟攻击者对内网的网络攻击，从而高效地发现内部网络中存在的脆弱点，实现自免疫防御.方案首先对内部网络信息进行窃取并加以处理，在攻击图的有向边上附加权值，然后通过改进的

-learning算法寻找最优攻击路径，实现全部最优攻击路径的获取，并返回最优攻击路径的攻击图和内部网络主机脆弱性分析结果.通过理论分析和实验结果表明，该方法兼顾寻找最优攻击路径的高效性、准确性的同时，还解决了最优攻击路径中存在环型回路、多条最优攻击路径的问题，充分利用内部网络脆弱性，提升自免疫安全防护能力.

Abstract

The approach of launching network attacks through optimal pathways has become a significant factor affecting the internal network security of various enterprises and organizations. Existing methods for exploring optimal attack pathways within internal networks mostly rely on attack graphs and often neglect the relationship between attack costs and benefits. Methods that utilize the

-learning algorithm to address attack pathways suffer from low utilization of network vulnerability information. To address these issues

this paper draws inspiration from the biological immune system and proposes a reinforcement learning-based dynamic self-immune attack generation method. This method simulates network attacks by intruders on an internal network

efficiently uncovering vulnerabilities within the internal network

thereby achieving self-immune defense. The proposed approach first acquires and processes internal network information

attaches weights to directed edges in the attack graph

and then employs an improved

-learning algorithm to discover optimal attack pathways. It successfully identifies all optimal attack pathways

providing attack graphs and an analysis of host vulnerabilities within these pathways. Theoretical analysis and experimental results demonstrate that this method not only efficiently and accurately identifies optimal attack pathways but also resolves issues such as ring loops and multiple optimal attack pathways. By making full use of internal network vulnerabilitie

it enhances self-immune security defenses.

关键词

Keywords

references

胡浩 , 叶润国 , 张红旗 , 等 . 基于攻击预测的网络安全态势量化方法 [J ] . 通信学报 , 2017 , 38 ( 10 ): 122 - 134 .

HU H , YE R G , ZHANG H Q , et al . Quantitative method for network security situation based on attack prediction [J ] . Journal on Communications , 2017 , 38 ( 10 ): 122 - 134 . (in Chinese)

闫峰 , 刘淑芬 , 冷煌 . 基于转换的攻击图分析方法研究 [J ] . 电子学报 , 2014 , 42 ( 12 ): 2477 - 2480 .

YAN F , LIU S F , LENG H . Study on analysis of attack graphs based on conversion [J ] . Acta Electronica Sinica , 2014 , 42 ( 12 ): 2477 - 2480 . (in Chinese)

叶子维 , 郭渊博 , 王宸东 , 等 . 攻击图技术应用研究综述 [J ] . 通信学报 , 2017 , 38 ( 11 ): 121 - 132 .

YE Z W , GUO Y B , WANG C D , et al . Survey on application of attack graph technology [J ] . Journal on Communications , 2017 , 38 ( 11 ): 121 - 132 . (in Chinese)

DACIER M , DESWARTE Y . Privilege Graph: An extension to the typed access matrix model [M ] // Computer Security — ESORICS 94 . Berlin : Springer , 1994 : 319 - 334 .

WANG L Y , YAO C , SINGHAL A , et al . Interactive analysis of attack graphs using relational queries [M ] // Data and Applications Security XX . Berlin : Springer , 2006 : 119 - 132 .

RAMAKRISHNAN C R , SEKAR R . Model-based analysis of configuration vulnerabilities1 [J ] . Journal of Computer Security , 2002 , 10 ( 1/2 ): 189 - 209 .

RITCHEY R W , AMMANN P . Using model checking to analyze network vulnerabilities [C ] // Proceeding 2000 IEEE Symposium on Security and Privacy . Piscataway : IEEE , 2002 : 156 - 165 .

臧艺超 , 周天阳 , 朱俊虎 , 等 . 领域独立智能规划技术及其面向自动化渗透测试的攻击路径发现研究进展 [J ] . 电子与信息学报 , 2020 , 42 ( 9 ): 2095 - 2107 .

ZANG Y C , ZHOU T Y , ZHU J H , et al . Domain-independent intelligent planning technology and its application to automated penetration testing oriented attack path discovery [J ] . Journal of Electronics & Information Technology , 2020 , 42 ( 9 ): 2095 - 2107 . (in Chinese)

李庆朋 , 王布宏 , 王晓东 , 等 . 基于最优攻击路径的网络安全增强策略研究 [J ] . 计算机科学 , 2013 , 40 ( 4 ): 152 - 154 .

LI Q P , WANG B H , WANG X D , et al . Approach on network security enhancement strategies based on optimal attack path [J ] . Computer Science , 2013 , 40 ( 4 ): 152 - 154 . (in Chinese)

杨本毅 . 基于攻击图的渗透测试方法 [J ] . 电子科技 , 2019 , 32 ( 10 ): 75 - 78 .

YANG B Y . Research on corrosion detection technology of power system grounding grid [J ] . Electronic Science and Technology , 2019 , 32 ( 10 ): 75 - 78 . (in Chinese)

NGUYEN T T , REDDI V J . Deep reinforcement learning for cyber security [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2023 , 34 ( 8 ): 3779 - 3795 .

KAELBLING L P , LITTMAN M L , MOORE A W . Reinforcement learning: A survey [J ] . Journal of Artificial Intelligence Research , 1996 , 4 : 237 - 285 .

CODY T , RAHMAN A , REDINO C , et al . Discovering exfiltration paths using reinforcement learning with attack graphs [C ] // 2022 IEEE Conference on Dependable and Secure Computing (DSC) . Piscataway : IEEE , 2022 : 1 - 8 .

曾庆伟 , 张国敏 , 邢长友 , 等 . 基于分层强化学习的智能化攻击路径发现方法 [J ] . 计算机科学 , 2023 , 50 ( 7 ): 308 - 316 .

ZENG Q W , ZHANG G M , XING C Y , et al . Intelligent attack path discovery based on hierarchical reinforcement learning [J ] . Computer Science , 2023 , 50 ( 7 ): 308 - 316 . (in Chinese)

CLIFTON J , LABER E . Q -learning: Theory and applications [J ] . Annual Review of Statistics and Its Application , 2020 , 7 : 279 - 301 .

李腾 , 曹世杰 , 尹思薇 , 等 . 应用 Q 学习决策的最优攻击路径生成方法 [J ] . 西安电子科技大学学报 , 2021 , 48 ( 1 ): 160 - 167 .

LI T , CAO S J , YIN S W , et al . Optimal method for the generation of the attack path based on the Q -learning decision [J ] . Journal of Xidian University , 2021 , 48 ( 1 ): 160 - 167 . (in Chinese)

胡昌振 , 陈韵 , 吕坤 . 一种基于 Q 学习的最佳攻击路径规划方法 : CN107317756A [P ] . 2017-11-03 .

YOUSEFI M , MTETWA N , ZHANG Y , et al . A reinforcement learning approach for attack graph analysis [C ] // 2018 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications / 12th IEEE International Conference on Big Data Science and Engineering (TrustCom/BigDataSE) . Piscataway : IEEE , 2018 : 212 - 217 .

JANG B , KIM M , HARERIMANA G , et al . Q -learning algorithms: A comprehensive classification and applications [J ] . IEEE Access , 2019 , 7 : 133653 - 133667 .

胡玮 . 网络拓扑自动发现 [D ] . 成都 : 电子科技大学 , 2012 .

HU W . Automatic Discovery of Network Topology [D ] . Chengdu : University of Electronic Science and Technology of China , 2012 . (in Chinese)

刘杰 , 王清贤 , 罗军勇 . 一种基于ICMP的逻辑层网络拓扑发现与分析方法 [J ] . 计算机应用 , 2008 , 28 ( 6 ): 1498 - 1500 .

LIU J , WANG Q X , LUO J Y . ICMP-based method for logical network topology discovery and analysis [J ] . Journal of Computer Applications , 2008 , 28 ( 6 ): 1498 - 1500 . (in Chinese)

PANSIOT J J , GRAD D . On routes and multicast trees in the Internet [J ] . ACM SIGCOMM Computer Communication Review , 1998 , 28 ( 1 ): 41 - 50 .

DONNET B , FRIEDMAN T . Internet topology discovery: A survey [J ] . IEEE Communications Surveys & Tutorials , 2007 , 9 ( 4 ): 56 - 69 .

陈晋音 , 胡书隆 , 邢长友 , 等 . 面向智能渗透攻击的欺骗防御方法 [J ] . 通信学报 , 2022 , 43 ( 10 ): 106 - 120 .

CHEN J Y , HU S L , XING C Y , et al . Deception defense method against intelligent penetration attack [J ] . Journal on Communications , 2022 , 43 ( 10 ): 106 - 120 . (in Chinese)

张涛 , 张文涛 , 代凌 , 等 . 基于序贯博弈多智能体强化学习的综合模块化航空电子系统重构方法 [J ] . 电子学报 , 2022 , 50 ( 4 ): 954 - 966 .

ZHANG T , ZHANG W T , DAI L , et al . Integrated modular avionics system reconstruction method based on sequential game multi-agent reinforcement learning [J ] . Acta Electronica Sinica , 2022 , 50 ( 4 ): 954 - 966 . (in Chinese)

WU Z N , TIAN L Q , WANG Y , et al . Network security defense decision-making method based on stochastic game and deep reinforcement learning [J ] . Security and Communication Networks , 2021 , 2021 : 1 - 13 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于冲突代价Bayesian权重的改进PBS多智能体路径规划算法

基于图组合优化的高效社区搜索

知识数据协同的多对手智能空中博弈策略设计

基于强化学习的免调参即插即用单光子图像重建方法