MASTER：基于知识注入与意图强化的大语言模型漏洞验证代码生成框架

李秋香; 张晗; 曲豫宾; 吴建平

doi:10.12263/DZXB.20260071

您当前的位置：

首页 >

文章列表页 >

MASTER：基于知识注入与意图强化的大语言模型漏洞验证代码生成框架

学术论文 | 更新时间：2026-07-30

- MASTER：基于知识注入与意图强化的大语言模型漏洞验证代码生成框架
- MASTER: A Large Language Model Framework for Proof of Concept Generation Via Knowledge Injection and Intention Reinforcement
- 电子学报 2026年54卷第4期页码：1802-1822
- 作者机构：
  
  1.清华大学计算机科学与技术系，北京 100084
  2.江苏工程职业技术学院信息工程学院，江苏南通 226007
- 作者简介：
  
  李秋香女，1981年9月出生于吉林省四平市。现为公安部第一研究所副研究员，清华大学计算机系博士研究生。主要研究方向为网络安全政策及相关技术。E-mail: liqx21@mails.tsinghua.edu.cn
  张晗男，1990年4月出生于山东省淄博市。现为清华大学副研究员。主要研究方向为互联网体系结构与网络空间安全领域基础理论与核心技术研究。E-mail: zhhan@tsinghua.edu.cn
  曲豫宾男， 1981年11月出生于河南省南阳市。现为江苏工程职业技术学院副教授。主要研究方向为大语言模型安全。E-mail: quyubin@hotmail.com
  吴建平男，1953年10月出生于山西省太原市。现为清华大学教授。主要研究方向为互联网体系结构与网络空间安全领域基础理论与核心技术研究。中国电子学会会员编号：E190184942M。E-mail: wjp@tsinghua.edu.cn
- 基金信息：
  
  国家自然科学基金(62572269)
- DOI：10.12263/DZXB.20260071
  中图分类号： TP393.08;TP311.5
- 收稿：2026-03-10，
  
  录用：2026-03-25，
  
  网络首发：2026-05-21，
  
  纸质出版：2026-04-25
- 稿件说明：
移动端阅览
李秋香, 张晗, 曲豫宾, 等. MASTER：基于知识注入与意图强化的大语言模型漏洞验证代码生成框架[J]. 电子学报, 2026, 54(04): 1802-1822.

LI Qiuxiang, ZHANG Han, QU Yubin, et al. MASTER: A Large Language Model Framework for Proof of Concept Generation Via Knowledge Injection and Intention Reinforcement[J]. Acta Electronica Sinica, 2026, 54(04): 1802-1822.
李秋香, 张晗, 曲豫宾, 等. MASTER：基于知识注入与意图强化的大语言模型漏洞验证代码生成框架[J]. 电子学报, 2026, 54(04): 1802-1822. DOI：10.12263/DZXB.20260071

LI Qiuxiang, ZHANG Han, QU Yubin, et al. MASTER: A Large Language Model Framework for Proof of Concept Generation Via Knowledge Injection and Intention Reinforcement[J]. Acta Electronica Sinica, 2026, 54(04): 1802-1822. DOI：10.12263/DZXB.20260071

摘要

自动化漏洞验证代码（Proof of Concept， PoC）的生成在网络安全评估、漏洞修复验证以及自动化防御体系构建中扮演着至关重要的角色。尽管大语言模型（Large Language Models，LLMs）在通用领域的代码合成任务中已展现出显著的效能优势，但由于高危漏洞逻辑的复杂性以及通用大模型垂域推理能力的局限，其在面对未见漏洞（unseen vulnerabilities）时，往往难以生成逻辑严密且高质量的PoC代码。本文聚焦于垂直领域代码生成场景，提出了一种名为MASTER（mastering automated synthesis of targeted exploits via reinforcement）的自动化PoC生成框架，通过构建领域知识注入与意图强化的协同机制突破大模型面向复杂PoC代码生成任务的泛化瓶颈。本研究首先构建了融合思维链（Chain-of-Thought，CoT）增强的领域专用漏洞数据集ExploitDB-CVE-CoT；随后，通过全参数监督微调（Supervised Fine-Tuning， SFT）实现深度的领域知识注入，使模型掌握漏洞利用的基础能力；进而，创新性地引入基于AI反馈的强化学习（Reinforcement Learning from AI Feedback， RLAIF），采用群组相对策略优化（Group Relative Policy Optimization，GRPO）算法，利用外部专家大模型作为动态评估器，深度强化模型针对复杂漏洞的逻辑推理与代码生成能力。实证结果表明，虽然仅进行SFT已能将针对通用漏洞披露（Common Vulnerabilities and Exposures，CVE）场景的生成成功率显著提升至57.5%~76%区间，但单一SFT存在非目标领域的灾难性遗忘与泛化能力缺陷；而RLAIF机制在改善泛化能力缺陷和提升模型整体性能方面发挥了积极作用。最终，MASTER框架在Qwen2.5和Llama-3.1等主流模型上将高危漏洞PoC的自动生成成功率进一步提升至88.8%~95.6%，且具备针对训练集外漏洞场景的较好的泛化能力。该成果为自动化渗透测试与防御评估提供了强大的技术支撑，有效拓宽了大语言模型在强逻辑、高度专业化网络安全任务中的应用边界。

Abstract

The automated generation of proof-of-concept (PoC) code for vulnerability verification plays a pivotal role in cybersecurity assessment

patch validation

and the construction of automated defense systems. Despite their success in general coding

LLMs still struggle with complex “unseen” PoCs due to limited domain-specific reasoning and the inherent difficulty of high-risk exploits. This paper focuses on code generation within vertical domains and proposes MASTER (mastering automated synthesis of targeted exploits via reinforcement)

an automated PoC generation framework. MASTER overcomes the generalization bottlenecks of LLMs in complex PoC synthesis through a synergistic mechanism of domain knowledge injection and intent reinforcement. Specifically

we first construct ExploitDB-CVE-CoT

a domain-specific vulnerability dataset enhanced with chain-of-thought (CoT) reasoning. Subsequently

deep domain knowledge injection is achieved via full-parameter supervised fine-tuning (SFT)

enabling the model to master fundamental exploit capabilities. Furthermore

we innovatively introduce reinforcement learning from AI feedback (RLAIF) utilizing the group relative policy optimization (GRPO) algorithm. This approach employs external expert LLMs as dynamic evaluators to deeply reinforce the model’s logical reasoning and code generation proficiency for complex vulnerabilities. Empirical results demonstrate that while SFT alone significantly improves the generation success rate for common vulnerabilities and exposures (CVE) scenarios to a range of 57.5%~76%

it suffers from catastrophic forgetting in non-target domains and generalization deficiencies. The RLAIF mechanism effectively mitigates these weaknesses and enhances overall performance. Ultimately

the MASTER framework elevates the automated PoC generation success rate for high-risk vulnerabilities to 88.8%~95.6% on mainstream models such as Qwen2.5 and Llama-3.1

exhibiting robust generalization across out-of-distribution vulnerability scenarios. This research provides potent technical support for automated penetration testing and defense evaluation

effectively extending the application boundaries of LLMs in high-logic

highly specialized cybersecurity tasks.

关键词

Keywords

references

Vaswani A , Shazeer N , Parmar N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : Curran Associates Inc. , 2017 : 6000 - 6010 .

Agarwal S , Almeida D , Askell A , et al . Training language models to follow instructions with human feedback [C ] // Advances in Neural Information Processing Systems 35 . Neural Information Processing Systems Foundation, Inc. (NeurIPS) , 2022 : 27730 - 27744 . DOI: 10.52202/068431-2011 http://dx.doi.org/10.52202/068431-2011

Ganguli D , Lovitt L , Kernion J , et al . Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned [PP/OL ] . V2. arXiv ( 2022-11-22 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2209.07858 https://doi.org/10.48550/arXiv.2209.07858 .

Qu Y B , Huang S , Nie P . A review of backdoor attacks and defenses in code large language models: Implications for security measures [J ] . Information and Software Technology , 2025 , 182 : 107707 . DOI: 10.1016/j.infsof.2025.107707 http://dx.doi.org/10.1016/j.infsof.2025.107707

Qu Y B , Huang S , Li Y Z , et al . BadCodePrompt: Backdoor attacks against prompt engineering of large language models for code generation [J ] . Automated Software Engineering , 2025 , 32 : 17 . DOI: 10.1007/s10515-024-00485-2 http://dx.doi.org/10.1007/s10515-024-00485-2

Kurita K , Michel P , Neubig G . Weight poisoning attacks on pre-trained models [PP/OL ] . V1. arXiv ( 2020-04-14 )[ 2026-03-10 ] . https://arxiv.org/abs/2004.0666 https://arxiv.org/abs/2004.0666 . DOI: 10.18653/v1/2020.acl-main.249 http://dx.doi.org/10.18653/v1/2020.acl-main.249

Qu Y B , Huang S , Yao Y M , et al . A simple yet practical backdoor prompt attack against black-box code summarization engines [J ] . Journal of Software: Evolution and Process , 2025 , 37 ( 8 ): e70032 . DOI: 10.1002/smr.70032 http://dx.doi.org/10.1002/smr.70032

Qu Y B , Huang S , Li L , et al . Beyond intentions: A critical survey of misalignment in LLMs [J ] . Computers , Materials & Continua, 2025 , 85 ( 1 ): 249 - 300 . DOI: 10.32604/cmc.2025.067750 http://dx.doi.org/10.32604/cmc.2025.067750

Xing S , Hong J Y , Wang Y F , et al . LLMs can get "brain rot": A pilot study on Twitter/X [PP/OL ] . V2. arXiv ( 2026-04-22 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2510.13928 https://doi.org/10.48550/arXiv.2510.13928 .

DIVYA . OAST-based exploit platform targets 200 CVEs via Google Cloud resources [EB/OL ] . ( 2025-11-29 )[ 2026-03-10 ] . https://cyberpress.org/oast-based-exploit-platform-targets-200-cves/ https://cyberpress.org/oast-based-exploit-platform-targets-200-cves/ .

Bai Y T , Kadavath S , Kundu S , et al . Constitutional AI: Harmlessness from AI feedback [PP/OL ] . V1. arXiv ( 2022-12-15 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2212.08073 https://doi.org/10.48550/arXiv.2212.08073 .

Lim J Y , Lim K M , Lee C P , et al . SFT: Few-shot learning via self-supervised feature fusion with transformer [J ] . IEEE Access , 2024 , 12 : 86690 - 86703 . DOI: 10.1109/ACCESS.2024.3416327 http://dx.doi.org/10.1109/ACCESS.2024.3416327

Christiano P F , Leike J , Brown T B , et al . Deep reinforcement learning from human preferences [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 4302 - 4310 .

Ermon S , Finn C , Manning C D , et al . Direct preference optimization: Your language model is secretly a reward model [C ] // Advances in Neural Information Processing Systems 36 . Neural Information Processing Systems Foundation, Inc. (NeurIPS) , 2023 : 53728 - 53741 . DOI: 10.52202/075280-2338 http://dx.doi.org/10.52202/075280-2338

Qu Y B , Huang S , Yao Y M . A survey on robustness attacks for deep code models [J ] . Automated Software Engineering , 2024 , 31 ( 2 ): 65 . DOI: 10.1007/s10515-024-00464-7 http://dx.doi.org/10.1007/s10515-024-00464-7

Chen M , Tworek J , Jun H , et al . Evaluating large language models trained on code [PP/OL ] . V2. arXiv ( 2021-07-14 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2107.03374 https://doi.org/10.48550/arXiv.2107.03374 .

Pearce H , Ahmad B , Tan B , et al . Asleep at the keyboard? Assessing the security of GitHub copilot’s code contributions [J ] . Communications of the ACM , 2025 , 68 ( 2 ): 96 - 105 . DOI: 10.1145/3610721 http://dx.doi.org/10.1145/3610721

Guo C Q , Li B , Lin Z N , et al . RedCode: Risky code execution and generation benchmark for code agents [C ] // Advances in Neural Information Processing Systems 37 . Neural Information Processing Systems Foundation, Inc. (NeurIPS) , 2024 : 106190 - 106236 . DOI: 10.52202/079017-3369 http://dx.doi.org/10.52202/079017-3369

DeepSeek-AI , Guo D Y , Yang D J , et al . DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning [PP/OL ] . V2. arXiv ( 2026-01-04 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2501.12948 https://doi.org/10.48550/arXiv.2501.12948 .

Lu S , Guo D , Ren S , et al . Codexglue: A machine learning benchmark dataset for code understanding and generation [PP/OL ] . V1. arXiv ( 2021-02-09 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2510.10148 https://doi.org/10.48550/arXiv.2510.10148 .

Austin J , Odena A , Nye M , et al . Program synthesis with large language models [PP/OL ] . V1. arXiv ( 2021-08-16 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2102.04664 https://doi.org/10.48550/arXiv.2102.04664 .

Avgerinos T , Cha S K , Rebert A , et al . Automatic exploit generation [J ] . Communications of the ACM , 2014 , 57 ( 2 ): 74 - 84 . DOI: 10.1145/2560217.2560219 http://dx.doi.org/10.1145/2560217.2560219

Zeng Y , Lin H P , Zhang J W , et al . How johnny can persuade LLMs to jailbreak them: Rethinking persuasion to challenge AI safety by humanizing LLMs [C ] // Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2024 : 14322 - 14350 . DOI: 10.18653/v1/2024.acl-long.773 http://dx.doi.org/10.18653/v1/2024.acl-long.773

Peng W Z , Ye L , Du X T , et al . PwnGPT: Automatic exploit generation based on large language models [C ] // Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2025 : 11481 - 11494 . DOI: 10.18653/v1/2025.acl-long.562 http://dx.doi.org/10.18653/v1/2025.acl-long.562

Zhao M Y , Li K X , Zhang L Y , et al . A systematic study on generating web vulnerability proof-of-concepts using large language models [PP/OL ] . V1. arXiv ( 2025-10-11 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2510.10148 https://doi.org/10.48550/arXiv.2510.10148 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

大语言模型在电子设计自动化中的应用综述

基于大语言模型的加密流量分析方法研究综述

一种融合双粒度知识库的仿生设计方案智能生成方法

通用知识与专用信息共驱动的具身智能抗干扰通信策略生成方法

PCDA-MPAT：物理约束数据增强与多径Transformer的无源物联网室内定位方法