MASTER: A Large Language Model Framework for Proof of Concept Generation Via Knowledge Injection and Intention Reinforcement

LI Qiuxiang; ZHANG Han; QU Yubin; WU Jianping

doi:10.12263/DZXB.20260071

您当前的位置：

首页 >

文章列表页 >

MASTER: A Large Language Model Framework for Proof of Concept Generation Via Knowledge Injection and Intention Reinforcement

PAPERS | 更新时间：2026-07-30

- MASTER: A Large Language Model Framework for Proof of Concept Generation Via Knowledge Injection and Intention Reinforcement
- ACTA ELECTRONICA SINICA Vol. 54, Issue 4, Pages: 1802-1822(2026)
- 作者机构：
  
  1.清华大学计算机科学与技术系，北京 100084
  2.江苏工程职业技术学院信息工程学院，江苏南通 226007
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62572269)
- DOI：10.12263/DZXB.20260071
  CLC： TP393.08;TP311.5
- Received：10 March 2026，
  
  Accepted：25 March 2026，
  
  Online First：21 May 2026，
  
  Published：25 April 2026
- 稿件说明：
移动端阅览
李秋香, 张晗, 曲豫宾, 等. MASTER：基于知识注入与意图强化的大语言模型漏洞验证代码生成框架[J]. 电子学报, 2026, 54(04): 1802-1822.

LI Qiuxiang, ZHANG Han, QU Yubin, et al. MASTER: A Large Language Model Framework for Proof of Concept Generation Via Knowledge Injection and Intention Reinforcement[J]. Acta Electronica Sinica, 2026, 54(04): 1802-1822.
李秋香, 张晗, 曲豫宾, 等. MASTER：基于知识注入与意图强化的大语言模型漏洞验证代码生成框架[J]. 电子学报, 2026, 54(04): 1802-1822. DOI：10.12263/DZXB.20260071

LI Qiuxiang, ZHANG Han, QU Yubin, et al. MASTER: A Large Language Model Framework for Proof of Concept Generation Via Knowledge Injection and Intention Reinforcement[J]. Acta Electronica Sinica, 2026, 54(04): 1802-1822. DOI：10.12263/DZXB.20260071

摘要

自动化漏洞验证代码（Proof of Concept， PoC）的生成在网络安全评估、漏洞修复验证以及自动化防御体系构建中扮演着至关重要的角色。尽管大语言模型（Large Language Models，LLMs）在通用领域的代码合成任务中已展现出显著的效能优势，但由于高危漏洞逻辑的复杂性以及通用大模型垂域推理能力的局限，其在面对未见漏洞（unseen vulnerabilities）时，往往难以生成逻辑严密且高质量的PoC代码。本文聚焦于垂直领域代码生成场景，提出了一种名为MASTER（mastering automated synthesis of targeted exploits via reinforcement）的自动化PoC生成框架，通过构建领域知识注入与意图强化的协同机制突破大模型面向复杂PoC代码生成任务的泛化瓶颈。本研究首先构建了融合思维链（Chain-of-Thought，CoT）增强的领域专用漏洞数据集ExploitDB-CVE-CoT；随后，通过全参数监督微调（Supervised Fine-Tuning， SFT）实现深度的领域知识注入，使模型掌握漏洞利用的基础能力；进而，创新性地引入基于AI反馈的强化学习（Reinforcement Learning from AI Feedback， RLAIF），采用群组相对策略优化（Group Relative Policy Optimization，GRPO）算法，利用外部专家大模型作为动态评估器，深度强化模型针对复杂漏洞的逻辑推理与代码生成能力。实证结果表明，虽然仅进行SFT已能将针对通用漏洞披露（Common Vulnerabilities and Exposures，CVE）场景的生成成功率显著提升至57.5%~76%区间，但单一SFT存在非目标领域的灾难性遗忘与泛化能力缺陷；而RLAIF机制在改善泛化能力缺陷和提升模型整体性能方面发挥了积极作用。最终，MASTER框架在Qwen2.5和Llama-3.1等主流模型上将高危漏洞PoC的自动生成成功率进一步提升至88.8%~95.6%，且具备针对训练集外漏洞场景的较好的泛化能力。该成果为自动化渗透测试与防御评估提供了强大的技术支撑，有效拓宽了大语言模型在强逻辑、高度专业化网络安全任务中的应用边界。

Abstract

The automated generation of proof-of-concept (PoC) code for vulnerability verification plays a pivotal role in cybersecurity assessment

patch validation

and the construction of automated defense systems. Despite their success in general coding

LLMs still struggle with complex “unseen” PoCs due to limited domain-specific reasoning and the inherent difficulty of high-risk exploits. This paper focuses on code generation within vertical domains and proposes MASTER (mastering automated synthesis of targeted exploits via reinforcement)

an automated PoC generation framework. MASTER overcomes the generalization bottlenecks of LLMs in complex PoC synthesis through a synergistic mechanism of domain knowledge injection and intent reinforcement. Specifically

we first construct ExploitDB-CVE-CoT

a domain-specific vulnerability dataset enhanced with chain-of-thought (CoT) reasoning. Subsequently

deep domain knowledge injection is achieved via full-parameter supervised fine-tuning (SFT)

enabling the model to master fundamental exploit capabilities. Furthermore

we innovatively introduce reinforcement learning from AI feedback (RLAIF) utilizing the group relative policy optimization (GRPO) algorithm. This approach employs external expert LLMs as dynamic evaluators to deeply reinforce the model’s logical reasoning and code generation proficiency for complex vulnerabilities. Empirical results demonstrate that while SFT alone significantly improves the generation success rate for common vulnerabilities and exposures (CVE) scenarios to a range of 57.5%~76%

it suffers from catastrophic forgetting in non-target domains and generalization deficiencies. The RLAIF mechanism effectively mitigates these weaknesses and enhances overall performance. Ultimately

the MASTER framework elevates the automated PoC generation success rate for high-risk vulnerabilities to 88.8%~95.6% on mainstream models such as Qwen2.5 and Llama-3.1

exhibiting robust generalization across out-of-distribution vulnerability scenarios. This research provides potent technical support for automated penetration testing and defense evaluation

effectively extending the application boundaries of LLMs in high-logic

highly specialized cybersecurity tasks.

关键词

Keywords

references

Vaswani A , Shazeer N , Parmar N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : Curran Associates Inc. , 2017 : 6000 - 6010 .

Agarwal S , Almeida D , Askell A , et al . Training language models to follow instructions with human feedback [C ] // Advances in Neural Information Processing Systems 35 . Neural Information Processing Systems Foundation, Inc. (NeurIPS) , 2022 : 27730 - 27744 . DOI: 10.52202/068431-2011 http://dx.doi.org/10.52202/068431-2011

Ganguli D , Lovitt L , Kernion J , et al . Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned [PP/OL ] . V2. arXiv ( 2022-11-22 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2209.07858 https://doi.org/10.48550/arXiv.2209.07858 .

Qu Y B , Huang S , Nie P . A review of backdoor attacks and defenses in code large language models: Implications for security measures [J ] . Information and Software Technology , 2025 , 182 : 107707 . DOI: 10.1016/j.infsof.2025.107707 http://dx.doi.org/10.1016/j.infsof.2025.107707

Qu Y B , Huang S , Li Y Z , et al . BadCodePrompt: Backdoor attacks against prompt engineering of large language models for code generation [J ] . Automated Software Engineering , 2025 , 32 : 17 . DOI: 10.1007/s10515-024-00485-2 http://dx.doi.org/10.1007/s10515-024-00485-2

Kurita K , Michel P , Neubig G . Weight poisoning attacks on pre-trained models [PP/OL ] . V1. arXiv ( 2020-04-14 )[ 2026-03-10 ] . https://arxiv.org/abs/2004.0666 https://arxiv.org/abs/2004.0666 . DOI: 10.18653/v1/2020.acl-main.249 http://dx.doi.org/10.18653/v1/2020.acl-main.249

Qu Y B , Huang S , Yao Y M , et al . A simple yet practical backdoor prompt attack against black-box code summarization engines [J ] . Journal of Software: Evolution and Process , 2025 , 37 ( 8 ): e70032 . DOI: 10.1002/smr.70032 http://dx.doi.org/10.1002/smr.70032

Qu Y B , Huang S , Li L , et al . Beyond intentions: A critical survey of misalignment in LLMs [J ] . Computers , Materials & Continua, 2025 , 85 ( 1 ): 249 - 300 . DOI: 10.32604/cmc.2025.067750 http://dx.doi.org/10.32604/cmc.2025.067750

Xing S , Hong J Y , Wang Y F , et al . LLMs can get "brain rot": A pilot study on Twitter/X [PP/OL ] . V2. arXiv ( 2026-04-22 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2510.13928 https://doi.org/10.48550/arXiv.2510.13928 .

DIVYA . OAST-based exploit platform targets 200 CVEs via Google Cloud resources [EB/OL ] . ( 2025-11-29 )[ 2026-03-10 ] . https://cyberpress.org/oast-based-exploit-platform-targets-200-cves/ https://cyberpress.org/oast-based-exploit-platform-targets-200-cves/ .

Bai Y T , Kadavath S , Kundu S , et al . Constitutional AI: Harmlessness from AI feedback [PP/OL ] . V1. arXiv ( 2022-12-15 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2212.08073 https://doi.org/10.48550/arXiv.2212.08073 .

Lim J Y , Lim K M , Lee C P , et al . SFT: Few-shot learning via self-supervised feature fusion with transformer [J ] . IEEE Access , 2024 , 12 : 86690 - 86703 . DOI: 10.1109/ACCESS.2024.3416327 http://dx.doi.org/10.1109/ACCESS.2024.3416327

Christiano P F , Leike J , Brown T B , et al . Deep reinforcement learning from human preferences [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 4302 - 4310 .

Ermon S , Finn C , Manning C D , et al . Direct preference optimization: Your language model is secretly a reward model [C ] // Advances in Neural Information Processing Systems 36 . Neural Information Processing Systems Foundation, Inc. (NeurIPS) , 2023 : 53728 - 53741 . DOI: 10.52202/075280-2338 http://dx.doi.org/10.52202/075280-2338

Qu Y B , Huang S , Yao Y M . A survey on robustness attacks for deep code models [J ] . Automated Software Engineering , 2024 , 31 ( 2 ): 65 . DOI: 10.1007/s10515-024-00464-7 http://dx.doi.org/10.1007/s10515-024-00464-7

Chen M , Tworek J , Jun H , et al . Evaluating large language models trained on code [PP/OL ] . V2. arXiv ( 2021-07-14 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2107.03374 https://doi.org/10.48550/arXiv.2107.03374 .

Pearce H , Ahmad B , Tan B , et al . Asleep at the keyboard? Assessing the security of GitHub copilot’s code contributions [J ] . Communications of the ACM , 2025 , 68 ( 2 ): 96 - 105 . DOI: 10.1145/3610721 http://dx.doi.org/10.1145/3610721

Guo C Q , Li B , Lin Z N , et al . RedCode: Risky code execution and generation benchmark for code agents [C ] // Advances in Neural Information Processing Systems 37 . Neural Information Processing Systems Foundation, Inc. (NeurIPS) , 2024 : 106190 - 106236 . DOI: 10.52202/079017-3369 http://dx.doi.org/10.52202/079017-3369

DeepSeek-AI , Guo D Y , Yang D J , et al . DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning [PP/OL ] . V2. arXiv ( 2026-01-04 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2501.12948 https://doi.org/10.48550/arXiv.2501.12948 .

Lu S , Guo D , Ren S , et al . Codexglue: A machine learning benchmark dataset for code understanding and generation [PP/OL ] . V1. arXiv ( 2021-02-09 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2510.10148 https://doi.org/10.48550/arXiv.2510.10148 .

Austin J , Odena A , Nye M , et al . Program synthesis with large language models [PP/OL ] . V1. arXiv ( 2021-08-16 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2102.04664 https://doi.org/10.48550/arXiv.2102.04664 .

Avgerinos T , Cha S K , Rebert A , et al . Automatic exploit generation [J ] . Communications of the ACM , 2014 , 57 ( 2 ): 74 - 84 . DOI: 10.1145/2560217.2560219 http://dx.doi.org/10.1145/2560217.2560219

Zeng Y , Lin H P , Zhang J W , et al . How johnny can persuade LLMs to jailbreak them: Rethinking persuasion to challenge AI safety by humanizing LLMs [C ] // Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2024 : 14322 - 14350 . DOI: 10.18653/v1/2024.acl-long.773 http://dx.doi.org/10.18653/v1/2024.acl-long.773

Peng W Z , Ye L , Du X T , et al . PwnGPT: Automatic exploit generation based on large language models [C ] // Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2025 : 11481 - 11494 . DOI: 10.18653/v1/2025.acl-long.562 http://dx.doi.org/10.18653/v1/2025.acl-long.562

Zhao M Y , Li K X , Zhang L Y , et al . A systematic study on generating web vulnerability proof-of-concepts using large language models [PP/OL ] . V1. arXiv ( 2025-10-11 )[ 2026-03-10 ] . https://doi.org/10.48550/arXiv.2510.10148 https://doi.org/10.48550/arXiv.2510.10148 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Survey on the Applications of Large Language Models in Electronic Design Automation

A Review of Encryption Traffic Analysis Methods Based on Large Language Models

An Intelligent Bio-Inspired Design Solution Generation Method Integrating Dual-Granularity Knowledge Bases

Few-Shot Specific Emitter Identification Method Based on Lossless VMD and Mamba

Multilevel Resistive Memory Enabled Parallel-Gradient Boolean Satisfiability Solver with Clause-Folding Scheme

Related Author

LIN Liyu

WANG Jing

GUO Yufeng

ZENG Xiaoyang

YAO Jiafei

LI Yan

LAI Yao

LI Zheng

Related Institution

School of Computing and Data Science, The University of Hong Kong

School of Integrated Circuits and Microelectronic Innovation, Fudan University

School of Integrated Circuit Science and Engineering, Nanjing University of Posts and Telecommunications

School of Cyberspace Science and Technology, Beijing Institute of Technology

Institute for Network Sciences and Cyberspace, Tsinghua University

⁰