基于动态关系原型的持续关系抽取技术

钟将; 戴启祝; 李雪

doi:10.12263/DZXB.20240472

您当前的位置：

首页 >

文章列表页 >

基于动态关系原型的持续关系抽取技术

学术论文 | 更新时间：2025-12-27

- 基于动态关系原型的持续关系抽取技术
- Continuous Relation Extraction Technique Based on Dynamic Relational Prototypes
- 电子学报 2025年53卷第9期页码：3287-3298
- 作者机构：
  
  1.重庆大学计算机学院，重庆 400044
  2.重庆大学信息物理社会可信服务计算（CPS）教育部重点实验室，重庆 400044
  3.昆士兰大学电气工程和计算机科学学院，布里斯班 4072
- 作者简介：
  
  [ "钟将男，1974年4月出生于重庆市.重庆大学计算机学院教授、博士生导师，在国内外发表学术论文120余篇. 主要研究方向为自然语言处理、数据挖掘、机器学习等.E-mail: zhongjiang@cqu.edu.cn" ]
  [ "戴启祝男，1995年4月出生于江苏省盐城市.2024年毕业于重庆大学计算机学院获博士学位，现为马上消费金融股份有限公司博士后.主要研究方向为自然语言处理.E-mail: 1092864172@qq.com" ]
  [ "李雪男，1955年出生于重庆市.昆士兰大学教授、博士生导师.主要研究方向为社交网络分析、分布式高速时变数据流的知识挖掘.E-mail: xueli@itee.uq.edu.au" ]
- 基金信息：
  
  国家自然科学基金(62176029)
- DOI：10.12263/DZXB.20240472
  中图分类号： TP3;TP301.6
- 收稿：2024-05-23，
  
  录用：2025-08-29，
  
  纸质出版：2025-09-25
- 稿件说明：
移动端阅览
钟将, 戴启祝, 李雪. 基于动态关系原型的持续关系抽取技术[J]. 电子学报, 2025, 53(09): 3287-3298.

ZHONG Jiang, DAI Qi-zhu, LI Xue. Continuous Relation Extraction Technique Based on Dynamic Relational Prototypes[J]. Acta Electronica Sinica, 2025, 53(09): 3287-3298.
钟将, 戴启祝, 李雪. 基于动态关系原型的持续关系抽取技术[J]. 电子学报, 2025, 53(09): 3287-3298. DOI：10.12263/DZXB.20240472

ZHONG Jiang, DAI Qi-zhu, LI Xue. Continuous Relation Extraction Technique Based on Dynamic Relational Prototypes[J]. Acta Electronica Sinica, 2025, 53(09): 3287-3298. DOI：10.12263/DZXB.20240472

摘要

持续关系抽取（Continuous Relation Extraction，CRE）在理解和适应不断变化的数据环境中扮演着至关重要的角色.传统的CRE技术通常面临两大难题：一是关系模式的持续演变，二是遗忘之前学习的关系的风险.尽管存储和重放旧关系典型示例的做法在减少遗忘方面已被证明是有效的，但反复重放这些固定且有限的样本可能导致过拟合.为了解决这一问题，本文提出了一种基于动态原型的持续关系抽取方法.该方法结合了密度聚类和生成式大型语言模型，以应对上述挑战，本文将其命名为密度聚类和生成式大型语言建模（Continuous Relation Extraction with Density based Clustering and Generative Large Language Model，CRE-DCGLLM）.具体而言，本文采用了密度聚类技术来提取记忆样本，缓解对先前任务的遗忘问题，并基于全量样本和记忆样本设计了动态关系原型.此外，本文通过生成式大语文模型为记忆样本生成伪样本用于重放训练，以解决因多次重放导致的模型过拟合问题.同时，本文还运用焦点知识蒸馏技术，以提升对变化中关系模式的适应性能.通过在FewRel数据集和TACRED数据集上进行的一系列实验，本文验证了该方法的有效性.实验结果显示，本文的方法在持续关系抽取的准确性和效率方面都取得了显著的提升，特别是在处理相似关系、防止知识遗忘以及克服过拟合等方面表现出了卓越的性能.

Abstract

Continuous relation extraction (CRE) plays a crucial role in understanding and adapting to the ever-changing data environment. Traditional CRE techniques often face two major challenges: the continuous evolution of relationship patterns and the risk of forgetting previously learned relationships. Although storing and replaying typical examples of old relationships has been proven effective in reducing forgetting

repeatedly replaying these fixed and limited samples can lead to overfitting. To address this issue

this paper proposes a dynamic prototype-based continuous relation extraction method that combines density clustering and generative large language models to tackle the aforementioned challenges

which is named a dynamic prototype-based continuous relation extraction method (Continuous Relation Extraction with Density based Clustering and Generative Large Language Model

CRE-DC GLLM) in this paper. Specifically

this paper employs density clustering technology to extract memory samples to alleviate the problem of forgetting previous tasks

and designs dynamic relationship prototypes based on full samples and memory samples. In addition

this paper uses a generative large model to generate pseudo-samples for memory samples for replay training

to solve the problem of model overfitting caused by multiple replays. At the same time

this paper also uses focused knowledge distillation technology to enhance the adaptability to changing relationship patterns. A series of experiments conducted on the FewRel dataset and the TACRED dataset have verified the effectiveness of this method. The experimental results show that this method has achieved significant improvements in the accuracy and efficiency of continuous relation extraction

especially in handling similar relationships

preventing knowledge forgetting

and overcoming overfitting

it has shown excellent performance.

关键词

Keywords

references

隗昊 , 唐焕玲 , 周爱 , 等 . 基于双路分段注意力神经张量网络的临床文本关系抽取 [J ] . 电子学报 , 2023 , 51 ( 3 ): 658 - 665 .

WEI H , TANG H L , ZHOU A , et al . Clinical relation extraction via dual piecewise attention neural tensor network [J ] . Acta Electronica Sinica , 2023 , 51 ( 3 ): 658 - 665 . (in Chinese)

王宇 , 王震 , 温立强 , 等 . DRE-3DC: 基于三维表征建模的篇章级关系抽取模型 [J ] . 电子学报 , 2024 , 52 ( 9 ): 2950 - 2960 .

WANG Y , WANG Z , WEN L Q , et al . DRE-3DC: Document-level relation extraction with three-dimensional representation combination modeling [J ] . Acta Electronica Sinica , 2024 , 52 ( 9 ): 2950 - 2960 . (in Chinese)

ZHOU P , SHI W , TIAN J , et al . Attention-based bidirectional long short-term memory networks for relation classification [C ] // Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2016 : 207 - 212 .

BALDINI SOARES L , FITZGERALD N , LING J , et al . Matching the blanks: Distributional similarity for relation learning [C ] // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2019 : 2895 - 2905 .

WANG H , XIONG W H , YU M , et al . Sentence embedding alignment for lifelong relation extraction [C ] // Proceedings of the 2019 Conference of the North . Stroudsburg : ACL , 2019 : 796 - 806 .

HAN X , DAI Y , GAO T Y , et al . Continual relation learning via episodic memory activation and reconsolidation [C ] // Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2020 : 6429 - 6440 .

WU T T , LI X K , LI Y-F , et al . Curriculum-meta learning for order-robust continual relation extraction [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 12 ): 10363 - 10369 .

CUI L , YANG D Q , YU J X , et al . Refining sample embeddings with relation prototypes to enhance continual relation extraction [C ] // Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing . Stroudsburg : ACL , 2021 : 232 - 243 .

HASSABIS D , KUMARAN D , SUMMERFIELD C , et al . Neuroscience-inspired artificial intelligence [J ] . Neuron , 2017 , 95 ( 2 ): 245 - 258 .

PFULB B , GEPPERTH A . Overcoming catastrophic forgetting with Gaussian mixture replay [C ] // 2021 International Joint Conference on Neural Networks . Piscataway : IEEE , 2021 : 1 - 9 .

ZENKE F , POOLE B , GANGULI S . Continual learning through synaptic intelligence [C ] // Proceedings of the 34th International Conference on Machine Learning - Volume 70 . New York : ACM , 2017 : 3987 - 3995 .

CHEN T Q , GOODFELLOW I , SHLENS J . Net2Net: Accelerating learning via knowledge transfer [EB/OL ] . ( 2016-04-23 )[ 2024-05-20 ] . https://arXiv.org/abs/1511.05641 https://arXiv.org/abs/1511.05641 .

FERNANDO C , BANARSE D , BLUNDELL C , et al . PathNet: Evolution channels gradient descent in super neural networks [EB/OL ] . ( 2017-01-30 )[ 2024-04-08 ] . https://arXiv.org/abs/1701.08734 https://arXiv.org/abs/1701.08734 .

CHAUDHRY A , RANZATO M , ROHRBACH M , et al . Efficient Lifelong Learning with A-GEM [EB/OL ] . ( 2019-01-09 )[ 2024-05-20 ] . https://arxiv.org/abs/1812.00420 https://arxiv.org/abs/1812.00420 .

VERWIMP E , DE LANGE M , TUYTELAARS T . Rehearsal revealed: The limits and merits of revisiting samples in continual learning [C ] // 2021 IEEE/CVF International Conference on Computer Vision . Piscataway : IEEE , 2022 : 9365 - 9374 .

DE LANGE M , ALJUNDI R , MASANA M , et al . A continual learning survey: Defying forgetting in classification tasks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 7 ): 3366 - 3385 .

HU C , YANG D , JIN H , et al . Improving continual relation extraction through prototypical contrastive learning [EB/OL ] . ( 2022-10-10 )[ 2024-05-20 ] . https://arxiv.org/abs/2210.04513 https://arxiv.org/abs/2210.04513 .

ZHAO K , XU H , YANG J G , et al . Consistent representation learning for continual relation extraction [C ] // Findings of the Association for Computational Linguistics . Stroudsburg : ACL , 2022 : 3402 - 3411 .

BAI J Z , BAI S , CHU Y F , et al . Qwen technical report [EB/OL ] . ( 2023-09-28 )[ 2024-05-20 ] . https://arXiv.org/abs/2309.16609 https://arXiv.org/abs/2309.16609 .

PARISI G I , KEMKER R , PART J L , et al . Continual lifelong learning with neural networks: A review [J ] . Neural Networks , 2019 , 113 : 54 - 71 .

HSU Y C , LIU Y C , RAMASAMY A , et al . Re-evaluating continual learning scenarios: A categorization and case for strong baselines [EB/OL ] . ( 2019-01-23 )[ 2024-04-08 ] . https://arXiv.org/abs/1810.12488 https://arXiv.org/abs/1810.12488 .

MALLYA A , LAZEBNIK S . PackNet: Adding multiple tasks to a single network by iterative pruning [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7765 - 7773 .

QIN Q , PENG H , HU W R , et al . BNS: Building network structures dynamically for continual learning [C ] // Advances in Neural Information Processing Systems 34 (NeurIPS 2021) . Trier : DBLP , 2021 : 20608 - 20620 .

赵基藤 , 李国正 , 汪鹏 , 等 . 基于监督对比重放的持续关系抽取 [J ] . 中文信息学报 , 2023 , 37 ( 11 ): 60 - 67, 80 .

ZHAO J T , LI G Z , WANG P , et al . Continual relation extraction via supervised contrastive replay [J ] . Journal of Chinese Information Processing , 2023 , 37 ( 11 ): 60 - 67, 80 . (in Chinese)

LOPEZ-PAZ D , RANZATO M . Gradient episodic memory for continual learning [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 6470 - 6479 .

REBUFFI S A , KOLESNIKOV A , SPERL G , et al . iCaRL: Incremental classifier and representation learning [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 5533 - 5542 .

MAI Z D , LI R W , KIM H , et al . Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Piscataway : IEEE , 2021 : 3584 - 3594 .

SUN F K , HO C H , LEE H Y . LAMOL: LAnguage modeling for lifelong language learning [EB/OL ] . ( 2019-12-23 )[ 2024-05-20 ] . https://arxiv.org/abs/1909.03329 https://arxiv.org/abs/1909.03329 .

D’AUTUME C de M , RUDER S , KONG L , et al . Episodic memory in lifelong language learning [EB/OL ] . ( 2019-11-26 )[ 2024-05-20 ] . https://arxiv.org/abs/1906.01076 https://arxiv.org/abs/1906.01076 .

WANG P Y , SONG Y F , LIU T Y , et al . Learning robust representations for continual relation extraction via adversarial class augmentation [C ] // Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : ACL , 2022 : 6264 - 6278 .

HUANG M Y , XIAO M , WANG L D , et al . DP-CRE: Continual relation extraction via decoupled contrastive learning and memory structure preservation [EB/OL ] . ( 2024-03-05 )[ 2024-05-20 ] . https://arxiv.org/abs/2403.02718 https://arxiv.org/abs/2403.02718 .

ZHAO W Z , CUI Y N , HU W . Improving continual relation extraction by distinguishing analogous semantics [C ] // Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2023 : 1162 - 1175 .

DEVLIN J , CHANG M W , LEE K , et al . BERT: Pre-training of deep bidirectional transformers for language understanding [EB/OL ] . ( 2019-05-24 )[ 2024-09-28 ] . https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805 .

RODRIGUEZ A , LAIO A . Clustering by fast search and find of density peaks [J ] . Science , 2014 , 344 ( 6191 ): 1492 - 1496 .

HAN X , ZHU H , YU P F , et al . FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation [C ] // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : ACL , 2018 : 4803 - 4809 .

ZHANG Y H , ZHONG V , CHEN D Q , et al . Position-aware attention and supervised data improve slot filling [C ] // Proceedings of the 2017 Conference on Empirical Methods in NaturalLanguage Processing . Stroudsburg : ACL , 2017 : 35 - 45 .

ZHANG H , LIANG B , YANG M , et al . Prompt-based prototypical framework for continual relation extraction [J ] . IEEE/ACM Transactions on Audio , Speech and Language Processing, 2022 , 30 : 2801 - 2813 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向流形数据的测地距离与余弦互逆近邻密度峰值聚类算法

基于z值的分布式密度峰值聚类算法

基于时序-脉幅聚类的未知雷达辐射源信号分选方法