DRE-3DC: Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling

WANG Yu; WANG Zhen; WEN Li-qiang; LI Wei-ping; ZHAO Wen

doi:10.12263/DZXB.20221187

您当前的位置：

首页 >

文章列表页 >

DRE-3DC: Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling

PAPERS | 更新时间：2025-12-08

- DRE-3DC: Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling
- ACTA ELECTRONICA SINICA Vol. 52, Issue 9, Pages: 2950-2960(2024)
- 作者机构：
  
  1.北京大学软件与微电子学院，北京 100871
  2.北京大学软件工程国家工程研究中心，北京100871
- 作者简介：
- 基金信息：
  
  National Key R&D Program of China(2020YFC0833300)
- DOI：10.12263/DZXB.20221187
  CLC： TP391.1
- Received：20 October 2022，
  
  Revised：2023-02-11，
  
  Published：25 September 2024
- 稿件说明：
移动端阅览
王宇, 王震, 温立强, 等. DRE-3DC: 基于三维表征建模的篇章级关系抽取模型[J]. 电子学报, 2024, 52(09): 2950-2960.

WANG Yu, WANG Zhen, WEN Li-qiang, et al. DRE-3DC: Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling[J]. Acta Electronica Sinica, 2024, 52(09): 2950-2960.
王宇, 王震, 温立强, 等. DRE-3DC: 基于三维表征建模的篇章级关系抽取模型[J]. 电子学报, 2024, 52(09): 2950-2960. DOI：10.12263/DZXB.20221187

WANG Yu, WANG Zhen, WEN Li-qiang, et al. DRE-3DC: Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling[J]. Acta Electronica Sinica, 2024, 52(09): 2950-2960. DOI：10.12263/DZXB.20221187

摘要

篇章级关系抽取任务旨在从非结构化文档的多个句子中提取事实，是构建领域知识库和知识问答应用的关键环节，相较于句子级关系抽取，该任务既要求模型能够基于文档结构特征捕获实体间的复杂交互，还要应对严重的关系类别长尾分布问题.现有基于表格的关系抽取模型主要对文档进行“实体/实体”二维建模，采用多层卷积网络或局部注意力机制提取实体间的交互特征，由于未显式对关系语义进行解耦建模，使得模型无法避免类别重叠影响和捕获关系的方向性特征，导致缺乏实体交互的充分语义信息.针对上述挑战，本文提出了一种基于三维表征建模的篇章级关系抽取模型DRE-3DC（Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling），对二维表格建模方式进行扩展，形成“实体/实体/关系”三维表征建模，采用基于形变卷积的三重注意力机制有效区分和聚合不同语义空间下的实体间及实体与关系的交互表征，自适应地增强模型对文档结构特征的聚合.同时，采用多任务学习方法增强模型对文档整体关系类别组合的感知来缓解篇章级关系抽取任务中的关系类别长尾分布问题.在DocRED和Revisit-DocRED两个篇章级关系抽取数据集上进行的实验结果表明，DRE-3DC模型性能良好，并通过消融实验、对比分析和实例分析，验证了本文所提方法的有效性.

Abstract

The task of document-level relation extraction aims to extract facts from multiple sentences of unstructured documents

which is a key step in the construction of domain knowledge graph and knowledge answering application. The task requires that the model not only capture the complex interactions between entities based on the structural features of documents

but also deal with the serious long-tail category distribution problem. Existing table-based relation extraction models try to solve this issue

but they mainly model documents in two-dimensional “entity/entity” space

and use multi-layer convolutional network or restricted self-attention mechanism to extract the interaction features between entities

which cannot avoid the influence of category overlap and capture the directional features of relationships

resulting in the lack of decoupled semantic information of interaction. For the above challenges

this paper proposes a new document-level relation extraction model

named DRE-3DC (Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling)

in which the “entity/entity” modeling extend to the form of three-dimensional “entity/entities/relationship” modeling method. Based on the deformable convolution in triple attention mechanism

the model effectively distinguishes and integrates the interaction features under different semantic space and adaptively captures the document structural features. At the same time

we propose a multi-task learning method to enhance the perception of relation category combination of documents to alleviate the long-tail distribution problem. The experimental results reveal better score on DocRED and Revisit-DocRED dataset respectively. The effectiveness of the proposed method was verified by ablation experiment

comparative analysis and example analysis.

关键词

Keywords

references

冯钧 , 魏大保 , 苏栋 , 等 . 文档级实体关系抽取方法研究综述 [J ] . 计算机科学 , 2022 , 49 ( 10 ): 224 - 242 .

FENG J , WEI D B , SU D , et al . Survey of document-level entity relation extraction methods [J ] . Computer Science , 2022 , 49 ( 10 ): 224 - 242 . (in Chinese)

冯建周 , 宋沙沙 , 王元卓 , 等 . 基于改进注意力机制的实体关系抽取方法 [J ] . 电子学报 , 2019 , 47 ( 8 ): 1692 - 1700 .

FENG J Z , SONG S S , WANG Y Z , et al . Entity relation extraction based on improved attention mechanism [J ] . Acta Electronica Sinica , 2019 , 47 ( 8 ): 1692 - 1700 . (in Chinese)

SAHU S , ANAND A , ORUGANTY K , et al . Relation extraction from clinical texts using domain invariant convolutional neural network [C ] // Proceedings of the 15th Workshop on Biomedical Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2016 : 206 - 215 .

JIA R , WONG C , POON H . Document-level N-ary relation extraction with multiscale representation learning [C ] // Proceedings of the 2019 Conference of the North . Stroudsburg : Association for Computational Linguistics , 2019 : 3693 - 3704 .

XU Y , YANG Z H , SONG Y W , et al . Star-BiLSTM-LAN for document-level mutation-disease relation extraction from biomedical literature [C ] // 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) . Piscataway : IEEE , 2020 : 357 - 362 .

WANG H , FOCKE C , SYLVESTER R , et al . Fine-tune bert for DocRED with two-step process [EB/OL ] . ( 2019-09-26 ) [ 2022-08-20 ] . https://arxiv.org/abs/1909.11898 https://arxiv.org/abs/1909.11898 .

ZHOU W X , HUANG K , MA T Y , et al . Document-level relation extraction with adaptive thresholding and localized context pooling [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 16 ): 14612 - 14620 .

XU B F , WANG Q , LYU Y J , et al . Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 16 ): 14149 - 14157 .

ZHANG L , CHENG Y D . NC-DRE: Leveraging non-entity clue information for document-level relation extraction [EB/OL ] . ( 2022-04-01 )[ 2022-08-10 ] . http://arxiv.org/abs/2204.00255 http://arxiv.org/abs/2204.00255 .

CHRISTOPOULOU F , MIWA M , ANANIADOU S . Connecting the dots: Document-level neural relation extraction with edge-oriented graphs [C ] // Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Stroudsburg : Association for Computational Linguistics , 2019 : 4924 - 4935 .

李志欣 , 孙亚茹 , 唐素勤 , 等 . 双路注意力引导图卷积网络的关系抽取 [J ] . 电子学报 , 2021 , 49 ( 2 ): 315 - 323 .

LI Z X , SUN Y R , TANG S Q , et al . Dual attention guided graph convolutional networks for relation extraction [J ] . Acta Electronica Sinica , 2021 , 49 ( 2 ): 315 - 323 . (in Chinese)

ZENG S , XU R X , CHANG B B , et al . Double graph based reasoning for document-level relation extraction [C ] // Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Stroudsburg : Association for Computational Linguistics , 2020 : 1630 - 1640 .

WANG H L , QIN K , LU G M , et al . Document-level relation extraction using evidence reasoning on RST-GRAPH [J ] . Knowledge-Based Systems , 2021 , 228 : 107274 .

ZHANG N Y , CHEN X , XIE X , et al . Document-level relation extraction as semantic segmentation [EB/OL ] . ( 2021-08-22 ) [ 2022-08-20 ] . https://arxiv.org/abs/2106. 03618 https://arxiv.org/abs/2106.03618 .

TAN Q Y , HE R D , BING L D , et al . Document-level relation extraction with adaptive focal loss and knowledge distillation [C ] // Findings of the Association for Computational Linguistics: ACL 2022 . Stroudsburg : Association for Computational Linguistics , 2022 : 1672 - 1681 .

ZHANG L , CHENG Y D . A densely connected criss-cross attention network for document-level relation extraction [EB/OL ] . ( 2022-03-26 )[ 2022-08-10 ] . http://arxiv.org/abs/2203.13953 http://arxiv.org/abs/2203.13953 .

DU Y K , MA T F , WU L F , et al . Improving long tailed document-level relation extraction via easy relation augmentation and contrastive learning [EB/OL ] . ( 2022-05-21 )[ 2022-08-10 ] . https://arxiv.org/abs/2205.10511 https://arxiv.org/abs/2205.10511 .

YAO Y , YE D M , LI P , et al . DocRED: A large-scale document-level relation extraction dataset [C ] // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 764 - 777 .

HUANG Q Z , HAO S B , YE Y , et al . Does recommend-revise produce reliable annotations? An analysis on missing instances in DocRED [C ] // Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 6241 - 6252 .

RONNEBERGER O , FISCHER P , BROX T . U-Net: Convolutional networks for biomedical image segmentation [M ] // Lecture Notes in Computer Science . Cham : Springer International Publishing , 2015 : 234 - 241 .

WANG H Y , ZHU Y K , GREEN B , et al . Axial-deeplab: Stand-alone axial-attention for panoptic segmentation [C ] // Computer Vision-ECCV 2020 . New York : ACM , 2020 : 108 - 126 .

HUANG Z L , WANG X G , HUANG L C , et al . CCNet: criss-cross attention for semantic segmentation [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2019 : 603 - 612 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [C ] // Proceedings of the 26th Annual Conference on Neural Information Processing Systems . Cambridge : MIT Press , 2012 : 1106 - 1114 .

YU F , KOLTUN V . Multi-scale context aggregation by dilated convolutions [EB/OL ] . ( 2015-11-23 )[ 2022-08-10 ] . http://arxiv.org/abs/1511.07122 http://arxiv.org/abs/1511.07122 .

DAI J F , QI H Z , XIONG Y W , et al . Deformable convolutional networks [C ] // 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 764 - 773 .

ZHU X Z , HU H , LIN S , et al . Deformable ConvNets V2: More deformable, better results [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 9308 - 9316 .

GUO Z J , ZHANG Y , LU W . Attention guided graph convolutional networks for relation extraction [C ] // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 241 - 251 .

GUO M H , XU T X , LIU J J , et al . Attention mechanisms in computer vision: A survey [J ] . Computational Visual Media , 2022 , 8 ( 3 ): 331 - 368 .

吴绿 , 张馨月 , 唐茉 , 等 . Focus+Context语义表征的场景图像分割 [J ] . 电子学报 , 2021 , 49 ( 3 ): 596 - 604 .

WU L , ZHANG X Y , TANG M , et al . Focus+Context semantic representation in scene segmentation [J ] . Acta Electronica Sinica , 2021 , 49 ( 3 ): 596 - 604 . (in Chinese)

ZHANG Z H , YU W H , YU M X , et al . A survey of multi-task learning in natural language processing: regarding task relatedness and training methods [EB/OL ] . ( 2022-04-07 ) [ 2022-08-10 ] . https://arxiv.org/abs/2204. 03508 https://arxiv.org/abs/2204.03508 .

COLLOBERT R , WESTON J . A unified architecture for natural language processing: Deep neural networks with multitask learning [C ] // Proceedings of the 25th international conference on Machine learning . New York : ACM , 2008 : 160 - 167 .

SUBRAMANIAN S , TRISCHLER A , BENGIO Y , et al . Learning general purpose distributed sentence representations via large scale multi-task learning [EB/OL ] . ( 2018-03-30 )[ 2022-08-10 ] . http://arxiv.org/abs/1804.00079 http://arxiv.org/abs/1804.00079 .

LIU X D , HE P C , CHEN W Z , et al . Multi-task deep neural networks for natural language understanding [C ] // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 4487 - 4496 .

MISRA I , SHRIVASTAVA A , GUPTA A , et al . Cross-stitch networks for multi-task learning [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 3994 - 4003 .

RUDER S , BINGEL J , AUGENSTEIN I , et al . Latent multi-task architecture learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 ( 1 ): 4822 - 4829 .

LIU P F , QIU X P , HUANG X J . Recurrent neural network for text classification with multi-task learning [C ] // Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence . New York : ACM , 2016 : 2873 - 2879 .

HASHIMOTO K , XIONG C M , TSURUOKA Y , et al . A joint many-task model: Growing a neural network for multiple NLP tasks [C ] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2017 : 1923 - 1933 .

PENG H , GAO T Y , HAN X , et al . Learning from context or names? An empirical study on neural relation extraction [C ] // Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Stroudsburg : Association for Computational Linguistics , 2020 : 3661 - 3672 .

MISRA D , NALAMADA T , ARASANIPALAI A U , et al . Rotate to attend: Convolutional triplet attention module [C ] // 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE , 2021 : 3139 - 3148 .

KINGMA D P , BA J . Adam: A method for stochastic optimization [EB/OL ] . ( 2014-12-22 ) [ 2022-08-10 ] . https://arxiv.org/abs/1412.6980 https://arxiv.org/abs/1412.6980 .

LI J Y , XU K , LI F , et al . MRN: A locally and globally mention-based reasoning network for document-level relation extraction [C ] // Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 . Stroudsburg : Association for Computational Linguistics , 2021 : 1359 - 1370 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multimodal Intent Recognition Based on Hierarchical Semantic-Consistency Learning

MoGE: Graph Context Enhanced Multi-Task Recommendation Method

Special Video Recognition Based on Semantic Embedding Learning

Multi-task Learning and Identity-constrained Generative Adversarial Network for Face Frontalization and Recognition

Related Author

WANG Lan

PENG Jun-jie

LI Zheng-yi

ZHANG Huan-xiang

CHEN Chao-chao

ZHANG Zhi-qiang

HU Bin-bin

ZHOU Jun

Related Institution

School of Innovation and Entrepreneurship Education, Inner Mongolia University of Science and Technology

School of Computer Engineering and Science, Shanghai University

Department of Machine Intelligence， Ant Group

College of Computer Science and Technology， Zhejiang University

School of Information and Communication， Communication University of China

⁰