DRE-3DC: 基于三维表征建模的篇章级关系抽取模型

王宇; 王震; 温立强; 李伟平; 赵文

doi:10.12263/DZXB.20221187

PDF(1263 KB)

电子学报 ›› 2024, Vol. 52 ›› Issue (9) : 2950-2960. DOI: 10.12263/DZXB.20221187

学术论文

DRE-3DC: 基于三维表征建模的篇章级关系抽取模型

作者信息 +

DRE-3DC: Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling

Author information +

文章历史 +

摘要

篇章级关系抽取任务旨在从非结构化文档的多个句子中提取事实，是构建领域知识库和知识问答应用的关键环节，相较于句子级关系抽取，该任务既要求模型能够基于文档结构特征捕获实体间的复杂交互，还要应对严重的关系类别长尾分布问题.现有基于表格的关系抽取模型主要对文档进行“实体/实体”二维建模，采用多层卷积网络或局部注意力机制提取实体间的交互特征，由于未显式对关系语义进行解耦建模，使得模型无法避免类别重叠影响和捕获关系的方向性特征，导致缺乏实体交互的充分语义信息.针对上述挑战，本文提出了一种基于三维表征建模的篇章级关系抽取模型DRE-3DC（Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling），对二维表格建模方式进行扩展，形成“实体/实体/关系”三维表征建模，采用基于形变卷积的三重注意力机制有效区分和聚合不同语义空间下的实体间及实体与关系的交互表征，自适应地增强模型对文档结构特征的聚合.同时，采用多任务学习方法增强模型对文档整体关系类别组合的感知来缓解篇章级关系抽取任务中的关系类别长尾分布问题.在DocRED和Revisit-DocRED两个篇章级关系抽取数据集上进行的实验结果表明，DRE-3DC模型性能良好，并通过消融实验、对比分析和实例分析，验证了本文所提方法的有效性.

Abstract

The task of document-level relation extraction aims to extract facts from multiple sentences of unstructured documents, which is a key step in the construction of domain knowledge graph and knowledge answering application. The task requires that the model not only capture the complex interactions between entities based on the structural features of documents, but also deal with the serious long-tail category distribution problem. Existing table-based relation extraction models try to solve this issue, but they mainly model documents in two-dimensional “entity/entity” space, and use multi-layer convolutional network or restricted self-attention mechanism to extract the interaction features between entities, which cannot avoid the influence of category overlap and capture the directional features of relationships, resulting in the lack of decoupled semantic information of interaction. For the above challenges, this paper proposes a new document-level relation extraction model, named DRE-3DC (Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling), in which the “entity/entity” modeling extend to the form of three-dimensional “entity/entities/relationship” modeling method. Based on the deformable convolution in triple attention mechanism, the model effectively distinguishes and integrates the interaction features under different semantic space and adaptively captures the document structural features. At the same time, we propose a multi-task learning method to enhance the perception of relation category combination of documents to alleviate the long-tail distribution problem. The experimental results reveal better score on DocRED and Revisit-DocRED dataset respectively. The effectiveness of the proposed method was verified by ablation experiment, comparative analysis and example analysis.

导出引用

王宇 , 王震 , 温立强 , 李伟平 , 赵文. DRE-3DC: 基于三维表征建模的篇章级关系抽取模型[J]. 电子学报, 2024, 52(9): 2950-2960. https://doi.org/10.12263/DZXB.20221187

WANG Yu , WANG Zhen , WEN Li-qiang , LI Wei-ping , ZHAO Wen. DRE-3DC: Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling[J]. Acta Electronica Sinica, 2024, 52(9): 2950-2960. https://doi.org/10.12263/DZXB.20221187

参考文献

原文顺序 | 文献年度倒序 | 文中引用次数倒序

1	冯钧, 魏大保, 苏栋, 等. 文档级实体关系抽取方法研究综述[J]. 计算机科学, 2022, 49(10): 224-242. FENG J, WEI D B, SU D, et al. Survey of document-level entity relation extraction methods[J]. Computer Science, 2022, 49(10): 224-242. (in Chinese) 本文引用 [1]

2	冯建周, 宋沙沙, 王元卓, 等. 基于改进注意力机制的实体关系抽取方法[J]. 电子学报, 2019, 47(8): 1692-1700. FENG J Z, SONG S S, WANG Y Z, et al. Entity relation extraction based on improved attention mechanism[J]. Acta Electronica Sinica, 2019, 47(8): 1692-1700. (in Chinese) 本文引用 [1]

3	SAHU S, ANAND A, ORUGANTY K, et al. Relation extraction from clinical texts using domain invariant convolutional neural network[C]//Proceedings of the 15th Workshop on Biomedical Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2016: 206-215. 本文引用 [1]

4	JIA R, WONG C, POON H. Document-level N-ary relation extraction with multiscale representation learning[C]//Proceedings of the 2019 Conference of the North. Stroudsburg: Association for Computational Linguistics, 2019: 3693-3704.

5	XU Y, YANG Z H, SONG Y W, et al. Star-BiLSTM-LAN for document-level mutation-disease relation extraction from biomedical literature[C]//2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Piscataway: IEEE, 2020: 357-362.

6	WANG H, FOCKE C, SYLVESTER R, et al. Fine-tune bert for DocRED with two-step process[EB/OL]. (2019-09-26) [2022-08-20]. https://arxiv.org/abs/1909.11898

7	ZHOU W X, HUANG K, MA T Y, et al. Document-level relation extraction with adaptive thresholding and localized context pooling[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(16): 14612-14620. 本文引用 [2]

8	XU B F, WANG Q, LYU Y J, et al. Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(16): 14149-14157. 本文引用 [2]

9	ZHANG L, CHENG Y D. NC-DRE: Leveraging non-entity clue information for document-level relation extraction[EB/OL]. (2022-04-01)[2022-08-10]. http://arxiv.org/abs/2204.00255 本文引用 [2]

CHRISTOPOULOU

, MIWA

, ANANIADOU

. Connecting the dots: Document-level neural relation extraction with edge-oriented graphs[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Stroudsburg: Association for Computational Linguistics, 2019: 4924-4935.

本文引用 [3]

11	李志欣, 孙亚茹, 唐素勤, 等. 双路注意力引导图卷积网络的关系抽取[J]. 电子学报, 2021, 49(2): 315-323. LI Z X, SUN Y R, TANG S Q, et al. Dual attention guided graph convolutional networks for relation extraction[J]. Acta Electronica Sinica, 2021, 49(2): 315-323. (in Chinese)

12	ZENG S, XU R X, CHANG B B, et al. Double graph based reasoning for document-level relation extraction[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg: Association for Computational Linguistics, 2020: 1630-1640. 本文引用 [3]

13	WANG H L, QIN K, LU G M, et al. Document-level relation extraction using evidence reasoning on RST-GRAPH[J]. Knowledge-Based Systems, 2021, 228: 107274. 本文引用 [2]

14	ZHANG N Y, CHEN X, XIE X, et al. Document-level relation extraction as semantic segmentation[EB/OL]. (2021-08-22) [2022-08-20]. https://arxiv.org/abs/2106. 03618 本文引用 [6]

15	TAN Q Y, HE R D, BING L D, et al. Document-level relation extraction with adaptive focal loss and knowledge distillation[C]//Findings of the Association for Computational Linguistics: ACL 2022. Stroudsburg: Association for Computational Linguistics, 2022: 1672-1681. 本文引用 [6]

16	ZHANG L, CHENG Y D. A densely connected criss-cross attention network for document-level relation extraction[EB/OL]. (2022-03-26)[2022-08-10]. http://arxiv.org/abs/2203.13953 本文引用 [4]

17	DU Y K, MA T F, WU L F, et al. Improving long tailed document-level relation extraction via easy relation augmentation and contrastive learning[EB/OL]. (2022-05-21)[2022-08-10]. https://arxiv.org/abs/2205.10511 本文引用 [1]

18	YAO Y, YE D M, LI P, et al. DocRED: A large-scale document-level relation extraction dataset[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 764-777. 本文引用 [3]

HUANG

Q Z

, HAO

S B

, YE

, et al. Does recommend-revise produce reliable annotations? An analysis on missing instances in DocRED[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: Association for Computational Linguistics, 2022: 6241-6252.

本文引用 [2]

20	RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241. 本文引用 [1]

21	WANG H Y, ZHU Y K, GREEN B, et al. Axial-deeplab: Stand-alone axial-attention for panoptic segmentation[C]//Computer Vision-ECCV 2020. New York: ACM, 2020: 108-126. 本文引用 [1]

22	HUANG Z L, WANG X G, HUANG L C, et al. CCNet: criss-cross attention for semantic segmentation[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 603-612. 本文引用 [1]

23	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 26th Annual Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2012: 1106-1114. 本文引用 [1]

24	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. (2015-11-23)[2022-08-10]. http://arxiv.org/abs/1511.07122 本文引用 [1]

25	DAI J F, QI H Z, XIONG Y W, et al. Deformable convolutional networks[C]//2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2017: 764-773. 本文引用 [1]

26	ZHU X Z, HU H, LIN S, et al. Deformable ConvNets V2: More deformable, better results[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 9308-9316. 本文引用 [2]

27	GUO Z J, ZHANG Y, LU W. Attention guided graph convolutional networks for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 241-251. 本文引用 [1]

28	GUO M H, XU T X, LIU J J, et al. Attention mechanisms in computer vision: A survey[J]. Computational Visual Media, 2022, 8(3): 331-368. 本文引用 [1]

29	吴绿, 张馨月, 唐茉, 等. Focus+Context语义表征的场景图像分割[J]. 电子学报, 2021, 49(3): 596-604. WU L, ZHANG X Y, TANG M, et al. Focus+Context semantic representation in scene segmentation[J]. Acta Electronica Sinica, 2021, 49(3): 596-604. (in Chinese) 本文引用 [1]

30	ZHANG Z H, YU W H, YU M X, et al. A survey of multi-task learning in natural language processing: regarding task relatedness and training methods[EB/OL]. (2022-04-07) [2022-08-10]. https://arxiv.org/abs/2204. 03508 本文引用 [1]

31	COLLOBERT R, WESTON J. A unified architecture for natural language processing: Deep neural networks with multitask learning[C]//Proceedings of the 25th international conference on Machine learning. New York: ACM, 2008: 160-167. 本文引用 [1]

32	SUBRAMANIAN S, TRISCHLER A, BENGIO Y, et al. Learning general purpose distributed sentence representations via large scale multi-task learning[EB/OL]. (2018-03-30)[2022-08-10]. http://arxiv.org/abs/1804.00079

33	LIU X D, HE P C, CHEN W Z, et al. Multi-task deep neural networks for natural language understanding[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 4487-4496. 本文引用 [1]

34	MISRA I, SHRIVASTAVA A, GUPTA A, et al. Cross-stitch networks for multi-task learning[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2016: 3994-4003. 本文引用 [1]

35	RUDER S, BINGEL J, AUGENSTEIN I, et al. Latent multi-task architecture learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 4822-4829.

36	LIU P F, QIU X P, HUANG X J. Recurrent neural network for text classification with multi-task learning[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. New York: ACM, 2016: 2873-2879. 本文引用 [1]

37	HASHIMOTO K, XIONG C M, TSURUOKA Y, et al. A joint many-task model: Growing a neural network for multiple NLP tasks[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 1923-1933. 本文引用 [1]

38	PENG H, GAO T Y, HAN X, et al. Learning from context or names? An empirical study on neural relation extraction[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg: Association for Computational Linguistics, 2020: 3661-3672. 本文引用 [1]

39	MISRA D, NALAMADA T, ARASANIPALAI A U, et al. Rotate to attend: Convolutional triplet attention module[C]//2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE, 2021: 3139-3148. 本文引用 [1]

40	KINGMA D P, BA J. Adam: A method for stochastic optimization[EB/OL]. (2014-12-22) [2022-08-10]. https://arxiv.org/abs/1412.6980 本文引用 [1]

41	LI J Y, XU K, LI F, et al. MRN: A locally and globally mention-based reasoning network for document-level relation extraction[C]//Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Stroudsburg: Association for Computational Linguistics, 2021: 1359-1370. 本文引用 [1]