
DRE-3DC: 基于三维表征建模的篇章级关系抽取模型
DRE-3DC: Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling
篇章级关系抽取任务旨在从非结构化文档的多个句子中提取事实,是构建领域知识库和知识问答应用的关键环节,相较于句子级关系抽取,该任务既要求模型能够基于文档结构特征捕获实体间的复杂交互,还要应对严重的关系类别长尾分布问题.现有基于表格的关系抽取模型主要对文档进行“实体/实体”二维建模,采用多层卷积网络或局部注意力机制提取实体间的交互特征,由于未显式对关系语义进行解耦建模,使得模型无法避免类别重叠影响和捕获关系的方向性特征,导致缺乏实体交互的充分语义信息.针对上述挑战,本文提出了一种基于三维表征建模的篇章级关系抽取模型DRE-3DC(Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling),对二维表格建模方式进行扩展,形成“实体/实体/关系”三维表征建模,采用基于形变卷积的三重注意力机制有效区分和聚合不同语义空间下的实体间及实体与关系的交互表征,自适应地增强模型对文档结构特征的聚合.同时,采用多任务学习方法增强模型对文档整体关系类别组合的感知来缓解篇章级关系抽取任务中的关系类别长尾分布问题.在DocRED和Revisit-DocRED两个篇章级关系抽取数据集上进行的实验结果表明,DRE-3DC模型性能良好,并通过消融实验、对比分析和实例分析,验证了本文所提方法的有效性.
The task of document-level relation extraction aims to extract facts from multiple sentences of unstructured documents, which is a key step in the construction of domain knowledge graph and knowledge answering application. The task requires that the model not only capture the complex interactions between entities based on the structural features of documents, but also deal with the serious long-tail category distribution problem. Existing table-based relation extraction models try to solve this issue, but they mainly model documents in two-dimensional “entity/entity” space, and use multi-layer convolutional network or restricted self-attention mechanism to extract the interaction features between entities, which cannot avoid the influence of category overlap and capture the directional features of relationships, resulting in the lack of decoupled semantic information of interaction. For the above challenges, this paper proposes a new document-level relation extraction model, named DRE-3DC (Document-Level Relation Extraction with Three-Dimensional Representation Combination Modeling), in which the “entity/entity” modeling extend to the form of three-dimensional “entity/entities/relationship” modeling method. Based on the deformable convolution in triple attention mechanism, the model effectively distinguishes and integrates the interaction features under different semantic space and adaptively captures the document structural features. At the same time, we propose a multi-task learning method to enhance the perception of relation category combination of documents to alleviate the long-tail distribution problem. The experimental results reveal better score on DocRED and Revisit-DocRED dataset respectively. The effectiveness of the proposed method was verified by ablation experiment, comparative analysis and example analysis.
篇章级关系抽取 / 三维表征 / 三重注意力 / 形变卷积网络 / 多任务学习 {{custom_keyword}} /
document-level relation extraction / three-dimensional representation / triplet attention / deformable convolution / multi-task learning {{custom_keyword}} /
表1 实验所用数据集的相关统计信息 |
统计信息 | DocRED | Revisit-DocRED |
---|---|---|
训练集 | 3 053 | 3 053 |
验证集 | 998 | 902 |
测试集 | 1 000 | 96 |
关系种类 | 97 | 97 |
平均实体数量 | 19.5 | 19.7 |
平均元组数量 | 12.6 | 34.5 |
表2 在DocRED数据集上的实验结果 |
数据集类型 | 模型 | 验证集 | 测试集 | ||
---|---|---|---|---|---|
Ign F1 | F1 | Ign F1 | F1 | ||
不使用远程 监督数据集 | NC-DRE-B-b[9] | 60.84 | 62.75 | 60.59 | 62.73 |
DocuNet-B-b[14] | 59.86 | 61.83 | 59.93 | 61.86 | |
KD-DocRE-B-b[15] | 60.08 | 62.03 | 60.04 | 62.08 | |
Dense-CCNet-B-b[16] | 60.72 | 62.74 | 60.46 | 62.55 | |
MRN-B-b[41] | 59.74 | 61.61 | 59.52 | 61.74 | |
Ours-B-b | 61.19 | 63.11 | 61.23 | 63.14 | |
DocuNet-Rb-l[14] | 62.23 | 64.12 | 62.39 | 64.55 | |
KD-DocRE-Rb-l[15] | 62.16 | 64.19 | 62.57 | 64.28 | |
Ours-Rb-l | 62.51 | 64.59 | 62.88 | 65.17 | |
使用远程 监督数据集 | DocuNet-NA-Rb-l[14] | 63.26 | 65.21 | 63.29 | 65.44 |
ATLOP-NA-Rb-l[7] | 63.41 | 65.33 | 63.54 | 65.47 | |
KD-DocRE-NA-Rb-l[15] | 63.38 | 65 64 | 63.63 | 65.71 | |
SSAN-NA-Rb-l[8] | 63.76 | 65.69 | 63.78 | 65.92 | |
Ours-NA-Rb-l | 64.24 | 66.34 | 63.93 | 66.19 |
表3 在Revisit⁃DocRED数据集上的实验结果 |
模型 | Revise | Scratch | ||||
---|---|---|---|---|---|---|
P | R | F1 | P | R | F1 | |
BiLSTM[18] | 50.2 | 46.7 | 48.4 | 66.6 | 22.8 | 33.9 |
GAIN[12] | 60.0 | 56.8 | 58.3 | 81.1 | 28.1 | 41.8 |
ATLOP[7] | 66.3 | 59.1 | 62.5 | 90.3 | 29.5 | 44.5 |
SSAN[8] | 63.1 | 61.3 | 62.2 | 84.5 | 30.1 | 44.5 |
DocuNet[14] | 66.9 | 59.9 | 63.2 | 89.1 | 29.3 | 44.1 |
KD-DocRE[15] | 65.4 | 62.9 | 64.1 | 88.4 | 29.4 | 44.2 |
Ours | 63.2 | 67.4 | 65.2 | 85.5 | 31.6 | 46.2 |
表4 在DocRED数据集上的消融实验结果 |
模型变体 | 验证集 | |||
---|---|---|---|---|
P | R | Ing F1 | F1 | |
Ours | 66.62 | 62.57 | 62.51 | 64.59 |
-Triplet Attn | 64.52 | 63.74 | 61.24 | 64.13 |
-Deform Conv | 64.21 | 63.46 | 61.05 | 63.83 |
-Global Relation | 65.24 | 62.31 | 62.02 | 64.08 |
表5 在Revisit⁃DocRED数据集上的消融实验结果 |
模型变体 | Scratch数据集 | ||
---|---|---|---|
P | R | F1 | |
Ours | 85.5 | 31.6 | 46.2 |
-Triplet Attn | 84.3 | 30.2 | 45.3 |
-Deform Conv | 83.7 | 31.4 | 45.7 |
-Global Relation | 87.6 | 30.8 | 45.6 |
表6 实例分析例句 |
例句1 | 文本 内容 | English is the language in Great Britain and United States. A Loyal Character Dancer was published by Soho Press in the United States. |
---|---|---|
包含的 三元组 | (Language,spokenIn,Britain), (Dancer,publisher,Press),(States,language,language),(Press,country,States) | |
例句2 | 文本 内容 | Cornell University in Ithaca, New York is the publisher of Administrative Science Quarterly. The University is affiliated with the Association of American Universities. |
包含的 三元组 | (University,city,York), (University,state,York), (University,affiliation,Universities), (Quarterly,publisher,University) | |
例句3 | 文本 内容 | Elliot See was born in Dallas, which is a country in Texas. He attended the University of Texas at Austin, which is affiliated to the University of Texas system. The University of Texas at Austin will be part of the Big 12 Conference competition. |
包含的 三元组 | (See, birthplace, Dallas), (See,almaMater,Austin), (Austin,compete in,Conference), (Dallas,partsType,Texas) |
表7 实例分析结果 |
例句 | 方法 | 输出 |
---|---|---|
例句1 | DocuNet | (Language, spokenIn, Britain), (Dancer, publisher, Press), (Dancer, country, States), |
本文方法 | (Language, spokenIn, Britain), (Dancer, publisher, Press), (States, language, language), (Press, country,States) | |
例句2 | KD-DocRE | (University, city, York), (University, affiliation, Universities), (Quarterly, publisher, University) |
本文方法 | (University, city, York), (University, state, York), (University, affiliation, Universities), (Quarterly, publisher, University) | |
例句3 | KD-DocRE | (See, birthplace, Dallas), (See, almaMater, Austin), (Austin, compete in, Conference) |
本文方法 | (See, birthplace, Dallas), (See, almaMater, Austin), (Austin, compete in, Conference), (Dallas, part, Texas) |
1 |
冯钧, 魏大保, 苏栋, 等. 文档级实体关系抽取方法研究综述[J]. 计算机科学, 2022, 49(10): 224-242.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
2 |
冯建周, 宋沙沙, 王元卓, 等. 基于改进注意力机制的实体关系抽取方法[J]. 电子学报, 2019, 47(8): 1692-1700.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
3 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
4 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
5 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
6 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
7 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
8 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
9 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
10 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
11 |
李志欣, 孙亚茹, 唐素勤, 等. 双路注意力引导图卷积网络的关系抽取[J]. 电子学报, 2021, 49(2): 315-323.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
12 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
13 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
14 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
15 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
16 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
17 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
18 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
19 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
20 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
21 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
22 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
23 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
24 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
25 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
26 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
27 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
28 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
29 |
吴绿, 张馨月, 唐茉, 等. Focus+Context语义表征的场景图像分割[J]. 电子学报, 2021, 49(3): 596-604.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
30 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
31 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
32 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
33 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
34 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
35 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
36 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
37 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
38 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
39 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
40 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
41 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
{{custom_ref.label}} |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
/
〈 |
|
〉 |