基于双路分段注意力神经张量网络的临床文本关系抽取

隗昊; 唐焕玲; 周爱; 张益嘉; 陈飞; 鲁明羽

doi:10.12263/DZXB.20210628

您当前的位置：

首页 >

文章列表页 >

基于双路分段注意力神经张量网络的临床文本关系抽取

学术论文 | 更新时间：2025-12-08

- 基于双路分段注意力神经张量网络的临床文本关系抽取
- Clinical Relation Extraction via Dual Piecewise Attention Neural Tensor Network
- 电子学报 2023年51卷第3期页码：658-665
- 作者机构：
  
  1.大连外国语大学软件学院,辽宁大连 116044
  2.大连海事大学信息科学技术学院,辽宁大连 116026
  3.山东工商学院计算机科学与技术学院,山东烟台 264005
- 作者简介：
  
  [ "隗昊男，讲师，博士，1993年生于山东济南．2017年于云南师范大学获得理学硕士学位，2021年于大连海事大学获得工学博士学位，研究方向为生物医学信息抽取、深度学习等． E-mail: wh1005@dlmu.edu.cn" ]
  [ "唐焕玲女，教授，博士，硕士生导师，1970年生于山东龙口．2004年于清华大学获得工学硕士学位，2009年于大连海事大学大学获得工学博士学位．从事机器学习、人工智能、数据挖掘等方向的理论及应用研究． E-mail: thl01@163.com" ]
  [ "周爱女，博士研究生，1990年生于黑龙江哈尔滨．2017年于香港理工大学获得硕士学位，现于大连海事大学攻读博士学位，研究方向为数据挖掘、机器学习等． E-mail: zhouai9070@163.com" ]
  [ "张益嘉男，副教授，博士，硕士生导师，1979年生于吉林长春．2009年于大连理工大学获得工学硕士学位，2014年于大连理工大学获得工学博士学位．从事自然语言处理、生物医学知识挖掘等方向的理论及应用研究． E-mail: zhangyijia@dlmu.edu.cn" ]
  [ "陈飞男，讲师，博士，硕士生导师，1979年出生于辽宁本溪. 2005年于大连海事大学获得工学硕士学位，2010年于大连海事大学获得管理学博士学位.从事人工智能、机器学习、数据挖掘等方向的理论及应用研究. E-mail: dlmucf@163.com" ]
  [ "鲁明羽男，教授，博士生导师．1963年生于黑龙江鸡西．1988年、2002年于清华大学分别获得工学硕士和工学博士学位，从事机器学习、人工智能、数据挖掘等方向的理论及应用研究． E-mail: lumingyu@dlmu.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(61976124;62072070)
- DOI：10.12263/DZXB.20210628
  中图分类号： TP391
- 收稿：2021-05-18，
  
  修回：2021-07-16，
  
  纸质出版：2023-03-25
- 稿件说明：
移动端阅览
隗昊,唐焕玲,周爱等.基于双路分段注意力神经张量网络的临床文本关系抽取[J].电子学报,2023,51(03):658-665.

WEI Hao,TANG Huan-ling,ZHOU Ai,et al.Clinical Relation Extraction via Dual Piecewise Attention Neural Tensor Network[J].ACTA ELECTRONICA SINICA,2023,51(03):658-665.
隗昊,唐焕玲,周爱等.基于双路分段注意力神经张量网络的临床文本关系抽取[J].电子学报,2023,51(03):658-665. DOI： 10.12263/DZXB.20210628.

WEI Hao,TANG Huan-ling,ZHOU Ai,et al.Clinical Relation Extraction via Dual Piecewise Attention Neural Tensor Network[J].ACTA ELECTRONICA SINICA,2023,51(03):658-665. DOI： 10.12263/DZXB.20210628.

摘要

目前，生物医学领域的关系提取工作已经取得了长足的发展，但是在面对句式复杂的临床医学文本时，由于存在大量长句以及句中实体对的高密度分布，限制了当前关系抽取模型性能的进一步提升.本文提出了一种基于张量权重矩阵的双向门控循环单元网络（Tensor-based Bidirectional Gated Recurrent Unit， Tensor-BiGRU）和分段注意力机制的关系抽取模型，基于张量权重矩阵改进BiGRU网络的编码方式，提升神经网络捕获底层特征的能力，而后提出了两种分段注意力机制，以提高模型捕获长句特征的性能.此外，当句子中有多个实体对时，引入实体对的语义信息特征来克服模型的性能下降.本文进一步提出一种权重自适应的交叉熵损失函数，用于提升模型面对数据集中不同关系类别的样本分布不平衡问题的泛化性.实验结果表明，在不依赖任何特征工程和高性能运算环境的情况下，本文模型在2010 i2b2/VA临床关系抽取数据集上实现了先进的性能.

Abstract

At present

biomedical relation extraction has made considerable progress. However

when dealt with complex clinical texts

due to the large number of long sentences and the high density distribution of entity pairs in the sentences

the existing methods of relation extraction still have defects. We propose a relation extraction model via tensor-based bidirectional gate recurrent unit (Tensor-BiGRU) and piecewise attention mechanism. The ability of BiGRU to extract the underlying features is enhanced based on tensor weight matrix. Two kinds of piecewise attention mechanisms are proposed to improve the performance of the model in capturing long sentence features.When the sentence has multiple entity pairs

the semantic representations of the entity pairs are introduced to overcome the performance degradation of the mode. A weight-adaptive cross-entropy loss function is proposed to improve the generalization of the model when the sample distribution of different relation categories in the dataset is unbalanced. The experimental results show that without relying on any feature engineering and high-performance computing environment

the model achieves advanced performance on the 2010 i2b2/VA clinical data set.

关键词

Keywords

references

李志欣 , 孙亚茹 , 唐素勤 , 等 . 双路注意力引导图卷积网络的关系抽取 [J]. 电子学报 , 2021 , 49 ( 2 ): 315 - 323 .

LI Z X , SUN Y R , TANG S Q , et al . Dual attention guided graph convolutional networks for relation extraction [J]. Acta Electronica Sinica , 2021 , 49 ( 2 ): 315 - 323 . (in Chinese)

冯建周 , 宋沙沙 , 王元卓 , 等 . 基于改进注意力机制的实体关系抽取方法 [J]. 电子学报 , 2019 , 47 ( 8 ): 1692 - 1700 .

FENG J Z , SONG S S , WANG Y Z , et al . Entity relation extraction based on improved attention mechanism [J]. Acta Electronica Sinica , 2019 , 47 ( 8 ): 1692 - 1700 . (in Chinese)

杨锦锋 , 于秋滨 , 关毅 , 等 . 电子病历命名实体识别和实体关系抽取研究综述 [J]. 自动化学报 , 2014 , 40 ( 8 ): 1537 - 1562 .

YANG J F , YU Q B , GUAN Y , et al . An overview of research on electronic medical record oriented named entity recognition and entity relation extraction [J]. Acta Automatica Sinica , 2014 , 40 ( 8 ): 1537 - 1562 . (in Chinese)

RINK B , HARABAGIU S , ROBERTS K . Automatic extraction of relations between medical concepts in clinical texts [J]. Journal of the American Medical Informatics Association , 2011 , 18 ( 5 ): 594 - 600 .

DE BRUIJN B , CHERRY C , KIRITCHENKO S , et al . Machine-learned solutions for three stages of clinical information extraction: The state of the art at i2b2 2010 [J]. Journal of the American Medical Informatics Association , 2011 , 18 ( 5 ): 557 - 562 .

SAHU S K , ANAND A , ORUGANTY K , et al . Relation extraction from clinical texts using domain invariant convolutional neural network [EB/OL]. ( 2016-06-30 ): https://arxiv.org/abs/1606.09370.

LUO Y . Recurrent neural networks for classifying relations in clinical notes [J]. Journal of Biomedical Informatics , 2017 , 72 : 85 - 95 .

HE B , GUAN Y , DAI R . Classifying medical relations in clinical text via convolutional neural networks [J]. Artificial Intelligence in Medicine , 2019 , 93 : 43 - 49 .

LI Z , YANG J S , GOU X , et al . Recurrent neural networks with segment attention and entity description for relation extraction from clinical texts [J]. Artificial Intelligence in Medicine , 2019 , 97 : 9 - 18 .

宁尚明 , 滕飞 , 李天瑞 . 基于多通道自注意力机制的电子病历实体关系抽取 [J]. 计算机学报 , 2020 , 43 ( 5 ): 916 - 929 .

NING S M , TENG F , LI T R . Multi-channel self-attention mechanism for relation extraction in clinical records [J]. Chinese Journal of Computers , 2020 , 43 ( 5 ): 916 - 929 . (in Chinese)

LE Q V , MIKOLOV T . Distributed representations of sentences and documents [EB/OL]. ( 2014-05-16 ): https://arxiv.org/abs/1405.4053

ZENG D J , LIU K , CHEN Y B , et al . Distant supervision for relation extraction via piecewise convolutional neural networks [C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2015 : 1753 - 1762 .

SOCHER R , PERELYGIN A , WU J Y , et al . Recursive deep models for semantic compositionality over a sentiment treebank [C]// EMNLP 2013 . Seattle : Association: for Computational Linguistics 2013: 1631 - 1642 .

ZHOU P , SHI W , TIAN J , et al . Attention-based bidirectional long short-term memory networks for relation classification [C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . Stroudsburg, PA, USA : Association for Computational Linguistics , 2016 : 207 - 212 .

ROSS A S , DOSHI-VELEZ F . Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients [EB/OL]. ( 2017-11-26 ): https://arxiv.org/abs/1711.09404.

UZUNER Ö , SOUTH B R , SHEN S , et al . 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text [J]. Journal of the American Medical Informatics Association , 2011 , 18 ( 5 ): 552 - 556 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

融合特征编码和短语交互感知的隐式篇章关系识别

基于改进注意力机制的实体关系抽取方法