Clinical Relation Extraction via Dual Piecewise Attention Neural Tensor Network

WEI Hao; TANG Huan-ling; ZHOU Ai; ZHANG Yi-jia; CHEN Fei; LU Ming-yu

doi:10.12263/DZXB.20210628

您当前的位置：

首页 >

文章列表页 >

Clinical Relation Extraction via Dual Piecewise Attention Neural Tensor Network

PAPERS | 更新时间：2025-12-08

- Clinical Relation Extraction via Dual Piecewise Attention Neural Tensor Network
- ACTA ELECTRONICA SINICA Vol. 51, Issue 3, Pages: 658-665(2023)
- 作者机构：
  
  1.大连外国语大学软件学院,辽宁大连 116044
  2.大连海事大学信息科学技术学院,辽宁大连 116026
  3.山东工商学院计算机科学与技术学院,山东烟台 264005
- 作者简介：
- 基金信息：
  
  Foundation Item(s):　National Natural Science Foundation of China(61976124;62072070)
- DOI：10.12263/DZXB.20210628
  CLC： TP391
- Received：18 May 2021，
  
  Revised：2021-07-16，
  
  Published：25 March 2023
- 稿件说明：
移动端阅览
隗昊,唐焕玲,周爱等.基于双路分段注意力神经张量网络的临床文本关系抽取[J].电子学报,2023,51(03):658-665.

WEI Hao,TANG Huan-ling,ZHOU Ai,et al.Clinical Relation Extraction via Dual Piecewise Attention Neural Tensor Network[J].ACTA ELECTRONICA SINICA,2023,51(03):658-665.
隗昊,唐焕玲,周爱等.基于双路分段注意力神经张量网络的临床文本关系抽取[J].电子学报,2023,51(03):658-665. DOI： 10.12263/DZXB.20210628.

WEI Hao,TANG Huan-ling,ZHOU Ai,et al.Clinical Relation Extraction via Dual Piecewise Attention Neural Tensor Network[J].ACTA ELECTRONICA SINICA,2023,51(03):658-665. DOI： 10.12263/DZXB.20210628.

摘要

目前，生物医学领域的关系提取工作已经取得了长足的发展，但是在面对句式复杂的临床医学文本时，由于存在大量长句以及句中实体对的高密度分布，限制了当前关系抽取模型性能的进一步提升.本文提出了一种基于张量权重矩阵的双向门控循环单元网络（Tensor-based Bidirectional Gated Recurrent Unit， Tensor-BiGRU）和分段注意力机制的关系抽取模型，基于张量权重矩阵改进BiGRU网络的编码方式，提升神经网络捕获底层特征的能力，而后提出了两种分段注意力机制，以提高模型捕获长句特征的性能.此外，当句子中有多个实体对时，引入实体对的语义信息特征来克服模型的性能下降.本文进一步提出一种权重自适应的交叉熵损失函数，用于提升模型面对数据集中不同关系类别的样本分布不平衡问题的泛化性.实验结果表明，在不依赖任何特征工程和高性能运算环境的情况下，本文模型在2010 i2b2/VA临床关系抽取数据集上实现了先进的性能.

Abstract

At present

biomedical relation extraction has made considerable progress. However

when dealt with complex clinical texts

due to the large number of long sentences and the high density distribution of entity pairs in the sentences

the existing methods of relation extraction still have defects. We propose a relation extraction model via tensor-based bidirectional gate recurrent unit (Tensor-BiGRU) and piecewise attention mechanism. The ability of BiGRU to extract the underlying features is enhanced based on tensor weight matrix. Two kinds of piecewise attention mechanisms are proposed to improve the performance of the model in capturing long sentence features.When the sentence has multiple entity pairs

the semantic representations of the entity pairs are introduced to overcome the performance degradation of the mode. A weight-adaptive cross-entropy loss function is proposed to improve the generalization of the model when the sample distribution of different relation categories in the dataset is unbalanced. The experimental results show that without relying on any feature engineering and high-performance computing environment

the model achieves advanced performance on the 2010 i2b2/VA clinical data set.

关键词

Keywords

references

李志欣 , 孙亚茹 , 唐素勤 , 等 . 双路注意力引导图卷积网络的关系抽取 [J]. 电子学报 , 2021 , 49 ( 2 ): 315 - 323 .

LI Z X , SUN Y R , TANG S Q , et al . Dual attention guided graph convolutional networks for relation extraction [J]. Acta Electronica Sinica , 2021 , 49 ( 2 ): 315 - 323 . (in Chinese)

冯建周 , 宋沙沙 , 王元卓 , 等 . 基于改进注意力机制的实体关系抽取方法 [J]. 电子学报 , 2019 , 47 ( 8 ): 1692 - 1700 .

FENG J Z , SONG S S , WANG Y Z , et al . Entity relation extraction based on improved attention mechanism [J]. Acta Electronica Sinica , 2019 , 47 ( 8 ): 1692 - 1700 . (in Chinese)

杨锦锋 , 于秋滨 , 关毅 , 等 . 电子病历命名实体识别和实体关系抽取研究综述 [J]. 自动化学报 , 2014 , 40 ( 8 ): 1537 - 1562 .

YANG J F , YU Q B , GUAN Y , et al . An overview of research on electronic medical record oriented named entity recognition and entity relation extraction [J]. Acta Automatica Sinica , 2014 , 40 ( 8 ): 1537 - 1562 . (in Chinese)

RINK B , HARABAGIU S , ROBERTS K . Automatic extraction of relations between medical concepts in clinical texts [J]. Journal of the American Medical Informatics Association , 2011 , 18 ( 5 ): 594 - 600 .

DE BRUIJN B , CHERRY C , KIRITCHENKO S , et al . Machine-learned solutions for three stages of clinical information extraction: The state of the art at i2b2 2010 [J]. Journal of the American Medical Informatics Association , 2011 , 18 ( 5 ): 557 - 562 .

SAHU S K , ANAND A , ORUGANTY K , et al . Relation extraction from clinical texts using domain invariant convolutional neural network [EB/OL]. ( 2016-06-30 ): https://arxiv.org/abs/1606.09370.

LUO Y . Recurrent neural networks for classifying relations in clinical notes [J]. Journal of Biomedical Informatics , 2017 , 72 : 85 - 95 .

HE B , GUAN Y , DAI R . Classifying medical relations in clinical text via convolutional neural networks [J]. Artificial Intelligence in Medicine , 2019 , 93 : 43 - 49 .

LI Z , YANG J S , GOU X , et al . Recurrent neural networks with segment attention and entity description for relation extraction from clinical texts [J]. Artificial Intelligence in Medicine , 2019 , 97 : 9 - 18 .

宁尚明 , 滕飞 , 李天瑞 . 基于多通道自注意力机制的电子病历实体关系抽取 [J]. 计算机学报 , 2020 , 43 ( 5 ): 916 - 929 .

NING S M , TENG F , LI T R . Multi-channel self-attention mechanism for relation extraction in clinical records [J]. Chinese Journal of Computers , 2020 , 43 ( 5 ): 916 - 929 . (in Chinese)

LE Q V , MIKOLOV T . Distributed representations of sentences and documents [EB/OL]. ( 2014-05-16 ): https://arxiv.org/abs/1405.4053

ZENG D J , LIU K , CHEN Y B , et al . Distant supervision for relation extraction via piecewise convolutional neural networks [C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2015 : 1753 - 1762 .

SOCHER R , PERELYGIN A , WU J Y , et al . Recursive deep models for semantic compositionality over a sentiment treebank [C]// EMNLP 2013 . Seattle : Association: for Computational Linguistics 2013: 1631 - 1642 .

ZHOU P , SHI W , TIAN J , et al . Attention-based bidirectional long short-term memory networks for relation classification [C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . Stroudsburg, PA, USA : Association for Computational Linguistics , 2016 : 207 - 212 .

ROSS A S , DOSHI-VELEZ F . Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients [EB/OL]. ( 2017-11-26 ): https://arxiv.org/abs/1711.09404.

UZUNER Ö , SOUTH B R , SHEN S , et al . 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text [J]. Journal of the American Medical Informatics Association , 2011 , 18 ( 5 ): 552 - 556 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Implicit Discourse Relation Recognition Integrating Feature Coding and Phrase Interaction Perception

Entity Relation Extraction Based on Improved Attention Mechanism

Related Author

LU Ming-yu

JIN Fang-yan

WANG Xiu-li

JIN Fang-yan

WU Hong-ying

GONG Hao

WANG Yuan-zhuo

Related Institution

Engineering Research Center of State Financial Security， Ministry of Education

College of Information， Central University of Finance and Economics

Institute of Computer Technology, Chinese Academy of Sciences

School of Information Science and Engineering Yanshan University Qinhuangdao Hebei China

Institute of Computer Technology Chinese Academy of Sciences Beijing China

⁰