A Biomedical Event Trigger Identification Method Based on Hybrid Neural Network and Attention Mechanism

REN Yong-gong; LIN Yu-zhu; TANG Yu-jie; YU Bo; HE Xin-yu

doi:10.12263/DZXB.20221361

您当前的位置：

首页 >

文章列表页 >

A Biomedical Event Trigger Identification Method Based on Hybrid Neural Network and Attention Mechanism

PAPERS | 更新时间：2025-12-08

- A Biomedical Event Trigger Identification Method Based on Hybrid Neural Network and Attention Mechanism
- ACTA ELECTRONICA SINICA Vol. 52, Issue 9, Pages: 3206-3216(2024)
- 作者机构：
  
  1.辽宁师范大学计算机与人工智能学院,辽宁大连 116081
  2.大连理工大学通信与工程博士后研究站,辽宁大连 116081
  3.大连永佳电子技术有限公司博士后工作站,辽宁大连 116081
- 作者简介：
- 基金信息：
  
  National Science Foundation of China(62006108;61976109);Liaoning Revitalization Talents Program(XLYC2006005);Research Project of Liaoning General Higher Education (Liao Jiao Tong [2022] No.166)(辽教通［2022］ 166号);Scientific Research Project of Liaoning Province(LJKZ0963);Research and Practice Project of Undergraduate Teaching Reform of Liaoning Normal University(LSJG202210);Liaoning Provincial Department of Science and Technology(2022JH2/101300271)
- DOI：10.12263/DZXB.20221361
  CLC： TP391
- Received：01 December 2022，
  
  Revised：2023-03-14，
  
  Published：25 September 2024
- 稿件说明：
移动端阅览
任永功, 林禹竹, 唐玉洁, 等. 基于混合神经网络和注意力机制的生物医学事件触发词识别方法[J]. 电子学报, 2024, 52(09): 3206-3216.

REN Yong-gong, LIN Yu-zhu, TANG Yu-jie, et al. A Biomedical Event Trigger Identification Method Based on Hybrid Neural Network and Attention Mechanism[J]. Acta Electronica Sinica, 2024, 52(09): 3206-3216.
任永功, 林禹竹, 唐玉洁, 等. 基于混合神经网络和注意力机制的生物医学事件触发词识别方法[J]. 电子学报, 2024, 52(09): 3206-3216. DOI：10.12263/DZXB.20221361

REN Yong-gong, LIN Yu-zhu, TANG Yu-jie, et al. A Biomedical Event Trigger Identification Method Based on Hybrid Neural Network and Attention Mechanism[J]. Acta Electronica Sinica, 2024, 52(09): 3206-3216. DOI：10.12263/DZXB.20221361

摘要

生物医学事件作为生物医学文本挖掘的重要组成部分，在生物医学研究和疾病的预防中发挥着重要作用.触发词识别是生物医学事件抽取的关键和前提步骤，旨在提取描述事件类型的关键词.传统方法在特征提取过程中过分依赖自然语言处理工具，导致耗费人工成本.另外，由于生物医学文献的特殊性—长文本语句多，导致长距离依赖问题比较明显.为了解决这些问题，我们提出了一种混合结构，由残差卷积神经网络和双向长短期神经网络、混合神经网络和多头注意力机制组成.该模型利用残差卷积神经网络提取单词级特征并利用双向长短期神经网络提取上下文语义信息.此外，本文通过空间域滑动窗口将长句划分为等长短句，在不破坏上下文信息的前提下，避免了长距离依赖.实验结果表明，本文提出的方法在生物医学事件抽取通用语料MLEE（Multi-Level Event Extraction）上取得了较好的效果，

值达到81.15%.

Abstract

Biomedical events

as an important part of biomedical text mining

play an important role in biomedical research and disease prevention. Trigger identification is the key and prerequisite step of biomedical event extraction

which

aims to extract the key words describing event types. Traditional trigger identification methods rely too much on natural language processing tools in the process of feature extraction

consuming a lot of manual cost. In addition

due to the particularity of biomedical literature—there are many long text sentences

the problem of long-distance dependence is obvious. To solve these problems

we propose a hybrid structure

which is composed of residual convolution neural network and bidirectional long short term memory

hybrid neural network and multi head attention mechanism. The proposed model uses residual convolution neural network to extract vocabulary-level features and bidirectional long short term memory to obtain contextual semantic information. Furthermore

spatial domain sliding windows divide long sentences into equal-length short sentences without damaging context information

which can avoid long-distance dependency without destroying the context information. The experimental results show that our method outperforms the state-of-the-art methods on the commonly used multi-level event extraction (MLEE) corpus

achieving 81.15%

-score.

关键词

Keywords

references

GUPTA P , SCHUTZE H , ANDRASSY B . Table filling multi-task recurrent neural network for joint entity and relation extraction [C ] // Proceedings of COLING 2016 , the 26th International Conference on Computational Linguistics . Osaka : The COLING 2016 Organizing Committee , 2016: 2537 - 2547 .

BENGIO Y , DUCHARME R , VINCENT P , et al . A neural probabilistic language model [J ] . Journal of machine learning research , 2003 , 3 : 1137 - 1155 .

MIKOLOV T , CHEN K , CORRADO G , et al . Efficient estimation of word representations in vector space [EB/OL ] . ( 2013-01-16 )[ 22 - 10 - 01 ] . http://arxiv.org/abs/1301.3781 http://arxiv.org/abs/1301.3781 .

MIKOLOV T , SUTSKEVER I , CHEN K , et al . Distributed representations of words and phrases and their compositionality [C ] // Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 . New York : ACM , 2013 : 3111 - 3119 .

MNIH A , HINTON G . A scalable hierarchical distributed language model [C ] // Proceedings of the 21st International Conference on Neural Information Processing Systems . New York : ACM , 2008 : 1081 - 1088 .

PYYSALO S , OHTA T , MIWA M , et al . Event extraction across multiple levels of biological organization [J ] . Bioinformatics , 2012 , 28 ( 18 ): i575 - i581 .

HE X Y , LI L S , LIU Y , et al . A two-stage biomedical event trigger detection method integrating feature selection and word embeddings [J ] . IEEE/ACM Transactions on Computational Biology and Bioinformatics , 2018 , 15 ( 4 ): 1325 - 1332 .

ZHOU D Y , ZHONG D Y , HE Y L . Event trigger identification for biomedical events extraction using domain knowledge [J ] . Bioinformatics , 2014 , 30 ( 11 ): 1587 - 1594 .

NIE Y F , RONG W G , ZHANG Y Y , et al . Embedding assisted prediction architecture for event trigger identification [J ] . Journal of Bioinformatics and Computational Biology , 2015 , 13 ( 3 ): 1541001 .

RAHUL P V S S , SAHU S K , ANAND A . Biomedical event trigger identification using bidirectional recurrent neural network based models [EB/OL ] . ( 2017-05-26 )[ 2022-10-01 ] . http://arxiv.org/abs/1705.09516 http://arxiv.org/abs/1705.09516 .

WEI H , ZHOU A , ZHANG Y J , et al . Biomedical event trigger extraction based on multi-layer residual BiLSTM and contextualized word representations [J ] . International Journal of Machine Learning and Cybernetics , 2022 , 13 ( 3 ): 721 - 733 .

CHEN Y B , XU L H , LIU K , et al . Event extraction via dynamic multi-pooling convolutional neural networks [C ] // Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2015 : 167 - 176 .

DIAO Y F , LIN H F , YANG L , et al . FBSN: A hybrid fine-grained neural network for biomedical event trigger identification [J ] . Neurocomputing , 2020 , 381 ( C ): 105 - 112 .

SHEN C , LIN H F , FAN X C , et al . Biomedical event trigger detection with convolutional highway neural network and extreme learning machine [J ] . Applied Soft Computing , 2019 , 84 : 105661 .

WANG J , ZHANG J H , AN Y , et al . Biomedical event trigger detection by dependency-based word embedding [C ] // Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) . Washington, DC : IEEE , 2015 : 429 - 432 .

LI L S , HUANG M Z , LIU Y , et al . Contextual label sensitive gated network for biomedical event trigger extraction [J ] . Journal of Biomedical Informatics , 2019 , 95 : 103221 .

张亚军 , 刘宗田 , 周文 . 基于深度信念网络的事件识别 [J ] . 电子学报 , 2017 , 45 ( 6 ): 1415 - 1423 .

ZHANG Y J , LIU Z T , ZHOU W . Event recognition based on deep belief network [J ] . Acta Electronica Sinica , 2017 , 45 ( 6 ): 1415 - 1423 . (in Chinese)

SUN L , SONG Y . Research on Classifying performance of SVM with modified kernel function in HCCR [C ] // ICNN&B'05 , International Conference on Neural Networks and Brain . Beijing : IEEE , 2005 , ( 3 ): 1720 - 1723 .

LI Y , YANG T . Word Embedding for understanding natural language: A survey [C ] // Guide to Big Data Applications . Cham : Springer , 2018 : 83 - 104 .

潘博 , 于重重 , 张青川 , 等 . 基于词性与词序的相关因子训练的word2vec改进模型 [J ] . 电子学报 , 2018 , 46 ( 8 ): 1976 - 1982 .

PAN B , YU C C , ZHANG Q C , et al . The improved model for word2vec based on part of speech and word order [J ] . Acta Electronica Sinica , 2018 , 46 ( 8 ): 1976 - 1982 . (in Chinese)

DEVLIN J , CHANG M W , LEE K , et al . BERT: Pre-training of deep bidirectional transformers for language understanding [C ] // Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics . Minneapolis : Association for Computational Linguistics , 2019 : 4171 - 4186 .

LEE J , YOON W , KIM S , et al . BioBERT: A pre-trained biomedical language representation model for biomedical text mining [J ] . Bioinformatics , 2020 , 36 ( 4 ): 1234 - 1240 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [EB/OL ] . ( 2017-06-12 )[ 2022-10-01 ] . http://arxiv.org/abs/1706.03762 http://arxiv.org/abs/1706.03762 .

KETKAR N . Introduction to PyTorch [M ] // Deep Learning with Python . Berkeley, CA : Apress , 2017 : 195 - 208 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

REN Yong-gong

LIN Yu-zhu

TANG Yu-jie

YU Bo

HE Xin-yu

Related Institution

School of Computer and Information Technology, Liaoning Normal University

Postdoctoral Workstation of Dalian Yongjia Electronic Technology Co., Ltd

⁰