基于混合神经网络和注意力机制的生物医学事件触发词识别方法

任永功; 林禹竹; 唐玉洁; 于博; 何馨宇

doi:10.12263/DZXB.20221361

您当前的位置：

首页 >

文章列表页 >

基于混合神经网络和注意力机制的生物医学事件触发词识别方法

学术论文 | 更新时间：2025-12-08

- 基于混合神经网络和注意力机制的生物医学事件触发词识别方法
- A Biomedical Event Trigger Identification Method Based on Hybrid Neural Network and Attention Mechanism
- 电子学报 2024年52卷第9期页码：3206-3216
- 作者机构：
  
  1.辽宁师范大学计算机与人工智能学院,辽宁大连 116081
  2.大连理工大学通信与工程博士后研究站,辽宁大连 116081
  3.大连永佳电子技术有限公司博士后工作站,辽宁大连 116081
- 作者简介：
  
  [ "任永功男，1972年出生.博士，教授，博士生导师，现为辽宁师范大学计算机与信息技术学院教授.主要研究方向为人工智能技术、数据库及数据挖掘技术等.在《计算机学报》等国际、国内计算机类核心刊物上发表论文60余篇，被SCI、EI、ISTP收录30余篇. E-mail: ygren@lnnu.edu.cn" ]
  [ "林禹竹　女，1996年出生，辽宁师范大学计算机与人工智能学院在读博士，主要研究方向为数据挖掘与智能计算、智慧教育与大数据技术." ]
  [ "唐玉洁女，1999年出生于辽宁省大连市.现为辽宁师范大学硕士研究生.主要研究方向为自然语言处理、文本挖掘和生物信息学. E-mail: 609500343@qq.com" ]
  [ "于博男，1996年出生.2022年毕业于辽宁师范大学，获得理学硕士学位.主要研究方向为数据挖掘、自然语言处理、文本挖掘和生物信息学. E-mail: yubochina@aliyun.com" ]
  何馨宇女，1983年出生.博士，副教授，硕士生导师，现为辽宁师范大学计算机与信息技术学院计算机科学与技术（师范）专业专任教师.主要研究方向为文本挖掘和生物信息学.目前已发表30余篇高水平学术论文，其中第一作者SCI、EI检索十余篇，包括国际顶级期刊IEEE Trans系列TCBB（CCF推荐B类SCI期刊）和领域顶级会议BIBM（CCF推荐B类会议）等. Email: hexinyu@lnnu.edu.cn
- 基金信息：
  
  国家自然科学基金(62006108;61976109);辽宁省“兴辽英才计划”项目(XLYC2006005);辽宁省普通高等教育本科教学改革研究项目(辽教通［2022］ 166号);辽宁省高等学校科学研究项目(LJKZ0963);辽宁师范大学本科教学改革研究与实践项目(LSJG202210);辽宁省科技厅重点研发项目(2022JH2/101300271)
- DOI：10.12263/DZXB.20221361
  中图分类号： TP391
- 收稿：2022-12-01，
  
  修回：2023-03-14，
  
  纸质出版：2024-09-25
- 稿件说明：
移动端阅览
任永功, 林禹竹, 唐玉洁, 等. 基于混合神经网络和注意力机制的生物医学事件触发词识别方法[J]. 电子学报, 2024, 52(09): 3206-3216.

REN Yong-gong, LIN Yu-zhu, TANG Yu-jie, et al. A Biomedical Event Trigger Identification Method Based on Hybrid Neural Network and Attention Mechanism[J]. Acta Electronica Sinica, 2024, 52(09): 3206-3216.
任永功, 林禹竹, 唐玉洁, 等. 基于混合神经网络和注意力机制的生物医学事件触发词识别方法[J]. 电子学报, 2024, 52(09): 3206-3216. DOI：10.12263/DZXB.20221361

REN Yong-gong, LIN Yu-zhu, TANG Yu-jie, et al. A Biomedical Event Trigger Identification Method Based on Hybrid Neural Network and Attention Mechanism[J]. Acta Electronica Sinica, 2024, 52(09): 3206-3216. DOI：10.12263/DZXB.20221361

摘要

生物医学事件作为生物医学文本挖掘的重要组成部分，在生物医学研究和疾病的预防中发挥着重要作用.触发词识别是生物医学事件抽取的关键和前提步骤，旨在提取描述事件类型的关键词.传统方法在特征提取过程中过分依赖自然语言处理工具，导致耗费人工成本.另外，由于生物医学文献的特殊性—长文本语句多，导致长距离依赖问题比较明显.为了解决这些问题，我们提出了一种混合结构，由残差卷积神经网络和双向长短期神经网络、混合神经网络和多头注意力机制组成.该模型利用残差卷积神经网络提取单词级特征并利用双向长短期神经网络提取上下文语义信息.此外，本文通过空间域滑动窗口将长句划分为等长短句，在不破坏上下文信息的前提下，避免了长距离依赖.实验结果表明，本文提出的方法在生物医学事件抽取通用语料MLEE（Multi-Level Event Extraction）上取得了较好的效果，

值达到81.15%.

Abstract

Biomedical events

as an important part of biomedical text mining

play an important role in biomedical research and disease prevention. Trigger identification is the key and prerequisite step of biomedical event extraction

which

aims to extract the key words describing event types. Traditional trigger identification methods rely too much on natural language processing tools in the process of feature extraction

consuming a lot of manual cost. In addition

due to the particularity of biomedical literature—there are many long text sentences

the problem of long-distance dependence is obvious. To solve these problems

we propose a hybrid structure

which is composed of residual convolution neural network and bidirectional long short term memory

hybrid neural network and multi head attention mechanism. The proposed model uses residual convolution neural network to extract vocabulary-level features and bidirectional long short term memory to obtain contextual semantic information. Furthermore

spatial domain sliding windows divide long sentences into equal-length short sentences without damaging context information

which can avoid long-distance dependency without destroying the context information. The experimental results show that our method outperforms the state-of-the-art methods on the commonly used multi-level event extraction (MLEE) corpus

achieving 81.15%

-score.

关键词

Keywords

references

GUPTA P , SCHUTZE H , ANDRASSY B . Table filling multi-task recurrent neural network for joint entity and relation extraction [C ] // Proceedings of COLING 2016 , the 26th International Conference on Computational Linguistics . Osaka : The COLING 2016 Organizing Committee , 2016: 2537 - 2547 .

BENGIO Y , DUCHARME R , VINCENT P , et al . A neural probabilistic language model [J ] . Journal of machine learning research , 2003 , 3 : 1137 - 1155 .

MIKOLOV T , CHEN K , CORRADO G , et al . Efficient estimation of word representations in vector space [EB/OL ] . ( 2013-01-16 )[ 22 - 10 - 01 ] . http://arxiv.org/abs/1301.3781 http://arxiv.org/abs/1301.3781 .

MIKOLOV T , SUTSKEVER I , CHEN K , et al . Distributed representations of words and phrases and their compositionality [C ] // Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 . New York : ACM , 2013 : 3111 - 3119 .

MNIH A , HINTON G . A scalable hierarchical distributed language model [C ] // Proceedings of the 21st International Conference on Neural Information Processing Systems . New York : ACM , 2008 : 1081 - 1088 .

PYYSALO S , OHTA T , MIWA M , et al . Event extraction across multiple levels of biological organization [J ] . Bioinformatics , 2012 , 28 ( 18 ): i575 - i581 .

HE X Y , LI L S , LIU Y , et al . A two-stage biomedical event trigger detection method integrating feature selection and word embeddings [J ] . IEEE/ACM Transactions on Computational Biology and Bioinformatics , 2018 , 15 ( 4 ): 1325 - 1332 .

ZHOU D Y , ZHONG D Y , HE Y L . Event trigger identification for biomedical events extraction using domain knowledge [J ] . Bioinformatics , 2014 , 30 ( 11 ): 1587 - 1594 .

NIE Y F , RONG W G , ZHANG Y Y , et al . Embedding assisted prediction architecture for event trigger identification [J ] . Journal of Bioinformatics and Computational Biology , 2015 , 13 ( 3 ): 1541001 .

RAHUL P V S S , SAHU S K , ANAND A . Biomedical event trigger identification using bidirectional recurrent neural network based models [EB/OL ] . ( 2017-05-26 )[ 2022-10-01 ] . http://arxiv.org/abs/1705.09516 http://arxiv.org/abs/1705.09516 .

WEI H , ZHOU A , ZHANG Y J , et al . Biomedical event trigger extraction based on multi-layer residual BiLSTM and contextualized word representations [J ] . International Journal of Machine Learning and Cybernetics , 2022 , 13 ( 3 ): 721 - 733 .

CHEN Y B , XU L H , LIU K , et al . Event extraction via dynamic multi-pooling convolutional neural networks [C ] // Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2015 : 167 - 176 .

DIAO Y F , LIN H F , YANG L , et al . FBSN: A hybrid fine-grained neural network for biomedical event trigger identification [J ] . Neurocomputing , 2020 , 381 ( C ): 105 - 112 .

SHEN C , LIN H F , FAN X C , et al . Biomedical event trigger detection with convolutional highway neural network and extreme learning machine [J ] . Applied Soft Computing , 2019 , 84 : 105661 .

WANG J , ZHANG J H , AN Y , et al . Biomedical event trigger detection by dependency-based word embedding [C ] // Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) . Washington, DC : IEEE , 2015 : 429 - 432 .

LI L S , HUANG M Z , LIU Y , et al . Contextual label sensitive gated network for biomedical event trigger extraction [J ] . Journal of Biomedical Informatics , 2019 , 95 : 103221 .

张亚军 , 刘宗田 , 周文 . 基于深度信念网络的事件识别 [J ] . 电子学报 , 2017 , 45 ( 6 ): 1415 - 1423 .

ZHANG Y J , LIU Z T , ZHOU W . Event recognition based on deep belief network [J ] . Acta Electronica Sinica , 2017 , 45 ( 6 ): 1415 - 1423 . (in Chinese)

SUN L , SONG Y . Research on Classifying performance of SVM with modified kernel function in HCCR [C ] // ICNN&B'05 , International Conference on Neural Networks and Brain . Beijing : IEEE , 2005 , ( 3 ): 1720 - 1723 .

LI Y , YANG T . Word Embedding for understanding natural language: A survey [C ] // Guide to Big Data Applications . Cham : Springer , 2018 : 83 - 104 .

潘博 , 于重重 , 张青川 , 等 . 基于词性与词序的相关因子训练的word2vec改进模型 [J ] . 电子学报 , 2018 , 46 ( 8 ): 1976 - 1982 .

PAN B , YU C C , ZHANG Q C , et al . The improved model for word2vec based on part of speech and word order [J ] . Acta Electronica Sinica , 2018 , 46 ( 8 ): 1976 - 1982 . (in Chinese)

DEVLIN J , CHANG M W , LEE K , et al . BERT: Pre-training of deep bidirectional transformers for language understanding [C ] // Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics . Minneapolis : Association for Computational Linguistics , 2019 : 4171 - 4186 .

LEE J , YOON W , KIM S , et al . BioBERT: A pre-trained biomedical language representation model for biomedical text mining [J ] . Bioinformatics , 2020 , 36 ( 4 ): 1234 - 1240 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [EB/OL ] . ( 2017-06-12 )[ 2022-10-01 ] . http://arxiv.org/abs/1706.03762 http://arxiv.org/abs/1706.03762 .

KETKAR N . Introduction to PyTorch [M ] // Deep Learning with Python . Berkeley, CA : Apress , 2017 : 195 - 208 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据