Entity Relation Extraction Based on Improved Attention Mechanism
FENG Jian-zhou1, SONG Sha-sha1, WANG Yuan-zhuo1,2, LIU Ya-kun1, WU Hong-ying1, GONG Hao1
1. School of Information Science and Engineering, Yanshan University, Qinhuangdao, Hebei 066000, China;
2. Institute of Computer Technology, Chinese Academy of Sciences, Beijing 100080, China
Abstract：Entity relation extraction is a crucial part of knowledge base construction.Among many methods of relationship extraction,the method of distant supervision combined with neural network model is satisfactory in terms of accuracy and other performance.However,there is often a large amount of noise data in the labeled corpus obtained by distant supervision,which has a great impact on the training of relationship extraction model.In this paper,we propose an entity relationship extraction model of convolutional neural network based on improved attention mechanism.Aiming at the sentence set containing the same entity pair,this model tries to find out all the positive instances that embody the relationship between the entity pair,construct the combined sentence vector,and discard the possible noise sentences,so as to minimize the impact of noise sentences and make full use of the semantic information of positive instances.Experimental results show that the accuracy of the proposed relation extraction model is better than that of the comparative relation extraction model.
[1] 陈宇,郑德权,赵铁军.基于Deep Belief Nets的中文名实体关系抽取[J].软件学报,2012,23(10):2572-2585. CHEN Yu,ZHENG De-quan,ZHAO Tie-jun.Chinese relation extraction based on deep belief nets[J].Journal of Software,2012,23(10):2572-2585.(in Chinese)
[2] Hasegawa T,Sekine S,Grishman R.Discovering relations among named entities from large corpora[A].Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics[C].USA:Association for Computational Linguistics,2004.415.
[3] Rozenfeld B,Feldman R.High-performance unsupervised relation extraction from large corpora[A].Proceedings of the Sixth International Conference on Data Mining (ICDM'06)[C].USA:IEEE,2006.1032-1037.
[4] Gonzalez E,Turmo J.Unsupervised relation extraction by massive clustering[A].Proceedings of the Ninth IEEE International Conference on Data Mining[C].USA:IEEE,2009.782-787.
[5] Brin S.Extracting patterns and relations from the world wide web[A].International Workshop on the World Wide Web and Databases[C].Berlin:Springer,1998.172-183.
[6] Agichtein E,Gravano L.Snowball:Extracting relations from large plain-text collections[A].Proceedings of the Fifth ACM Conference on Digital Libraries[C].USA:ACM,2000.85-94.
[7] Liu X,Yu N.Multi-type web relation extraction based on bootstrapping[A].International Conference on Information Engineering(ICIE)[C].USA:IEEE,2010.24-27.
[8] Chen J,Ji D,Tan C L,et al.Relation extraction using label propagation based semi-supervised learning[A].Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics[C].USA:Association for Computational Linguistics,2006.129-136.
[9] Moens M F.Information Extraction:Algorithms and Prospects in a Retrieval Context[M].Germany:Springer,2006.315-317.
[10] Kambhatla N.Combining lexical,syntactic,and semantic features with maximum entropy models for extracting relations[A].Proceedings of the ACL 2004 on Interactive Poster and Demonstration Dessions[C].Spain:Association for Computational Linguistics,2004.22.
[11] Jiang J,Zhai C X.A systematic exploration of the feature space for relation extraction[A].Proceedings of NAACL HLT[C].USA:Association for Computational Linguistics,2007.113-120.
[12] Bunescu R C,Mooney R J.A shortest path dependency kernel for relation extraction[A].Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing[C].USA:Association for Computational Linguistics,2005.724-731.
[13] Mintz M,Bills S,Snow R,et al.Distant supervision for relation extraction without labeled data[A].Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing[C].Singapore:Association for Computational Linguistics,2009.1003-1011.
[14] Zeng D,Liu K,Chen Y,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[A].Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing[C].Portugal:Association for Computational Linguistics,2015.1753-1762.
[15] Lin Y,Shen S,Liu Z,et al.Neural Relation Extraction with Selective Attention over Instances[A].Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1)[C].Germany:Association for Computational Linguistics,2016.2124-2133.
[16] Guoliang Ji,Kang Liu,Shizhu He,Jun Zhao.Distant supervision for relation extraction with sentence-level attention and entity descriptions[A].Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17)[C].USA:AAAI,2017.3060-3066.
[17] Xiaocheng Feng,Jiang Guo,Bing Qin,Ting Liu,Yongjie Liu.Effective deep memory networks for distant supervised relation extraction[A].Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17)[C].Australia:IJCAI,2017.4003-4008.
[18] Collobert R,Weston J,Bottou L,et al.Natural language processing (almost) from scratch[J].Journal of Machine Learning Research,2011,12(Aug):2493-2537.
[19] Hinton G E,Srivastava N,Krizhevsky A,et al.Improving neural networks by preventing co-adaptation of feature detectors[J].arXiv preprint arXiv:1207.0580,2012.
[20] Riedel S,Yao L,McCallum A.Modeling relations and their mentions without labeled text[A].Joint European Conference on Machine Learning and Knowledge Discovery in Databases[C].Berlin:Springer,2010.148-163.
[21] Hoffmann R,Zhang C,Ling X,et al.Knowledge-based weak supervision for information extraction of overlapping relations[A].Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies(Volume 1)[C].USA:Association for Computational Linguistics,2011.541-550.
[22] Surdeanu M,Tibshirani J,Nallapati R,et al.Multi-instance multi-label learning for relation extraction[A].Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning[C].USA:Association for Computational Linguistics,2012.455-465.
[23] Zeiler M D.ADADELTA:an adaptive learning rate method[J].arXiv preprint arXiv:1212.5701,2012.
2019, Vol. 47 
