基于高效用神经网络的文本分类方法

doi:10.3969/j.issn.0372-2112.2020.02.008

电子学报 ›› 2020, Vol. 48 ›› Issue (2): 279-284.DOI: 10.3969/j.issn.0372-2112.2020.02.008

所属专题：自然语言处理技术；自然语言处理：技术与应用

基于高效用神经网络的文本分类方法

吴玉佳, 李晶, 宋成芳, 常军

武汉大学计算机学院, 湖北武汉 430072

收稿日期:2018-10-08 修回日期:2019-10-22 出版日期:2020-02-25
- 通讯作者:
- 李晶
- 作者简介:
- 吴玉佳 男,1986年11月出生,湖北广水人,武汉大学博士生,计算机应用技术专业.研究方向:数据挖掘,自然语言处理,深度学习.E-mail:wuyujia@whu.edu.cn
- 基金资助:
- 国家重点基础研究发展规划 (973计划）项目 (No.2012CB719905）; 国家自然科学基金 (No.41201404）; 中央高校基本科研业务费专项资金项目 (No.2042015gf0009）

High Utility Neural Networks for Text Classification

WU Yu-jia, LI Jing, SONG Cheng-fang, CHANG Jun

School of Computer Science, Wuhan University, Wuhan, Hubei 430072, China

Received:2018-10-08 Revised:2019-10-22 Online:2020-02-25 Published:2020-02-25
- Corresponding author:
- LI Jing
- Supported by:
- Program of National Program on Key Basic Research Project (973 Program) (No.2012CB719905); National Natural Science Foundation of China (No.41201404); Fundamental Research Funds for the Central Universities (No.2042015gf0009)

摘要/Abstract

摘要： 现有的基于深度学习的文本分类方法没有考虑文本特征的重要性和特征之间的关联关系，影响了分类的准确率.针对此问题，本文提出一种基于高效用神经网络（High Utility Neural Networks，HUNN）的文本分类模型，可以有效地表示文本特征的重要性及其关联关系.利用高效用项集挖掘（Mining High Utility Itemsets，MHUI）算法获取数据集中各个特征的重要性以及共现频率.其中，共现频率在一定程度上反映了特征之间的关联关系.将MHUI作为HUNN的挖掘层，用于挖掘每个类别数据中重要性和关联性强的文本特征.然后将这些特征作为神经网络的输入，再经过卷积层进一步提炼类别表达能力更强的高层次文本特征，从而提高模型分类的准确率.通过在6个公开的基准数据集上进行实验分析，提出的算法优于卷积神经网络（Convolutional Neural Networks，CNN），循环神经网络（Recurrent Neural Networks，RNN），循环卷积神经网络（Recurrent Convolutional Neural Networks，RCNN），快速文本分类（Fast Text Classifier，FAST），分层注意力网络（Hierarchical Attention Networks，HAN）等5个基准算法.

关键词: 数据挖掘, 关联规则, 高效用项集, 自然语言处理, 文本分类, 神经网络

Abstract: The existing text classification methods based on deep learning do not consider the importance and association of text features. The association between the text features perhaps affects the accuracy of the classification. To solve this problem, in this study, a framework based on high utility neural networks (HUNN) for text classification were proposed. Which can effectively mine the importance of text features and their association. Mining high utility itemsets (MHUI) from databases is an emerging topic in data mining. It can mine the importance and the co-occurrence frequency of each feature in the dataset.The co-occurrence frequency of the feature reflects the association between the text features. Using MHUI as the mining layer of HUNN, it is used to mine strong importance and association text features in each type, select these text features as input to the neural networks. And then acquire the high-level features with strong ability of categorical representation through the convolution layer for improving the accuracy of model classification. The experimental results showed that the proposed model performed significantly better on six different public datasets compared with convolutional neural networks (CNN), recurrent neural networks (RNN), recurrent convolutional neural networks (RCNN), fast text classifier (FAST), and hierarchical attention networks (HAN).

Key words: data mining, association rule, high utility itemset, natural language processing, text classification, neural networks

中图分类号:

TP311

吴玉佳, 李晶, 宋成芳, 等. 基于高效用神经网络的文本分类方法[J]. 电子学报, 2020, 48(2): 279-284.

WU Yu-jia, LI Jing, SONG Cheng-fang, et al. High Utility Neural Networks for Text Classification[J]. Acta Electronica Sinica, 2020, 48(2): 279-284.

参考文献

[1] 胡小娟,刘磊,邱宁佳.基于主动学习和否定选择的垃圾邮件分类算法[J] 电子学报,2018,46(1):203-209. HU Xiao-juan,LIU Lei,QIU Ning-jia.A novel spam categorization algorithm based on active learning method and negative selection algorithm[J].Acta Electronica Sinica,2018,46(1):203-209.
[2] Y Kim.Convolutional neural networks for sentence classification[A].Conference on Empirical Methods in Natural Language Processing[C].Doha,Qatar:ACL,2014.1746-1751.
[3] T Mullen,N Collier.Sentiment analysis using support vector machines with diverse information sources[A].Conference on Empirical Methods in Natural Language Processing[C].Barcelona,Spain:ACL,2004.412-418.
[4] S Tan,X Cheng,Y Wang,et al.Adapting naive Bayes to domain adaptation for sentiment analysis[A].The 31th European Conference on IR Research [C].Toulouse,France:Springer,2009.337-349.
[5] S Wawre,S Deshmukh.Sentiment classification using machine learning techniques[J].International Journal of Science and Research,2016,5(4):819-821.
[6] A Maas,R Daly,P Pham,et al.Learning word vectors for sentiment analysis[A].The 49th Annual Meeting of the Association for Computational Linguistics [C] Portland,Oregon,USA:ACL,2011.142-150.
[7] Trstenjak B,Mikac S,Donko D.KNN with TF-IDF based framework for text categorization[J].Procedia Engineering,2014,69(1):1356-1364.
[8] Johnson R,Zhang T.Deeppyramid convolutional neural networks for text categorization[A].55nd Annual Meeting of the Association for Computational Linguistics[C].Vancouver,Canada:ACL,2017.562-570.
[9] Ammar Ismael Kadhim.Survey on supervised machine learning techniques for automatic text classification[J].Artificial Intelligence Review,2019,52(1):273-292.
[10] R Johnson and T Zhang.Effective use of word order for text categorization with convolutional neural networks[A].Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies[C].Denver,Colorado,USA:NAACL,2015.103-112.
[11] W Yih,X He,C Meek.Semantic parsing for single-relation question answering[A].52nd Annual Meeting of the Association for Computational Linguistics[C].Baltimore,Maryland,USA:ACL,2014.643-648.
[12] Y Shen,X He,J Gao,et al.Learning semantic representations using convolutional neural networks for web search[A].Proceedings of the 23rd International Conference on World Wide Web [C].Seoul,Korea:ACM,2014.373-374.
[13] D Tang,B Qin,T Liu.Learning semantic representations of users and products for document level sentiment classification[A].53rd Annual Meeting of the Association for Computational Linguistics[C].Beijing,China:ACL,2015.1014-1023.
[14] Batmaz Z,Yurekli A,Bilge A,et al.A review on deep learning for recommender systems:challenges and remedies[J].Artificial Intelligence Review,2018,52(1):1-37.
[15] Zhang X,Zhao J,Lecun Y.Character-level convolutional networks for text classification[A].Advances in Neural Information Processing Systems [C].Montreal,Canada:MIT Press,2015.649-657.
[16] Joulin A,Grave E,Bojanowski P,et al.Bag of tricks for efficient text classification[A].Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics[C].Valencia,Spain:EACL,2017.427-431.
[17] P Liu,X Qiu,X Huang.Recurrent neural network for text classification with multi-task learning [A].The 26th International Joint Conference on Artificial Intelligence[C].Melbourne,Australia:Morgan Kaufmann,2017.1480-1489.
[18] Lai S,Xu L,Liu K,et al.Recurrent convolutional neural networks for text classification[A].Conference of the Association for the Advancement of Artificial Intelligence[C].Austin,Texas,USA:AAAI,2015.2267-2273.
[19] Yang Z,Yang D,Dyer C,et al.Hierarchical attention networks for document classification[A].Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies[C].San Diego,California,USA:NAACL,2016.1480-1489.
[20] T Mikolov,I Sutskever,K Chen,et al.Distributed representations of words and phrases and their compositionality[A].Advances in Neural Information Processing Systems [C].Lake Tahoe,Nevada,USA:MIT Press,2013:3111-3119.
[21] J Pennington,R Socher,C Manning.Glove:global vectors for word representation[A].Conference on Empirical Methods in Natural Language Processing [C].Doha,Qatar:ACL,2014.1532-1543.
[22] Pedro Domingos.A few useful things to know about machine learning[J].Communications of the ACM,2012,55(10):78-87.
[23] S Kiritchenko,X Zhu,S Mohammad.Sentiment analysis of short informal texts[J].Journal of Artificial Intelligence Research,2014,50(1):723-762.
[24] L Qu,G Ifrim,G Weikum.The bag-of-opinions method for review rating prediction from sparse text patterns[A].23rd International Conference on Computational Linguistics[C].Beijing,China:ACM,2010.913-921.
[25] R Socher,A Perelygin,J Wu,et al.Recursive deep models for semantic compositionality over a sentiment treebank[A].Conference on Empirical Methods in Natural Language Processing[C].Seattle,Washington,USA:ACL,2013.1631-1642.
[26] N Kalchbrenner,E Grefenstette,P Blunsom.A convolutional neural network for modelling sentences[A].52rd Annual Meeting of the Association for Computational Linguistics[C].Baltimore,Maryland,USA:ACL,2014.655-665.
[27] Quoc V,T Mikolov.Distributed representations of sentences and documents[A].The 31st International Conference on Machine Learning[C].Beijing,China:ACM,2014.1188-1196.
[28] C.F.Ahmed,S.K.Tanbeer,B.S.Jeong et al.Efficient tree structures for high utility pattern mining in incremental databases [J].IEEE Transactions on Knowledge and Data Engineering,2009,21(12):1708-1721.
[29] M.C.Liu,J.F.Qu.Mining high utility itemsets without candidate generation[A].Proc of the 12th International Conference on Information and Knowledge Management[C],Maui,HI,USA,ACM,2012.55-64.
[30] 黄坤,吴玉佳,李晶.基于差集的高效用项集挖掘方法[J].电子学报,2018,46(8):1804-1814. HUANG Kun,WU Yu-jia,LI Jing.Mining high utility itemsets using diffsets[J].Acta Electronica Sinica,2018,46(8):1804-1814.

基于高效用神经网络的文本分类方法

High Utility Neural Networks for Text Classification

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	黄赟, 张帆, 郭威, 陈立, 羊光. 一种基于数据标准差的卷积神经网络量化方法[J]. 电子学报, 2023, 51(3): 639-647.
[2]	张金颢, 周恒, 张守龙, 蒋廷勇, 王胜涛, 刘真. 基于仿真及神经网络的大型电磁脉冲模拟器近区场计算[J]. 电子学报, 2023, 51(3): 712-719.
[3]	吕杭, 蒋明峰, 李杨, 张鞠成, 王志康. 基于混合时频域特征的卷积神经网络心律失常分类方法的研究[J]. 电子学报, 2023, 51(3): 701-711.
[4]	王子为, 鲁继文, 周杰. 基于自适应梯度优化的二值神经网络[J]. 电子学报, 2023, 51(2): 257-266.
[5]	许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.
[6]	杨利平, 侯振威, 辜小花, 郝峻永. 弱标签声音事件检测的空间-通道特征表征与自注意池化[J]. 电子学报, 2023, 51(2): 297-306.
[7]	王神龙, 雍宇, 吴晨睿. 基于伪孪生神经网络的低纹理工业零件6D位姿估计[J]. 电子学报, 2023, 51(1): 192-201.
[8]	张永梅, 孙捷. 基于动静态特征双输入神经网络的咳嗽声诊断COVID-19算法[J]. 电子学报, 2023, 51(1): 202-212.
[9]	丁琪, 田萱, 孙国栋. 基于注意力增强的热点感知新闻推荐模型[J]. 电子学报, 2023, 51(1): 93-104.
[10]	袁海英, 成君鹏, 曾智勇, 武延瑞. Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J]. 电子学报, 2023, 51(1): 180-191.
[11]	王硕, 王坚, 王亚男, 宋亚飞. 一种基于特征融合的恶意代码快速检测方法[J]. 电子学报, 2023, 51(1): 57-66.
[12]	沈鹏飞, 徐臻, 王英. 基于嵌套生成对抗学习的网络嵌入[J]. 电子学报, 2022, 50(9): 2155-2163.
[13]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[14]	毛国君, 王者浩, 黄山, 王翔. 基于剪边策略的图残差卷积深层网络模型[J]. 电子学报, 2022, 50(9): 2205-2214.
[15]	张少波, 原刘杰, 毛新军, 朱更明. 基于本地差分隐私的K-modes聚类数据隐私保护方法[J]. 电子学报, 2022, 50(9): 2181-2188.