High Utility Neural Networks for Text Classification

doi:10.3969/j.issn.0372-2112.2020.02.008

PDF(2194 KB)

ACTA ELECTRONICA SINICA ›› 2020, Vol. 48 ›› Issue (2) : 279-284. DOI: 10.3969/j.issn.0372-2112.2020.02.008

High Utility Neural Networks for Text Classification

WU Yu-jia, LI Jing, SONG Cheng-fang, CHANG Jun

Author information +

History +

Abstract

The existing text classification methods based on deep learning do not consider the importance and association of text features. The association between the text features perhaps affects the accuracy of the classification. To solve this problem, in this study, a framework based on high utility neural networks (HUNN) for text classification were proposed. Which can effectively mine the importance of text features and their association. Mining high utility itemsets (MHUI) from databases is an emerging topic in data mining. It can mine the importance and the co-occurrence frequency of each feature in the dataset.The co-occurrence frequency of the feature reflects the association between the text features. Using MHUI as the mining layer of HUNN, it is used to mine strong importance and association text features in each type, select these text features as input to the neural networks. And then acquire the high-level features with strong ability of categorical representation through the convolution layer for improving the accuracy of model classification. The experimental results showed that the proposed model performed significantly better on six different public datasets compared with convolutional neural networks (CNN), recurrent neural networks (RNN), recurrent convolutional neural networks (RCNN), fast text classifier (FAST), and hierarchical attention networks (HAN).

Key words

data mining / association rule / high utility itemset / natural language processing / text classification / neural networks

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

WU Yu-jia, LI Jing, SONG Cheng-fang, CHANG Jun. High Utility Neural Networks for Text Classification[J]. Acta Electronica Sinica, 2020, 48(2): 279-284. https://doi.org/10.3969/j.issn.0372-2112.2020.02.008

References

[1] 胡小娟,刘磊,邱宁佳.基于主动学习和否定选择的垃圾邮件分类算法[J] 电子学报,2018,46(1):203-209. HU Xiao-juan,LIU Lei,QIU Ning-jia.A novel spam categorization algorithm based on active learning method and negative selection algorithm[J].Acta Electronica Sinica,2018,46(1):203-209.
[2] Y Kim.Convolutional neural networks for sentence classification[A].Conference on Empirical Methods in Natural Language Processing[C].Doha,Qatar:ACL,2014.1746-1751.
[3] T Mullen,N Collier.Sentiment analysis using support vector machines with diverse information sources[A].Conference on Empirical Methods in Natural Language Processing[C].Barcelona,Spain:ACL,2004.412-418.
[4] S Tan,X Cheng,Y Wang,et al.Adapting naive Bayes to domain adaptation for sentiment analysis[A].The 31th European Conference on IR Research [C].Toulouse,France:Springer,2009.337-349.
[5] S Wawre,S Deshmukh.Sentiment classification using machine learning techniques[J].International Journal of Science and Research,2016,5(4):819-821.
[6] A Maas,R Daly,P Pham,et al.Learning word vectors for sentiment analysis[A].The 49th Annual Meeting of the Association for Computational Linguistics [C] Portland,Oregon,USA:ACL,2011.142-150.
[7] Trstenjak B,Mikac S,Donko D.KNN with TF-IDF based framework for text categorization[J].Procedia Engineering,2014,69(1):1356-1364.
[8] Johnson R,Zhang T.Deeppyramid convolutional neural networks for text categorization[A].55nd Annual Meeting of the Association for Computational Linguistics[C].Vancouver,Canada:ACL,2017.562-570.
[9] Ammar Ismael Kadhim.Survey on supervised machine learning techniques for automatic text classification[J].Artificial Intelligence Review,2019,52(1):273-292.
[10] R Johnson and T Zhang.Effective use of word order for text categorization with convolutional neural networks[A].Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies[C].Denver,Colorado,USA:NAACL,2015.103-112.
[11] W Yih,X He,C Meek.Semantic parsing for single-relation question answering[A].52nd Annual Meeting of the Association for Computational Linguistics[C].Baltimore,Maryland,USA:ACL,2014.643-648.
[12] Y Shen,X He,J Gao,et al.Learning semantic representations using convolutional neural networks for web search[A].Proceedings of the 23rd International Conference on World Wide Web [C].Seoul,Korea:ACM,2014.373-374.
[13] D Tang,B Qin,T Liu.Learning semantic representations of users and products for document level sentiment classification[A].53rd Annual Meeting of the Association for Computational Linguistics[C].Beijing,China:ACL,2015.1014-1023.
[14] Batmaz Z,Yurekli A,Bilge A,et al.A review on deep learning for recommender systems:challenges and remedies[J].Artificial Intelligence Review,2018,52(1):1-37.
[15] Zhang X,Zhao J,Lecun Y.Character-level convolutional networks for text classification[A].Advances in Neural Information Processing Systems [C].Montreal,Canada:MIT Press,2015.649-657.
[16] Joulin A,Grave E,Bojanowski P,et al.Bag of tricks for efficient text classification[A].Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics[C].Valencia,Spain:EACL,2017.427-431.
[17] P Liu,X Qiu,X Huang.Recurrent neural network for text classification with multi-task learning [A].The 26th International Joint Conference on Artificial Intelligence[C].Melbourne,Australia:Morgan Kaufmann,2017.1480-1489.
[18] Lai S,Xu L,Liu K,et al.Recurrent convolutional neural networks for text classification[A].Conference of the Association for the Advancement of Artificial Intelligence[C].Austin,Texas,USA:AAAI,2015.2267-2273.
[19] Yang Z,Yang D,Dyer C,et al.Hierarchical attention networks for document classification[A].Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies[C].San Diego,California,USA:NAACL,2016.1480-1489.
[20] T Mikolov,I Sutskever,K Chen,et al.Distributed representations of words and phrases and their compositionality[A].Advances in Neural Information Processing Systems [C].Lake Tahoe,Nevada,USA:MIT Press,2013:3111-3119.
[21] J Pennington,R Socher,C Manning.Glove:global vectors for word representation[A].Conference on Empirical Methods in Natural Language Processing [C].Doha,Qatar:ACL,2014.1532-1543.
[22] Pedro Domingos.A few useful things to know about machine learning[J].Communications of the ACM,2012,55(10):78-87.
[23] S Kiritchenko,X Zhu,S Mohammad.Sentiment analysis of short informal texts[J].Journal of Artificial Intelligence Research,2014,50(1):723-762.
[24] L Qu,G Ifrim,G Weikum.The bag-of-opinions method for review rating prediction from sparse text patterns[A].23rd International Conference on Computational Linguistics[C].Beijing,China:ACM,2010.913-921.
[25] R Socher,A Perelygin,J Wu,et al.Recursive deep models for semantic compositionality over a sentiment treebank[A].Conference on Empirical Methods in Natural Language Processing[C].Seattle,Washington,USA:ACL,2013.1631-1642.
[26] N Kalchbrenner,E Grefenstette,P Blunsom.A convolutional neural network for modelling sentences[A].52rd Annual Meeting of the Association for Computational Linguistics[C].Baltimore,Maryland,USA:ACL,2014.655-665.
[27] Quoc V,T Mikolov.Distributed representations of sentences and documents[A].The 31st International Conference on Machine Learning[C].Beijing,China:ACM,2014.1188-1196.
[28] C.F.Ahmed,S.K.Tanbeer,B.S.Jeong et al.Efficient tree structures for high utility pattern mining in incremental databases [J].IEEE Transactions on Knowledge and Data Engineering,2009,21(12):1708-1721.
[29] M.C.Liu,J.F.Qu.Mining high utility itemsets without candidate generation[A].Proc of the 12th International Conference on Information and Knowledge Management[C],Maui,HI,USA,ACM,2012.55-64.
[30] 黄坤,吴玉佳,李晶.基于差集的高效用项集挖掘方法[J].电子学报,2018,46(8):1804-1814. HUANG Kun,WU Yu-jia,LI Jing.Mining high utility itemsets using diffsets[J].Acta Electronica Sinica,2018,46(8):1804-1814.

Funding

Program of National Program on Key Basic Research Project (973 Program) (No.2012CB719905); National Natural Science Foundation of China (No.41201404); Fundamental Research Funds for the Central Universities (No.2042015gf0009)