熵权约束稀疏表示的短文本分类算法

doi:10.3969/j.issn.0372-2112.2020.11.006

电子学报 ›› 2020, Vol. 48 ›› Issue (11): 2131-2137.DOI: 10.3969/j.issn.0372-2112.2020.11.006

熵权约束稀疏表示的短文本分类算法

脱婷¹, 马慧芳^1,2,3, 李志欣³, 赵卫中⁴

1. 西北师范大学计算机科学与工程学院, 甘肃兰州 730070;
2. 桂林电子科技大学广西可信软件重点实验室, 广西桂林 541004;
3. 广西师范大学广西多源信息挖掘与安全重点实验室, 广西桂林 541004;
4. 华中师范大学计算机学院, 湖北武汉 430079

收稿日期:2018-07-30 修回日期:2020-07-01 出版日期:2020-11-25
- 通讯作者:
- 马慧芳
- 作者简介:
- 脱婷女,1990年9月出生,甘肃庆阳人.自2016年进入西北师范大学计算机科学与工程学院学习,现为硕士研究生,主要从事自然语言处理与分类算法方面研究.E-mail:nwnutuot@yeah.net;李志欣 男,1971年10月出生,广西桂林人.博士,博士生导师.现为广西师范大学计算机科学与信息工程学院教授,主要从事图像理解与机器学习等方面的研究.E-mail:lizx@gxnu.edu.cn;赵卫中 男,1981年10月出生,山东菏泽人.博士,硕士生导师,现为华中师范大学计算机学院副教授,主要从事机器学习与数据挖掘等方面研究工作.E-mail:zhaoweizhong@gmail.com
- 基金资助:
- 国家自然科学基金 (No.61762078，No.61363058，No.61663004，No.61966004，No.61762079）; 广西可信软件重点实验室研究课 (No.kx202003）; 广西多源信息挖掘与安全重点实验室开放基金 (No.MIMS18-08）; 西北师范大学2019年度青年教师科研能力提升计划 (No.NWNU-LKQN2019-2）

Effectively Classifying Short Texts by Entropy Weighted Constraints Sparse Representation

TUO Ting¹, MA Hui-fang^1,2,3, LI Zhi-xin³, ZHAO Wei-zhong⁴

1. College of Computer Science and Engineering, Northwest Normal University, Lanzhou, Gansu 730070, China;
2. Guangxi Key Laboratory of Trusted Software, Guilin University of Electronic Technology, Guilin, Guangxi 541004, China;
3. Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, Guangxi 541004, China;
4. School of Computer Central China Normal University, Wuhan, Hubei 430079, China

Received:2018-07-30 Revised:2020-07-01 Online:2020-11-25 Published:2020-11-25
- Corresponding author:
- MA Hui-fang
- Supported by:
- National Natural Science Foundation of China (No.61762078, No.61363058, No.61663004, No.61966004, No.61762079); Research Project of Guangxi Key Laboratory of Trusted Software (No.kx202003); Guangxi Key Laboratory Foundation for Multi-source Information Mining and Security (No.MIMS18-08); 2019 Key Project of Young Teachers Research Ability Enhancement Program of Northwest Normal University (No.NWNU-LKQN2019-2)

摘要/Abstract

摘要： 针对短文本特征稀疏性问题，提出一种熵权约束稀疏表示的短文本分类方法.考虑到初始字典维数较高，首先，利用Word2vec工具将字典中的词表示成词向量形式，然后根据加权向量平均值对原始字典进行降维.其次，利用一种快速特征子集选择算法去除字典中不相关和冗余短文本，得到过滤后的字典.再次，基于稀疏表示理论在过滤后的字典上，为目标函数设计一种熵权约束的稀疏表示方法，引入拉格朗日乘数法求得目标函数的最优值，从而得到每个类的子空间.最后，在学习到的子空间下通过计算待分类短文本与每个类中短文本的距离，并根据三种分类规则对短文本进行分类.在真实数据集上的大量实验结果表明，本文提出的方法能够有效缓解短文本特征稀疏问题且优于现有短文本分类方法.

关键词: 短文本分类, 词向量, 熵, 稀疏表示

Abstract: Aiming at the problem of short text feature sparsity, a short text sparse representation classification method based on entropy weighted constraint is proposed. Considering that the initial dictionary dimension is high, firstly, the word in the dictionary is represented as a word vector form via using the Word2vec tool, and then the original dictionary is reduced according to the average weighted vectors. Secondly, a fast feature subset selection algorithm is adopted to remove the irrelevant and redundant short texts in the dictionary, and the filtered dictionary can then be obtained. Thirdly, based on the sparse representation theory, an improved entropy-weighted sparse representation method is designed for the objective function, and the Lagrange multiplier method is introduced to obtain the optimal value of the objective function, thus the subspace of each class is obtained. Finally, the distance between the short text to be classified and the short text in each class is calculated under the subspace, and the short text is classified according to three classification rules. A large number of experimental results on real data sets show that the proposed method can effectively alleviate the short text feature sparse problem and exhibits better performance than the existing short text classification methods.

Key words: short text classification, word embedding, entropy, sparse representation

中图分类号:

TP393.092

脱婷, 马慧芳, 李志欣, 等. 熵权约束稀疏表示的短文本分类算法[J]. 电子学报, 2020, 48(11): 2131-2137.

TUO Ting, MA Hui-fang, LI Zhi-xin, et al. Effectively Classifying Short Texts by Entropy Weighted Constraints Sparse Representation[J]. Acta Electronica Sinica, 2020, 48(11): 2131-2137.

参考文献

[1] Wang J,Wang Z,Zhang D,et al.Combining knowledge with deep convolutional neural networks for short text classification[A].Proceedings of the 26th International Joint Conference on Artificial Intelligence[C].New York:ACM,2017.2915-2921.
[2] Li C,Wang H,Zhang Z,et al.Enhancing topic modeling for short texts with auxiliary word embeddings[J].ACM Transactions on Information Systems,2017,36(2):11:1-11:30.
[3] 张雄,陈福才,黄瑞阳.基于双词主题模型的半监督实体消歧方法研究[J].电子学报,2018,46(3):607-613. ZHANG Xiong,CHEN Fu-cai,HUANG Rui-yang.Semi-supervised entity disambiguation method research based on biterm topic model[J].Acta Electronica Sinica,2018,46(3):607-613.(in Chinese)
[4] 马慧芳,刘芳,夏琴,等.基于加权超图随机游走的文献关键词提取算法[J].电子学报,2018,46(6):1410-1414. MA Hui-fang,LIU Fang,XIA Qin,et al.Keywords extraction algorithm based on weighted hypergraph random walk[J].Acta Electronica Sinica,2018,46(6):1410-1414.(in Chinese)
[5] Mikolov T,Sutskever I,Chen K,et al.Distributed representations of words and phrases and their compositionality[A].Proceedings of Advances in Neural Information Processing Systems 27th Annual Conference on Neural Information Processing Systems[C].Cambridge:MIT Press,2013.3111-3119.
[6] Boom C,Canneyt S,Demeester T,et al.Representation learning for very short texts using weighted word embedding aggregation[J].Pattern Recognition Letters,2016,80(C):150-156.
[7] Gao L,Zhou S,Guan J.Effectively classifying short texts by structured sparse representation with dictionary filtering[J].Information Sciences,2015,323:130-142.
[8] Jing L,Ng M K,Huang J Z.An entropy weighting k-Means algorithm for subspace clustering of high-dimensional sparse data[J].IEEE Transactions on Knowledge and Data Engineering,2007,19(8):1026-1041.
[9] Song Q,Ni J,Wang G.A fast clustering-based feature subset selection algorithm for high-dimensional data[J].IEEE Transactions on Knowledge and Data Engineering,2013,25(1):1-14.
[10] Sharma N,Dileep A D,Thenkanidiyoor V.Text classification using hierarchical sparse representation classifiers[A].Proceedings of the 16th IEEE International Conference on Machine Learning and Applications[C].Piscataway:IEEE,2017.1015-1019.
[11] Sharma N,Sharma A,Thenkanidiyoor V,et al.Text classification using combined sparse representation classifiers and support vector machines[A].International Symposium on Computational and Business Intelligence[C].Piscataway:IEEE,2016.181-185.

[1]	曹源, 宋迪, 胡小溪, 孙永奎. 基于改进时域多尺度散布熵与支持向量机的转辙机故障诊断[J]. 电子学报, 2023, 51(1): 117-127.
[2]	李明爱, 张圆圆. 基于连续小波变换和符号传递熵的脑功能网络构建方法[J]. 电子学报, 2022, 50(7): 1600-1608.
[3]	饶宁, 许华, 蒋磊, 宋佰霖, 史蕴豪. 基于多智能体深度强化学习的分布式协同干扰功率分配算法[J]. 电子学报, 2022, 50(6): 1319-1330.
[4]	易令, 李泽平. 基于深度强化学习的码率自适应算法研究[J]. 电子学报, 2022, 50(5): 1192-1200.
[5]	张宏伟, 张小虎, 曹勇. 贝叶斯序贯重要性积分滤波器[J]. 电子学报, 2022, 50(4): 823-831.
[6]	李凯, 张可心. 结构α-熵的加权高斯混合模型的子空间聚类[J]. 电子学报, 2022, 50(3): 718-725.
[7]	陈孟元, 丁陵梅, 张玉坤. 基于改进关键帧选取策略的快速PL-SLAM算法[J]. 电子学报, 2022, 50(3): 608-618.
[8]	胡钢, 牛琼, 许丽鹏, 卢志宇, 过秀成. 基于网络超链接信息熵的节点重要性序结构演化建模分析[J]. 电子学报, 2022, 50(11): 2638-2644.
[9]	魏钰轩, 陈莹. 基于自适应层信息熵的卷积神经网络压缩[J]. 电子学报, 2022, 50(10): 2398-2408.
[10]	冯西安, 寇思玮, 谭伟杰, 毕杨. 水声信号处理中的稀疏表示理论及应用[J]. 电子学报, 2021, 49(9): 1840-1851.
[11]	李彦苍, 巩翔宇. 基于信息熵的改进狮群算法及其在组合优化中的应用[J]. 电子学报, 2021, 49(8): 1577-1585.
[12]	张昱, 刘开峰, 张全新, 王艳歌, 高凯龙. 基于组合-卷积神经网络的中文新闻文本分类[J]. 电子学报, 2021, 49(6): 1059-1067.
[13]	禤韵怡, 杨春玲. 基于帧间组稀疏的两阶段递归增强视频压缩感知重构网络[J]. 电子学报, 2021, 49(3): 435-442.
[14]	王毅, 李晓梦, 耿国华, 周琳, 段焱中. 基于直觉模糊熵的混合粒子群优化算法[J]. 电子学报, 2021, 49(12): 2381-2389.
[15]	沈忱, MIHAYLOVA Lyudmila. 跳变马尔可夫系统的最大混合相关熵状态估计[J]. 电子学报, 2021, 49(11): 2225-2233.

熵权约束稀疏表示的短文本分类算法

Effectively Classifying Short Texts by Entropy Weighted Constraints Sparse Representation

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics