基于特征项分布的信息熵及特征动态加权概念漂移检测模型

doi:10.3969/j.issn.0372-2112.2015.07.016

电子学报 ›› 2015, Vol. 43 ›› Issue (7): 1356-1361.DOI: 10.3969/j.issn.0372-2112.2015.07.016

所属专题：机器学习—特征选择

基于特征项分布的信息熵及特征动态加权概念漂移检测模型

孙雪¹, 李昆仑², 韩蕾¹, 白晓亮¹

1. 河北大学工商学院, 河北保定 071002;
2. 河北大学电子信息工程学院, 河北保定 071000

收稿日期:2014-01-03 修回日期:2014-06-06 出版日期:2015-07-25
- 作者简介:
- 孙雪 女,1981年1月出生于天津市.现为河北大学工商学院老师.从事模式识别与人工智能,机器学习与数据挖掘,信息安全方面的研究. E-mail:sunxue@hbu.edu.cn;李昆仑 男,1962年7月出生于河北保定市.教授,硕士生导师.主要研究方向为模式识别与人工智能,机器学习与数据挖掘,信息安全,生物信息技术等. E-mail:likunlun@hbu.edu.cn
- 基金资助:
- 国家自然基金 (No.61073121); 河北省自然基金 (No.F2013201170); 国家科技支撑计划 (No.2013BAK07B04); 河北省高等学校科学技术研究重点项目 (No.ZD2014008); 河北大学青年基金 (No.2010Q17)

Construction of the Concept Drift Detection Model Based on the Information Entropy of Feature Distribution and Dynamic Weighting Algorithm

SUN Xue¹, LI Kun-lun², HAN Lei¹, BAI Xiao-liang¹

1. Industrial and Commercial College, Hebei University, Baoding, Hebei 071000, China;
2. College of Electronic and Information Engineering, Hebei University, Baoding, Hebei 071002, China

Received:2014-01-03 Revised:2014-06-06 Online:2015-07-25 Published:2015-07-25
- Supported by:
- National Natural Science Foundation of China (No.61073121); Natural Science Foundation of Hebei Province (No.F2013201170); National Key Technology Research and Development Program of the Ministry of Science and Technology (No.2013BAK07B04); Key Project of Science and Technology Research in Higher Education of Hebei Province (No.ZD2014008); Youth foundation of Hebei University (No.2010Q17)

摘要/Abstract

摘要：

现有的概念漂移算法大多建立在数据流的分类模型上,忽略了特征空间与样本空间的分布特点,以及特征选择和加权的重要性.针对此问题提出了一种基于特征项分布的信息熵及特征动态加权算法,从概念漂移的动态演化性出发,根据样本和特征空间的拟合程度,运用特征信息熵理论对数据流中的概念漂移现象进行捕捉,以实现新旧概念的过渡.利用改进的隐含Dirichlet模型特征动态加权算法,以解决当前特征与历史特征的权重确定和无效特征的裁剪问题.在公开的语料库CCERT和Trec06上的测试实验证明了所提出算法的有效性.

关键词: 概念漂移, LDA模型, 特征项分布, 信息熵

Abstract:

Most of the existing concept drift algorithm focuses on the classification model data streams,some of which overlook the distribution of the feature space and sample space,and the importance of feature selection and weighting.To solve this problem,we propose a dynamic information entropy and feature weighting algorithm based on the distribution of feature items from the dynamic evolution of the concept drift departure.To realize the concept transition,we capture the concept drifting of the data stream by the information entropy,according to the fitness degree between the sample and feature space.We improve the feature dynamic weighting latent dirichlet model,to overcome the problem of the current and historical feature weight assignment,as well as cropping the invalid features.Furthermore,the validity of the proposed algorithm was confirmed by the test in open corpus CCERT and Trec06.

Key words: concept drift, latent dirichlet allocation(LDA), feature distribution, information entropy

中图分类号:

孙雪, 李昆仑, 韩蕾, 等. 基于特征项分布的信息熵及特征动态加权概念漂移检测模型[J]. 电子学报, 2015, 43(7): 1356-1361.

SUN Xue, LI Kun-lun, HAN Lei, et al. Construction of the Concept Drift Detection Model Based on the Information Entropy of Feature Distribution and Dynamic Weighting Algorithm[J]. Acta Electronica Sinica, 2015, 43(7): 1356-1361.

参考文献

[1] WIDMER G,KUBAT M.Learning in the presence of concept drift and hidden contexts[J].Machine Learning,1996,(23):69-101.
[2] HOENS T R,POLIKAR R,et al.Learning from streaming data with concept drift and imbalance:an overview[J].Progress in Artificial Intelligence,2012,1(1):89-101.
[3] 文益民,强保华,等.概念漂移数据流分类研究综述[J].智能系统学报,2013,8(2):95-104. WEN Yi-min,QIANG Bao-hua,et al.A survey of the classification of data streams with concept drift [J].CAAI Transactions on Intelligent Systems,2013,8(2):95-104.(in Chinese)
[4] 柴玉梅,张卓,等.基于频繁概念直乘分布的全局闭频繁项集挖掘算法[J].计算机学报,2012,35(5):990-1000. CHAI Yu-mei,ZHANG Zhuo,et al.An algorithm for mining global closed frequent itemsets based on distributed frequent concept direct product[J].Chinese Journal of Computers,2012,35(5):990-1000.(in Chinese)
[5] 孙岳,毛国君,等.基于多分类器的数据流中的概念漂移挖掘[J].自动化学报,2008,34(1):93-97. SUN Yue,MAO Guo-jun,et al.Mining concept drifts from data streams based on multi-classifiers[J].Acta Automatica Sinica,2008,34(1):93-97.(in Chinese)
[6] 欧阳震诤,罗建书,等.一种不平衡数据流集成分类模型[J].电子学报,2010,38(1):184-189. OUYANG Zhen-zheng,LUO Jian-shu,et al.An ensemble classifier frame work for mining imbalanced data streams[J].Acta Electronica Sinica,2010,38(1):184-189.(in Chinese)
[7] ELWELL R,POLIKAR R.Incremental learning of concept drift in non-stationary environments [J].IEEE Transactions on Neural Networks,2011,22(10):1517-1531.
[8] 朱群,张玉红,等.一种基于双层窗口的概念漂移数据流分类算法[J].自动化学报,2011,9(37):1077-1084. ZHU Qun,ZHANG Yu-hong,et al.A double-window-based classification algorithm for concept drifting data streams[J].Acta Automatica Sinica,2011,9(37):1077-1084.(in Chinese)
[9] 徐文华,覃征,常扬.基于半监督学习的数据流集成分类算法[J].模式识别与人工智能,2012,25(2):292-299. XU Wen-hua,QIN Zheng,CHANG Yang.Semi-supervised learning based ensemble classifier for stream data[J].Pattern Recognition and Artificial Intelligence,2012,25(2):292-299.(in Chinese)
[10] PIOTR S,MICHAL W.Concept drift detection and model selection with simulated recurrence and ensembles of statistical detectors[J].Journal of Universal Computer Science,2013,19(4):462-483.
[11] PAULO M G,ROBERTO S M.RCD:A recurring concept drift framework[J].Pattern Recognition Letters,2013,34 (9):1018-1025.
[12] DEWAN M F,LI Z,et al.An adaptive ensemble classifier for mining concept drifting data streams[J].Expert Systems with Applications,2013,40(15):5895-5906.
[13] KLINKENBERG R.Learning drifting concepts:example selection vs.example weighting [J].Intelligent Data Analysis,2004,8(3):281-300.
[14] PETER V,ABRANHAM B.Entropy-based concept drift detection[A].Han J W.Proceedings of the 6th International Conference on Data Mining[C].Houston:IEEE Computer Society,2006.1113-1118.
[15] 于剑,石洪波,等.关于极大熵聚类算法的收敛性定理的反例[J].中国科学E辑:技术科学,2003,33(6):531-536.
[16] BLEI D M,NG A Y,et al.Latent dirichlet allocation [J].Journal of Machine Learning Research,2003,(3):993-1022.
[17] 石晶,范猛,等.基于LDA模型的主题分析[J].自动化学报,2009,35(12):1586-1592. SHI Jing,FAN Meng,et al.Topic analysis based on LDA model[J].Acta Automatica Sinica,2009,35(12):1586-1592.(in Chinese)

基于特征项分布的信息熵及特征动态加权概念漂移检测模型

Construction of the Concept Drift Detection Model Based on the Information Entropy of Feature Distribution and Dynamic Weighting Algorithm

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	张宏伟, 张小虎, 曹勇. 贝叶斯序贯重要性积分滤波器[J]. 电子学报, 2022, 50(4): 823-831.
[2]	陆克中, 陈超凡, 蔡桓, 吴定明. 面向概念漂移和类不平衡数据流的在线分类算法[J]. 电子学报, 2022, 50(3): 585-597.
[3]	陈孟元, 丁陵梅, 张玉坤. 基于改进关键帧选取策略的快速PL-SLAM算法[J]. 电子学报, 2022, 50(3): 608-618.
[4]	胡钢, 牛琼, 许丽鹏, 卢志宇, 过秀成. 基于网络超链接信息熵的节点重要性序结构演化建模分析[J]. 电子学报, 2022, 50(11): 2638-2644.
[5]	魏钰轩, 陈莹. 基于自适应层信息熵的卷积神经网络压缩[J]. 电子学报, 2022, 50(10): 2398-2408.
[6]	李彦苍, 巩翔宇. 基于信息熵的改进狮群算法及其在组合优化中的应用[J]. 电子学报, 2021, 49(8): 1577-1585.
[7]	张鹏, 叶剑, 张鹏. 基于概念漂移检测的大数据交易过程模型优化方法[J]. 电子学报, 2019, 47(7): 1465-1474.
[8]	王树亮, 毕大平, 阮怀林, 杜明洋, 潘继飞. 基于信息熵准则的认知雷达机动目标跟踪算法[J]. 电子学报, 2019, 47(6): 1277-1284.
[9]	邓大勇, 葛雅雯, 黄厚宽. 属性约简簇的优化选择[J]. 电子学报, 2019, 47(5): 1111-1120.
[10]	程玉胜, 赵大卫, 王一宾, 裴根生. 非平衡化标签补全核极限学习机多标签学习[J]. 电子学报, 2019, 47(3): 719-725.
[11]	吴宁博, 彭长根, 牟其林. 面向关联属性的差分隐私信息熵度量方法[J]. 电子学报, 2019, 47(11): 2337-2343.
[12]	邓大勇, 卢克文, 黄厚宽, 邓志轩. 概念的属性约简及异构数据概念漂移探测[J]. 电子学报, 2018, 46(5): 1234-1239.
[13]	邓大勇, 薛欢欢, 苗夺谦, 卢克文. 属性约简准则与约简信息损失的研究[J]. 电子学报, 2017, 45(2): 401-407.
[14]	李玉鑑, 张亚红. 三元变量间一维流形依赖关系的检测[J]. 电子学报, 2016, 44(3): 639-645.
[15]	谭丽, 孙季丰. 基于码书索引变换的高通量DNA序列数据压缩算法[J]. 电子学报, 2015, 43(5): 1007-1013.