基于随机子空间和AdaBoost的自适应集成方法

doi:10.3969/j.issn.0372-2112.2013.04.031

电子学报 ›› 2013, Vol. 41 ›› Issue (4): 810-814.DOI: 10.3969/j.issn.0372-2112.2013.04.031

基于随机子空间和AdaBoost的自适应集成方法

姚旭, 王晓丹, 张玉玺, 邢雅琼

空军工程大学防空反导学院,陕西西安 710051

收稿日期:2012-05-28 修回日期:2012-10-09 出版日期:2013-04-25
- 作者简介:
- 姚旭 女,1982年10月出生,河北昌黎人,博士生.主要研究方向为智能信息处理和机器学习等. E-mail:ffxy132@163.com 王晓丹 女,1966年10月出生,陕西汉中人,教授,博士生导师.主要研究方向为模式识别,智能信息处理和机器学习等.
- 基金资助:
- 国家自然科学基金 (No.60975026,No.61273275)

A Self-Adaption Ensemble Algorithm Based on Random Subspace and AdaBoost

YAO Xu, WANG Xiao-dan, ZHANG Yu-xi, XING Ya-qiong

School of Air and Missile Defense, Air Force Engineering University, Xi'an, Shaanxi 710051, China

Received:2012-05-28 Revised:2012-10-09 Online:2013-04-25 Published:2013-04-25
- Supported by:
- National Natural Science Foundation of China (No.60975026, No.61273275)

摘要/Abstract

摘要： 如何构造差异性大且精确度高的基分类器是集成学习的重点,为此提出一种新的集成学习方法——利用PSO寻找使得AdaBoost依样本权重抽取的数据集分类错误率最小化的最优特征权重分布,依据此最优权重分布对特征随机抽样生成随机子空间,并应用于AdaBoost的训练过程中.这就在增加分类器间差异性的同时保证了基分类器的准确度.最后用多数投票法融合各基分类器的决策结果,并通过仿真实验验证该方法的有效性.

关键词: 集成学习, 随机子空间, AdaBoost算法, 粒子群优化

Abstract: It is an open issue how to generate base classifiers with higher diversity and accuracy for ensemble learning.In this paper,a novel algorithm is proposed to solve this problem---particle swarm optimization is used to search for an optimal feature weight distribution which makes the classification error rate of training data sample by the distribution in AdaBoost minimal.Then,the feature subspace is constructed according to the optimal feature weight distribution,which is applied into the training process of AdaBoost.Thus,the accuracy of base classifier is advanced;meanwhile,the diversity between classifiers is improved.Finally,majority voting method is utilized to fuse the base classifiers' results and experiments have been done to attest the validity of the proposed algorithm.

Key words: ensemble learning, random subspace, AdaBoost algorithm, particle swarm optimization

中图分类号:

TP391.4

姚旭, 王晓丹, 张玉玺, 等. 基于随机子空间和AdaBoost的自适应集成方法[J]. 电子学报, 2013, 41(4): 810-814.

YAO Xu, WANG Xiao-dan, ZHANG Yu-xi, et al. A Self-Adaption Ensemble Algorithm Based on Random Subspace and AdaBoost[J]. Acta Electronica Sinica, 2013, 41(4): 810-814.

参考文献

[1] Dietterich TG.Machine learning research:four current directions[J].AI Magazine ,1997,18(4):97-136.
[2] Liu Kun-Hong.Cancer classification using rotation forest[J].Computers in Biol ogy and Medicine,2008,38(5):601-610.
[3] Kim TK.Boosted manifold principal angles for image set-based recognition[J].P attern Recognition,2007,40(9):2475-2484.
[4] Schapire RE.BoosTexter:A boosting-based system for text categorization[J].Mac hine Learning,2000,39(2-3):135-168.
[5] L Breiman.Bagging predictors[J].Machine Learning,1996,24(2):123-140.
[6] Y Freund.Experiments with a new boosting algorithm[A].Proceedings of the 13th International Conference on Machine Learning[C].Italy:Morgan Kaufm ann,1996.148-156.
[7] Ho,TK.The random subspace method for constructing decision forests[J].IEEE Tra nsaction on Pattern Analysis and Machine Intelligence.1998,20(8):832-844.
[8] 文学志,等.一种基于类Haar特征和改进AdaBoost分类器的车辆识别算法[J].电子学报,201 1,38(5):1121-1126. Wen Xue-zhi,et al.An algorithm based on Haar-like features and improved AdaBoo st classifier for vehicle recognition[J].Acta Electronica Sinica,2011,38(5):11 21-1126.(in Chinese)
[9] 王改革,等.基于Elman-AdaBoost强预测器的目标威胁评估模型及算法[J].电子学报,2012 ,40(5):901-906. Wang Gai-ge,et al.The model and algorithm for the target threat assessment base d on Elman-AdaBoost strong predictor[J].Acta Electronica Sinica,2012,40(5):90 1-906.(in Chinese)
[10] Ankit Desai,PM Jadav.An empirical evaluation of AdaBoost extensions for cost-se nsitive classification[J].International Journal of Computer Applications,2012, 44(13):34-41.
[11] 王雪松,高阳,程玉虎.基于随机子空间-正交局部保持投影的支持向量机[J].电子学报,20 11,39(8):1746-1750. Wang Xue-song,Gao Yang,Cheng Yu-hu.Support vector machine based on random subs pace and orthogonal locality preserving projection[J].Acta Electronica Sinica, 2011,39(8):1746-1750.(in Chinese)
[12] LI Kuncheva,JJ Rodríguez,CO Plumpton,et al.Random subspace ensembles for fMRI c lassification [J].IEEE Transactions on medical imaging,2010,29(2):531-542.
[13] JM Yang,BC Kuo,PT Yu,et al.A dynamic subspace method for hyper spectral image cl assification[J].IEEE Transactions on geoscience and remote sensing,2010,48(7): 2840-2853.
[14] N Garcia-Pedrajas,et al.Boosting random subspace method[J].Neural Networks,20 08,21(9):1344-1362.
[15] Kennedy J,Eberhart R.Particle swarm optimization[A].Proceedings of IEEE Intern ational Conference on Neural Networks[C].Perth:IEEE Press,1995.1942-1948.
[16] HU Wang,LI Zhi-shu.A simpler and more effective particle swarm optimization alg orithm[J].Journal of Software,2007,18(4):861-868.
[17] 高岳林,任子晖.带有变异算子的自适应粒子群优化算法[J].计算机工程与应用,2007,43(2 5):43-47. Gao Yue-lin,Ren Zi-hui.Adaptive particle swarm optimization algorithm with mut ation operator[J].Computer Engineering and Applications,2007,43(25):43-47.(in Chinese)
[18] 王晓丹,等.一种基于AdaBoost的SVM分类器[J].空军工程大学学报(自然科学版),2006,7(6 ):54-57. Wang Xiao-dan,et al.A combined SVM classifier based on AdaBoost[J].Journal of Air Force Engineering University (Natural Science Edition),2006,7(6):54-57.(in Chinese)
[19] GI Webb.MultiBoosting:A technique for combining boosting and wagging[J].Machin e Learning,2000,40(2):159-196.
[20] J Demsar.Statistical comparisons of classifiers over multiple data sets[J].Jou rnal of Machine Learning Research,2006,7(1):1-30.

基于随机子空间和AdaBoost的自适应集成方法

A Self-Adaption Ensemble Algorithm Based on Random Subspace and AdaBoost

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	郑锐, 汪秋云, 林卓庞, 靖蓉琦, 姜政伟, 傅建明, 汪姝玮. 一种基于威胁情报层次特征集成的挖矿恶意软件检测方法[J]. 电子学报, 2022, 50(11): 2707-2715.
[2]	张梦, 郑建宏, 刘香燕, 何云. 基于集成学习的全双工中继系统安全中继选择方案研究[J]. 电子学报, 2021, 49(9): 1852-1856.
[3]	管永明, 王刚, 骆凯波, 吕梁, 吕晓雯, 史玉良. 基于极端不平衡学习的泛化低压异常箱表关系识别研究与应用[J]. 电子学报, 2021, 49(8): 1507-1514.
[4]	韩红桂, 阿音嘎, 张璐, 乔俊飞. 自适应分解式多目标粒子群优化算法[J]. 电子学报, 2020, 48(7): 1245-1254.
[5]	王海东, 刘云敬, 康琳, 武迎春. 射频能量捕获传感网移动能量源的布置策略研究[J]. 电子学报, 2020, 48(12): 2367-2375.
[6]	李泽, 田增山, 王中春, 王亚. 基于粒子群优化的多径辅助室内定位算法[J]. 电子学报, 2020, 48(10): 1952-1960.
[7]	孙辉, 邓志诚, 赵嘉, 王晖, 谢海华. 混合均值中心反向学习粒子群优化算法[J]. 电子学报, 2019, 47(9): 1809-1818.
[8]	刘金平, 何捷舟, 马天雨, 张五霞, 唐朝晖, 徐鹏飞. 基于KELM选择性集成的复杂网络环境入侵检测[J]. 电子学报, 2019, 47(5): 1070-1078.
[9]	黄锦旺, 吕善翔, 李广明, 袁华强. 基于增殖系数的混沌信号提取算法[J]. 电子学报, 2019, 47(4): 855-861.
[10]	张行, 常颖, 宋康, 李春国, 杨绿溪. 一种新颖的水声信道参数估计算法[J]. 电子学报, 2019, 47(2): 509-512.
[11]	龙文, 蔡绍洪, 焦建军, 伍铁斌. 一种改进的灰狼优化算法[J]. 电子学报, 2019, 47(1): 169-175.
[12]	韩红桂, 武淑君. 基于收敛速度和多样性的多目标粒子群种群规模优化设计[J]. 电子学报, 2018, 46(9): 2263-2269.
[13]	刘兆广, 纪秀花, 刘云霞. 一种快速收敛的非参数粒子群优化算法及其收敛性分析[J]. 电子学报, 2018, 46(7): 1669-1674.
[14]	韩红桂, 卢薇, 乔俊飞. 一种基于多样性信息和收敛度的多目标粒子群优化算法[J]. 电子学报, 2018, 46(2): 315-324.
[15]	闫涛, 刘凤娴, 陈斌. 基于量子混沌粒子群优化算法的分数阶超混沌系统参数估计[J]. 电子学报, 2018, 46(2): 333-340.