基于自适应图学习的半监督特征选择

doi:10.12263/DZXB.20210415

电子学报 ›› 2022, Vol. 50 ›› Issue (7): 1643-1652.DOI: 10.12263/DZXB.20210415

基于自适应图学习的半监督特征选择

江兵兵¹, 何文达¹, 吴兴宇², 项俊浩¹, 洪立斌¹, 盛伟国¹

^1.杭州师范大学信息科学与技术学院，浙江杭州 311121
^2.中国科学技术大学计算机科学与技术学院，安徽合肥 230027

收稿日期:2021-03-30 修回日期:2021-11-25 出版日期:2022-07-25
- 作者简介:
- 江兵兵男，1991年生，安徽颍上人.2019年毕业于中国科学技术大学计算机应用技术专业，获工学博士学位.现为杭州师范大学信息科学与技术学院硕士生导师，讲师.主要研究方向为半监督学习、特征选择、多视图学习和贝叶斯学习.E-mail: jiangbb@hznu.edu.cn
  何文达男，1996年生，浙江海盐人.杭州师范大学信息科学与技术学院在读硕士研究生，主要研究方向为半监督学习、特征选择等.E-mail: wendahe1996@stu.hznu.edu.cn
  盛伟国（通讯作者）男，1977年生，浙江瑞安人.2002年毕业于英国诺丁汉大学，获信息技术硕士学位；2005年毕业于英国布鲁耐尔大学，获计算机科学博士学位.现为杭州师范大学信息科学与技术学院教授，博士生导师.主要研究方向为智能算法理论和设计及其在数据挖掘、模式识别和信息安全等.E-mail: weiguouk@hotmail.com
- 基金资助:
- 国家自然科学基金 (62006065); 杭州师范大学科研启动项目 (20204003)

Semi-Supervised Feature Selection with Adaptive Graph Learning

JIANG Bing-bing¹, HE Wen-da¹, WU Xing-yu², XIANG Jun-hao¹, HONG Li-bin¹, SHENG Wei-guo¹

^1.School of Information Science and Technology, Hangzhou Normal University, Hangzhou, Zhejiang 311121, China
^2.School of Computer Science and Technology, University of Science and Technology, Hefei, Anhui 230027, China

Received:2021-03-30 Revised:2021-11-25 Online:2022-07-25 Published:2022-07-30
- Supported by:
- National Natural Science Foundation of China (62006065); Scientific Research Start‐up Foundation of Hangzhou Normal University (20204003)

摘要/Abstract

摘要：

随着数据特征维数的增加，如何在少量有标签和大量无标签高维样本的情况下选择相关的特征子集已成为特征选择领域的热点问题.针对现有半监督特征选择算法直接忽略特征选择与局部结构学习之间的相互作用，从而难以有效获取样本分布结构的问题，本文提出了一种基于自适应图学习的半监督特征选择（Semi-supervised Feature Selection with Adaptive Graph learning，SFSAG）算法.利用标签传播将特征空间的稀疏投影学习和近邻图的构建有效地结合起来，实现在选择相关特征的同时还能学习样本的局部结构；自适应地利用样本在投影特征空间中的相似性信息构建可靠的近邻图，从而有效降低噪声特征的干扰并选择更具判别性的特征子集.多种数据集上的实验验证了SFSAG的有效性及其相对于现有半监督特征选择算法的优越性.

关键词: 特征选择, 自适应图学习, 半监督学习, 标签传播, L_2，1稀疏正则化

Abstract:

With the increasing feature dimensionality, how to select a relevant feature subset in the case of a few labeled and large amount of unlabeled high-dimensional samples has become a hot issue in feature selection. However, existing semi-supervised feature selection algorithms directly ignore the interaction between feature selection and local structure learning, making it difficult to obtain the distribution structure information. To these ends, a semi-supervised feature selection algorithm with adaptive graph learning(SFSAG) is developed in this paper. Firstly, the label propagation is used to link the tasks of sparse projection learning on the original feature space and construction of affinity graph, such that the feature selection and local structure learning can be performed simultaneously. Then, a reliable neighbor graph is adaptively constructed by using the similarity information of samples in the projected feature space, which largely alleviates the adverse effects of noisy dimensions and facilitates selecting more discriminative features. Extensive experiments are conducted on various datasets, and the results demonstrate the effectiveness of the proposed SFSAG and its superiority in comparison with the state-of-the-art feature selection algorithms.

Key words: feature selection, adaptive graph learning, semi-supervised learning, label propagation, L_2,1sparse regularization

中图分类号:

TP391.4

江兵兵, 何文达, 吴兴宇, 项俊浩, 洪立斌, 盛伟国. 基于自适应图学习的半监督特征选择[J]. 电子学报, 2022, 50(7): 1643-1652.

JIANG Bing-bing, HE Wen-da, WU Xing-yu, XIANG Jun-hao, HONG Li-bin, SHENG Wei-guo. Semi-Supervised Feature Selection with Adaptive Graph Learning[J]. Acta Electronica Sinica, 2022, 50(7): 1643-1652.

图/表 7

参考文献 23

1	LIH F, JIANGT, ZHANGK S. Efficient and robust feature extraction by maximum margin criterion[J]. IEEE Transactions on Neural Networks, 2006, 17(1): 157-165.
2	蒋盛益, 郑琪, 张倩生. 基于聚类的特征选择方法[J]. 电子学报, 2008, 36(S1): 157-160.
	JIANGS Y, ZHENGQ, ZHANGQ S. Clustering-based feature selection[J]. Acta Electronica Sinica, 2008, 36(S1): 157-160. (in Chinese)
3	SONGX N, ZANGJ G, HANY H, et al. Semi-supervised feature selection via hierarchical regression for web image classification[J]. Multimedia Systems, 2016, 22(1): 41-49.
4	刘艳芳, 李文斌, 高阳. 基于自适应邻域嵌入的无监督特征选择算法[J]. 计算机研究与发展, 2020, 57(8): 1639-1649.
	LIUY F, LIW B, GAOY. Adaptive neighborhood embedding based unsupervised feature selection[J]. Journal of Computer Research and Development, 2020, 57(8): 1639-1649. (in Chinese)
5	SHEIKHPOURR, SARRAMM A, GHARAGHANIS, et al. A Survey on semi-supervised feature selection methods[J]. Pattern Recognition, 2017, 64: 141-158.
6	ZHAOH F, LIQ, WANGZ, et al. Joint adaptive graph learning and discriminative analysis for unsupervised feature selection[J]. Cognitive Computation, 2021: 1-11.
7	余游, 冯林, 王格格, 等. 一种基于伪标签的半监督少样本学习模型[J]. 电子学报, 2019, 47(11): 2284-2291.
	YUY, FENGL, WANGG G, et al. A few-shot learning model based on semi-supervised with pseudo label[J]. Acta Electronica Sinica, 2019, 47(11): 2284-2291. (in Chinese)
8	ZHAOJ D, LUK, HEX F. Locality sensitive semi-supervised feature selection[J]. Neurocomputing, 2008, 71(10/11/12): 1842-1849.
9	LIUY, NIEF P, WUJ G, et al. Efficient semi-supervised feature selection with noise insensitive trace ratio criterion[J]. Neurocomputing, 2013, 105: 12-18.
10	CHANGX J, NIEF P, YANGY, et al. A convex formulation for semi-supervised multi-label feature selection[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence. Québec City: AAAI Press, 2014: 1171-1177.
11	CHENX J, YUANG W, NIEF P, et al. Semi-supervised feature selection via rescaled linear regression[C]//Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Melbourne: IJCAI, 2017: 1525-1531.
12	BELKINM, NIYOGIP, SINDHWANIV. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples[J]. Journal of Machine Learning Research, 2006, 7: 2399-2434.
13	MAZ G, NIEF P, YANGY, et al. Discriminating joint feature analysis for multimedia data understanding[J]. IEEE Transactions on Multimedia, 2012, 14(6): 1662-1672.
14	LUOT J, HOUC P, NIEF P, et al. Semi-supervised feature selection via insensitive sparse regression with application to video semantic recognition[J]. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(10): 1943-1956.
15	SHEIKHPOURR, SARRAMM A, GHARAGHANIS, et al. A robust graph-based semi-supervised sparse feature selection method[J]. Information Sciences, 2020, 531: 13-30.
16	NIEF P, ZHUW, LIX L. Structured graph optimization for unsupervised feature selection[J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 33(3): 1210-1222.
17	ZHANGZ, ZHAOM B, CHOWT W S. Graph based constrained semi-supervised learning framework via label propagation over adaptive neighborhood[J]. IEEE Transactions on Knowledge and Data Engineering, 2015, 27(9): 2362-2376.
18	JIANGB B, WUX Y, YUK, et al. Joint semi-supervised feature selection and classification through Bayesian approach[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 3983-3990.
19	陈兵飞, 江兵兵, 周熙人, 等. 基于稀疏贝叶斯的流形学习[J]. 电子学报, 2018, 46(1): 98-103.
	CHENB F, JIANGB B, ZHOUX R, et al. Manifold learning based on sparse Bayesian approach[J]. Acta Electronica Sinica, 2018, 46(1): 98-103. (in Chinese)
20	NIEF P, WANGX Q, HUANGH. Clustering and projected clustering with adaptive neighbors[C]//Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. New York: ACM, 2014: 977-986.
21	XUY Y, WANGJ, ANS, et al. Semi-supervised multi-label feature selection by preserving feature-label space consistency[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management. Torino: ACM, 2018: 783-792.
22	HUANGJ, NIEF P, HUANGH. A new simplex sparse learning model to measure data similarity for clustering[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence. Buenos Aires: AAAI Press, 2015: 3569-3575
23	HOUC P, NIEF P, TAOH, et al. Multi-view unsupervised feature selection with adaptive similarity and view weight[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(9): 1998-2011.

数据集名称	样本数	特征数	类别数
Binalpha^[16]	1 404	320	36
Coil20^[16]	1 440	1 024	20
Flower-17	1 360	1 239	17
Isolet	1 560	617	26
JAFFE^[22]	213	1 024	10
MSRA25^[20]	1 799	256	12
MSRC^[23]	210	512	7
USPST^[16]	2 007	256	10

数据集名称	样本数	特征数	类别数
Binalpha^[16]	1 404	320	36
Coil20^[16]	1 440	1 024	20
Flower-17	1 360	1 239	17
Isolet	1 560	617	26
JAFFE^[22]	213	1 024	10
MSRA25^[20]	1 799	256	12
MSRC^[23]	210	512	7
USPST^[16]	2 007	256	10

数据集名称	模型	10%	20%	30%	40%	50%
Binalpha	CSFS	26.68±2.20	32.61±1.96	38.25±2.75	40.81±3.70	42.13±3.01
	LSDF	32.25±2.02	36.13±2.62	38.82±3.17	40.45±2.16	42.93±2.02
	SFSS	34.91±2.46	39.09±2.15	43.15±2.18	45.20±2.34	47.48±3.30
	TRCFS	32.82±1.79	37.42±2.67	42.39±2.76	44.87±1.84	45.81±3.02
	SFSAG	35.90±2.66	40.11±2.21	43.82±2.29	45.95±2.54	47.19±2.87
Coil20	CSFS	77.27±1.73	83.74±2.39	88.04±1.92	89.42±3.62	88.82±0.35
	LSDF	79.33±3.13	82.47±3.00	83.72±2.15	84.26±2.14	84.86±1.82
	SFSS	79.42±3.37	84.82±2.86	87.90±1.76	89.71±1.56	90.21±1.47
	TRCFS	80.48±2.81	84.72±2.96	86.39±2.27	88.08±1.89	86.93±2.31
	SFSAG	81.80±3.08	87.46±2.29	89.39±1.12	90.70±1.19	91.44±1.61
Flower-17	CSFS	26.50±4.21	32.18±1.81	34.72±1.26	38.97±1.43	40.44±3.06
	LSDF	27.68±2.29	33.13±2.37	37.95±1.88	40.02±2.21	41.70±2.09
	SFSS	26.66±2.69	32.13±3.05	37.08±1.86	38.99±2.58	41.75±2.28
	TRCFS	27.65±2.12	33.14±2.32	38.12±2.02	40.05±2.27	41.65±2.11
	SFSAG	27.98±2.25	33.73±2.71	37.69±2.08	39.84±2.05	42.22±2.33
Isolet	CSFS	61.45±2.70	74.57±3.88	77.48±1.87	78.85±3.25	81.24±1.08
	LSDF	61.36±4.20	64.65±3.15	65.90±2.59	66.51±2.53	66.71±1.73
	SFSS	71.60±3.06	76.92±2.08	78.86±2.30	77.99±2.22	79.08±2.30
	TRCFS	64.52±3.40	70.57±2.53	72.43±2.68	72.79±3.47	75.10±3.82
	SFSAG	73.96±3.11	79.74±1.93	82.59±2.29	83.80±1.95	83.68±2.38
JAFFE	CSFS	66.25±9.73	91.88±9.84	98.75±2.04	100.00±0.00	98.75±1.71
	LSDF	87.81±6.89	94.06±4.17	96.88±1.96	96.80±1.79	97.50±1.47
	SFSS	87.97±5.53	97.11±2.11	98.44±1.76	99.30±1.56	99.77±0.76
	TRCFS	88.75±4.80	95.31±3.13	97.97±2.10	98.44±1.83	98.67±1.46
	SFSAG	94.06±2.80	98.67±1.37	99.45±0.76	99.61±0.86	99.92±0.35
MSRA25	CSFS	98.44±0.75	99.52±0.21	99.52±0.31	99.89±0.17	99.96±0.08
	LSDF	96.19±2.69	98.66±0.75	99.15±0.05	99.25±0.52	99.41±0.32
	SFSS	98.93±0.91	99.51±0.29	99.69±0.23	99.81±0.18	99.81±0.17
	TRCFS	96.31±2.19	98.71±0.84	99.15±0.46	99.24±0.60	99.44±0.35
	SFSAG	99.23±0.53	99.65±0.26	99.78±0.22	99.83±0.16	99.84±0.17
MSRC	CSFS	56.51±9.30	70.48±14.70	81.90±4.14	83.49±5.79	87.62±2.84
	LSDF	63.89±6.07	73.25±4.64	76.03±5.82	75.48±4.79	78.89±5.60
	SFSS	65.48±5.87	74.76±8.30	79.76±5.07	80.16±5.39	83.33±4.14
	TRCFS	64.44±5.84	73.49±4.72	76.83±5.51	78.57±3.80	79.05±4.42
	SFSAG	70.71±7.59	79.29±6.40	83.02±4.81	83.41±4.23	86.03±4.23
USPST	CSFS	81.16±2.09	83.75±1.21	84.31±2.50	85.31±1.10	85.80±0.99
	LSDF	77.30±2.82	80.16±2.44	81.07±2.44	82.76±1.66	82.60±1.94
	SFSS	81.11±1.37	83.02±2.17	84.09±1.81	84.89±1.65	84.62±1.45
	TRCFS	78.31±2.33	81.10±2.11	81.99±2.20	82.40±2.22	82.59±1.96
	SFSAG	81.82±1.98	83.76±1.63	84.68±1.53	85.62±1.19	85.50±1.60

数据集名称	模型	10%	20%	30%	40%	50%
Binalpha	CSFS	26.68±2.20	32.61±1.96	38.25±2.75	40.81±3.70	42.13±3.01
	LSDF	32.25±2.02	36.13±2.62	38.82±3.17	40.45±2.16	42.93±2.02
	SFSS	34.91±2.46	39.09±2.15	43.15±2.18	45.20±2.34	47.48±3.30
	TRCFS	32.82±1.79	37.42±2.67	42.39±2.76	44.87±1.84	45.81±3.02
	SFSAG	35.90±2.66	40.11±2.21	43.82±2.29	45.95±2.54	47.19±2.87
Coil20	CSFS	77.27±1.73	83.74±2.39	88.04±1.92	89.42±3.62	88.82±0.35
	LSDF	79.33±3.13	82.47±3.00	83.72±2.15	84.26±2.14	84.86±1.82
	SFSS	79.42±3.37	84.82±2.86	87.90±1.76	89.71±1.56	90.21±1.47
	TRCFS	80.48±2.81	84.72±2.96	86.39±2.27	88.08±1.89	86.93±2.31
	SFSAG	81.80±3.08	87.46±2.29	89.39±1.12	90.70±1.19	91.44±1.61
Flower-17	CSFS	26.50±4.21	32.18±1.81	34.72±1.26	38.97±1.43	40.44±3.06
	LSDF	27.68±2.29	33.13±2.37	37.95±1.88	40.02±2.21	41.70±2.09
	SFSS	26.66±2.69	32.13±3.05	37.08±1.86	38.99±2.58	41.75±2.28
	TRCFS	27.65±2.12	33.14±2.32	38.12±2.02	40.05±2.27	41.65±2.11
	SFSAG	27.98±2.25	33.73±2.71	37.69±2.08	39.84±2.05	42.22±2.33
Isolet	CSFS	61.45±2.70	74.57±3.88	77.48±1.87	78.85±3.25	81.24±1.08
	LSDF	61.36±4.20	64.65±3.15	65.90±2.59	66.51±2.53	66.71±1.73
	SFSS	71.60±3.06	76.92±2.08	78.86±2.30	77.99±2.22	79.08±2.30
	TRCFS	64.52±3.40	70.57±2.53	72.43±2.68	72.79±3.47	75.10±3.82
	SFSAG	73.96±3.11	79.74±1.93	82.59±2.29	83.80±1.95	83.68±2.38
JAFFE	CSFS	66.25±9.73	91.88±9.84	98.75±2.04	100.00±0.00	98.75±1.71
	LSDF	87.81±6.89	94.06±4.17	96.88±1.96	96.80±1.79	97.50±1.47
	SFSS	87.97±5.53	97.11±2.11	98.44±1.76	99.30±1.56	99.77±0.76
	TRCFS	88.75±4.80	95.31±3.13	97.97±2.10	98.44±1.83	98.67±1.46
	SFSAG	94.06±2.80	98.67±1.37	99.45±0.76	99.61±0.86	99.92±0.35
MSRA25	CSFS	98.44±0.75	99.52±0.21	99.52±0.31	99.89±0.17	99.96±0.08
	LSDF	96.19±2.69	98.66±0.75	99.15±0.05	99.25±0.52	99.41±0.32
	SFSS	98.93±0.91	99.51±0.29	99.69±0.23	99.81±0.18	99.81±0.17
	TRCFS	96.31±2.19	98.71±0.84	99.15±0.46	99.24±0.60	99.44±0.35
	SFSAG	99.23±0.53	99.65±0.26	99.78±0.22	99.83±0.16	99.84±0.17
MSRC	CSFS	56.51±9.30	70.48±14.70	81.90±4.14	83.49±5.79	87.62±2.84
	LSDF	63.89±6.07	73.25±4.64	76.03±5.82	75.48±4.79	78.89±5.60
	SFSS	65.48±5.87	74.76±8.30	79.76±5.07	80.16±5.39	83.33±4.14
	TRCFS	64.44±5.84	73.49±4.72	76.83±5.51	78.57±3.80	79.05±4.42
	SFSAG	70.71±7.59	79.29±6.40	83.02±4.81	83.41±4.23	86.03±4.23
USPST	CSFS	81.16±2.09	83.75±1.21	84.31±2.50	85.31±1.10	85.80±0.99
	LSDF	77.30±2.82	80.16±2.44	81.07±2.44	82.76±1.66	82.60±1.94
	SFSS	81.11±1.37	83.02±2.17	84.09±1.81	84.89±1.65	84.62±1.45
	TRCFS	78.31±2.33	81.10±2.11	81.99±2.20	82.40±2.22	82.59±1.96
	SFSAG	81.82±1.98	83.76±1.63	84.68±1.53	85.62±1.19	85.50±1.60

[1]	肖进胜, 张舒豪, 陈云华, 王元方, 杨力衡. 双向特征融合与特征选择的遥感影像目标检测[J]. 电子学报, 2022, 50(2): 267-272.
[2]	刘浩阳, 林耀进, 刘景华, 吴镒潾, 毛煜, 李绍滋. 由粗到细的分层特征选择[J]. 电子学报, 2022, 50(11): 2778-2789.
[3]	周伯阳, 郭志民, 王延松, 阮伟, 吴春明, 周宁, 张伟, 程国振. 基于多尺度低秩模型的电力无线接入网异常流量检测方法[J]. 电子学报, 2020, 48(8): 1552-1557.
[4]	尤殿龙, 郭松, 赵春慧, 原福永, 申利民, 陈真. 面向分类的流特征在线特征选择算法[J]. 电子学报, 2020, 48(2): 321-332.
[5]	方佳艳, 刘峤. 具有同步化特征选择的迭代紧凑非平行支持向量聚类算法[J]. 电子学报, 2020, 48(1): 44-58.
[6]	余游, 冯林, 王格格, 徐其凤. 一种基于伪标签的半监督少样本学习模型[J]. 电子学报, 2019, 47(11): 2284-2291.
[7]	黄健航, 雷迎科. 基于半监督矩形网络的通信电台个体识别[J]. 电子学报, 2019, 47(1): 1-8.
[8]	盛凯, 刘忠, 周德超, 魏启航, 冯成旭. 一种基于证据理论的多类半监督分类算法[J]. 电子学报, 2018, 46(11): 2642-2649.
[9]	周孟, 朱福喜. 基于情感标签的极性分类[J]. 电子学报, 2017, 45(4): 1018-1024.
[10]	刘金平, 陈青, 张进, 唐朝晖. 基于集成学习的交互式图像分割[J]. 电子学报, 2016, 44(7): 1649-1655.
[11]	周炫余, 刘娟, 邵鹏, 卢笑, 罗飞. 基于测度优化Laplacian SVM的中文指代消解方法[J]. 电子学报, 2016, 44(12): 3064-3072.
[12]	刘世超, 朱福喜, 冯曦. 复杂网络的重叠社区及社区间的结构洞识别[J]. 电子学报, 2016, 44(11): 2600-2606.
[13]	陈小红, 李霞, 王娜. 高维多目标优化中基于稀疏特征选择的目标降维方法[J]. 电子学报, 2015, 43(7): 1300-1307.
[14]	张健沛, 邓琨, 杨静, 刘星妍. 基于边标签传播的复杂网络社区识别方法[J]. 电子学报, 2015, 43(6): 1113-1118.
[15]	高文, 钱亚冠, 吴春明, 郭晔, 朱凯, 陈双喜. 网络流量特征选择方法中的分治投票策略研究[J]. 电子学报, 2015, 43(4): 795-799.

基于自适应图学习的半监督特征选择

Semi-Supervised Feature Selection with Adaptive Graph Learning

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 23

相关文章 15

编辑推荐

Metrics