基于自编码器和超图的半监督宽度学习系统

doi:10.12263/DZXB.20210105

电子学报 ›› 2022, Vol. 50 ›› Issue (3): 533-539.DOI: 10.12263/DZXB.20210105

基于自编码器和超图的半监督宽度学习系统

王雪松¹^,², 张翰林¹^,², 程玉虎¹^,²()

^1.地下空间智能控制教育部工程研究中心, 江苏徐州 221116
^2.中国矿业大学信息与控制工程学院, 江苏徐州 221116

收稿日期:2021-01-13 修回日期:2021-03-17 出版日期:2022-03-25
- 通讯作者:
- 程玉虎
- 作者简介:
- 王雪松女，1974年生，安徽泗县人.现为地下空间智能控制教育部工程研究中心主任，中国矿业大学信息与控制工程学院教授、博士生导师.研究领域为机器学习及模式识别.E-mail:wangxuesongcumt@163.com
  张翰林男，1998年生，山东聊城人.现为中国矿业大学信息与控制工程学院硕士研究生.研究领域为图像处理与分析.E-mail:18252111368@163.com
  程玉虎（通讯作者）男，1973年生，安徽淮南人.现为中国矿业大学信息与控制工程学院教授、博士生导师.研究领域为机器学习及模式识别.
- 基金资助:
- 国家自然科学基金 (61976215)

Autoencoder and Hypergraph-Based Semi-Supervised Broad Learning System

WANG Xue-song¹^,², ZHANG Han-lin¹^,², CHENG Yu-hu¹^,²()

^1.Engineering Research Center of Intelligent Control for Underground Space, Ministry of Education, Xuzhou, Jiangsu 221116, China
^2.School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, Jiangsu 221116, China

Received:2021-01-13 Revised:2021-03-17 Online:2022-03-25 Published:2022-03-25
- Corresponding author:
- CHENG Yu-hu
- Supported by:
- National Natural Science Foundation of China (61976215)

摘要/Abstract

摘要：

常规宽度学习系统（Broad Learning System, BLS）通常采用的线性稀疏特征提取方法难以对数据的复杂非线性特征进行有效表征.此外，当标记样本量较少时，BLS的泛化性能难以得到保证.为此，提出一种基于自编码器和超图的半监督宽度学习系统（Autoencoder and Hypergraph-based Semi-supervised BLS, AH-SBLS）.主要步骤为：首先，使用包括标记样本和无标记样本在内的全部样本训练自编码器，利用训练好的自编码器自动提取数据的复杂非线性特征；其次，将自编码器特征层中的特征作为AH-SBLS的特征节点并对其进行宽度拓展；然后，构造半监督超图以挖掘标记样本和无标记样本间的高阶流形关系，并将超图正则项引入宽度学习系统的目标函数中；最后，利用岭回归对目标函数进行求解，实现对无标记样本的类别预测.在图像分类实验上的结果表明，AH-SBLS能够实现半监督分类且获得较高的分类精度.

关键词: 半监督, 宽度学习系统, 自编码器, 超图, 图像分类

Abstract:

The linear sparse feature extraction method used in the classical broad learning system(BLS) is difficult to extract the complex nonlinear features of data effectively. In addition, when the number of labeled samples is small, the generalization ability of BLS cannot be guaranteed. To solve these problems, a novel autoencoder and hypergraph-based semi-supervised BLS(AH-SBLS) is proposed. The main steps of AH-SBLS are described as follows. Firstly, we use all labeled and unlabeled samples to train the autoencoder, and then the trained autoencoder is used to extract the features of input data automatically. Secondly, the extracted features are viewed as the feature nodes of AH-SBLS and are further broadened. In the third step, a semi-supervised hypergraph is constructed to express the high-order manifold relationship between labeled and unlabeled samples, and the hypergraph regularization term is introduced into the objective function of AH-SBLS. Finally, the objective function of AH-SBLS is solved by ridge regression and thus the labels of unlabeled samples can be predicted. Experimental results of image classification show that AH-SBLS can achieve higher classification accuracy in semi-supervised classification tasks.

Key words: semi-supervised, broad learning system, autoencoder, hypergraph, image classification

中图分类号:

TP181

王雪松, 张翰林, 程玉虎. 基于自编码器和超图的半监督宽度学习系统[J]. 电子学报, 2022, 50(3): 533-539.

Xue-song WANG, Han-lin ZHANG, Yu-hu CHENG . Autoencoder and Hypergraph-Based Semi-Supervised Broad Learning System[J]. Acta Electronica Sinica, 2022, 50(3): 533-539.

图/表 8

参考文献 22

1	CHENC L P, LIUZ. Broad learning system: an effective and efficient incremental learning system without the need for deep architecture[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(99): 10⁃24.
2	SUIS, CHENC L P, TONGS, FENGS. Finite-time adaptive quantized control of stochastic nonlinear systems with input quantization: a broad learning system based identification method[J]. IEEE Transactions on Industrial Electronics, 2020, 67(10): 8555⁃8565.
3	CHUF, LIANGT, CHENC L P, WANGX, MAX. Weighted broad learning system and its application in nonlinear industrial process modeling[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(8):3017⁃3031.
4	HANM, LIW, FENGS, QIUT, CHENC L P. Maximum information exploitation using broad learning system for large-scale chaotic time-series prediction[J]. IEEE Transactions on Neural Networks and Learning Systems,2021,32(6): 2320⁃2329.
5	KONGY, WANGX, CHENGY, CHENC L P. Hyperspectral imagery classification based on semi-supervised broad learning system[J]. Remote Sensing, 2018, 10(5): 685.
6	SHAOY, SANGN, GAOC, MAL. Spatial and class structure regularized sparse representation graph for semi-supervised hyperspectral image classification[J]. Pattern Recognition, 2018, 81: 81⁃94.
7	ZHAOH, ZHENGJ, DENGW, SONGY. Semi-supervised broad learning system based on manifold regularization and broad network[J]. IEEE Transactions on Circuits and Systems I: Regular Papers, 2020, 67(3): 983⁃994.
8	BELKINM, NIYOGIP, SINDHWANIV. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples[J]. Journal of Machine Learning Research, 2006, 7: 2399⁃2434.
9	ANISA, ELG A, AVESTIMEHRA S, ORTEGAA. A sampling theory perspective of graph-based semi-supervised learning[J]. IEEE Transactions on Information Theory, 2019, 65(4): 2322⁃2342.
10	冀中, 樊帅飞. 基于超图排序算法的视频摘要[J]. 电子学报, 2017, 45(5): 1035⁃1043.
	JIZhong, FANShuai-fei. Video summarization based on hypergraph ranking[J]. Acta Electronica Sinica, 2017, 45(5): 1035⁃1043. (in Chinese)
11	HINTONG E, SALAKHUTDINOVR. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504⁃507.
12	高妮, 高岭, 贺毅岳, 王海. 基于自编码网络特征降维的轻量级入侵检测模型[J]. 电子学报, 2017, 45(3): 730⁃739.
	GAONi, GAOLing, HEYi-yue, WANGHai. A lightweight intrusion detection model based on autoencoder network with feature reduction[J]. Acta Electronica Sinica, 2017, 45(3): 730⁃739. (in Chinese)
13	翟颖, 陈渤. 基于稳健变分自编码模型的雷达高分辨距离像目标识别算法[J]. 电子学报, 2020, 48(6): 1149⁃1155.
	ZHAIYing, CHENBo. Study of radar high range resolution profiles target recognition based on auto-encoder[J]. Acta Electronica Sinica, 2020, 48(6): 1149⁃1155. (in Chinese)
14	李东瑾, 杨瑞娟, 李晓柏, 董睿杰. 基于栈式稀疏降噪自编码网络的辐射源调制识别[J]. 电子学报, 2020, 48(6): 1198⁃1204.
	LIDong-jin, YANGRui-juan, LIXiao-bo, DONGRui-jie. Emitter signal modulation recognition based on stacked sparse denoising auto-encoders[J]. Acta Electronica Sinica, 2020, 48(6): 1198⁃1204. (in Chinese)
15	RUMMELHARTD E, HINTONG E, WILLIAMSR J. Learning internal representations by error propagation[J]. Readings in Cognitive Science, 1988, 323(2): 399⁃421.
16	QIUD, ZHENGL, ZHUJ, et al. Multiple improved residual networks for medical image super-resolution[J]. Future Generation Computer Systems, 2021: 200⁃208.
17	LECUNY, BOTTOUL. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278⁃2324.
18	LECUNY, HUANGF J, BOTTOUL. Learning methods for generic object recognition with invariance to pose and lighting[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR2004. Washington, DC, USA: IEEE, 2004:97⁃104.
19	VINCENTP, LAROCHELLEH, BENGIOY, MANZAGOLP A. Extracting and composing robust features with denoising autoencoders[C]//Proceedings of the 25th International Conference on Machine Learning. New York: Academic Press, 2008: 1096⁃1103.
20	HINTONG E, OSINDEROS, TEHY W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527⁃1554.
21	SALAKHUTDINOVR, HINTONG E. Deep boltzmann machines[J]. Journal of Machine Learning Research, 2009, 5(2): 1967⁃2006.
22	HANX, GHAEMIM S, ANDOK, PETERSONL S, GAUDILLIEREB,et al. Differential dynamics of the maternal immune system in healthy pregnancy and preeclampsia[J]. Frontiers in Immunology, 2019, 10: 1305. DOI: 10.3389/fimmu.2019.01305.

模型	λ	ρ	k	α	特征节点数	增强节点数
BLS	10^-8	-	-	-	100	11000
A-BLS1	10^-14	-	-	0.005	100	5000
A-BLS	1	-	-	10^-3	2250	3000
ss-BLS	1	10^-2	-	-	100	5000
AH-SBLS1	10^-3	10^-2	7	0.005	100	5000
AH-SBLS	1	10^-2	7	10^-3	2250	3000

模型	λ	ρ	k	α	特征节点数	增强节点数
BLS	10^-8	-	-	-	100	11000
A-BLS1	10^-14	-	-	0.005	100	5000
A-BLS	1	-	-	10^-3	2250	3000
ss-BLS	1	10^-2	-	-	100	5000
AH-SBLS1	10^-3	10^-2	7	0.005	100	5000
AH-SBLS	1	10^-2	7	10^-3	2250	3000

模型	λ	ρ	k	α	特征节点数	增强节点数
BLS	10^-12	-	-	-	1000	9000
A-BLS1	1	-	-	10^-4	900	3000
A-BLS	1	-	-	0.003	5000	3000
ss-BLS	10^-3	10^-4	-	-	900	3000
AH-SBLS1	10^-2	10^-2	5	10^-4	900	3000
AH-SBLS	10^-2	10^-2	5	0.003	5000	3000

模型	λ	ρ	k	α	特征节点数	增强节点数
BLS	10^-12	-	-	-	1000	9000
A-BLS1	1	-	-	10^-4	900	3000
A-BLS	1	-	-	0.003	5000	3000
ss-BLS	10^-3	10^-4	-	-	900	3000
AH-SBLS1	10^-2	10^-2	5	10^-4	900	3000
AH-SBLS	10^-2	10^-2	5	0.003	5000	3000

模型	SAE	SDA	DBN	DBM	BLS
精度（%）	98.60	98.72	98.87	99.05	98.74
耗时（s）	36448.40	37786.03	53219.77	121455.69	29.92
模型	A-BLS1	A-BLS	ss-BLS	AH-SBLS1	AH-SBLS
精度（%）	98.93	99.22	92.54	93.43	94.57
耗时（s）	55.29	63.04	238.41	255.88	264.04

基于自编码器和超图的半监督宽度学习系统

Autoencoder and Hypergraph-Based Semi-Supervised Broad Learning System

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 22

相关文章 15

编辑推荐

Metrics

模型	SAE	SDA	DBN	DBM	BLS
精度（%）	86.26	87.62	88.47	89.65	89.27
耗时（s）	60504.34	65747.69	87280.42	182183.53	41.47
模型	A-BLS1	A-BLS	ss-BLS	AH-SBLS1	AH-SBLS
精度（%）	92.37	92.86	73.25	87.56	89.29
耗时（s）	32.36	67.86	226.97	264.58	311.95

[1]	曹阳, 高旻, 余俊良, 范琪琳, 荣文戈, 文俊浩. 基于双图混合随机游走的社会化推荐模型[J]. 电子学报, 2023, 51(2): 286-296.
[2]	许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.
[3]	王子为, 鲁继文, 周杰. 基于自适应梯度优化的二值神经网络[J]. 电子学报, 2023, 51(2): 257-266.
[4]	江兵兵, 何文达, 吴兴宇, 项俊浩, 洪立斌, 盛伟国. 基于自适应图学习的半监督特征选择[J]. 电子学报, 2022, 50(7): 1643-1652.
[5]	赵旭东, 陶然, 李伟, 张蒙蒙. 高光谱图像分数域信息提取理论与方法进展[J]. 电子学报, 2022, 50(12): 2874-2883.
[6]	孙新, 盖晨, 申长虹, 张颖捷. 基于短语向量和主题加权的关键词抽取方法[J]. 电子学报, 2021, 49(9): 1682-1690.
[7]	席亮, 刘涵, 樊好义, 张凤斌. 基于深度对抗学习潜在表示分布的异常检测模型[J]. 电子学报, 2021, 49(7): 1257-1265.
[8]	刘芳, 韩笑. 基于小波变换和深度网络的着陆地貌图像分类[J]. 电子学报, 2021, 49(11): 2171-2176.
[9]	江泽涛, 秦嘉奇, 张少钦. 参数池化卷积神经网络图像分类方法[J]. 电子学报, 2020, 48(9): 1729-1734.
[10]	王雄, 董一鸿, 潘剑飞, 陈华辉, 钱江波. 基于编辑行为码的图概要技术[J]. 电子学报, 2020, 48(12): 2434-2443.
[11]	邵航, 黄海亮, 郭雨晨, 戴琼海. 一种基于可信度估计单元的图像分类噪声抑制深度学习策略[J]. 电子学报, 2020, 48(10): 1969-1975.
[12]	魏志强, 毕海霞, 刘霞. 基于深度卷积神经网络的图上半监督极化SAR图像分类算法[J]. 电子学报, 2020, 48(1): 66-74.
[13]	曹晔. 一种基于局部排序的约束稀疏编码的图像分类方法[J]. 电子学报, 2019, 47(4): 832-836.
[14]	余游, 冯林, 王格格, 徐其凤. 一种基于伪标签的半监督少样本学习模型[J]. 电子学报, 2019, 47(11): 2284-2291.
[15]	葛疏雨, 高子淋, 张冰冰, 李培华. 基于核化双线性卷积网络的细粒度图像分类[J]. 电子学报, 2019, 47(10): 2134-2141.