一种基于伪标签的半监督少样本学习模型

doi:10.3969/j.issn.0372-2112.2019.11.007

电子学报 ›› 2019, Vol. 47 ›› Issue (11): 2284-2291.DOI: 10.3969/j.issn.0372-2112.2019.11.007

一种基于伪标签的半监督少样本学习模型

余游, 冯林, 王格格, 徐其凤

四川师范大学计算机科学学院, 四川成都 610101

收稿日期:2019-05-06 修回日期:2019-07-01 出版日期:2019-11-25
- 通讯作者:
- 冯林
- 作者简介:
- 余游男,1993年出生于四川遂宁,现为四川师范大学计算机科学学院硕士研究生,主要研究方向为深度学习与数据挖掘.E-mail:454665275@qq.com;王格格 女,1995年出生于重庆市,现为四川师范大学计算机科学学院硕士研究生,主要研究方向为人工智能与数据挖掘.E-mail:347673996@qq.com;徐其凤 女,1993年出生于四川宜宾,现为四川师范大学计算机科学学院硕士研究生,主要研究方向为深度学习.E-mail:616876864@qq.com
- 基金资助:
- 国家科技支撑计划项目 (No.2014BAH11F01,No.2014BAH11F02); 四川师范大学2019年研究生优秀论文培育基金 (No.川师研201903-36)

A Few-Shot Learning Model Based on Semi-Supervised with Pseudo Label

YU You, FENG Lin, WANG Ge-ge, XU Qi-feng

Department of Computer Science, Sichuan Normal University, Chengdu, Sichuan 610101, China

Received:2019-05-06 Revised:2019-07-01 Online:2019-11-25 Published:2019-11-25

摘要/Abstract

摘要： 如何将带有大量标记数据的源域知识模型迁移至带有少量标记数据的目标域是少样本学习研究领域的热点问题.针对现有的少样本学习算法在源域数据与目标域数据的特征分布差异较大时存在的泛化能力较弱的问题，提出一种基于伪标签的半监督少样本学习模型FSLSS（Few-Shot Learning based on Semi-Supervised）.首先，利用pytorch深度学习框架建立一个关系型深度学习网络，并使用源域数据对网络进行预训练；然后，使用此网络对目标域数据进行分类预测，将分类概率最大的类标签作为数据的伪标签；最后，利用目标域的伪标签数据和源域的真实标签数据对网络进行混合训练，并重复伪标签标记与混合训练过程.实验结果表明，相对于现有主流少样本学习算法，FSLSS模型有更好的泛化能力及知识迁移效果.

关键词: 少样本学习, 半监督学习, 伪标签, 迁移学习

Abstract: How to migrate a source domain knowledge model with a large amount of tagged data to a target domain with a small amount of tagged data is a hot issue in few-shot learning.For the problems that the existing few-shot learning algorithm have weak generalization ability when the difference between the feature distribution of the source domain data and the target domain data is large,a few-shot learning model based on semi-supervised FSLSS(Few-Shot Learning based on Semi-Supervised) is proposed.Firstly,a relational deep learning network is established by using the pytorch framework,and the network is pre-trained by the source domain data.Then,the network is used to predict the target domain data,and the label with the highest classification probability is used as the data's pseudo label.Finally,the network is hybrid trained using the pseudo label data of the target domain and the real label data of the source domain,then repeating the pseudo-labeled and hybrid trained process.The experimental results show that the FSLSS model has better generalization ability and knowledge transfer effect than the existing few-shot learning algorithms.

Key words: few-shot learning, semi-supervised learning, pseudo label, transfer learning

中图分类号:

TP181

余游, 冯林, 王格格, 徐其凤. 一种基于伪标签的半监督少样本学习模型[J]. 电子学报, 2019, 47(11): 2284-2291.

YU You, FENG Lin, WANG Ge-ge, XU Qi-feng. A Few-Shot Learning Model Based on Semi-Supervised with Pseudo Label[J]. Acta Electronica Sinica, 2019, 47(11): 2284-2291.

参考文献

[1] Yann L,Yoshua B,Geoffrey Hinton.Deep learning[J].Nature,2015,521(7553):436-444.
[2] Lin X,Rivenson Y,Yardimci N T,et al.All-optical machine learning using diffractive deep neural networks[J].Science,2018,361(6406):1004-1008.
[3] Fazeli N,Oller M,Wu J,et al.See,feel,act:Hierarchical learning for complex manipulation skills with multisensory fusion[J].Science Robotics,2019,4(26):eaav3123.
[4] 李小薪,梁荣华.有遮挡人脸识别综述:从子空间回归到深度学习[J].计算机学报,2018,41(01):177-207. Li Xiao-shui,Liang Ronghua.Overview of occluded face recognition:from subspace regression to deep learning[J].Chinese Journal of Computers,2018,41(01):177-207.(in Chinese)
[5] 赵嘉,等.深度学习萤火虫算法[J].电子学报,2018,46(11):2633-2641. Zhao Jia,et al.Deep learning firefly algorithm[J].Acta Electronica Sinica,2018,46(11):2633-2641.(in Chinese)
[6] Pan S J,Yang Q.A survey on transfer learning[J].IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[7] Li FF,Robert F,Pietro P.A bayesian approach to unsupervised one-shot learning of object categories[A].Ninth IEEE International Conference on Computer Vision[C].Sydney,Australia:IEEE,2003.1134-1141.
[8] Jake S,Kevin S,Ricard S Z.Prototypical networks for few-shot learning[A].Neural Information Processing Systems[C].Long Beach,California,2017.
[9] Maas A,Kemp C.One-shot learning with bayesian networks[A].31st Annual Conference of the Cognitive Science Society[C].Amsterdam,The Netherlands,2009.
[10] 舒醒,等.基于边际Fisher准则和迁移学习的小样本集分类器设计算法[J].自动化学报,2016,42(9):1313-1321. Shu Xing,et al.Smallsample set classifier design algorithm based on marginal fisher criterion and migration learning[J].Acta Automatica Sinica,2016,42(9):1313-1321.(in Chinese)
[11] Gregory K,Richard Z,Ruslan S.Siamese neural networks for one-shot image recognition[A].International Conference on Machine Learning Deep Learning workshop[C].Lille,France,2015.
[12] Oriol V,Charles B,Timothy L,et al.Matching networks for one shot learning[A].Advances in Neural Information Processing Systems[C].Barcelona,Spain,2016.3630-3638.
[13] Flood S,YANG Y X,Zhang L,et al.Learning to compare:relation network for few-shot learning[Z/DB].arXiv preprint arXiv:1711.06025v2,2018.
[14] Adam S Sergey B,Matthew B M.One-shot learning with memory-augmented neural networks[A].International Conference on Machine Learning[C].New York,USA,2016.
[15] Sachin R,Hugo L.Optimization as a model for few-shot learning[A].International Conference on Learning Representations[C].Toulon,France,2017.
[16] Chelsea F,Pieter A,Sergey L.Model-agnostic meta-learning for fast adaptation of deep networks[A].International Conference on Machine Learning[C].Sydney,Australia,2017.
[17] Zhou Z H,Li M.Semi-supervised learning by disagreement[J].Knowledge and Information Systems,2010,24(3):415-439.
[18] 刘建伟,刘媛,罗雄麟.半监督学习方法[J].计算机学报,2015,38(08):1592-1617. Liu Jianwei,et al.Semi-supervised learning method[J].Chinese Journal of Computers,2015,38(08):1592-1617.(in Chinese)
[19] Ben D,Lu T,Pai D.Does unlabeled data provably help? Worset-case analysis of the sample complexity of semi-supervised learning[A].Proceedings of the 21st Annual Conference on Learning Theory[C].Helsinki,Finland,2008.33-44.
[20] Balcan M F,Blum A.A discriminative model for semi-super-vised learning[J].Journal of the Association for Computing Machinery,2010,57(3):19.
[21] Hinton G,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
[22] Jo Y J,Park S,Jung J H,et al.Holographic deep learning for rapid optical screening of anthrax spores[J].Science Advances,2017,3(8):e1700606.
[23] Lin X,Rivenson Y,Yardimci N T,et al.All-optical machine learning using diffractive deep neural networks[J].Science,2018,361(6406):1004-1008.
[24] Hirschberg J,Manning C D.Advances in natural language processing[J].Science,2015,349(6245):261-266.
[25] Alex K,I S,Geoffrey H.Imagenet classification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,1:1097-1105.
[26] Lee,Dong H.Pseudo-Label:The simple and efficient semi-supervised learning method for deep neural networks[A].ICML 2013 Workshop on Challenges in Representation Learning[C].Paris,France,2013.2-10.
[27] Brenden L,Ruslan S,Jason G,et al.One shot learning of simple visual concepts[A].Cognitive Science Society Annual Conference[C].Paris,France,2011.
[28] Gretton A.A kernel two-sample test[J].Journal of Machine Learning Research,2012,13(1):723-773.

一种基于伪标签的半监督少样本学习模型

A Few-Shot Learning Model Based on Semi-Supervised with Pseudo Label

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	邵志文, 周勇, 谭鑫, 马利庄, 刘兵, 姚睿. 基于深度学习的表情动作单元识别综述[J]. 电子学报, 2022, 50(8): 2003-2017.
[2]	江兵兵, 何文达, 吴兴宇, 项俊浩, 洪立斌, 盛伟国. 基于自适应图学习的半监督特征选择[J]. 电子学报, 2022, 50(7): 1643-1652.
[3]	李维刚, 甘平, 谢璐, 李松涛. 基于样本对元学习的小样本图像分类方法[J]. 电子学报, 2022, 50(2): 295-304.
[4]	陈文俊, 杨春玲. 图像压缩感知的特征域优化及自注意力增强神经网络重构算法[J]. 电子学报, 2022, 50(11): 2629-2637.
[5]	张小恒, 张馨月, 李勇明, 王品, 刘玉川. 面向帕金森病语音诊断的非监督两步式卷积稀疏迁移学习算法[J]. 电子学报, 2022, 50(1): 177-184.
[6]	李勇斌, 王彬, 邵高平, 邵帅. 一种基于RCGAN的水声通信信号降噪方法[J]. 电子学报, 2022, 50(1): 54-62.
[7]	唐焕玲, 郑涵, 刘艳红, 马思源, 窦全胜, 鲁明羽. Tr-SLDA:一种面向交叉领域的迁移主题模型[J]. 电子学报, 2021, 49(3): 605-613.
[8]	张锦, 李阳, 任传伦, 黄炼, 王帅辉, 段晔鑫, 潘志松, 谢钧. 基于帧间高级特征差分的跨场景视频前景分割算法[J]. 电子学报, 2021, 49(10): 2032-2040.
[9]	王格格, 郭涛, 余游, 苏菡. 基于生成对抗网络的无监督域适应分类模型[J]. 电子学报, 2020, 48(6): 1190-1197.
[10]	张红斌, 蒋子良, 熊其鹏, 武晋鹏, 邬任重, 袁天, 姬东鸿. 基于改进的有效区域基因选择与跨模态语义挖掘的图像属性标注[J]. 电子学报, 2020, 48(4): 790-799.
[11]	赵鹏, 王美玉, 纪霞, 刘慧婷. 基于张量表示的域适配的迁移学习中特征表示方法[J]. 电子学报, 2020, 48(2): 359-368.
[12]	权宇, 李志欣, 张灿龙, 马慧芳. 融合深度扩张网络和轻量化网络的目标检测模型[J]. 电子学报, 2020, 48(2): 390-397.
[13]	刘真, 田靖玉, 苑宝鑫, 孙永奇. 基于知识聚合和迁移的跨领域推荐算法[J]. 电子学报, 2020, 48(10): 1928-1932.
[14]	季鼎承, 蒋亦樟, 王士同. 基于域与样例平衡的多源迁移学习方法[J]. 电子学报, 2019, 47(3): 692-699.
[15]	任卓君, 陈光, 卢文科. 基于N-gram特征的恶意代码可视化方法[J]. 电子学报, 2019, 47(10): 2108-2115.