A Few-Shot Learning Model Based on Semi-Supervised with Pseudo Label

YU You, FENG Lin, WANG Ge-ge, XU Qi-feng

ACTA ELECTRONICA SINICA ›› 2019, Vol. 47 ›› Issue (11) : 2284-2291.

PDF(1284 KB)
CIE Homepage  |  Join CIE  |  Login CIE  |  中文 
PDF(1284 KB)
ACTA ELECTRONICA SINICA ›› 2019, Vol. 47 ›› Issue (11) : 2284-2291. DOI: 10.3969/j.issn.0372-2112.2019.11.007

A Few-Shot Learning Model Based on Semi-Supervised with Pseudo Label

  • YU You, FENG Lin, WANG Ge-ge, XU Qi-feng
Author information +

Abstract

How to migrate a source domain knowledge model with a large amount of tagged data to a target domain with a small amount of tagged data is a hot issue in few-shot learning. For the problems that the existing few-shot learning algorithm have weak generalization ability when the difference between the feature distribution of the source domain data and the target domain data is large, a few-shot learning model based on semi-supervised FSLSS (Few-Shot Learning based on Semi-Supervised) is proposed. Firstly, a relational deep learning network is established by using the pytorch framework,and the network is pre-trained by the source domain data. Then, the network is used to predict the target domain data, and the label with the highest classification probability is used as the data's pseudo label. Finally, the network is hybrid trained using the pseudo label data of the target domain and the real label data of the source domain, then repeating the pseudo-labeled and hybrid trained process. The experimental results show that the FSLSS model has better generalization ability and knowledge transfer effect than the existing few-shot learning algorithms.

Key words

few-shot learning / semi-supervised learning / pseudo label / transfer learning

Cite this article

Download Citations
YU You, FENG Lin, WANG Ge-ge, XU Qi-feng. A Few-Shot Learning Model Based on Semi-Supervised with Pseudo Label[J]. Acta Electronica Sinica, 2019, 47(11): 2284-2291. https://doi.org/10.3969/j.issn.0372-2112.2019.11.007

References

[1] Yann L,Yoshua B,Geoffrey Hinton.Deep learning[J].Nature,2015,521(7553):436-444.
[2] Lin X,Rivenson Y,Yardimci N T,et al.All-optical machine learning using diffractive deep neural networks[J].Science,2018,361(6406):1004-1008.
[3] Fazeli N,Oller M,Wu J,et al.See,feel,act:Hierarchical learning for complex manipulation skills with multisensory fusion[J].Science Robotics,2019,4(26):eaav3123.
[4] 李小薪,梁荣华.有遮挡人脸识别综述:从子空间回归到深度学习[J].计算机学报,2018,41(01):177-207. Li Xiao-shui,Liang Ronghua.Overview of occluded face recognition:from subspace regression to deep learning[J].Chinese Journal of Computers,2018,41(01):177-207.(in Chinese)
[5] 赵嘉,等.深度学习萤火虫算法[J].电子学报,2018,46(11):2633-2641. Zhao Jia,et al.Deep learning firefly algorithm[J].Acta Electronica Sinica,2018,46(11):2633-2641.(in Chinese)
[6] Pan S J,Yang Q.A survey on transfer learning[J].IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[7] Li FF,Robert F,Pietro P.A bayesian approach to unsupervised one-shot learning of object categories[A].Ninth IEEE International Conference on Computer Vision[C].Sydney,Australia:IEEE,2003.1134-1141.
[8] Jake S,Kevin S,Ricard S Z.Prototypical networks for few-shot learning[A].Neural Information Processing Systems[C].Long Beach,California,2017.
[9] Maas A,Kemp C.One-shot learning with bayesian networks[A].31st Annual Conference of the Cognitive Science Society[C].Amsterdam,The Netherlands,2009.
[10] 舒醒,等.基于边际Fisher准则和迁移学习的小样本集分类器设计算法[J].自动化学报,2016,42(9):1313-1321. Shu Xing,et al.Smallsample set classifier design algorithm based on marginal fisher criterion and migration learning[J].Acta Automatica Sinica,2016,42(9):1313-1321.(in Chinese)
[11] Gregory K,Richard Z,Ruslan S.Siamese neural networks for one-shot image recognition[A].International Conference on Machine Learning Deep Learning workshop[C].Lille,France,2015.
[12] Oriol V,Charles B,Timothy L,et al.Matching networks for one shot learning[A].Advances in Neural Information Processing Systems[C].Barcelona,Spain,2016.3630-3638.
[13] Flood S,YANG Y X,Zhang L,et al.Learning to compare:relation network for few-shot learning[Z/DB].arXiv preprint arXiv:1711.06025v2,2018.
[14] Adam S Sergey B,Matthew B M.One-shot learning with memory-augmented neural networks[A].International Conference on Machine Learning[C].New York,USA,2016.
[15] Sachin R,Hugo L.Optimization as a model for few-shot learning[A].International Conference on Learning Representations[C].Toulon,France,2017.
[16] Chelsea F,Pieter A,Sergey L.Model-agnostic meta-learning for fast adaptation of deep networks[A].International Conference on Machine Learning[C].Sydney,Australia,2017.
[17] Zhou Z H,Li M.Semi-supervised learning by disagreement[J].Knowledge and Information Systems,2010,24(3):415-439.
[18] 刘建伟,刘媛,罗雄麟.半监督学习方法[J].计算机学报,2015,38(08):1592-1617. Liu Jianwei,et al.Semi-supervised learning method[J].Chinese Journal of Computers,2015,38(08):1592-1617.(in Chinese)
[19] Ben D,Lu T,Pai D.Does unlabeled data provably help? Worset-case analysis of the sample complexity of semi-supervised learning[A].Proceedings of the 21st Annual Conference on Learning Theory[C].Helsinki,Finland,2008.33-44.
[20] Balcan M F,Blum A.A discriminative model for semi-super-vised learning[J].Journal of the Association for Computing Machinery,2010,57(3):19.
[21] Hinton G,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
[22] Jo Y J,Park S,Jung J H,et al.Holographic deep learning for rapid optical screening of anthrax spores[J].Science Advances,2017,3(8):e1700606.
[23] Lin X,Rivenson Y,Yardimci N T,et al.All-optical machine learning using diffractive deep neural networks[J].Science,2018,361(6406):1004-1008.
[24] Hirschberg J,Manning C D.Advances in natural language processing[J].Science,2015,349(6245):261-266.
[25] Alex K,I S,Geoffrey H.Imagenet classification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,1:1097-1105.
[26] Lee,Dong H.Pseudo-Label:The simple and efficient semi-supervised learning method for deep neural networks[A].ICML 2013 Workshop on Challenges in Representation Learning[C].Paris,France,2013.2-10.
[27] Brenden L,Ruslan S,Jason G,et al.One shot learning of simple visual concepts[A].Cognitive Science Society Annual Conference[C].Paris,France,2011.
[28] Gretton A.A kernel two-sample test[J].Journal of Machine Learning Research,2012,13(1):723-773.

Funding

Project of National Key Technology R&D Program (No.2014BAH11F01, No.2014BAH11F02); 2019 Cultivation Fund Program for Excellent Dissertation in Sichuan Normal University (No.川师研201903-36)
PDF(1284 KB)

2064

Accesses

0

Citation

Detail

Sections
Recommended

/