卷积自编码器中粗粒度池化特征提取研究

doi:10.3969/j.issn.0372-2112.2017.10.012

电子学报 ›› 2017, Vol. 45 ›› Issue (10): 2390-2401.DOI: 10.3969/j.issn.0372-2112.2017.10.012

卷积自编码器中粗粒度池化特征提取研究

罗畅¹, 王洁¹, 王鹏飞¹, 肖军¹, 肖红²

1. 空军工程大学防空反导学院, 陕西西安 710051;
2. 94691部队, 福建连城 366202

收稿日期:2016-03-29 修回日期:2016-08-28 出版日期:2017-10-25
- 作者简介:
- 罗畅,男.1988年8月出生,四川广安人.分别于2011年和2013年在空军工程大学获工学学士和工学硕士学位.现为空军工程大学防空反导学院博士研究生,研究方向为非监督特征提取、遥感图像处理.E-mail:luochang1988@126.com;王洁,女.1963年11月出生,陕西渭南人.1984年、1991年和2004年分别在西安工业大学、北京理工大学和空军工程大学获工学学士、工学硕士和工学博士学位.现为空军工程大学防空反导学院教授、博士生导师,研究方向为智能控制与决策、遥感图像处理、自主目标识别.E-mail:wjlc123@sina.com
- 基金资助:
- 国家自然科学基金 (No.71501184）

Coarse-Grained Pooled Features Learning in Convolutional Autoencoders

LUO Chang¹, WANG Jie¹, WANG Peng-fei¹, XIAO Jun¹, XIAO Hong²

1. Air and Missile Defense College, Air Force Engineering University, Xi'an, Shaanxi 710051, China;
2. Unit 94691 PLA, Liancheng, Fujian 366202, China

Received:2016-03-29 Revised:2016-08-28 Online:2017-10-25 Published:2017-10-25
- Supported by:
- National Natural Science Foundation of China (No.71501184)

摘要/Abstract

摘要： 卷积自编码器（Convolutional Auto Encoder，CAE）提取的粗粒度池化特征具有一定范围内旋转和平移的不变性，因而得到广泛使用.然而，目前CAE仍主要依靠经验调节内部参数以获取满足要求的粗粒度池化特征.本文将CAE看作一个整体，从概率上分析了影响其表现的具体原因，构建了一个通用框架用于调节其中的主要参数以获取更好的粗粒度特征.首先从概率上权衡了粗粒度特征在池化层上的判别性与不变性，并在CAE中选择合适的卷积范围和白化参数.然后通过分析池化域内特征的稀疏度选择相应的池化方法以获取具有更好可分离性的粗粒度池化特征.在两个公开数据库（STL-10和CIFAR-10）的实验结果表明本文提出的方法可以指导CAE提取到更好的粗粒度池化特征并在多类分类任务中表现得更好.

关键词: 粗粒度特征, 池化, 卷积自编码器, 非监督学习, 深度学习

Abstract: Coarse-grained pooled features obtained from convolutional autoencoder (CAE) achieve scale and shift invariances and have been widely used recently.However,in most previous works coarse-grained pooled features are obtained by empirically modulating parameters in CAE.In this paper,we see the CAE as a whole,find the probabilistic factors affecting the performance of it,and formulate a general framework to regulate parameters in it to obtain better coarse-grained representation.Firstly,the discrimination-invariance tradeoff of coarse-grained features is probabilistically evaluated in the pooled feature maps.Furthermore,the proper convolved filter scales and appropriate whitening parameters are suggested in a CAE.Secondly,pooling approaches are combined with the sparsity degree in pooling regions,and we propose the preferable pooling approach in different cases.Experimental results on two independent benchmark datasets (STL-10 and CIFAR-10) demonstrate that our framework can guide CAEs to extract better coarse-grained pooled features and performs better in multi-class classification task.

Key words: coarse-grained features, pooling, convolutional autoencoder, unsupervised learning, deep learning

中图分类号:

TP181

罗畅, 王洁, 王鹏飞, 等. 卷积自编码器中粗粒度池化特征提取研究[J]. 电子学报, 2017, 45(10): 2390-2401.

LUO Chang, WANG Jie, WANG Peng-fei, et al. Coarse-Grained Pooled Features Learning in Convolutional Autoencoders[J]. Acta Electronica Sinica, 2017, 45(10): 2390-2401.

参考文献

[1] JORDAN M I,MITCHELL T M.Machine learning:trends,perspectives,and prospects[J].Science,2015,17(7):255-260.
[2] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenet classification with deep convolutional neural networks[A].Proceedings of Advances in Neural Information Processing Systems[C].Lake Tahoe:NIPS,2012.1097-1105.
[3] HINTON G,et al.Deep neural networks for acoustic modeling in speech recognition:the shared views of four research groups[J].IEEE Signal Processing Magazine,2012,29(6):82-97.
[4] DU B,et al.Stacked convolutional denoising auto-encoders for feature representation[J].IEEE Transactions on Cybern,2016,16(3):1-11.
[5] WANG Y,et al.An efficient and effective convolutional auto-encoder extreme learning machine network for 3D feature learning[J].Neurocomputing,2016,22(1):988-998.
[6] LI Z,FAN Y,WANG F.Convolutional autoencoder-based color image classification using chroma subsampling in YCbCr space[A].Proceedings of 8th International Congress on Image and Signal Processing (CISP)[C].Shenyang:IEEE,2015.351-355.
[7] GENG J,FAN J,WANG H,et al.High-resolution SAR image classification via deep convolutional autoencoders[J].IEEE Geoscience and Remote Sensing Letters,2015,12(11):2351-2355.
[8] LENG B,GUO S,ZHANG X,et al.3D object retrieval with stacked local convolutional autoencoder[J].Signal Processing,2015,112:119-128.
[9] HAMESTER D,BARROS P,WERMTER S.Face expression recognition with a 2-channel convolutional neural network[A].Proceedings of 2015 International Joint Conference on Neural Networks (IJCNN)[C].Killarney:IEEE,2015.1-8.
[10] HOLDEN D,SAITO J,KOMURA T,et al.Learning motion manifolds with convolutional autoencoders[A].Proceedings of SIGGRAPH Asia 2015 Technical Briefs[C].New York:ACM,2015.18.
[11] LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-444.
[12] LI P,PENG L,WEN J.Rejecting character recognition errors using CNN based confidence estimation[J].Chinese Journal of Electronics,2016,25(3):520-526.
[13] CIRESAN D C,MEIER U,MASCI J,et al.Flexible,high performance convolutional neural networks for image classification[A].Proceedings of International Joint Conference on Artificial Intelligence (IJCAI)[C].USA:AAAI,2011.1237-1242.
[14] LEE C Y,GALLAGHER P W,TU Z.Generalizing pooling functions in convolutional neural networks:Mixed,gated,and tree[A].Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS)[C].Cadiz:JMLR,2016.464-472.
[15] CHEN L,ROTTENSTEINER F,HEIPKE C.Feature descriptor by convolution and pooling autoencoders[J].The International Archives of Photogrammetry,Remote Sensing and Spatial Information Sciences,2015,40(3):31-38.
[16] MALINOWSKI M,FRITZ M.Learnable pooling regions for image classification[A].Workshop at International Conference on Learning Representations[C].USA:arXiv Preprint,2013,arXiv:1301.3516.
[17] JIA Y,HUANG C,DARRELL T.Beyond spatial pyramids:Receptive field learning for pooled image features[A].Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Providence:IEEE,2012.3370-3377.
[18] SIMONYAN K,VEDALDI A,ZISSERMAN A.Descriptor learning using convex optimisation[A].Proceedings of 2012 European Conference on Computer Vision (ECCV)[C].Florence:Springer,2012.243-256.
[19] XU C,VASCONCELOS N.Learning receptive fields for pooling from tensors of feature response[A].Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Columbus:IEEE,2014.835-842.
[20] GONG Y,WANG L,GUO R,et al.Multi-scale orderless pooling of deep convolutional activation features[A].Proceedings of 2014 European Conference on Computer Vision (ECCV)[C].Zurich:Springer,2014.392-407.
[21] RYAN FANELLO S,NOCETI N,CILIBERTO C,et al.Ask the image:supervised pooling to preserve feature locality[A].Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Columbus:IEEE,2014.851-858.
[22] LI C,REITER A,HAGER G D.Beyond spatial pooling:fine-grained representation learning in multiple domains[A].Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Boston:IEEE,2015.4913-4922.
[23] HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
[24] SERMANET P,EIGEN D,ZHANG X,et al.Overfeat:Integrated recognition,localization and detection using convolutional networks[A].Workshop at International Conference on Learning Representations[C].USA:arXiv preprint,2013,arXiv:1312.6229.
[25] BO L,REN X,FOX D.Unsupervised feature learning for RGB-D based object recognition[A].Proceedings of the 13th International Symposium on Experimental Robotics[C].Berlin:Springer,2013.387-402.
[26] SOCHER R,HUVAL B,BATH B,et al.Convolutional-recursive deep learning for 3d object classification[A].Proceedings of the Advances in Neural Information Processing Systems[C].Lake Tahoe:NIPS,2012.665-673.
[27] COATES A,NG A Y,LEE H.An analysis of single-layer networks in unsupervised feature learning[A].Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS)[C].Fort Lauderdale:JMLR,2011.215-223.
[28] KRIZHEVSKY A.Learning Multiple Layers of Features From Tiny Images[D].Toronto:University of Toronto,2009.
[29] MALALUR S S,MANRY M T,JESUDHAS P.Multiple optimal learning factors for the multi-layer perceptron[J].Neurocomputing,2015,3(2):1490-1501.
[30] ELAWADY M.Sparse coral classification using deep convolutional neural networks[A].Workshop at International Conference on Learning Representations[C].USA:arXiv preprint,2015,arXiv:1511.0906.
[31] WANG W,YANG X,OOI B C,et al.Effective deep learning-based multi-modal retrieval[J].The VLDB Journal,2016,25(1):79-101.
[32] NG A Y,et al.Deep Learning[OL].http://deeplearning.stanford.edu/wiki/index.php[2014-11-18].
[33] ZEILER M D,FERGUS R.Stochastic pooling for regularization of deep convolutional neural networks[A].Workshop at International Conference on Learning Representations[C].USA:arXiv preprint,2013,arXiv:1301.3557.
[34] LIU L,SHEN C,VAN DEN HENGEL A.The treasure beneath convolutional layers:Cross-convolutional-layer pooling for image classification[A].Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Boston:IEEE,2015.4749-4757.
[35] BOUREAU Y L,PONCE J,LECUN Y.A theoretical analysis of feature pooling in visual recognition[A].Proceedings of the 27th International Conference on Machine Learning (ICML)[C].Haifa:ELSEVIER,2010.111-118.
[36] LI Z,FAN Y,LIU W.The effect of whitening transformation on pooling operations in convolutional autoencoders[J].EURASIP Journal on Advances in Signal Processing,2015,2015(37):1-11.
[37] LIAN Z,JING X,WANG X,et al.Drop connect regularization method with sparsity constraint for neural networks[J].Chinese Journal of Electronics,2016,25(1):152-158.
[38] MAATEN L,HINTON G.Visualizing data using t-SNE[J].Journal of Machine Learning Research,2008,9(9):2579-2605.
[39] VAN DER MAATEN L.Accelerating t-SNE using tree-based algorithms[J].Journal of Machine Learning Research,2014,15(1):3221-3245.

卷积自编码器中粗粒度池化特征提取研究

Coarse-Grained Pooled Features Learning in Convolutional Autoencoders

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	郑云飞, 王晓兵, 张雄伟, 曹铁勇, 孙蒙. 基于金字塔知识的自蒸馏HRNet目标分割方法[J]. 电子学报, 2023, 51(3): 746-756.
[2]	李钦, 刘伟, 牛朝阳, 宝音图, 惠周勃. 低信噪比下基于分裂EfficientNet网络的雷达信号调制方式识别[J]. 电子学报, 2023, 51(3): 675-686.
[3]	范兵兵, 何庭建, 张聪炫, 陈震, 黎明. 联合遮挡约束与残差补偿的特征金字塔光流计算方法[J]. 电子学报, 2023, 51(3): 648-657.
[4]	杨利平, 侯振威, 辜小花, 郝峻永. 弱标签声音事件检测的空间-通道特征表征与自注意池化[J]. 电子学报, 2023, 51(2): 297-306.
[5]	张聿远, 闫文君, 张立民. 基于多模态特征融合网络的空时分组码识别算法[J]. 电子学报, 2023, 51(2): 489-498.
[6]	许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.
[7]	王神龙, 雍宇, 吴晨睿. 基于伪孪生神经网络的低纹理工业零件6D位姿估计[J]. 电子学报, 2023, 51(1): 192-201.
[8]	张永梅, 孙捷. 基于动静态特征双输入神经网络的咳嗽声诊断COVID-19算法[J]. 电子学报, 2023, 51(1): 202-212.
[9]	袁海英, 成君鹏, 曾智勇, 武延瑞. Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J]. 电子学报, 2023, 51(1): 180-191.
[10]	李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128-138.
[11]	郭晓轩, 冯其波, 冀振燕, 郑发家, 杨燕燕. 多线激光光条图像缺陷分割模型研究[J]. 电子学报, 2023, 51(1): 172-179.
[12]	贾童瑶, 卓力, 李嘉锋, 张菁. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231-245.
[13]	何滢婕, 刘月峰, 边浩东, 郭威, 张小燕. 基于Informer的电池荷电状态估算及其稀疏优化方法[J]. 电子学报, 2023, 51(1): 50-56.
[14]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[15]	李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.