深度卷积神经网络鉴别正交特征生成及其应用

doi:10.3969/j.issn.0372-2112.2018.10.010

电子学报 ›› 2018, Vol. 46 ›› Issue (10): 2376-2383.DOI: 10.3969/j.issn.0372-2112.2018.10.010

深度卷积神经网络鉴别正交特征生成及其应用

杨勃, 邵泉铭, 李文彬, 郭观七, 方欣

湖南理工学院信息与通信工程学院, 湖南岳阳 414000

收稿日期:2017-10-15 修回日期:2018-01-08 出版日期:2018-10-25
- 通讯作者:
- 方欣
- 作者简介:
- 杨勃,男,1974年出生于湖南岳阳.现为湖南理工学院副教授、硕士生导师.主要研究方向为统计机器学习、深度学习和脑影像分析.E-mail:ybmengshen@163.com;邵泉铭,男,1991年出生于四川巴中.现为湖南理工学院硕士研究生.主要研究方向为深度学习.E-mail:783295011@qq.com
- 基金资助:
- 湖南省教育厅科学研究重点项目 (No.17A089，No.15A079）; 湖南省科技计划 (No.2016TP1021）

Deep Convolutional Neural Networks Controlled by Discriminatively Orthogonal Feature Generation and Its Application

YANG Bo, SHAO Quan-ming, LI Wen-bin, GUO Guan-qi, FANG Xin

School of Information & Communication Engineering, Hunan Institute of Science and Technology, Yueyang, Hunan 414000, China

Received:2017-10-15 Revised:2018-01-08 Online:2018-10-25 Published:2018-10-25
- Corresponding author:
- FANG Xin

摘要/Abstract

摘要： 针对现有深度卷积神经网络在小样本学习时的泛化性问题，本文提出一种鉴别正交特征生成方法.该方法通过正则化技术对网络非负中间层特征输出的异类正交度和同类相关度进行优化，生成具有稀疏特性的网络中间层鉴别正交特征.为有效调节稀疏度以控制网络容量，采用正则化系数自适应调节方式逼近预设特征稀疏度目标.为提高特征生成计算效率，进一步设计了随机2类别鉴别正交特征生成反向传播规则.随后在数据集MNIST上进行了小样本手写体数字识别对比实验，验证了本文方法的稀疏度调节特性和网络表达容量控制能力.通过反卷积可视化，进一步发现本文方法还具有衍生出的局部鉴别区域聚焦特性.最后，将鉴别正交特征生成卷积网络应用到老年痴呆症3D磁共振影像分析上.实验结果表明，本文方法用于老年痴呆症诊断，不仅诊断效果更好，而且利用其良好的局部聚焦性，还成功定位了老年痴呆症与健康对照组典型差异脑区.

关键词: 深度卷积网络, 鉴别正交特征生成, 脑影像分析, 核磁共振

Abstract: To improve the generalization of deep convolutional neural networks (CNN), we proposed a discriminatively orthogonal feature generation method. By regularizing nonnegative outputs, orthogonal degree and correlation degree were optimized simultaneously, which helps to generate discriminatively orthogonal and sparse features. To adjust sparse degree for controlling network capacity, the technique of auto-adjusting regularization coefficient was proposed. To improve computational efficiency, a stochastically 2-class discriminatively orthogonal feature generation rule was further designed. Subsequently, a comparative experiment was conducted on handwritten digit set MNIST. In this experiment, the sparsity adjustment property of our method was verified. By means of deconvolution technique for visualization, it was further found that our method has a good property of focusing on local discriminant areas. Finally, our method was applied to Alzheimer's Disease MRI image analysis. The experimental results showed that our method outperforms some other representative methods and locates the importantly discriminant brain regions successfully.

Key words: deep convolutional neural networks, discriminatively orthogonal feature generation, brain image analysis, nuclear magnetic resonance

中图分类号:

TP391.4

杨勃, 邵泉铭, 李文彬, 郭观七, 方欣. 深度卷积神经网络鉴别正交特征生成及其应用[J]. 电子学报, 2018, 46(10): 2376-2383.

YANG Bo, SHAO Quan-ming, LI Wen-bin, GUO Guan-qi, FANG Xin. Deep Convolutional Neural Networks Controlled by Discriminatively Orthogonal Feature Generation and Its Application[J]. Acta Electronica Sinica, 2018, 46(10): 2376-2383.

参考文献

[1] Lecun Y,Bengio Y,Hinton G.Deep learning[J].Nature,2015,521(7553):436-444.
[2] Schmidhuber J.Deep learning in neural networks:An overview[J].Neural Networks,2015,61(1):85-117.
[3] Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[A].International Conference on Neural Information Processing Systems[C].Doha,Qatar:Curran Associates Inc,2012.1097-1105.
[4] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[A].Computer Vision and Pattern Recognition[C].USA,Boston:IEEE,2015.3431-3440.
[5] Dai J,Li Y,He K,et al.R-FCN:Object Detection via Region-based Fully Convolutional Networks[Z].Preprint arXiv:1605.06409,2016.
[6] Lin K,Yang H F,Hsiao J H,et al.Deep learning of binary hash codes for fast image retrieval[A].Computer Vision and Pattern Recognition Workshops[C].USA,Boston:IEEE,2015.27-35.
[7] Halevy A,Norvig P,Pereira F.The unreasonable effectiveness of data[J].IEEE Intelligent Systems,2009,24(2):8-12.
[8] Sun C,Shrivastava A,Singh S,et al.Revisiting Unreasonable Effectiveness of Data in Deep Learning Era[Z].Preprint arXiv:1707.02968,2017.
[9] Vapnik V N.Statistical Learning Theory[M].[S.l.]:Publishing House of Electronics Industry,2015.
[10] 庄福振,罗平,何清,等.迁移学习研究进展[J].软件学报,2015,26(1):26-39. Zhuang Fu-Zhen,LuoPing,He Qing,et al.Survey on transfer learning research[J].Journal of Software,2015,26(1):26-39.(in Chinese)
[11] Koch G.Siamese Neural Networks for One-Shot Image Recognition[D].University of Toronto,2015.
[12] Vinyals O,Blundell C,Lillicrap T,et al.Matching networks for one shot learning[A].Proceedings of Advances in Neural Information Processing Systems[C].Spain,Barcelona,2016.3630-3638.
[13] Srivastava N,Hinton G,Krizhevsky A,et al.Dropout:A simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15(1):1928-1958.
[14] Hinton G E.Connectionist learning procedures[J].Artificial Intelligence,1989,40(1-3):185-234.
[15] Yosinski J,Clune J,Bengio Y,et al.How transferable are features in deep neural networks?[A].International Conference on Neural Information Processing Systems[C].Canada,Montreal,2014.3320-3328.
[16] Azizpour H,Razavian A S,Sullivan J,et al.From generic to specific deep representations for visual recognition[A].Computer Vision and Pattern Recognition Workshops[C].USA,Boston:IEEE,2015.36-45.
[17] Jia Y,Shelhamer E,Donahue J,et al.Caffe:Convolutional Architecture for Fast Feature Embedding[Z].Preprint arXiv:1408.5093,2014.
[18] Lecun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[19] Szegedy C,Liu W,Jia Y,et al.Going deeper with convolutions[A].Computer Vision and Pattern Recognition Workshops[C].USA,Boston:IEEE,2015.1-9.
[20] He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[A].Computer Vision and Pattern Recognition Workshops[C].USA,Las Vegas:IEEE,2016.770-778.
[21] Zeiler M D,Fergus R.Visualizing and understanding convolutional networks[A].European Conference on Computer Vision[C].Swiss,Zurich:2014,8689:818-833.
[22] 孙晓,潘汀.基于兴趣区域深度神经网络的静态面部表情识别[J].电子学报,2017,45(5):1189-1197. SUN Xiao,PAN Ting.Static facial expression recognition system using ROI deep neural networks[J].Acta Electronica Sinica,2017,45(5):1189-1197.(in Chinese)
[23] 胡正平,陈俊岭.多层融合深度局部PCA子空间稀疏优化特征提取模型[J].电子学报,2017,45(10):2383-2389. HU Zheng-ping,CHEN Jun-ling.Feature extraction model based on multi-layered deep local subspace sparse optimization[J].Acta Electronica Sinica,2017,45(10):2383-2389.(in Chinese)
[24] Ewers M,Walsh C,Trojanowski J Q,et al.Prediction of conversion from mild cognitive impairment to Alzheimer's disease dementia based upon biomarkers and neuropsychological test performance[J].Neurobiology of Aging,2012,33(7):1203-1214.
[25] Westman E,Muehlboeck J S,Simmons A.Combining MRI and CSF measures for classification of Alzheimer's disease and prediction of mild cognitive impairment conversion[J].Neuroimage,2012,62(1):229-238.
[26] Liu M,Zhang D,Shen D.Ensemble sparse classification of Alzheimer's disease[J].Neuroimage,2012,60(2):1106-1116.
[27] Li R,Zhang W,Suk H I,et al.Deep learning based imaging data completion for improved brain disease diagnosis[A].International Conference on Medical Image Computing and Computer-Assisted Intervention[C].USA,Cambridge:Springer,2014.305-312.
[28] Mueller S G,Weiner M W,Thal L J,et al.The Alzheimer's disease neuroimaging initiative[J].Neuroimaging Clinics of North America,2005,15(4):869-869.
[29] Johnson N A,Jahng G H,Weiner M W,et al.Pattern of cerebral hypoperfusion in Alzheimer disease and mild cognitive impairment measured with arterial spin-labeling MR imaging:initial experience[J].Radiology,2005,234(3):851-859.

深度卷积神经网络鉴别正交特征生成及其应用

Deep Convolutional Neural Networks Controlled by Discriminatively Orthogonal Feature Generation and Its Application

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

[1]	葛婷, 牟宁, 李黎. 基于softmax回归与图割法的脑肿瘤分割算法[J]. 电子学报, 2017, 45(3): 644-649.
[2]	詹天明;张军;韦志辉;肖亮;孙玉宝. 基于参数化互信息的脑MR图像分割与偏移场矫正模型及快速算法[J]. 电子学报, 2011, 39(12): 2807-2812.
[3]	刘凡;高上凯;高小榕. 基准点法和表面匹配法加权结合的MRI图像和EEG电极的配准[J]. 电子学报, 2000, 28(12): 19-21.