Abstract：To improve the generalization of deep convolutional neural networks (CNN),we proposed a discriminatively orthogonal feature generation method.By regularizing nonnegative outputs,orthogonal degree and correlation degree were optimized simultaneously,which helps to generate discriminatively orthogonal and sparse features.To adjust sparse degree for controlling network capacity,the technique of auto-adjusting regularization coefficient was proposed.To improve computational efficiency,a stochastically 2-class discriminatively orthogonal feature generation rule was further designed.Subsequently,a comparative experiment was conducted on handwritten digit set MNIST.In this experiment,the sparsity adjustment property of our method was verified.By means of deconvolution technique for visualization,it was further found that our method has a good property of focusing on local discriminant areas.Finally,our method was applied to Alzheimer's Disease MRI image analysis.The experimental results showed that our method outperforms some other representative methods and locates the importantly discriminant brain regions successfully.
[1] Lecun Y,Bengio Y,Hinton G.Deep learning[J].Nature,2015,521(7553):436-444.
[2] Schmidhuber J.Deep learning in neural networks:An overview[J].Neural Networks,2015,61(1):85-117.
[3] Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[A].International Conference on Neural Information Processing Systems[C].Doha,Qatar:Curran Associates Inc,2012.1097-1105.
[4] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[A].Computer Vision and Pattern Recognition[C].USA,Boston:IEEE,2015.3431-3440.
[5] Dai J,Li Y,He K,et al.R-FCN:Object Detection via Region-based Fully Convolutional Networks[Z].Preprint arXiv:1605.06409,2016.
[6] Lin K,Yang H F,Hsiao J H,et al.Deep learning of binary hash codes for fast image retrieval[A].Computer Vision and Pattern Recognition Workshops[C].USA,Boston:IEEE,2015.27-35.
[7] Halevy A,Norvig P,Pereira F.The unreasonable effectiveness of data[J].IEEE Intelligent Systems,2009,24(2):8-12.
[8] Sun C,Shrivastava A,Singh S,et al.Revisiting Unreasonable Effectiveness of Data in Deep Learning Era[Z].Preprint arXiv:1707.02968,2017.
[9] Vapnik V N.Statistical Learning Theory[M].[S.l.]:Publishing House of Electronics Industry,2015.
[10] 庄福振,罗平,何清,等.迁移学习研究进展[J].软件学报,2015,26(1):26-39. Zhuang Fu-Zhen,LuoPing,He Qing,et al.Survey on transfer learning research[J].Journal of Software,2015,26(1):26-39.(in Chinese)
[11] Koch G.Siamese Neural Networks for One-Shot Image Recognition[D].University of Toronto,2015.
[12] Vinyals O,Blundell C,Lillicrap T,et al.Matching networks for one shot learning[A].Proceedings of Advances in Neural Information Processing Systems[C].Spain,Barcelona,2016.3630-3638.
[13] Srivastava N,Hinton G,Krizhevsky A,et al.Dropout:A simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15(1):1928-1958.
[14] Hinton G E.Connectionist learning procedures[J].Artificial Intelligence,1989,40(1-3):185-234.
[15] Yosinski J,Clune J,Bengio Y,et al.How transferable are features in deep neural networks?[A].International Conference on Neural Information Processing Systems[C].Canada,Montreal,2014.3320-3328.
[16] Azizpour H,Razavian A S,Sullivan J,et al.From generic to specific deep representations for visual recognition[A].Computer Vision and Pattern Recognition Workshops[C].USA,Boston:IEEE,2015.36-45.
[17] Jia Y,Shelhamer E,Donahue J,et al.Caffe:Convolutional Architecture for Fast Feature Embedding[Z].Preprint arXiv:1408.5093,2014.
[18] Lecun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[19] Szegedy C,Liu W,Jia Y,et al.Going deeper with convolutions[A].Computer Vision and Pattern Recognition Workshops[C].USA,Boston:IEEE,2015.1-9.
[20] He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[A].Computer Vision and Pattern Recognition Workshops[C].USA,Las Vegas:IEEE,2016.770-778.
[21] Zeiler M D,Fergus R.Visualizing and understanding convolutional networks[A].European Conference on Computer Vision[C].Swiss,Zurich:2014,8689:818-833.
[22] 孙晓,潘汀.基于兴趣区域深度神经网络的静态面部表情识别[J].电子学报,2017,45(5):1189-1197. SUN Xiao,PAN Ting.Static facial expression recognition system using ROI deep neural networks[J].Acta Electronica Sinica,2017,45(5):1189-1197.(in Chinese)
[23] 胡正平,陈俊岭.多层融合深度局部PCA子空间稀疏优化特征提取模型[J].电子学报,2017,45(10):2383-2389. HU Zheng-ping,CHEN Jun-ling.Feature extraction model based on multi-layered deep local subspace sparse optimization[J].Acta Electronica Sinica,2017,45(10):2383-2389.(in Chinese)
[24] Ewers M,Walsh C,Trojanowski J Q,et al.Prediction of conversion from mild cognitive impairment to Alzheimer's disease dementia based upon biomarkers and neuropsychological test performance[J].Neurobiology of Aging,2012,33(7):1203-1214.
[25] Westman E,Muehlboeck J S,Simmons A.Combining MRI and CSF measures for classification of Alzheimer's disease and prediction of mild cognitive impairment conversion[J].Neuroimage,2012,62(1):229-238.
[26] Liu M,Zhang D,Shen D.Ensemble sparse classification of Alzheimer's disease[J].Neuroimage,2012,60(2):1106-1116.
[27] Li R,Zhang W,Suk H I,et al.Deep learning based imaging data completion for improved brain disease diagnosis[A].International Conference on Medical Image Computing and Computer-Assisted Intervention[C].USA,Cambridge:Springer,2014.305-312.
[28] Mueller S G,Weiner M W,Thal L J,et al.The Alzheimer's disease neuroimaging initiative[J].Neuroimaging Clinics of North America,2005,15(4):869-869.
[29] Johnson N A,Jahng G H,Weiner M W,et al.Pattern of cerebral hypoperfusion in Alzheimer disease and mild cognitive impairment measured with arterial spin-labeling MR imaging:initial experience[J].Radiology,2005,234(3):851-859.
2018, Vol. 46 
