残差神经网络及其在医学图像处理中的应用研究

doi:10.3969/j.issn.0372-2112.2020.07.024

电子学报 ›› 2020, Vol. 48 ›› Issue (7): 1436-1447.DOI: 10.3969/j.issn.0372-2112.2020.07.024

所属专题：机器学习与智慧医疗

残差神经网络及其在医学图像处理中的应用研究

周涛^1,2,3, 霍兵强¹, 陆惠玲², 任海玲⁴

1. 北方民族大学计算机科学与工程学院, 宁夏银川 750021;
2. 宁夏医科大学理学院, 宁夏银川 750004;
3. 宁夏智能信息与大数据处理重点实验室, 宁夏银川 750021;
4. 宁夏医科大学公共卫生与管理学院, 宁夏银川 750004

收稿日期:2019-07-19 修回日期:2020-01-11 出版日期:2020-07-25
- 通讯作者:
- 周涛
- 作者简介:
- 霍兵强 男,1994年生,河北石家庄人.硕士,现为北方民族大学计算机学院研究生,主研领域:智能医学影像图像处理,深度学习.E-mail:2916656832@qq.com;陆惠玲 女,1976年,河北定兴县人.副教授,硕士,主要研究医学图像分析处理、机器学习;任海玲 女,1993生,四川人.现为宁夏医科大学公共卫生与管理学院研究生,主研领域为智能医学影像图像处理及计算机辅助诊断.E-mail:renhailingnxmu@163.com
- 基金资助:
- 国家自然科学基金 (No.61561040）; 宁夏高等学校一流学科建设 (数学学科） (No.NXYLXK2017B09）; 北方民族大学引进人才科研启动项目 (No.2020KYQD08）; 宁夏312优秀人才项目; 北方民族大学创新创业项目 (No.YCX19075）

Research on Residual Neural Network and Its Application on Medical Image Processing

ZHOU Tao^1,2,3, HUO Bing-qiang¹, LU Hui-ling², REN Hai-ling⁴

1. School of Computer Science and Engineering, North Minzu University, Yinchuan, Ningxia 750021, China;
2. School of Science, Ningxia Medical University, Yinchuan, Ningxia 750004, China;
3. Ningxia Key Laboratory of Intelligent Information and Big Data Processing, Yinchuan, Ningxia 750021, China;
4. School of Public Health and Management, Ningxia Medical University, Yinchuan, Ningxia 750004, China

Received:2019-07-19 Revised:2020-01-11 Online:2020-07-25 Published:2020-07-25
- Corresponding author:
- ZHOU Tao
- Supported by:
- National Natural Science Foundation of China (No.61561040); First-Class Discipline Construction Program of Colleges and Universities in Ningxia Hui Autonomous Region (Mathematics) (No.NXYLXK2017B09); Recruited Talent Research Program of North Minzu University (No.2020KYQD08); Ningxia 312 Outstanding Talents Project; Innovation and Entrepreneurship Project of North Minzu University (No.YCX19075)

摘要/Abstract

摘要： 残差神经网络（ResNet）是近几年来深度学习研究中的热点，在计算机视觉领域取得较好成就.本文对残差神经网络从以下几个方面进行总结：第一，阐述残差神经网络的基本结构和工作原理；第二，在模型发展方面，以时间为顺序总结了残差神经网络的8种网络模型；第三，在结构优化方面，从残差神经网络的卷积层、池化层、残差单元、全连接层以及整个网络5个方面进行总结；最后，将ResNet应用到医学图像处理领域，主要从图像识别和图像分割2个方面探讨.本文对残差神经网络的原理、模型、结构进行了系统地总结，对残差神经网络的研究发展具有一定的积极意义.

关键词: 残差神经网络, 网络结构, 医学图像

Abstract: Residual neural network (ResNet) has witnessed tremendous amount of attention in deep learning research over the last few years and has made great achievements in computer vision. In this paper, the ResNet is summarized in the following aspects: Firstly, the basic structure and working principle of the ResNet are expounded; Secondly, in model development, the eight network models of the ResNet are summarized in time sequence; Thirdly, in structural optimization, the research progress is described from five aspects of ResNet, including convolutional layer, pooling layer, residual unit, fully connected layer and the whole network; Finally, the application of ResNet in medical images processing is mainly discussed from two aspects of image recognition and image segmentation. In this paper, the principles, models, and structures of ResNet are systematically summarized, which has positive significance to the research and development of ResNet.

Key words: residual neural network, network structure, medical image

中图分类号:

TP399

周涛, 霍兵强, 陆惠玲, 等. 残差神经网络及其在医学图像处理中的应用研究[J]. 电子学报, 2020, 48(7): 1436-1447.

ZHOU Tao, HUO Bing-qiang, LU Hui-ling, et al. Research on Residual Neural Network and Its Application on Medical Image Processing[J]. Acta Electronica Sinica, 2020, 48(7): 1436-1447.

参考文献

[1] 肖焕辉,袁程朗,冯仕庭,等.基于深度学习的癌症计算机辅助分类诊断研究进展[J].国际医学放射杂志,2019,42(1):22-25. XIAO Huan-hui,YUAN Cheng-lang,FENG Shi-ting,et al.Research progress of computer aided diagnosis in cancer based on deep learning[J].Int J Med Radiol,2019,42(1):22-25.(in Chinese)
[2] He K M,Zhang X Y,Ren S Q,Sun J.Deep residual learning for image recognition[A].Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Las Vegas,NV,USA:IEEE,2016.770-778.
[3] 田娟秀,刘国才,谷珊珊,等.医学图像分析深度学习方法研究与挑战[J].自动化学报,2018,44(3):401-424. TIAN Juan-xiu,LIU Guo-cai,GU Shan-shan,et al.Deep learning in medical image analysis and its challenges[J].Acta Automatica Sinica,2018,44(3):401-424.(in Chinese)
[4] Wells III W M.Medical image analysis-past,present,and future[J].Medical Image Analysis,2016,33:4-6.
[5] Geert Litjens,Thijs Kooi,Babak Ehteshami Bejnordi,et al.A survey on deep learning in medical image analysis[J].M edical Image Analysis,2017,42:60-88.
[6] AI医疗影像辅助诊断系统发展研究报告[R].海松医疗基金, http://www.oceanpinecap.com/documents/view20180205.pdf.
[7] Andreas Maier,Christopher Syben,Tobias Lasser,et al.A gentle introduction to deep learning in medical image processing[J].Computer Vision and Pattern Recognition,2019,29(2):86-101.
[8] Nibali A,He Z,Wollersheim D.Pulmonary nodule classification with deep residual networks[J].Int J Comput Assist Radiol Surg,2017,12:1799-1808.
[9] 亢寒,张荣国,陈宽.基于深度学习的医学图像分割技术[J].人工智能,2018,4:30-37.
[10] Michal Drozdzal,Eugene Vorontsov,Gabriel Chartrand,et al.The importance of skip connections in biomedical image segmentation[J].arXiv Preprint,2016,arXiv:1608.04117.
[11] He K,Zhang X,Ren S,et al.Identity mappings in deep residual networks[A].Proceedings of the 2016 Computer Vision-ECCV 14th European Conference[C].Amsterdam,Netherlands,2016.630-645.
[12] 梁蒙蒙,周涛,张飞飞,等.卷积神经网络及其在医学图像分析中的应用研究[J].生物医学工程学杂志,2018,35(6):977-985. LIANG Meng-meng,ZHOU Tao,ZHANG Fei-fei,et al.Research on convolutional neural network and its application on medical image[J].Journal of Biomedical Engineering,2018,35(6):977-985.(in Chinese)
[13] Lecun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].IEEE,1998,86(11):2278-2324.
[14] Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[A].International Conference on Neural Information Processing Systems[C].Lake Tahoe,USA:Curran Associates,2012.1097-1105.
[15] Szegedy C,Liu W,Jia Y Q,et al.Going deeper with convolutions[A].Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition[C].Boston,MA,USA:IEEE,2015.1-9.
[16] Ke Zhang,Miao Sun,Tony X,et al.Review:RoR-ResNet of ResNet/Multilevel ResNet (Image Classification)[J].arXiv Preprint,2017,arXiv:1608.02908.
[17] Zagoruyko S,Komodakis N.Wide residual networks[A].Proceedings of the 2016 British Machine Vision Conference[C].York,UK:BMVC,2016.1-12.
[18] Sasha Targ,Diogo Almeida,Kevin Lyman.Resnet in resnet:generalizing residual architectures[J].arXiv Preprint,2016,arXiv:1603.08029.
[19] Gao Huang,Yu Sun,Zhuang Liu,et al.Deep networks with stochastic depth[J].arXiv Preprint,2016,arXiv:1603.09382.
[20] Gao Huang,Zhuang Liu,Laurens van der Maaten,et al.Densely connected convolutional networks[J].arXiv Preprint,2017,arXiv:1608.06993.
[21] Yunpeng Chen,Jianan Li,Huaxin Xiao,et al.Dual path networks[J].arXiv Preprint,2017,arxiv:1707.01629.
[22] Szegedy C,Vanhoucke V,Ioffffe S,et al.Rethinking the inception architecture for computer vision[A].Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition[C].Las Vegas,NV,USA:IEEE,2016.2818-2826.
[23] Szegedy C,Ioffffe S,Vanhoucke V,et al.Inception-v4,Inception-ResNet and the impact of residual connections on learning[J].arXiv Preprint,2016,arXiv:1602.07261.
[24] Zoph B,Le Q V.Neural architecture search with reinforcement learning[J].arXiv Preprint,2016,arXiv:1611.01578.
[25] Barret Zoph,Vijay Vasudevan,Jonathon Shlens,et al.Learning transferable architectures for scalable image recognition[J].arXiv Preprint,2017,arXiv:1707.07012.
[26] Saining Xie,Ross Girshick,Piotr Dollár,et al.Aggregated residual transformations for deep neural networks[J].arXiv Preprint,2017,arXiv:1611.05431.
[27] Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[J].arXiv Preprint,2014,arXiv:1409.1556.
[28] Lin M,Chen Q,Yan S C.Network in network[J].arXiv Preprint,2014,arXiv:1312.4400.
[29] Dai J F,Qi H Z,Xiong Y W,et al.Deformable convolutional networks[J].arXiv Preprint,2017,arXiv:1703.06211.
[30] 彭亚丽,张鲁,张钰,等.基于深度反卷积神经网络的图像超分辨率算法[J].软件学报,2018,29(4):926-934. PENG Ya-li,ZHANG Lu,ZHANG Yu,et al.Deep deconvolution neural network for image super-resolution[J].Journal of Software,2018,29(4):926-934.(in Chinese)
[31] Hao Chen,Qi Dou,Lequan Yu,et al.VoxResNet:Deep voxel wise residual networks for brain segmentation from 3D MR images[J].Neuro Image,2018,170:446-455.
[32] He K M,Zhang X Y,Ren S Q,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
[33] Nal Kalchbrenner,Edward Grefenstette,Phil Blunsom.A convolutional neural network for modelling sentences[J].arXiv Preprint,2015,arXiv:1404.2188.
[34] Zhiqiang Tong,Gouhei Tanaka.Hybrid pooling for enhancement of generalization ability in deep convolutional neural networks[J].Neurocomputing,2019,333:76-85.
[35] 张振焕,周彩兰,梁媛.基于残差的优化卷积神经网络服装分类算法[J].计算机工程与科学,2018,40(2):355-360. ZHANG Zhen-huan,ZHOU Cai-lan,LIANG Yuan.An optimized clothing classification algorithm based on residual convolutional neural network[J].Computer Engineering & Science,2018,40(2):355-360.(in Chinese)
[36] 汪家明,卢涛.多尺度残差深度神经网络的卫星图像超分辨率算法[J].武汉工程大学学报,2018,40(04):439-443. WANG Jia-ming,LU Tao.Satellite imagery super-resolution algorithm via multi-scale residual deep neural network[J].Journal of Wuhan Institute of Technology,2018,40(04):439-443.(in Chinese)
[37] Yifeng Xu,Huigang Wang,Xing Liu,et al.An improved multi-branch residual network based on random multiplier and adaptive cosine learning rate method[J].Journal of Visual Communication and Image Representation,2019,59:363-370.
[38] Linlin Shen,Xi Jia,Yuexiang Li.Deep cross residual network for HEp-2 cell staining pattern classification[J].Pattern Recognition,2018,82:68-78.
[39] Alexandre Boulch.Reducing parameter number in residual networks by sharing weights[J].Pattern Recognition Letters,2018,103:53-59.
[40] 陈良甫,杨曾.一种基于残差网络的多任务模型[J].中国集成电路,2017,26(8):64-71. CHEN Liang-fu,YANG Zeng.A residual network based multi-task model[J].China Integrated Circult,2017,26(8):64-71.(in Chinese)
[41] 戴加明,佟继周.基于深度残差网络的星系形态分类[J].天文学进展,2018,36(20):385-397. DAI Jia-ming,TONG Ji-zhou.Galaxy morphology classification with deep convolutional neural networks[J].Progress in Astronomy,2018,36(20):385-397.(in Chinese)
[42] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[A].Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition[C].Boston,Massachusetts,USA:IEEE,2015.3431-3440.
[43] Sreela S R,Sumam Mary Idicula.Action recognition in still images using residual neural network features[J].Procedia Computer Science,2018,143:563-569.
[44] Dandan Zhu,Ye Luo,Lei Dai,et al.Salient object detection via a local and global method based on deep residual network[J].Journal of Visual Communication and Image Representation,2018,54:1-9.
[45] Haijun Lei,Tao Han,Feng Zhou,et al.A deeply supervised residual network for HEp-2 cell classification via cross-modal transfer learning[J].Pattern Recognition,2018,79:290-302.
[46] Themos Stafylakisa,Muhammad Haris Khanab,Georgios Tzimiropoulosa.Pushing the boundaries of audiovisual word recognition using residual networks and LSTMs[J].Computer Vision and Image Understanding,2018,176-177:22-32.
[47] Hoang Ngan Le T,Chi Nhan Duong,Ligong Han,et al.Deep contextual recurrent residual networks for scene labeling[J].Pattern Recognition,2018,80:32-41.
[48] Hyojoo Son,Hyunchul Choi,Hyeonwoo Seong,et al.Detection of construction workers under varying poses and changing background in image sequences via very deep residual networks[J].Automation in Construction,2019,99:27-38.
[49] Junbo Zhang,Yu Zheng,Dekang Qi,et al.Predicting citywide crowd flows using deep spatio-temporal residual networks[J].Artificial Intelligence,2018,259:147-166.
[50] 郝旭政,柴争义.一种改进的深度残差网络行人检测方法[J].计算机应用研,2019,36(6):http://www.arocmag.com/article/02-2019-06-057.html. HAO Xu-zheng,CHAI Zheng-yi.Improved pedestrian detection method based on depth residual network[J].Application Research of Computers,2019,36(6):http://www.arocmag.com/article/02-2019-06-057.html.(in Chinese)
[51] 郑光远,刘峡壁,韩光辉.医学影像计算机辅助检测与诊断系统综述[J].软件学报,2018,29(5):1471-1514. ZHENG Guang-yuan,LIU Xia-bi,HAN Guang-hui.Survey on medical image computer aided detection and diagnosis systems[J].Journal of Software,2018,29(5):1471-1514.(in Chinese)
[52] 刘振丙,方旭升,杨辉华,等.基于多尺度残差神经网络的阿尔茨海默病诊断分类[J].山东大学学报(工学版),2018,48(06):1-7+18. LIU Zhen-bing,FANG Xu-sheng,YANG Hui-hua,et al.The diagnosis of Alzheimer's disease classification based on multi-scale residual neutral network[J].Journal of Shandong University (Engineering Science),2018,48(06):1-7+18.(in Chinese)
[53] 应自炉,龙祥.多尺度密集残差网络的单幅图像超分辨率重建[J].中国图象图形学报,2019,24(3):0410-0419. YING Zi-lu,LONG Xiang.Single-image super-resolution construction based on multi-scale dense residual network[J].Journal of Image and Graphics,2019,24(3):0410-0419.(in Chinese)
[54] Songtao Guo,Zhouwang Yang.Multi-Channel-ResNet:An integration framework towards skin lesion analysis[J].Informatics in Medicine Unlocked,2018,12:67-74.
[55] Ruikai Zhang,Yali Zheng,Carmen C,et al.Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker[J].Pattern Recognition,2018,83:209-219.
[56] Ziba Gandomkar,Patrick C.Brennan,et al.MuDeRN:Multi-category classification of breast histopathological image using deep residual networks[J].Artificial Intelligence in Medicine,2018,88:14-24.
[57] Feng Sha,Seid Miad Zandavi,Yuk Ying Chung.Fast deep parallel residual network for accurate super resolution image processing[J].Expert Systems with Applications,2019,128:157-168.
[58] 姚宇瑾,张利.基于混合损失联合调优与多尺度分类相结合的肺结节检测算法[J].计算机应用研究,2019,36(9):http://www.arocmag.com/article/02-2019-09-020.html. YAO Yu-jin,ZHANG Li.Pulmonary nodule detection via hybrid loss based joint fine-tuning and multi-scale classification[J].Application Research of Computers,2019,36(9):http://www.arocmag.com/article/02-2019-09-020.html.(in Chinese)
[59] 胡海根,孔祥勇,周乾伟,等.基于深层卷积残差网络集成的黑色素瘤分类方法[J].计算机科学,2019,46(5):248-253. HU Hai-gen,KONG Xiang-yong,ZHOU Qian-wei,et al.Melanoma classification method by integrating deep convolutional residual network[J].Computer Science,2019,46(5):248-253.(in Chinese)
[60] 戴垚均,宋成利,闫士举.基于密集网络改进的肺结节良恶性分类模型[J].中国医学影像技术,2018,34(7):1104-1109. DAI Yao-jun,SONG Cheng-li,YAN Shi-ju.Benign or malignant lung nodules classification model based on modified DenseNet[J].China J Med Imaging Technol,2018,34(7):1104-1109.(in Chinese)
[61] 李航,余镇,倪东,等.基于深度残差网络的皮肤镜图像黑色素瘤的识别[J].中国生物医学工程学报,2018,37(3):275-282. LI Hang,YU Zhen,NI Dong,et al.Melanoma recognition indermoscopy images via deep residual network[J].Chinese Journal of Biomedical Engineering,2018,37(3):274-282.(in Chinese)
[62] 罗希平,田捷.一种改进的交互式医学图像序列分割方法[J].电子学报,2003,31(1):30-32. LUO Xi-ping,TIAN Jie.A modified interactive segmentation of medical image series[J].Acta Electronica Sinica,2003,31(1):30-32.(in Chinese)
[63] Mahendra Khened,Varghese Alex Kollerathu,Ganapathy Krishnamurthi.Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers[J].Medical Image Analysis,2019,51:21-45.
[64] Rui Zhang,Lin Huang,Wei Xia,et al.Multiple supervised residual network for osteosarcoma segmentation in CT images[J].Computerized Medical Imaging and Graphics,2018,63:1-8.
[65] Juan Mo,Lei Zhang,Yangqin Feng.Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks[J].Neurocomputing,2018,290:161-171.
[66] Guofeng Tong,Yong Li,Huairong Chen,et al.Improved U-NET network for pulmonary nodules segmentation[J].Optik,2018,174:460-469.
[67] Md Zahangir Alom,Mahmudul Hasan,Chris Yakopcic,et al.Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation[J].arXiv Preprint,2018,arXiv:1082.06955.
[68] 徐光柱,胡松,陈莎,等.U-Net与DenseNet相结合的视网膜血管提取[J].中国图象图形学报,2019,24(9):1569-1580. XU Guang-zhu,HU Song,CHEN Sha,et al.Retinal blood vessel extraction by combining U-net and Dense-net[J].Jourmal of Image and Graphics,2019,24(9):1569-1580.(in Chinese)
[69] 周沛,陈后金,于泽宽,等.跨模态医学图像预测综述[J].电子学报,2019,47(1):221-226. ZHOU Pei,CHEN Hou-Jin,YU Ze-kuan,et al.Review of cross-modality medical image prediction[J].Acta Electronica Sinica,2019,47(1):220-226.(in Chinese)

残差神经网络及其在医学图像处理中的应用研究

Research on Residual Neural Network and Its Application on Medical Image Processing

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	刘少鹏, 赵慧民, 洪佳明, 吴晓航, 许发宝, 欧阳佳, 梁鹏, 熊建斌. 面向医学图像生成的鲁棒条件生成对抗网络[J]. 电子学报, 2023, 51(2): 427-437.
[2]	张淑军, 彭中, 李辉. SAU-Net：基于U-Net和自注意力机制的医学图像分割方法[J]. 电子学报, 2022, 50(10): 2433-2442.
[3]	李居朋, 王颖慧, 李刚. 医学图像关键点检测深度学习方法研究与挑战[J]. 电子学报, 2022, 50(1): 226-237.
[4]	谭翔元, 高晓光, 贺楚超. 基于马尔科夫毯约束的最优贝叶斯网络结构学习算法[J]. 电子学报, 2019, 47(9): 1898-1904.
[5]	潘剑飞, 董一鸿, 陈华辉, 钱江波, 戴明洋. 基于结构紧密性的重叠社区发现算法[J]. 电子学报, 2019, 47(1): 145-152.
[6]	袁绍锋, 杨丰, 刘树杰, 季飞, 黄靖. 基于局部形状结构分类的心血管内超声图像中-外膜边界检测[J]. 电子学报, 2018, 46(7): 1601-1608.
[7]	葛婷, 牟宁, 李黎. 基于softmax回归与图割法的脑肿瘤分割算法[J]. 电子学报, 2017, 45(3): 644-649.
[8]	戴文战, 姜晓丽, 李俊峰. 基于人眼视觉特性的NSCT医学图像自适应融合[J]. 电子学报, 2016, 44(8): 1932-1939.
[9]	汪友生. 血管壁小应变数学模型研究[J]. 电子学报, 2014, 42(10): 2086-2091.
[10]	蔺想红, 李志强, 魏伟一. 进化神经网络间接编码方法的计算框架研究[J]. 电子学报, 2013, 41(5): 852-858.
[11]	唐雨, 何明升, 王英赫. 基于用户偏好的嵌入性网络结构分析[J]. 电子学报, 2012, 40(7): 1339-1344.
[12]	向涛;余晨韵;屈晋宇;罗小松. 基于改进AES加密算法的DICOM医学图像安全性研究[J]. 电子学报, 2012, 40(2): 406-411.
[13]	刘燕杰;卢振泰;冯前进;陈武凡. 基于KL距离加权和局部邻域信息的CV模型[J]. 电子学报, 2011, 39(6): 1447-1451.
[14]	徐艳蕾;赵继印;李敏;赵婷婷. 基于顺序形态学的医学图像插值算法的研究[J]. 电子学报, 2010, 38(5): 1002-1007.
[15]	彭景林, 章兢, 李树涛. 基于改进PV插值和混合优化算法的医学图像配准[J]. 电子学报, 2006, 34(5): 962-965.