基于深度学习的图像实例分割技术研究进展

doi:10.3969/j.issn.0372-2112.2020.12.025

摘要/Abstract

摘要： 随着深度学习算法在图像分割领域的成功应用，在图像实例分割方向上涌现出一大批优秀的算法架构.这些架构在分割效果、运行速度等方面都超越了传统方法.本文围绕图像实例分割技术的最新研究进展，对现阶段经典网络架构和前沿网络架构进行梳理总结，结合常用数据集和权威评价指标对各个架构的分割效果进行比较和分析.最后，对目前图像实例分割技术面临的挑战以及可能的发展趋势进行了展望.

关键词: 深度学习, 图像分割, 实例分割

Abstract: With the successful application of deep learning algorithms in the field of image segmentation,a large number of excellent algorithm architectures have emerged in the direction of image instance segmentation.These architectures surpass the traditional methods in terms of segmentation effects and running speed.This paper focuses on the latest research progress of image instance segmentation technology,summarizes the current classic network architecture and cutting-edge network architecture,and uses common datasets and authoritative evaluation indicators to compare and analyze the segmentation effects of each architecture.Finally,the challenges and possible development trends of image instance segmentation technology are prospected.

Key words: deep learning, image segmentation, instance segmentation

中图分类号:

TP183

梁新宇, 林洗坤, 权冀川, 肖铠鸿. 基于深度学习的图像实例分割技术研究进展[J]. 电子学报, 2020, 48(12): 2476-2486.

LIANG Xin-yu, LIN Xi-kun, QUAN Ji-chuan, XIAO Kai-hong. Research on the Progress of Image Instance Segmentation Based on Deep Learning[J]. Acta Electronica Sinica, 2020, 48(12): 2476-2486.

参考文献

[1] 郑冰.面向肺部CT影像表征的多层语义检索[D].哈尔滨:哈尔滨工程大学,2013.
[2] 南洋.基于深度学习的粗标记胃癌病理切片图像分割算法[D].长沙:湖南大学,2018.
[3] 杨少戈.基于深度学习的冠脉造影图像分割[D].北京:北京邮电大学,2019.
[4] 杨远航.面向深度学习的医学影像分析系统及其在胃镜视频分割中的实践[D].杭州:浙江大学,2018.
[5] 赵旭.基于医学先验的多尺度乳腺超声肿瘤实例分割方法[D].哈尔滨:哈尔滨工业大学,2019.
[6] 张晓林.基于卷积神经网络的腹部CT图像分割[D].镇江:江苏大学,2019.
[7] 宫进昌,赵尚义,王远军.基于深度学习的医学图像分割研究进展[J].中国医学物理学杂志,2019,36(04):420-424. GONG Jin-chang,ZHAO Shang-yi,WANG Yuan-jun.Research progress of medical image segmentation based on deep learning[J].Chinese Journal of Medical Physics,2019,36(04):420-424.(in Chinese)
[8] 白辰甲.基于计算机视觉和深度学习的自动驾驶方法研究[D].哈尔滨:哈尔滨工业大学,2017.
[9] 邓琉元,杨明,王春香,等.基于环视相机的无人驾驶汽车实例分割方法[J].华中科技大学学报(自然科学版),2018,46(12):24-29. DENG Liu-yuan,YANG Ming,WANG Chun-xiang,et al.Method for segmentation of unmanned car instance based on surround view camera[J].Journal of Huazhong University of Science & Technology (Natural Science Edition),2018,46(12):24-29.(in Chinese)
[10] 姜立标,台啟龙.基于实例分割方法的复杂场景下车道线检测[J].机械设计与制造工程,2019,48(05):113-118. JIANG Li-biao,TAI Qi-long.Lane line detection in complex scenes based on instance segmentation method[J].Mechanical Design and Manufacturing Engineering,2019,48(05):113-118.(in Chinese)
[11] 惠健,秦其明,许伟,等.基于多任务学习的高分辨率遥感影像建筑实例分割[J].北京大学学报(自然科学版),2019,55(06):1067-1077. HUI Jian,QIN Qi-ming,XU Wei,et al.Segmentation of high-resolution remote sensing image building instance based on multi-task learning[J].Journal of Peking University (Natural Science Edition),2019,55(06):1067-1077.(in Chinese)
[12] 李澜.基于Mask R-CNN的高分辨率光学遥感影像的目标检测与实例分割[D].武汉:武汉大学,2018.
[13] 帅靖文.自然场景中的文本检测研究[D].成都:电子科技大学,2018.
[14] 张小爽.基于实例分割的场景图像文字检测[D].杭州:浙江大学,2018.
[15] 邓丹.PixelLink:基于实例分割的自然场景文本检测算法[D].杭州:浙江大学,2018.
[16] 谢元澄,于增源,姜海燕,等.小麦麦穗几何表型测量的精准分割方法研究[J].南京农业大学学报,2019,42(05):956-966. XIE Yuan-cheng,YU Zeng-yuan,JIANG Hai-yan,et al.Research on accurate segmentation method of wheat ear geometric phenotype measurement[J].Journal of Nanjing Agricultural University,2019,42(05):956-966.(in Chinese)
[17] 乔虹,冯全,赵兵,等.基于Mask R-CNN的葡萄叶片实例分割[J].林业机械与木工设备,2019,47(10):15-22. QIAO Hong,FENG Quan,ZHAO Bing,et al.Grape leaf instance segmentation based on Mask R-CNN[J].Forestry Machinery and Woodworking Equipment,2019,47(10):15-22.(in Chinese)
[18] 冈萨雷斯.数字图像处理[M].北京:电子工业出版社,2011. Gonzal R.Digital Image Processing[M].Beijing:Publishing House of Electronics Industry,2011.(in Chinese)
[19] 苏雯.语义分割及其在图像检索中的应用[D].合肥:中国科学技术大学,2018.
[20] Hinton G E,Salakhutdinov R R.Reducing the dimensionality of data with neural networks[J].Science,2006,313(5786):504-507.
[21] Hariharan B,Arbeláez P,Girshick R,et al.Simultaneous detection and segmentation[A].European Conference on Computer Vision[C].Cham:Springer,2014.297-312.
[22] Arbeláez P,Pont-Tusent J,Barron J T,et al.Multiscale combinatorial grouping[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].USA:IEEE,2014.328-335.
[23] LeCun Y,Bottou L,Bengio Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[24] Duda R O,Hart P E,Stork D G.Pattern Classification[M].USA:John Wiley & Sons,2012.
[25] Neubeck A,Van Gool L.Efficient non-maximum suppression[A].18th International Conference on Pattern Recognition (ICPR'06)[C].Hong Kong:IEEE,2006.850-855.
[26] Pinheiro P O,Collobert R,Dollár P.Learning to segment object candidates[A].Advances in Neural Information Processing Systems[C].Canada:NIPS,2015.1990-1998.
[27] Pinheiro P O,Lin T Y,Collobert R,et al.Learning to refine object segments[A].European Conference on Computer Vision[C].Cham:Springer,2016.75-91.
[28] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Boston:IEEE,2015.3431-3440.
[29] Dai J,He K,Li Y,et al.Instance-sensitive fully convolutional networks[A].European Conference on Computer Vision[C].Cham:Springer,2016.534-549.
[30] Li Y,Qi H,Dai J,et al.Fully convolutional instance-aware semantic segmentation[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Honolulu:IEEE,2017.2359-2367.
[31] Ren S,He K,Girshick R,et al.Faster R-CNN:Towardsreal-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1149.
[32] Girshick R.Fast r-cnn[A].Proceedings of the IEEE International Conference on Computer Vision[C].Santiago:IEEE,2015.1440-1448.
[33] He K,Gkioxari G,Dollár P,et al.Mask r-cnn[A].Proceedings of the IEEE International Conference on Computer Vision[C].Venice Italy:IEEE,2017.2961-2969.
[34] Liu S,Qi L,Qin H,et al.Path aggregation network for instance segmentation[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Salt Lake City:IEEE,2018.8759-8768.
[35] Chen X,Girshick R,He K,et al.Tensormask:A foundation for dense object segmentation[A].Proceedings of the IEEE International Conference on Computer Vision[C].Seoul:IEEE,2019.2061-2069.
[36] Lazarow J,Lee K,Tu Z.Learning instance occlusion for panoptic segmentation[A].Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition[C].Seattle,WA:IEEE,2020.10720-10729.
[37] 张国光.基于神经网络的有遮挡图像分割方法[J].电子科技,2015,28(05):132-135,139. ZHANG Guo-Guang.An occluded image segmentation method based on neural network[J].Electronic Technology,2015,28(05):132-135,139.(in Chinese)
[38] 师晓利,尚怡君,褚玉晓.安防监控中人员遮挡区域的有效图像分割研究[J].计算机仿真,2015,32(06):452-455. SHI Xiao-li,SHANG Yi-jun,CHU Yu-xiao.Research on effective image segmentation of people's occlusion area in security surveillance[J].Computer Simulation,2015,32(06):452-455.(in Chinese)
[39] 刘伟.基于Kinect遮挡条件下行人的深度图像分割[J].重庆邮电大学学报(自然科学版),2014,26(02):271-275. LIU Wei.Depth image segmentation of pedestrians based on Kinect occlusion conditions[J].Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition),2014,26(02):271-275.(in Chinese)
[40] 李晶晶.局部遮挡物体的轮廓修复算法研究[D].南昌:南昌航空大学,2014.
[41] Everingham M,Eslami S A,Van G L,et al.The pascal visual object classes challenge:A retrospective[J].International Journal on Computer Vision,2014,11(1):98-136.
[42] Silberman N,Hoiem D,Kohli P,et al.Indoor segmentation and support inference from rgbd images[A].European Conference on Computer Vision[C].Berlin,Heidelberg:Springer,2012.746-760.
[43] Chen X,Mottaghi R,Liu X,et al.Detect what you can:Detecting and representing objects using holistic models and body parts[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Columbus,OH:IEEE,2014.1971-1978.
[44] Lin T Y,Maire M,Belongie S,et al.Microsoft coco:Common objects in context[A].European Conference on Computer Vision[C].Cham:Springer,2014.740-755.
[45] CORDTS M,OMRAN M,RAMOS S,et al.The cityscapes dataset for semantic urban scene understanding[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Las Vegas,NV:IEEE,2016.3213-3223.
[46] Zhou B,Zhao H,Puig X,et al.Scene parsing through ade20k dataset[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Honolulu:IEEE,2017.633-641.
[47] Zhang Y,Chen H,He Y,et al.Road segmentation for all-day outdoor robot navigation[J].Neurocomputing,2018,314:316-325.
[48] Turpin A,Sholer F.User performance versus precision measures for simple search tasks[A].Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval[C].Seattle Washington:ACM,2006.11-18.
[49] 诸葛瑞彬.基于卷积神经网络的单幅图像去雨研究[D].桂林:广西师范大学,2019.
[50] Fu X,Liang B,Huang Y,et al.Lightweight pyramid networks for image deraining[J].IEEE Transactions on Neural Networks and Learning Systems,2020,31(6):1794-1807.
[51] 安鹤男,涂志伟,张昌林,等.单张图像去雨的多流细节加强网络[J].计算机系统应用,2019,28(11):202-207. AN He-nan,TU Zhi-wei,ZHANG Chang-lin,et al.Multi-stream detail enhancement network to remove rain from a single image[J].Computer System Applications,2019,28(11):202-207.(in Chinese)
[52] 寇大磊,钱敏,权冀川,等.基于多尺度卷积网络的快速图像去雾算法[J].计算机工程与应用,2020,56(20):191-198. KOU Da-lei,QIAN Min,QUAN Ji-chuan,et al.Fast image defogging algorithm based on multi-scale convolutional network[J].Computer Engineering and Applications,2020,56(20):191-198.(in Chinese)
[53] Gao Y,Li Q,Li J.Single image dehazing via a dual-fusion method[J].Image and Vision Computing,2020,94:103868.
[54] Wu Y,Qin Y,Wang Z,et al.Densely pyramidal residual network for UAV-based railway images dehazing[J].Neurocomputing,2020,371:124-136.
[55] 赵阳,王剑,曹浩男.基于自适应改进的遥感图像去雾算法研究[J].电子设计工程,2019,27(19):164-169. ZHAO Yang,WANG Jian,CAO Hao-nan.Research on remote sensing image defogging algorithm based on adaptive improvement[J].Electronic Design Engineering,2019,27(19):164-169.(in Chinese)
[56] 丁春玲.基于CNN网络的图像去雾霾技术研究[J].西安文理学院学报(自然科学版),2019,22(05):57-60. DING Chun-ling.Research on image removal technology based on CNN network[J].Journal of Xi'an University of Arts and Science (Natural Science Edition),2019,22(05):57-60.(in Chinese)
[57] 许骏.面向火灾场景的图像去烟雾系统研究[D].上海:东华大学,2016.
[58] 寇大磊,权冀川,张仲伟.基于深度学习的目标检测框架进展研究[J].计算机工程与应用,2019,55(11):25-34. KOU Da-lei,QUAN Ji-chuan,ZHANG Zhong-wei.Research on the progress of target detection framework based on deep learning[J].Computer Engineering and Applications,2019,55(11):25-34.(in Chinese)
[59] Liu C,Chen L C,Schroff F,et al.Auto-deeplab:Hierarchical neural architecture search for semantic image segmentation[A].Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition[C].Long Beach:IEEE,2019.82-92.