一种基于改进的MobileNetV2网络语义分割算法

doi:10.3969/j.issn.0372-2112.2020.09.015

电子学报 ›› 2020, Vol. 48 ›› Issue (9): 1769-1776.DOI: 10.3969/j.issn.0372-2112.2020.09.015

一种基于改进的MobileNetV2网络语义分割算法

孟琭¹, 徐磊¹, 郭嘉阳²

1. 东北大学信息科学与工程学院, 辽宁沈阳 110000;
2. 辛辛那提大学电气工程与计算机系, 俄亥俄州辛辛那提 45221

收稿日期:2019-09-10 修回日期:2019-12-27 出版日期:2020-09-25
- 作者简介:
- 孟琭男,1982年2月出生于辽宁沈阳.现为东北大学信息科学与工程学院副教授.主要研究方向为人工智能、图像处理.E-mail:menglu@ise.neu.edu.cn
  徐磊男,1997年11月出生于辽宁大连.研究生.主要研究方向为计算机视觉.E-mail:19s001019@stu.hit.edu.cn
  郭嘉阳 男,1985年生于福建厦门.现为美国辛辛那提大学电气工程与计算机系博士后,IEEE会员.主要研究方向为人工智能,机器学习,医学图像处理,信号处理等.E-mail:guojy@mail.uc.edu
- 基金资助:
- 国家自然科学基金 (No.61973058）; 教育部中央高校基本科研基金 (No.N2004020）

Semantic Segmentation Algorithm Based on Improved MobileNetV2

MENG Lu¹, XU Lei¹, GUO Jia-yang²

1. College of Information Science and Engineering, Northeastern University, Shenyang, Liaoning 110000, China;
2. Department of Electrical Engineering and Computer Science, University of Cincinnati, Cincinnati, Ohio 45221, USA

Received:2019-09-10 Revised:2019-12-27 Online:2020-09-25 Published:2020-09-25

摘要/Abstract

摘要： 基于金字塔卷积神经网络的语义分割算法准确率很高，但是其计算资源消耗巨大、算法执行时间长、无法满足实时性要求.为了解决这个问题，本文做出了以下改进：（1）用MobileNet替换原网络的结构，减少了网络运算时间和内存开销；（2）引入编码器-解码器结构提高输出图像的分辨率，进一步细化分割结果；（3）针对高分辨率图像推断时间过长的问题，本文设计了多级图像输入方法，降低了网络推断高分辨率图像所消耗的时间.本文在VOC 2012数据集和Cityscapes数据集上进行了测试，并与FCN、SegNet、DeepLab、PSPNet以及DFN等语义分割模型对比.实验结果表明，本文设计的语义分割算法在VOC 2012数据集上达到了76.1%的mIoU，在Cityscapes数据集上达到了74.1%的mIoU，略低于传统语义分割算法；处理一张分辨率为1024×512的图片需要18ms，少于传统语义分割算法，满足了实时性要求，达到了准确率与计算资源消耗之间的平衡.

关键词: 语义分割, 卷积神经网络, 金字塔网络, 快速语义分割, MobileNet, 编码器-解码器

Abstract: The algorithm of semantic segmentation based on pyramid convolution neural network has high accuracy,but it consumes a lot of computing resources,takes a long time to execute,and cannot meet the real-time requirements.To overcome these shortcomings,this paper made the following improvements:(1) replacing the original network structure with MobileNet in order to reduce the computation time and memory consumption;(2) using encoder-decoder structure to improve the resolution of the output image and further refine the segmentation results;(3) using a multi-level image input method,which can reduce the time consumed by network inference of high-resolution image.Our method was tested on the VOC 2012 dataset and the Cityscapes dataset compared with other state-of-the-art semantic segmentation models such as FCN(Fully Convolutional Networks),SegNet,DeepLab,PSPNet and DFN(Discriminative Feature Network).Experimental results showed that our method achieved mIoU of 76.1% on the VOC 2012 dataset,and achieved mIOU of 74.1% on the Cityscapes dataset,which was a little lower than the traditional semantic segmentation algorithms.It took 18ms for our method to predict a 1024×512 picture,which achieved a balance between accuracy and computational resource consumption.

Key words: semantic segmentation, convolution neural network, pyramid network, fast semantic segmentation, MobileNet, encoder-decoder

中图分类号:

TP391

孟琭, 徐磊, 郭嘉阳. 一种基于改进的MobileNetV2网络语义分割算法[J]. 电子学报, 2020, 48(9): 1769-1776.

MENG Lu, XU Lei, GUO Jia-yang. Semantic Segmentation Algorithm Based on Improved MobileNetV2[J]. Acta Electronica Sinica, 2020, 48(9): 1769-1776.

参考文献

[1] 陈鸿翔.基于卷积神经网络的图像语义分割[D].杭州:浙江大学,2016.
[2] Thoma M.A survey of semantic segmentation[J].IEEE Access,2016,5:1-11.
[3] 陈天华,郑司群,于峻川.采用改进DeepLab网络的遥感图像分割[J].测控技术,2018,37(11):40-45. CHEN Tianhua,ZHEN Siqun,YU Junchuan.Remote sensing image segmentation based on improved deeplab network[J].Measurement and Control Technology,2018,37(11):40-45.(in Chinese)
[4] Siam M,Elkerdawy S,Jagersand M,et al.Deep semantic segmentation for automated driving:Taxonomy,roadmap and challenges[J].IEEE Access,2017,1:1-8.
[5] 李琳辉,钱波,连静,等.基于卷积神经网络的交通场景语义分割方法研究[J].通信学报,2018,39(4):123-130. LI Linhui,QIAN Bo,LIAN Jing,et al.Study on traffic scene semantic segmentation method based on convolutional neural network[J].Journal of Communications,2018,39(4):123-130.(in Chinese)
[6] Zeng Z,Xie W,Zhang Y,et al.RIC-Unet:Animproved neural network based on Unet for nuclei segmentation in histology images[J].IEEE Access,2019,7:21420-21428.
[7] 林志伟,涂伟豪,黄嘉航,等.深度语义分割的无人机图像植被识别[J].山地学报,2018,36(6):953-963.
[8] Malmberg F,Lindblad J,Sladoje N,Nystrom I.A graph-based framework for sub-pixel image segmentation[J].Theoretical Computer Science,2011,412(15):1338-1349.
[9] Mahipal S C,Rajiv K.Performance Analysis of fuzzy C-means clustering methods for MRI image segmentation[J].Procedia Computer Science,2016,89:749-758.
[10] Geman S.Stochastic relaxation,Gibbs distribution,and Bayesian restoration of images[J].IEEE Trans Pattern Anal Mach Intell,1984,6(6):721-741.
[11] Caesar H,Uijlings J,Ferrari V.Region-based semantic segmentation with end-to-end training[J].IEEE Access,2016,4:1-13.
[12] Long J,Shelhamer E,Darrell T.Fullyconvolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2014,39(4):640-651.
[13] Simonyan K,Zisserman A.VeryDeep Convolutional Networks for Large-scale Image Recognition[EB/OL].https://arxiv.org/abs/1409.1556,2014.
[14] Szegedy C,Liu W,Jia Y,et al.Going Deeper With Convolutions[EB/OL].https://arxiv.org/abs/1409.4842,2014.
[15] Yu F,Koltun V.Multi-scale Context Aggregation by Dilated Convolutions[EB/OL].https://arxiv.org/abs/1511.07122,2014.
[16] Ronneberger O,Fischer P,Brox T.U-Net:convolutional networks for biomedical image segmentation[J].IEEE Access,2015,11:1-10.
[17] He K,Zhang X,Ren S,et al.Deepresidual learning for image recognition[J].IEEE Access,2015,9:1-16.
[18] Chen L C,Papandreou G,Kokkinos I,et al.DeepLab:semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2016,40(4):834-848.
[19] Chen L C,Papandreou G,Schroff F,et al.Rethinking Atrous Convolution for Semantic Image Segmentation[EB/OL].https://arxiv.org/abs/1706.05587,2017.
[20] Zhao H,Shi J,Qi X,et al.Pyramidscene parsing network[J].IEEE Access,2016,9:1-13.
[21] Hu J,Shen L,Albanie S,et al.Squeeze-and-excitation Networks[EB/OL].https://arxiv.org/abs/1709.01507,2019.
[22] Yu C,Wang J,Peng C,et al.Learning adiscriminative feature network for semantic segmentation[J].IEEE Access,2018,3:1-11.
[23] Badrinarayanan V,Kendall A,Cipolla R.SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Access,2015,7:1-18.
[24] He K,Zhang X,Ren S,et al.Identitymappings in deep residual networks[J].IEEE Access,2016,11:1-9.
[25] Huang G,Sun Y,Liu Z,et al.Deepnetworks with stochastic depth[J].IEEE Access,2016,4:1-8.
[26] Howard A G,Zhu M,Chen B,et al.MobileNets:efficient convolutional neural networks for mobile vision applications[J].IEEE Access,2017,6:1-14.
[27] Sandler M,Howard A,Zhu M,et al.Mobile NetV2:inverted residuals and linear bottlenecks[J].IEEE Access,2018,5:1-12.
[28] Zhao H,Qi X,Shen X,et al.ICNet forreal-time semantic segmentation on high-resolution images[J].IEEE Access,2017,3:1-21.
[29] Abadi,Martín,Agarwal,Ashish,Barham,Paul,Brevdo,et al.TensorFlow:large-scale machine learning on heterogeneous distributed systems[J].IEEE Access,2016,7:1-46.
[30] Hanchao Li,Pengfei Xiong,Haoqing Fan,Jian Sun.DFANet:Deep Feature Aggregation for Real-time Semantic Segmentation[EB/OL].https://arxiv.org/abs/1904.02216,2019.
[31] Hengshuang Zhao,XiaoJuan Qi,Xiaoyong Shen,Jianping Shi,Jiaya Jia.ICNet forreal-time semantic segmentation on high-resolution images[A].15th European Conference on Computer Vision[C].New York:IEEE,2018.418-434.
[32] Yu Wang,Quan Zhou,Xiaofu Wu.ESNet:An Efficient Symmetric Network for Real-time Semantic Segmentation[EB/OL].https://arxiv.org/abs/1906.09826,2019.

一种基于改进的MobileNetV2网络语义分割算法

Semantic Segmentation Algorithm Based on Improved MobileNetV2

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[2]	毛国君, 王者浩, 黄山, 王翔. 基于剪边策略的图残差卷积深层网络模型[J]. 电子学报, 2022, 50(9): 2205-2214.
[3]	袁海英, 曾智勇, 成君鹏. 面向灵活并行度的稀疏卷积神经网络加速器[J]. 电子学报, 2022, 50(8): 1811-1818.
[4]	王相海, 赵晓阳, 王鑫莹, 赵克云, 宋传鸣. 非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建[J]. 电子学报, 2022, 50(7): 1753-1765.
[5]	张志文, 刘天歌, 聂鹏举. 基于实景数据增强和双路径融合网络的实时街景语义分割算法[J]. 电子学报, 2022, 50(7): 1609-1620.
[6]	张波, 陆云杰, 秦东明, 邹国建. 一种卷积自编码深度学习的空气污染多站点联合预测模型[J]. 电子学报, 2022, 50(6): 1410-1427.
[7]	丁毅, 沈薇, 李海生, 钟琼慧, 田明宇, 李洁. 面向CNN的区块链可信隐私服务计算模型[J]. 电子学报, 2022, 50(6): 1399-1409.
[8]	张云, 化青龙, 姜义成, 徐丹. 基于混合型复数域卷积神经网络的三维转动舰船目标识别[J]. 电子学报, 2022, 50(5): 1042-1049.
[9]	廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217.
[10]	卓亚琦, 魏家辉, 李志欣. 基于双注意模型的图像描述生成方法研究[J]. 电子学报, 2022, 50(5): 1123-1130.
[11]	崔亚奇, 何友, 唐田田, 熊伟. 一种深度学习航迹关联方法[J]. 电子学报, 2022, 50(3): 759-763.
[12]	伍邦谷, 张苏林, 石红, 朱鹏飞, 王旗龙, 胡清华. 基于多分支结构的不确定性局部通道注意力机制[J]. 电子学报, 2022, 50(2): 374-382.
[13]	陈文俊, 杨春玲. 图像压缩感知的特征域优化及自注意力增强神经网络重构算法[J]. 电子学报, 2022, 50(11): 2629-2637.
[14]	魏博文, 全红艳. 基于语义与形态特征融合的语义分割网络[J]. 电子学报, 2022, 50(11): 2688-2697.
[15]	肖斌, 陈嘉博, 毕秀丽, 张俊辉, 李伟生, 王国胤, 马旭. 基于一维卷积神经网络与循环神经网络串联的心音分析方法[J]. 电子学报, 2022, 50(10): 2425-2432.