基于全卷积神经网络的非对称并行语义分割模型

doi:10.3969/j.issn.0372-2112.2019.05.012

电子学报 ›› 2019, Vol. 47 ›› Issue (5): 1058-1064.DOI: 10.3969/j.issn.0372-2112.2019.05.012

所属专题：优秀论文(2022)

基于全卷积神经网络的非对称并行语义分割模型

李宝奇, 贺昱曜, 何灵蛟, 强伟

西北工业大学航海学院, 陕西西安 710072

收稿日期:2018-06-04 修回日期:2019-01-17 出版日期:2019-05-25
- 通讯作者:
- 李宝奇
- 作者简介:
- 贺昱曜 男,1956年生,陕西富平人.教授,西北工业大学博士生导师,主要研究方向:精确制导预仿真,智能控制与智能优化理论,图像处理理论与算法.E-mail:heyyao@nwpu.edu.cn;何灵蛟 男,1994年12月生,甘肃会宁人.现于西北工业大学航海学院攻读硕士学位,研究方向为图像增强、图像语义分割及目标检测与识别.E-mail:helingjiao@mail.nwpu.edu.cn;强伟男,1986年12月生,陕西延安人.现于西北工业大学航海学院攻读硕士学位,研究方向为图像分类、图像语义分割及目标检测与识别等深度学习理论.E-mail:xgd2017qw@mail.nwpu.edu.cn
- 基金资助:
- 国家自然科学基金 (No.61271143）

Asymmetric Parallel Semantic Segmentation Model Based on Full Convolutional Neural Network

LI Bao-qi, HE Yu-yao, HE Ling-jiao, QIANG Wei

School of Marine Science and Technology, Northwestern Polytechnical University, Xi'an, Shaanxi 710072, China

Received:2018-06-04 Revised:2019-01-17 Online:2019-05-25 Published:2019-05-25

摘要/Abstract

摘要： 针对RGB图像具有丰富的色彩细节特征，红外图像对目标轮廓、尺寸、边界等外形特征有较高敏感度的特点，提出了一种非对称并行语义分割模型APFCN（Asymmetric Parallelism Fully Convolutional Networks）.APFCN上路设计了一个卷积核尺寸非统一的五层空洞卷积网络来提取红外图像目标高层轮廓特征；下路沿用卷积加池化网络提取RGB图像三个尺度上的细节特征；后端将红外图像高层特征与RGB图像三个尺度的细节特征进行融合，并将4倍上采样后的融合特征作为语义分割输出.结果表明，APFCN在像素精度和交并比等方面均优于FCN（输入为RGB图像或红外图像），适用于背景一致下地面目标的语义分割任务.

关键词: 语义分割, 全卷积神经网络, 非对称并行全卷积神经网络, 空洞卷积, 空洞率

Abstract: Aiming at that RGB image is rich in color details of scene and infrared image is sensitive to outline、size and boundary of target,a novel semantic segmentation model APFCN (Asymmetric Parallelism Fully Convolutional Networks) is proposed.In the upper part of APFCN,a five layer dilation convolution network,where the five kernel sizes are not uniform,is designed used to extract the high-level targets contour features of infrared image.In the lower part of APFCN,a classical CNN network is used to extract three scale features of RGB images.At the back of APFCN,the high level features of the infrared image are fused with the three scale features of the RGB image,and the fused features after 4 times upper sampling is used as the semantic segmentation output of APFCN.The results show that APFCN is better than FCN (input RGB image or infrared image) in PA (Pixel Accuracy) and MIoU (Mean Intersection over Union).APFCN is suitable for the semantic segmentation task of ground targets with consistent background colors.

Key words: semantic segmentation, fully convolution neural network, asymmetric parallelism fully convolutional networks, dilation convolution, dilation rate

中图分类号:

TP183

李宝奇, 贺昱曜, 何灵蛟, 强伟. 基于全卷积神经网络的非对称并行语义分割模型[J]. 电子学报, 2019, 47(5): 1058-1064.

LI Bao-qi, HE Yu-yao, HE Ling-jiao, QIANG Wei. Asymmetric Parallel Semantic Segmentation Model Based on Full Convolutional Neural Network[J]. Acta Electronica Sinica, 2019, 47(5): 1058-1064.

参考文献

[1] He X,Zemel R S,Carreiraperpiñán M Á.Multiscale conditional random fields for image labeling[A].Proceedings of Computer Vision and Pattern Recognition[C].Washington:IEEE,2004.695-702.
[2] Yang L,Meer P,Foran D J.Multiple class segmentation using A unified framework over mean-shift patches[A].Proceedings of Computer Vision and Pattern Recognition[C].Minneapolis:IEEE,2007.1-8.
[3] Pantofaru C,Schmid C,Hebert M.Object recognition by integrating multiple image segmentations[A].Proceedings of European Conference on Computer Vision[C].Berlin:Springer,2008.481-494.
[4] Gould S,Fulton R,Koller D.Decomposing a scene into geometric and semantically consistent regions[A].Proceedings of International Conference on Computer Vision[C].Kyoto:IEEE,2009.1-8.
[5] Kumar M P,Koller D.Efficiently selecting regions for scene understanding[A].Proceedings of Computer Vision and Pattern Recognition[C].San Francisco:IEEE,2010.3217-3224.
[6] Jain A,Gupta A,Davis L S.Piecing together the segmentation jigsaw using context[A].Proceedings of Computer Vision and Pattern Recognition[C].Colorado:IEEE,2011.2001-2008.
[7] Kohli P,Ladický L,Torr P H S.Robust higher order potentials for enforcing label consistency[A].Proceedings of Computer Vision and Pattern Recognition[C].Anchorage:IEEE,2008.1-8.
[8] Russell C.Associative hierarchical CRFs for object class image segmentation[A].Proceedings of International Conference on Computer Vision[C].Kyoto:IEEE,2009.739-746.
[9] Ladicky L,Russell C,Kohli P,et al.Graph cut based inference with co-occurrence statistics[A].Proceedings of European Conference on Computer Vision[C].Berlin:Springer,2010.239-253.
[10] Hinton G.Where do features come from?[J].Cognitive Science,2014,38(6):1078-101.
[11] Lecun Y,Bengio Y,Hinton G.Deep learning[J].Nature,2015,521(7553):436-444.
[12] Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[A].Proceedings of Neural Information Processing Systems[C].Doho:ACM,2012.1097-1105.
[13] Szegedy C,Liu W,Jia Y,et al.Going deeper with convolutions[A].Proceedings of Computer Vision and Pattern Recognition[C].Boston:IEEE,2015.1-9.
[14] Farabet C,Couprie C,Najman L,et al.Learning hierarchical features for scene labeling[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(8):1915-1929.
[15] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[A].Proceedings of Computer Vision and Pattern Recognition[C].Boston:IEEE,2015.3431-3440.
[16] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,39(4):640-651.
[17] Yu F,Koltun V.Multi-scale context aggregation by dilated convolutions[A].Proceedings of International Conference on Learning Representations[C].San Juan:ICLR,2016.arXiv:1511.07122.
[18] Chen L C,Papandreou G,Kokkinos I,et al.DeepLab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(4):834-848.
[19] Wang P,Chen P,Yuan Y,et al.Understanding convolution for semantic segmentation[A].Proceedings of Computer Vision and Pattern Recognition[C].Lake Tahoe:IEEE,2018.1451-1460.
[20] 周则明,胡彪,孟勇,陈超迁,罗其祥.基于流形特征与形状先验的红外直升机图像分割[J].电子学报,2018,46(4):834-839. ZHOU Ze-ming,HU Biao,MENG Yong,CHEN Chao-qian,LUO Qi-xiang.Infrared helicopter image segmentation based on manifold feature and shape priori[J].Acta Electronica Sinica,2018,46(4):834-839.(in Chinese)
[21] 廉蔺,李国辉,张军,涂丹.基于参数分步估计的红外与可见光图像自动配准算法[J].电子学报,2012,40(9):1829-1838. LIAN Lin,LI Guo-hui,ZHANG Jun,TU Dan.An automatic algorithm for infrared and visible image registration based on pParameter Step Estimation[J].Acta Electronica Sinica,2012,40(9):1829-1838.(in Chinese)
[22] 韩崇昭,朱洪艳,段战胜.多源信息融合[M].北京:清华大学出版社,2010.394-449. Han Chong-zhao,Zhu Hong-yan,Duan Zhan-sheng.Multi-Source Informatin Fusion[M].Beijing:Tsinghua University Press,2010.394-449.(in Chinese)

基于全卷积神经网络的非对称并行语义分割模型

Asymmetric Parallel Semantic Segmentation Model Based on Full Convolutional Neural Network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

[1]	张志文, 刘天歌, 聂鹏举. 基于实景数据增强和双路径融合网络的实时街景语义分割算法[J]. 电子学报, 2022, 50(7): 1609-1620.
[2]	魏博文, 全红艳. 基于语义与形态特征融合的语义分割网络[J]. 电子学报, 2022, 50(11): 2688-2697.
[3]	徐频捷, 陈逸杰, 李之南, 赵地. 基于事件驱动的车道线识别算法研究[J]. 电子学报, 2021, 49(7): 1379-1385.
[4]	孟琭, 徐磊, 郭嘉阳. 一种基于改进的MobileNetV2网络语义分割算法[J]. 电子学报, 2020, 48(9): 1769-1776.
[5]	陈震, 马龙, 张聪炫, 黎明, 吴俊劼, 江少锋. 基于语义分割的双目场景流估计[J]. 电子学报, 2020, 48(4): 631-636.
[6]	李宝奇, 贺昱曜, 强伟, 何灵蛟. 基于并行附加特征提取网络的SSD地面小目标检测模型[J]. 电子学报, 2020, 48(1): 84-91.
[7]	罗会兰, 张云. 基于深度网络的图像语义分割综述[J]. 电子学报, 2019, 47(10): 2211-2220.