Abstract：Aiming at that RGB image is rich in color details of scene and infrared image is sensitive to outline、size and boundary of target,a novel semantic segmentation model APFCN (Asymmetric Parallelism Fully Convolutional Networks) is proposed.In the upper part of APFCN,a five layer dilation convolution network,where the five kernel sizes are not uniform,is designed used to extract the high-level targets contour features of infrared image.In the lower part of APFCN,a classical CNN network is used to extract three scale features of RGB images.At the back of APFCN,the high level features of the infrared image are fused with the three scale features of the RGB image,and the fused features after 4 times upper sampling is used as the semantic segmentation output of APFCN.The results show that APFCN is better than FCN (input RGB image or infrared image) in PA (Pixel Accuracy) and MIoU (Mean Intersection over Union).APFCN is suitable for the semantic segmentation task of ground targets with consistent background colors.
李宝奇, 贺昱曜, 何灵蛟, 强伟. 基于全卷积神经网络的非对称并行语义分割模型[J]. 电子学报, 2019, 47(5): 1058-1064.
LI Bao-qi, HE Yu-yao, HE Ling-jiao, QIANG Wei. Asymmetric Parallel Semantic Segmentation Model Based on Full Convolutional Neural Network. Acta Electronica Sinica, 2019, 47(5): 1058-1064.
[1] He X,Zemel R S,Carreiraperpiñán M Á.Multiscale conditional random fields for image labeling[A].Proceedings of Computer Vision and Pattern Recognition[C].Washington:IEEE,2004.695-702.
[2] Yang L,Meer P,Foran D J.Multiple class segmentation using A unified framework over mean-shift patches[A].Proceedings of Computer Vision and Pattern Recognition[C].Minneapolis:IEEE,2007.1-8.
[3] Pantofaru C,Schmid C,Hebert M.Object recognition by integrating multiple image segmentations[A].Proceedings of European Conference on Computer Vision[C].Berlin:Springer,2008.481-494.
[4] Gould S,Fulton R,Koller D.Decomposing a scene into geometric and semantically consistent regions[A].Proceedings of International Conference on Computer Vision[C].Kyoto:IEEE,2009.1-8.
[5] Kumar M P,Koller D.Efficiently selecting regions for scene understanding[A].Proceedings of Computer Vision and Pattern Recognition[C].San Francisco:IEEE,2010.3217-3224.
[6] Jain A,Gupta A,Davis L S.Piecing together the segmentation jigsaw using context[A].Proceedings of Computer Vision and Pattern Recognition[C].Colorado:IEEE,2011.2001-2008.
[7] Kohli P,Ladický L,Torr P H S.Robust higher order potentials for enforcing label consistency[A].Proceedings of Computer Vision and Pattern Recognition[C].Anchorage:IEEE,2008.1-8.
[8] Russell C.Associative hierarchical CRFs for object class image segmentation[A].Proceedings of International Conference on Computer Vision[C].Kyoto:IEEE,2009.739-746.
[9] Ladicky L,Russell C,Kohli P,et al.Graph cut based inference with co-occurrence statistics[A].Proceedings of European Conference on Computer Vision[C].Berlin:Springer,2010.239-253.
[10] Hinton G.Where do features come from?[J].Cognitive Science,2014,38(6):1078-101.
[11] Lecun Y,Bengio Y,Hinton G.Deep learning[J].Nature,2015,521(7553):436-444.
[12] Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[A].Proceedings of Neural Information Processing Systems[C].Doho:ACM,2012.1097-1105.
[13] Szegedy C,Liu W,Jia Y,et al.Going deeper with convolutions[A].Proceedings of Computer Vision and Pattern Recognition[C].Boston:IEEE,2015.1-9.
[14] Farabet C,Couprie C,Najman L,et al.Learning hierarchical features for scene labeling[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(8):1915-1929.
[15] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[A].Proceedings of Computer Vision and Pattern Recognition[C].Boston:IEEE,2015.3431-3440.
[16] Long J,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,39(4):640-651.
[17] Yu F,Koltun V.Multi-scale context aggregation by dilated convolutions[A].Proceedings of International Conference on Learning Representations[C].San Juan:ICLR,2016.arXiv:1511.07122.
[18] Chen L C,Papandreou G,Kokkinos I,et al.DeepLab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(4):834-848.
[19] Wang P,Chen P,Yuan Y,et al.Understanding convolution for semantic segmentation[A].Proceedings of Computer Vision and Pattern Recognition[C].Lake Tahoe:IEEE,2018.1451-1460.
[20] 周则明,胡彪,孟勇,陈超迁,罗其祥.基于流形特征与形状先验的红外直升机图像分割[J].电子学报,2018,46(4):834-839. ZHOU Ze-ming,HU Biao,MENG Yong,CHEN Chao-qian,LUO Qi-xiang.Infrared helicopter image segmentation based on manifold feature and shape priori[J].Acta Electronica Sinica,2018,46(4):834-839.(in Chinese)
[21] 廉蔺,李国辉,张军,涂丹.基于参数分步估计的红外与可见光图像自动配准算法[J].电子学报,2012,40(9):1829-1838. LIAN Lin,LI Guo-hui,ZHANG Jun,TU Dan.An automatic algorithm for infrared and visible image registration based on pParameter Step Estimation[J].Acta Electronica Sinica,2012,40(9):1829-1838.(in Chinese)
[22] 韩崇昭,朱洪艳,段战胜.多源信息融合[M].北京:清华大学出版社,2010.394-449. Han Chong-zhao,Zhu Hong-yan,Duan Zhan-sheng.Multi-Source Informatin Fusion[M].Beijing:Tsinghua University Press,2010.394-449.(in Chinese)
2019, Vol. 47 
