基于全卷积网络的语义显著性区域检测方法研究

doi:10.3969/j.issn.0372-2112.2017.11.004

电子学报 ›› 2017, Vol. 45 ›› Issue (11): 2593-2601.DOI: 10.3969/j.issn.0372-2112.2017.11.004

基于全卷积网络的语义显著性区域检测方法研究

郑云飞^1,2,3, 张雄伟¹, 曹铁勇¹, 孙蒙¹

1. 解放军陆军工程大学, 江苏南京 210007;
2. 解放军炮兵防空兵学院, 安徽合肥 230031;
3. 安徽省偏振成像与探测重点实验室, 安徽合肥 230031

收稿日期:2016-08-11 修回日期:2016-12-08 出版日期:2017-11-25
- 通讯作者:
- 曹铁勇
- 作者简介:
- 郑云飞,男,1983年生,解放军陆军工程大学博士研究生,解放军炮兵防空兵学院讲师.研究方向为图像与视频的显著性检测.E-mail:yfzheng83@163.com;张雄伟,男,解放军陆军工程大学教授,研究方向为多媒体信息处理;孙蒙,男,比利时鲁汶大学博士,解放军陆军工程大学讲师,研究方向为多媒体信息处理,机器学习.
- 基金资助:
- 国家自然科学基金 (No.61471394）; 国家青年自然科学基金 (No.61402519）; 江苏省自然科学基金 (No.BK2012510，No.BK20140071，No.BK20140074）

The Semantic Salient Region Detection Algorithm Based on the Fully Convolutional Networks

ZHENG Yun-fei^1,2,3, ZHANG Xiong-wei¹, CAO Tie-yong¹, SUN Meng¹

1. The Army Engineering University of PLA, Nanjing, Jiangsu 210007, China;
2. The Army Artilery and Defense Academy of PLA, Hefei, Anhui 230031, China;
3. The Key Laboratory of Polarization Imaging Detection Technology, Hefei, Anhui 230031, China

Received:2016-08-11 Revised:2016-12-08 Online:2017-11-25 Published:2017-11-25

摘要/Abstract

摘要： 基于底层视觉特征和先验知识的显著性区域检测算法难以检测一些复杂的显著性目标，人的视觉系统能分辨这些目标是由于其中包含丰富的语义知识.本文构建了一个基于全卷积结构的语义显著性区域检测网络，用数据驱动的方式构建从图像底层特征到人类语义认知的映射，提取语义显著性区域.针对网络提取的语义显著性区域的缺点，本文进一步引入颜色信息、目标边界信息、空间一致性信息获得准确的超像素级前景和背景概率.最后提出一个优化模型融合前景和背景概率信息、语义信息、空间一致性信息得到最终的显著性区域图.在6个数据集上与15种最新算法的比较实验证明了本文算法的有效性和鲁棒性.

关键词: 语义信息, 全卷积网络, 颜色外观模型, 显著性区域检测

Abstract: The existing salient region detection algorithms based on visual stimulus and prior knowledge are difficult to detect some complicated salient regions.The human vision system can distinguish these complicated salient regions because of the rich semantic knowledge in the human visual system.We construct a semantic salient region detection network using the fully convolutional structure.Learning the mapping from the low-level features to the human semantic cognition,our network can extract semantic salient region effectively.Aiming to the defects of the semantic salient region map,we introduce the color information,object boundary information and spatial consistency information to derive accurate superpixel-level foreground and background probability.At last,we fuse the foreground and background probability,semantic information and spatial consistency information to derive the final salient region map.The experiments comparing with the state-of-the-art 15 algorithms on 6 data sets demonstrate the effectiveness of our algorithm.

Key words: semantic information, fully convolutional network, color appearance model, salient region detection

中图分类号:

TP391.4

郑云飞, 张雄伟, 曹铁勇, 孙蒙. 基于全卷积网络的语义显著性区域检测方法研究[J]. 电子学报, 2017, 45(11): 2593-2601.

ZHENG Yun-fei, ZHANG Xiong-wei, CAO Tie-yong, SUN Meng. The Semantic Salient Region Detection Algorithm Based on the Fully Convolutional Networks[J]. Acta Electronica Sinica, 2017, 45(11): 2593-2601.

参考文献

[1] Koch K,McLean J,Segev R,Freed M A,Berry M J,Balasubramanian V,Sterling P.How much the eye tells the brain[J].Current Biology,2006,25(16):1428-1434.
[2] 姜维,卢朝阳,李静,刘晓佩,姚超.基于视觉显著性与文字置信图的场景文字的背景抑制方法[J].电子学报,2015,42(9):62-68. JIANG Wei,LU Zhao-yang,LI Jing,LIU Xiao-pei,YAO Chao.Visual saliency and text confidence map based background suppression for scene text[J].Acta Electronica Sinica,2015,42(9):62-68.(in Chinese)
[3] 赵宏伟,李清亮,刘萍萍.基于分级显著信息的空间编码方法[J].电子学报,2015,43(1):1863-1867. ZHAO Hong-wei,LI Qing-liang,LIU Ping-ping.Spatial encoding based on hierarchical salient information[J].Acta Electronica Sinica,2015,43(1):1863-1867.(in Chinese)
[4] Borji A,Itti L.State-of-the-art in visual attention modeling[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2013,35(1):185-207.
[5] Itti L,Koch C,Niebur E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[6] Cheng M M,Mitra N J,Huang X L,Torr P H S,Hu S M.Global contrast based salient region detection[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2015,37(3):569-582.
[7] Bao L,Lu J J,Li Y,Shi Y W.A saliency detection model using shearlet transform[J].Multimedia Tools and Applications,2015,74(11):4045-4058.
[8] Bao L,Zhang X W,Zheng Y F,Y Li.Video saliency detection using 3D shearlet transform[J],Multimedia Tools and Applications,2015,75(13):7716-7778.
[9] Perazzi F,Krahenbuhl P,Pritch Y,Hornung A.Saliency filters:contrast based filtering for salient region detection[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Providence:IEEE Press,2012.733-740.
[10] 钱晓亮,郭雷,韩军伟,等.一种基于加权稀疏编码的频域视觉显著性检测算法[J].电子学报,2013,41(6):1159-1165. QIAN Xiao-liang,GUO Lei,HAN Jun-wei,CHENG Gong,YAO Xi-wen.A spectral algorithm based on weighted sparse Coding for visual saliency detection[J].Acta Electronica Sinica,2013,41(6):1159-1165. (in Chinese)
[11] Zhu W J,Liang S,Wei Y C,J Sun.Saliency optimization from robust background detection[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Columbus:IEEE Press,2014.2814-2821.
[12] Shen X H,Wu Y.A unified approach to salient object detection via low rank matrix recovery[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Providence:IEEE Press,2012.853-860.
[13] Jiang H Zu,Wang J D,Yuan Z J,Wu Y,N N Zheng,S P Li.Salient object detection:a discriminative regional feature integration approach[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Portland:IEEE Press,2013.2083-2090.
[14] Kim J H,Han D,Tai Y W,J Kim.Salient region detection via high dimensional color transform[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Columbus:IEEE Press,2014.883-890.
[15] Tong N,Lu H C,Ruan X,M H Yang.Salient object detection via bootstrap learning[A],IEEE Conference on Computer Vision and Pattern Recognition[C].Boston:IEEE Press,2015.1884-1892.
[16] Li H Y,Lu H C,Lin Z,Shen X H,Price B.Inner and inter label propagation:salient object detection in the wild[J].IEEE Transactions on Image Processing,2015,10(24):3176-3186.
[17] Tong N,Lu H C,Zhang Y,Ruan X.Salient object detection via global and local cues[J].Pattern Recognition,2015,48:3258-3267.
[18] Einhäuser W G,Spain M,Perona P.Objects predict fixations better than early saliency[J].Journal of Vision,2008,8(14):1-26.
[19] Henderson J M,Hollingworth A.High-level scene perception[J].Annu Rev Psychol,1999,50:243-71.
[20] Athanasiadis T,Mylonas P,Avrithis Y,Kollias S.Semantic image segmentation and object labeling[J].IEEE Transaction on Circuits and Systems for Video Technology,2007,17(3):298-312.
[21] Long J,Shelhamer E,T Darrell.Fully convolutional networks for semantic segmentation[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Boston:IEEE Press,2015.3431-3440.
[22] Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[J/OL].http://arxiv.org/pdf/1409.1556.
[23] Cheng M M,Prisacariu V A,Zheng S,Torry P H S,Rother C.Densecut:densely connected CRFs for realtime grabcut[J].Computer Graphics Forum,2015,34(7):193-201.
[24] Achanta R,Shaji A,Smith K,Lucchi A,Fua P,Susstrunk.Slic superpixels compared to state-of-the-art superpixel methods[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2012,34(11):2274-2281.
[25] Boykov Y,Veksler O,Zabih R.Fast approximate energy minimization via graph cuts[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2001,23(11):1222-1239.
[26] Boykov Y,Kolmogorov V.An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2004,26(9):1124-1137.
[27] Liu T,Yuan Z,Sun J,Wang J,Zheng N,Tang X,Shum H Y.Learning to detect a salient object[J].IEEE Transaction on Pattern Analysis and Machine Intelligence,2011,33(2):353-367.
[28] Batra D,Kowdle A,Parikh D,Luo J,Chen T.Icoseg:interactive co-segmentation with intelligent scribble guidance[A].IEEE Conference on Computer Vision and Pattern Recognition[C].San Francisco:IEEE Press,2010.3169-3176.
[29] Li G B,Yu Y Z.Visual saliency based on multiscale deep features[A],IEEE Conference on Computer Vision and Pattern Recognition[C].Boston:IEEE Press,2015.5455-5463.
[30] Martin D,Fowlkes C,Tal D,Malik J.A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[A].International Conference on Computer Vision[C].Vancouver:Oxford University Press,2001.416-423.
[31] Yan Q,Xu L,Shi J,Jia J.Hierarchical saliency detection[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Portland:IEEE Press,2013.1155-1162.
[32] Li J,Martin D L,An X J,He H.Saliency detection based on frequency and spatial domain analysis[A].British Machine Vision Conference[C].Dundee:BMVA Press,2011.
[33] Radhakrishna A,Sheila,Francisco E,Sabine S.Frequency-tuned salient region detection[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Miami:IEEE Press,2009.1597-1604.
[34] Jiang P,Vasconcelos N,Peng J L.Generic promotion of diffusion-based salient object detection[A].International Conference on Computer Vision[C].Santiago:Oxford University Press,2015.217-225.
[35] Yang C,Zhang L H,Lu H C.Graph-regularized saliency detection with convex-hull-based center prior[J].IEEE Signal Processing Letters.2013,7(20):637-640.
[36] Li C Y,Yuan Y C,Cai W D,Xia Y,Feng D D.Robust saliency detection via regularized random walks ranking[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Boston:IEEE Press,2015.2710-2717.
[37] Yang C,Zhang L H,Lu H C,Ruan X,Yang M H.Saliency detection via graph-based manifold ranking[A].IEEE Conference on Computer Vision and Pattern Recognition[C].Portland:IEEE Press,2013.3166-3173.
[38] Tong N,Lu H C,Zhang L H,Ruan X.Saliency detection with multi-scale superpixels[J].IEEE Signal Processing Letters.2014,9(21):1035-1039.
[39] Sun J,Lu H C,Liu X P.Saliency region detection based on markov absorption probabilities[J].IEEE Transactions on Image Processing,2015,5(24):1639-1649.

基于全卷积网络的语义显著性区域检测方法研究

The Semantic Salient Region Detection Algorithm Based on the Fully Convolutional Networks

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 5

编辑推荐

Metrics

[1]	江泽涛, 简雄, 刘小艳, 曾聪文. 一种改进的二进制哈希编码方法[J]. 电子学报, 2019, 47(2): 462-469.
[2]	罗会兰, 张云. 基于深度网络的图像语义分割综述[J]. 电子学报, 2019, 47(10): 2211-2220.
[3]	张瑞杰, 李弼程, 魏福山. 基于多尺度上下文语义信息的图像场景分类算法[J]. 电子学报, 2014, 42(4): 646-652.
[4]	赵永威, 郭志刚, 李弼程, 高毫林, 陈刚. 基于随机化视觉词典组和上下文语义信息的目标检索方法[J]. 电子学报, 2012, 40(12): 2472-2480.
[5]	刘硕研;须德;冯松鹤;刘镝;裘正定. 一种基于上下文语义信息的图像块视觉单词生成算法[J]. 电子学报, 2010, 38(5): 1156-1161.