基于深度学习的显著性目标检测方法综述

罗会兰; 袁璞; 童康

doi:10.12263/DZXB.20200651

您当前的位置：

首页 >

文章列表页 >

基于深度学习的显著性目标检测方法综述

综述评论 | 更新时间：2025-12-08

- 基于深度学习的显著性目标检测方法综述
- Review of the Methods for Salient Object Detection Based on Deep Learning
- 电子学报 2021年49卷第7期页码：1417-1427
- 作者机构：
  
  1.江西理工大学信息工程学院，江西赣州 341000
  2.南京航空航天大学电子信息工程学院, 江苏南京211106
- 作者简介：
  
  [ "罗会兰女，1974年9月生于江西上高.2008年获浙江大学工学博士学位.现为江西理工大学图像处理实验室教授、硕士生导师.主要从事机器学习、模式识别等方面的研究.E-mail：luohuilan@sina.com" ]
  [ "袁璞女，1997年5月生于江西吉安.2018年进入江西理工大学.在读硕士研究生，研究方向为图像修复、显著性目标检测. E-mail：emmanuel_97@163.com" ]
  [ "童康男，1992年生于江苏南京，博士研究生，主要研究方向为计算机视觉与模式识别.E-mail：tkangcv@nuaa.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(61862031;61462035);江西省赣州市“科技创新人才计划”
- DOI：10.12263/DZXB.20200651
  中图分类号： TP389.1
- 收稿：2020-07-03，
  
  修回：2020-09-23，
  
  纸质出版：2021-07-25
- 稿件说明：
移动端阅览
罗会兰,袁璞,童康.基于深度学习的显著性目标检测方法综述[J].电子学报,2021,49(07):1417-1427.

LUO Hui-lan,YUAN Pu,TONG Kang.Review of the Methods for Salient Object Detection Based on Deep Learning[J].ACTA ELECTRONICA SINICA,2021,49(07):1417-1427.
罗会兰,袁璞,童康.基于深度学习的显著性目标检测方法综述[J].电子学报,2021,49(07):1417-1427. DOI： 10.12263/DZXB.20200651.

LUO Hui-lan,YUAN Pu,TONG Kang.Review of the Methods for Salient Object Detection Based on Deep Learning[J].ACTA ELECTRONICA SINICA,2021,49(07):1417-1427. DOI： 10.12263/DZXB.20200651.

摘要

显著性目标检测旨在对图像中最显著的对象进行检测和分割，是计算机视觉任务中重要的预处理步骤之一，且在信息检索、公共安全等领域均有广泛的应用.本文对近期基于深度学习的显著性目标检测模型进行了系统综述，从检测粒度的角度出发，综述了将深度学习引入显著性目标检测领域之后的研究成果.首先，从三个方面对显著性目标检测方法进行了论述：稀疏检测方法，密集检测方法以及弱监督学习下的显著性目标检测方法.然后，简要介绍了用于显著性目标检测研究的主流数据集和常用性能评价指标，并对各类主流模型在三个使用最广泛的数据集上进行了性能比较分析.最后，本文分析了显著性目标检测领域目前存在的问题，并对今后可能的研究趋势进行了展望.

Abstract

Salient object detection aims to detect and segment the most salient objects in the image. It is one of the important preprocessing steps in computer vision tasks

and it has a wide range of applications in information retrieval

public safety and other fields. This paper systematically reviews the recent research on the salient object detection models based on deep learning. From the perspective of detection granularity

the research results of applying deep learning into the field of salient object detection are reviewed. First

the salient object detection methods are discussed from three aspects: sparse detection methods

dense detection methods and weakly-supervised learning methods. Then

the mainstream data sets and common performance evaluation indicators used for salient object detection research are briefly introduced

and the performance of various mainstream models on the three most widely used data sets are compared and analyzed. Finally

this paper analyzes the current problems in the field of salient object detection and prospects for possible future research trends.

关键词

Keywords

references

Itti L , Koch C , Niebur E . A model of saliency-based visual attention for rapid scene analysis [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence , 1998 , 20 ( 11 ): 1254 - 1259 .

Liu T , Zheng N-N , Sun J , et al . Learning to detect a salient object [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Minneapolis, MN, USA : IEEE , 2007 . 1 - 8 .

Fang H G S , Iandola F , et al . From captions to visual concepts and back [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Boston, Massachusetts, USA : IEEE , 2015 . 1473 - 1482 .

K X , J B , R K . Show, attend and tell: Neural image caption generation with visual attention [A]. International Conference on Machine Learning [C]. Lille, France : International Machine Learning Society , 2015 . 2048 - 2057 .

Zhang F , Du B , Zhang L . Saliency-guided unsupervised feature learning for scene classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2015 , 53 ( 4 ): 2175 - 2184 .

Zhu J-Y , Wu J , Xu Y , et al . Unsupervised object class discovery via saliency-guided multiple class learning [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2014 , 37 ( 4 ): 862 - 875 .

Wang W , Shen J , Yang R , et al . Saliency-aware video object segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 40 ( 1 ): 20 - 33 .

毕威 , 黄伟国 , 张永萍 , 等 . 基于图像显著轮廓的目标检测 [J]. 电子学报 , 2017 , 45 ( 8 ): 1902 - 1910 .

BI Wei , HUANG Wei-guo , ZHANG Yong-ping , et al . Object detection based on salient contour of image [J]. Acta Electronica Sinica , 2017 , 45 ( 8 ): 1902 - 1910 . (in Chinese)

Das A A H , Zitnick L , et al . Human attention in visual question answering: Do humans and deep networks look at the same regions? [J]. Computer Vision and Image Understanding , 2017 , 163 : 90 - 100 .

Borji A , Cheng M-M , Jiang H , et al . Salient object detection: a benchmark [J]. IEEE Transactions on Image Processing , 2015 , 24 ( 12 ): 5706 - 5722 .

Borji A , Itti L . State-of-the-art in visual attention modeling [J]. IEEE Trans Pattern Anal Mach Intell, 2013 , 35 ( 1 ): 185 - 207 .

Kumar N . Thresholding in salient object detection: a survey [J]. Multimedia Tools and Applications , 2018 , 77 ( 15 ): 19139 - 19170 .

Han J , Zhang D , Cheng G , et al . Advanced deep-learning techniques for salient and category-specific object detection: a survey [J]. IEEE Signal Processing Magazine , 2018 , 35 ( 1 ): 84 - 100 .

Borji A , Cheng M-M , Hou Q , et al . Salient object detection: a survey [J]. Computational Visual Media , 2019 , 5 ( 2 ): 117 - 150 .

吴加莹 , 杨赛 , 堵俊 , 等 . 自底向上的显著性目标检测研究综述 [J]. 计算机科学 , 2019 , 46 ( 03 ): 48 - 52 .

WU Jia-ying , YANG Sai , DU Jun , et al . Review of bottom-up salient object detection [J]. Chinese Journal of Computers , 2019 , 46 ( 03 ): 48 - 52 . (in Chinese)

Wang W , Lai Q , Fu H , et al . Salient Object Detection in the Deep Learning Era: An In-depth Survey [EB/OL]. http:// arXiv.org/abs/1904.09146 http://arXiv.org/abs/1904.09146 , 2019 .

Yuan Y , Li C , Kim J , et al . Dense and sparse labeling with multidimensional features for saliency detection [J]. IEEE Transactions on Circuits and Systems for Video Technology , 2016 , 28 ( 5 ): 1130 - 1143 .

Long J , Shelhamer E , Darrell T . Fully convolutional networks for semantic segmentation [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Boston, Massachusetts, USA : IEEE , 2015 . 3431 - 3440 .

He S , Lau R W H , Liu W , et al . Supercnn: a superpixelwise convolutional neural network for salient object detection [J]. International Journal of Computer Vision , 2015 , 115 ( 3 ): 330 - 344 .

Zhao R , Ouyang W , Li H , et al . Saliency detection by multi-context deep learning [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Boston, Massachusetts, USA : IEEE , 2015 . 1265 - 1274 .

Lee G T Y W , Kim J . Deep saliency with encoded low level distance map and high level features [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Las Vegas, NV, USA : IEEE , 2016 . 660 - 668 .

Achanta R , Shaji A , Smith K , et al . SLIC superpixels compared to state-of-the-art superpixel methods [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012 , 34 ( 11 ): 2274 - 2282 .

Kim J , Pavlovic V . A shape preserving approach for salient object detection using convolutional neural networks [A]. International Conference on Pattern Recognition [C]. Cancun, Mexico : IEEE , 2016 . 609 - 614 .

Wang L , Lu H , Ruan X , et al . Deep networks for saliency detection via local estimation and global search [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Boston, Massachusetts, USA : IEEE , 2015 . 3183 - 3192 .

Li G , Yu Y . Visual saliency based on multiscale deep features [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Boston, Massachusetts, USA : IEEE , 2015 . 5455 - 5463 .

Zhang J , Sclaroff S , Lin Z , et al . Unconstrained salient object detection via proposal subset optimization [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Las Vegas, NV, USA : IEEE , 2016 . 5733 - 5742 .

Uijlings J R , Van De Sande K E , Gevers T , et al . Selective search for object recognition [J]. International Journal of Computer Vision , 2013 , 104 ( 2 ): 154 - 171 .

Krähenbühl P , Koltun V . Geodesic object proposals [A]. European Conference on Computer Vision [C]. Zurich, Switzerland : Springer , 2014 . 725 - 739 .

Zhang P , Wang D , Lu H , et al . Learning uncertain convolutional features for accurate saliency detection [A]. IEEE International Conference on Computer Vision [C]. Honolulu, Hawaii, USA : IEEE , 2017 . 212 - 221 .

Wang L , Wang L , Lu H , et al . Salient object detection with recurrent fully convolutional networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2018 , 41 ( 7 ): 1734 - 1746 .

Wang T , Zhang L , Wang S , et al . Detect globally, refine locally: a novel approach to saliency detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Salt Lake City, Utah, USA : IEEE , 2018 . 3127 - 3135 .

Kuen J , Wang Z , Wang G . Recurrent attentional networks for saliency detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Las Vegas, NV, USA : IEEE , 2016 . 3668 - 3677 .

Xu X , Chen J , Zhang H , et al . Dual pyramid network for salient object detection [J]. Neurocomputing , 2020 , 375 : 113 - 123 .

Li X , Zhao L , Wei L , et al . Deepsaliency: multi-task deep neural network model for salient object detection [J]. IEEE Trans Image Process, 2016 , 25 ( 8 ): 3919 - 3930 .

Hu P , Shuai B , Liu J , et al . Deep level sets for salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Honolulu, HI, USA : IEEE , 2017 . 2300 - 2309 .

Tang Y , Wu X . Saliency detection via combining region-level and pixel-level predictions with cnns [A]. European Conference on Computer Vision [C]. Amsterdam, The Netherlands : Springer , 2016 . 809 - 825 .

Li G Y Y . Deep contrast learning for salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Las Vegas, NV, USA : IEEE , 2016 . 478 - 487 .

Krähenbühl P , Koltun V . Efficient inference in fully connected crfs with gaussian edge potentials [A]. Advances in Neural Information Processing Systems [C]. Granada, Andalusia, Spain : Curran Associates, Inc , 2011 . 109 - 117 .

Ronneberger O , Fischer P , Brox T . U-net: convolutional networks for biomedical image segmentation [A]. International Conference on Medical Image Computing and Computer-assisted Intervention [C]. Munich, Germany : Springer , 2015 . 234 - 241 .

Zhiming Luo A M , Andrew Achkar , Justin Eichel , Shaozi Li , and Pierre-Marc Jodoin . Non-local deep features for salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Honolulu, HI, USA : IEEE , 2017 . 6609 - 6617 .

Zhang L , Dai J , Lu H , et al . A bi-directional message passing model for salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Salt Lake City, Utah, USA : IEEE , 2018 . 1741 - 1750 .

Liu N , Han J . Dhsnet: deep hierarchical saliency network for salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Las Vegas, NV, USA : IEEE , 2016 . 678 - 686 .

Zhang P , Wang D , Lu H , et al . Amulet: aggregating multi-level convolutional features for salient object detection [A]. IEEE International Conference on Computer Vision [C]. Honolulu, Hawaii, USA : IEEE , 2017 . 202 - 211 .

Liu N , Han J , Yang M-H . Picanet: learning pixel-wise contextual attention for saliency detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Salt Lake City, Utah, USA : IEEE , 2018 . 3089 - 3098 .

Zhang X , Wang T , Qi J , et al . Progressive attention guided recurrent network for salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Salt Lake City, Utah, USA : IEEE , 2018 . 714 - 722 .

Chen Z , Xu Q , Cong R , et al . Global context-aware progressive aggregation network for salient object detection [A]. Association for the Advancement of Artificial Intelligence [C]. New York, USA : AAAI , 2020 .

Zhao T , Wu X . Pyramid feature attention network for saliency detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Long Beach, California, USA : IEEE , 2019 . 3085 - 3094 .

Noori M , Mohammadi S , Majelan S G , et al . DFNet: discriminative feature extraction and integration network for salient object detection [J]. Engineering Applications of Artificial Intelligence , 2020 , 89 : 103419 .

Liu J-J , Hou Q , Cheng M-M , et al . A simple pooling-based design for real-time salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Long Beach, California, USA : IEEE , 2019 .

Wang B , Chen Q , Zhou M , et al . Progressive feature polishing network for salient object detection .[A]. Association for the Advancement of Artificial Intelligence [C]. New York, USA : AAAI , 2020 .

Feng M , Lu H , Ding E . Attentive feedback network for boundary-aware salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Long Beach, California, USA : IEEE , 2019 . 1623 - 1632 .

Zhao J-X , Liu J , Fan D-P , et al . EGNet: edge guidance network for salient object detection [A]. IEEE International Conference on Computer Vision [C]. Seoul, Korea : IEEE , 2019 .

Han L , Li X , Dong Y . Convolutional edge constraint-based U-Net for salient object detection [J]. IEEE Access , 2019 , 7 : 48890 - 48900 .

Qin X , Zhang Z , Huang C , et al . Basnet: boundary-aware salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Long Beach, California, USA : IEEE , 2019 . 7479 - 7489 .

Zhang D , Han J , Zhang Y . Supervision by fusion: towards unsupervised learning of deep salient object detector [A]. IEEE International Conference on Computer Vision [C]. Honolulu, Hawaii, USA : IEEE , 2017 . 4048 - 4056 .

Zhang J , Zhang T , Dai Y , et al . Deep unsupervised saliency detection: a multiple noisy labeling perspective [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Salt Lake City, Utah, USA : IEEE , 2018 . 9029 - 9038 .

Nguyen T , Dax M , Mummadi C K , et al . Deepusps: deep robust unsupervised saliency prediction via self-supervision [A]. Advances in Neural Information Processing Systems [C]. Vancouver, Canada : Morgan Kaufmann Publishers Inc. , 2019 . 204 - 214 .

Quan R , Han J , Zhang D , et al . Unsupervised salient object detection via inferring from imperfect saliency models [J]. IEEE Transactions on Multimedia , 2017 , 20 ( 5 ): 1101 - 1112 .

Wang L , Lu H , Wang Y , et al . Learning to detect salient objects with image-level supervision [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Honolulu, HI, USA : IEEE , 2017 . 136 - 145 .

Zhang L , Zhang J , Lin Z , et al . Capsal: leveraging captioning to boost semantics for salient object detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Long Beach, California, USA : IEEE , 2019 . 6024 - 6033 .

Lin T-Y , Maire M , Belongie S , et al . Microsoft coco: common objects in context [A]. European Conference on Computer Vision [C]. Zurich, Switzerland : Springer , 2014 . 740 - 755 .

Jiang M , Huang S , Duan J , et al . Salicon: saliency in context [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Boston, Massachusetts, USA : IEEE , 2015 . 1072 - 1080 .

Qian M , Qi J , Zhang L , et al . Language-aware weak supervision for salient object detection [J]. Pattern Recognition , 2019 , 96 : 106955 .

Zeng Y , Zhuge Y , Lu H , et al . Multi-source weak supervision for saliency detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Long Beach, California, USA : IEEE , 2019 . 6074 - 6083 .

Zhang J , Yu X , Li A , et al . Weakly-supervised salient object detection via scribble annotations [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Seattle, USA : IEEE , 2020 . 12546 - 12555 .

Cheng M-M , Zhang G-X , Mitra N J , et al . Global contrast based salient region detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. NW Washington, DC, USA : IEEE , 2011 . 409 - 416 .

Movahedi V , Elder J H . Design and perceptual validation of performance measures for salient object segmentation [A]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops [C]. San Francisco, CA, USA : IEEE , 2010 . 49 - 56 .

Martin D , Fowlkes C , Tal D , et al . A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics [A]. Proceedings of the 8th IEEE International Conference on Computer Vision [C]. Vancouver, BC, Canada : IEEE , 2001 . 416 - 423 .

Yan Q , Xu L , Shi J , et al . Hierarchical saliency detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Portland, Oregon : IEEE , 2013 . 1155 - 1162 .

Everingham M , Winn J . The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Development Kit [M]. UK : The VOC Workshop , 2012 .

Li Y , Hou X , Koch C , et al . The secrets of salient object segmentation [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Columbus, Ohio, USA : IEEE , 2014 . 280 - 287 .

Everingham M , Gool L V , Williams C K I , et al . The pascal visual object classes (voc) challenge [J]. International Journal of Computer Vision , 2010 , 88 ( 2 ): 303 - 338 .

Yang C , Zhang L , Lu H , et al . Saliency detection via graph-based manifold ranking [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Portland, Oregon : IEEE , 2013 . 3166 - 3173 .

Deng J , Dong W , Socher R , et al . Imagenet: a large-scale hierarchical image database [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Miami, FL, USA : IEEE , 2009 . 248 - 255 .

Xiao J , Hays J , Ehinger K A , et al . Sun database: large-scale scene recognition from abbey to zoo [A]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition [C]. San Francisco, CA, USA : IEEE , 2010 . 3485 - 3492 .

Achanta R , Hemami S , Estrada F , et al . Frequency-tuned salient region detection [A]. IEEE Conference on Computer Vision and Pattern Recognition [C]. Miami, FL, USA : IEEE , 2009 . 1597 - 1604 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于非一般类算子融合方法及硬件架构设计

MalMKNet：一种用于恶意代码分类的多尺度卷积神经网络

基于特征膨胀卷积模块的轻量化技术研究

Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计

一种轻量级的多尺度通道注意图像超分辨率重建网络