虚拟现实全景图像显著性检测研究进展综述

doi:10.3969/j.issn.0372-2112.2019.07.024

电子学报 ›› 2019, Vol. 47 ›› Issue (7): 1575-1583.DOI: 10.3969/j.issn.0372-2112.2019.07.024

虚拟现实全景图像显著性检测研究进展综述

丁颖^1,2, 刘延伟¹, 刘金霞³, 刘科栋¹, 王利明¹, 徐震¹

1. 中国科学院信息工程研究所, 北京 100864;
2. 中国科学院大学网络安全学院, 北京 100049;
3. 浙江万里学院, 浙江宁波 315100

收稿日期:2018-08-22 修回日期:2018-11-28 出版日期:2019-07-25
- 通讯作者:
- 刘延伟
- 作者简介:
- 丁颖女.1995年5月出生,河南安阳人.2016年毕业于郑州大学计算机科学与技术专业,2016年进入中国科学院大学信息工程研究所,现为硕士研究生,从事全景图像显著性检测方面的有关研究.E-mail:dingying@iie.ac.cn
- 基金资助:
- 国家自然科学基金 (No.61771469）

An Overview of Research Progress on Saliency Detection of Panoramic VR Images

DING Ying^1,2, LIU Yan-wei¹, LIU Jin-xia³, LIU Ke-dong¹, WANG Li-ming¹, XU Zhen¹

1. Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100864, China;
2. School of Cyber Security, University of Chinese Academy of Sciences, Beijing 100049, China;
3. Zhejiang Wanli University, Ningbo, Zhejiang 315100, China

Received:2018-08-22 Revised:2018-11-28 Online:2019-07-25 Published:2019-07-25

摘要/Abstract

摘要： 随着虚拟现实处理技术的发展，虚拟现实全景图像的显著性检测成为近年来学术界和工业界关注的研究热点.本文分析虚拟现实全景图像的特性，综述虚拟现实全景图像显著性检测算法的研究进展.将已有的虚拟现实全景图像显著性检测算法进行分类、分析以及对比，本文总结了当前虚拟现实全景图像显著性检测面临的挑战，并对其发展趋势进行展望.

关键词: 虚拟现实, VR全景图像, 显著性检测

Abstract: With the development of virtual reality (VR),the saliency detection for panoramic VR image has been a hot research topic in both academic and industry worlds.After analyzing the particular characteristics of panoramic VR image,this paper summarizes the research progress of the saliency detection algorithms of panoramic VR image.Existing VR image saliency detection algorithms are firstly classified,analyzed and compared.Then,the current research challenges and future directions of VR saliency detection are discussed.

Key words: virtual reality, VR panoramic image, saliency detection

中图分类号:

TP391

丁颖, 刘延伟, 刘金霞, 刘科栋, 王利明, 徐震. 虚拟现实全景图像显著性检测研究进展综述[J]. 电子学报, 2019, 47(7): 1575-1583.

DING Ying, LIU Yan-wei, LIU Jin-xia, LIU Ke-dong, WANG Li-ming, XU Zhen. An Overview of Research Progress on Saliency Detection of Panoramic VR Images[J]. Acta Electronica Sinica, 2019, 47(7): 1575-1583.

参考文献

[1] 解凯,郭恒业,张田文.图像Mosaics技术综述[J].电子学报,2004,32(4):630-634. XIE Kai,GUO Heng-ye,ZHANG Tian-wen.A survey of image mosaics technology[J].Acta Electronica Sinica,2004,32(4):630-634.(in Chinese)
[2] LUZ G,ASCENSO J,BRITES C,et al.Saliency-driven omni-directional imaging adaptive coding:modeling and assessment[A].Proceedings of the International Workshop on Multimedia Signal[C].USA:IEEE,2017.1-6.
[3] LIU K D,LIU Y W,LIU J X,et al.Joint source encoding and networking optimization for panoramic video streaming over LTE-A downlink[A].Proceedings of the GLOBECOM[C].USA:IEEE,2017.1-7.
[4] SERRANO A,SITZMANN V,RUIZ-BORAU J,et al.Movie editing and cognitive event segmentation in virtual reality video[J].ACM Transactions on Graphics,2017,36(4):1-7.
[5] BORJI A,CHENG M M,JIANG H,et al.Salient object detection:a survey[J].ArXiv Preprint,2014,2(4):1411-5878.
[6] JUDD T,DURAMD F,TORRALBA A.A benchmark of computational models of saliency to predict human fixations[J/OL].Technical Report,MIT,2012,http://hdl.handle.net/1721.1/68590.
[7] GUTIERREZ J,DAVID E,RAI Y,et al.Toolbox and dataset for the development of saliency and scanpath models for omni-directional/360° still images[J].Signal Processing Image Communication,2018,69:1-7.
[8] HE Y,VISHWANATH B,XIU X,et al.AHG8:interdigital's projection format conversion tool[S].
[9] KOCH C,ULLMAN S.Shifts in Selective Visual Attention:Towards the Underlying Neural Circuitry[M].Springer,Dordrecht:Matters of Intelligence,1987.115-141.
[10] ITTI L,KOCH C,NIEBUR E.A model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,V20(11):1254-1259.
[11] HAREL J,KOCH C,PERONA P.Graph-based visual saliency[A].Proceedings of Advances in Neural Information Processing Systems[C].USA:IEEE,2007.545-552.
[12] ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency-tuned salient region detection[A].Proceedings of Computer Vision and Pattern Recognition[C].USA:IEEE,2009.1597-1604.
[13] HOU X,ZHANG L.Saliency detection:a spectral residual approach[A].Proceedings of Computer Vision and Pattern Recognition[C].USA:IEEE,2007.1-8.
[14] ZHANG J,SCLAROFF S.Saliency detection:a boolean map approach[A].Proceedings of International Conference on Computer Vision[C].USA:IEEE,2013.153-160.
[15] KRIZHEVSHY A,SUTSKEVER I,HINTON G E.Imagenet classification with deep convolutional neural networks[A].Proceedings of International Conference on Neural Information[C].Qatar:Neural Information Processing,2012.1097-1105.
[16] MATHIEU M,LeCun Y,FERGUS R,et al.OverFeat:integrated recognition,localization and detection using convolutional networks[J].ArXiv Preprint,2013,2(4):1-7.
[17] LING J,ZHANG K,ZHANG Y,et al.A saliency prediction model on 360 degree images using color dictionary based sparse representation[J].Signal Processing Image Communication,2018,69:60-68.
[18] LEBRETON P,RAAKE A.GBVS360,BMS360,ProSal:extending existing saliency prediction models from 2D to omni-directional images[J].Signal Processing Image Communication,2018,69:69-78.
[19] STARTSEV M,DORR M.360-aware saliency estimation with conventional image saliency predictors[J].Signal Processing Image Communication,2018,69:43-52.
[20] BATTISTI F,BALDONI S,BRIZZI M,et al.A feature-based approach for saliency estimation of omni-directional images[J].Signal Processing Image Communication,2018,69:53-59.
[21] DING Y,LIU Y,LIU J,et al.Panoramic image saliency detection by fusing visual frequency feature and viewing behavior pattern[A].Proceedings of the Pacific-Rim Conference on Multimedia[C].China:MTAP,2018.418-429.
[22] ZHU Y,ZHAI G,MIN X.The prediction of head and eye movement for 360 degree images[J].Signal Processing Image Communication,2018,69:15-25.
[23] BOGDANOVA I,BUR A,HUGLI H.Visual attention on the sphere[J].Computer Vision & Image Understanding,2010,114(1):100-110.
[24] MAUGEY T,MEUR O L,LIU Z.Saliency-based navigation in omni-directional image[A].Proceedings of Multimedia Signal Processing[C].USA:IEEE,2017.1-6.
[25] ABREU A,OZCINAR C,SMOLIC A.Look around you:saliency maps for omni-directional images in VR applications[A].Proceedings of Quality of Multimedia Experience[C].USA:IEEE,2017.1-6.
[26] UPENIK E,EBRAHIMI T.A simple method to obtain visual attention data in head mounted virtual reality[A].Proceedings of Multimedia & Expo Workshops[C].USA:IEEE,2017.73-78.
[27] SITZMANN V,SERRANO A,PAVEL A,et al.Saliency in VR:how do people explore virtual environments?[A].Proceedings of Transactions on Visualization and Computer Graphics[C].USA:IEEE,2018.1633-1642.
[28] MONROY R,LUTZ S,CHALASANI T,et al.SalNet360:saliency maps for omni-directional images with CNN[J].Signal Processing Image Communication,2018,69:26-34.
[29] ASSENS M,MCGUINNESS K,GIRO-I-NIETO X,et al.SaltiNet:scan-path prediction on 360 degree images using saliency volumes[J].International Conference on Computer Vision Workshops,2017,3:2331-2338.
[30] XU M,SONG Y,WANG J,et al.Modeling attention in panoramic video:a deep reinforcement learning approach[J/OL].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018.https://arxiv.org/abs/1710.10755v1.
[31] ZHANG Z,XU Y,YU J,et al.Saliency detection in 360° videos[A].Proceedings of the European Conference on Computer Vision[C].Germany:Springer,2018.488-503.
[32] CORBILLON X,DE Simone F,SIMON G.360-Degree video head movement dataset[A].Proceedings of the 8th ACM on Multimedia Systems Conference[C].New York:ACM,2017.199-204.
[33] WU C,TAN Z,WANG Z,et al.A dataset for exploring user behaviors in VR spherical video streaming[A].Proceedings of the 8th ACM on Multimedia Systems Conference[C].New York:ACM,2017.193-198.
[34] LO W,FAN C,LEE J,et al.360° video viewing dataset in head-mounted virtual reality[A].Proceedings of the 8th ACM on Multimedia Systems Conference[C].New York:ACM,2017.211-216.

虚拟现实全景图像显著性检测研究进展综述

An Overview of Research Progress on Saliency Detection of Panoramic VR Images

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 11

编辑推荐

Metrics

[1]	陈星宇, 叶锋, 黄添强, 翁彬, 陈家祯, 林文忠. 融合小型深度生成模型的显著性检测[J]. 电子学报, 2021, 49(4): 768-774.
[2]	李庆武, 马云鹏, 周亚琴, 邢俊. 基于无监督栈式降噪自编码网络的显著性检测算法[J]. 电子学报, 2019, 47(4): 871-879.
[3]	杨小冈, 李维鹏, 马玛双. 概率框架下多特征显著性检测算法[J]. 电子学报, 2019, 47(11): 2378-2385.
[4]	张峰, 钟宝江. 基于兴趣目标的图像检索[J]. 电子学报, 2018, 46(8): 1915-1923.
[5]	李梦, 刘星, 任泽民. 联合Sobolev梯度的相场变分模型应用于图像显著性检测[J]. 电子学报, 2018, 46(7): 1683-1690.
[6]	杨兴明, 王雨廷, 谢昭, 吴克伟. 基于“轮廓-区域”多层互补特性的显著性检测[J]. 电子学报, 2018, 46(11): 2688-2695.
[7]	钱晓亮, 郭雷, 韩军伟, 程塨, 姚西文. 一种基于加权稀疏编码的频域视觉显著性检测算法[J]. 电子学报, 2013, 41(6): 1159-1165.
[8]	彭宇行, 张拥军, 李思昆. 分布式虚拟环境的时空一致性研究[J]. 电子学报, 2005, 33(2): 313-316.
[9]	解凯;郭恒业;张田文. 图像Mosaics技术综述[J]. 电子学报, 2004, 32(4): 630-634.
[10]	解凯;郭恒业;张田文. 一种具有径向畸变校正的图像整合算法[J]. 电子学报, 2004, 32(10): 1726-1729.
[11]	鲁敏, 郁文贤, 鲍虎军, 匡纲要. 基于机电跟踪的三维虚拟演播室系统[J]. 电子学报, 2003, 31(S1): 2035-2039.