基于新视角合成的视频摘要交互式浏览

doi:10.3969/j.issn.0372-2112.2015.11.019

PDF(5432 KB)

电子学报 ›› 2015, Vol. 43 ›› Issue (11) : 2263-2270. DOI: 10.3969/j.issn.0372-2112.2015.11.019

学术论文

基于新视角合成的视频摘要交互式浏览

徐超¹, 聂勇伟², 葛红美¹, 周国富³

作者信息 +

Novel-View Synthesis Based Interactive Video Synopsis Browsing

XU Chao¹, NIE Yong-wei², GE Hong-mei¹, ZHOU Guo-fu³

Author information +

文章历史 +

摘要

现有视频摘要方法大多仅在固定视角下对视频内容进行压缩,限制了视频内容的浏览空间,导致压缩后的视频内容相互碰撞和遮挡.最近的方法改变视频内容的时空关系,但导致摘要结果与原始视频的相似度降低.本文研究基于新视角合成技术的交互式摘要视频浏览算法,通过为视频内容构建新的观察视角,增加视频浏览空间,从本质上减少视频内容碰撞和遮挡的可能性.为完全避免碰撞现象,提出了特定视角下投影遮挡消除技术,仅需较小地改变视频内容的时空关系,保证摘要视频与原始视频内容的一致性.最后,为方便用户交互式浏览摘要视频,构建了观察视角自动生成算法,根据当前摘要内容实时推荐并调整到最优观察视角.实验结果和用户调查表明本文提出的交互式视频摘要算法能有效压缩视频冗余信息,提高了摘要视频准确度、易理解程度.

Abstract

Most of the previous video synopsis methods compact the video under a fixed viewpoint,which limits the browsing space of the video.Recent method changes the spatiotemporal relationships between video objects with the side-effect of reducing the accuracy of synopsized video.This paper presents a novel-view synthesis based interactive video synopsis browsing method.Our method constructs wider browsing space that can inherently reduce the probability of collision and occlusion occurrences.To completely remove collision and occlusion artifacts,we propose a projective occlusion removing algorithm for a specified viewpoint,slightly changing the relationships between video objects.Finally,to facilitate user interactive browsing the video,we further propose an automatic viewpoint suggestion method,to compute and adjust to the optimal viewpoint in real-time according to current synopsized contents.The experiments and user study demonstrate the effectiveness of our method.

导出引用

徐超, 聂勇伟, 葛红美, 周国富. 基于新视角合成的视频摘要交互式浏览[J]. 电子学报, 2015, 43(11): 2263-2270. https://doi.org/10.3969/j.issn.0372-2112.2015.11.019

XU Chao, NIE Yong-wei, GE Hong-mei, ZHOU Guo-fu. Novel-View Synthesis Based Interactive Video Synopsis Browsing[J]. Acta Electronica Sinica, 2015, 43(11): 2263-2270. https://doi.org/10.3969/j.issn.0372-2112.2015.11.019

中图分类号： TP37

参考文献

[1] 孙维智,邓婵.视频摘要研究概述[J]. 信息通信,2014,(9):128-129. Sun Weizi,Deng Can.Video abstraction research introduction[J]. Information & Communications,2014,(9):128-129.(in Chinese)
[2] Pritch Y,Rav-Acha A,Peleg S.Nonchronological video synopsis and indexing[J]. IEEE Trans,2008,PAMI-30(11):1971-1984.
[3] 刘彩云,曹建荣,李洪艳.基于对象的视频摘要技术[J]. 计算机系统应用,2012,21(1):204-207. Liu Caiyun,Cao Jianrong,Li Hongyan.Object-based video synopsis technology[J]. Computer Systems & Applications,2012,21(1):204-207.(in Chinese)
[4] 王亚沛,李仁旺,刘翔.对象和关键帧相结合的监控视频摘要提取方法[J]. 工业控制计算机,2015,28(3):11-13. Wang Yapei,Li Renwang,Liuxiang.Extraction method of surveillance video synopsis combines objects and keyframes[J]. Industrial Control Computer,2015,28(3):11-13.(in Chinese)
[5] Y N,C X,H S,et al.Compact video synopsis via global spatiotemporal optimization[J]. IEEE Trans,2013,TVCG-19(10):1664-1676.
[6] 程文刚,须德,蒋轶玮,即丛妍.一种新的动态视频摘要生成方法[J]. 电子学报,2005,33(8):1461-1466. Cheng Wengang,Xu De,Jiang Yiwei,Lang Congyan.A novel approach to dynamic video abstraction[J]. Acta Electronica Sinica,2005,33(8):1461-1466.(in Chinese)
[7] 张玉培,孔敏,翟素兰,罗斌.基于镜头标记与动态滑动窗口的视频摘要生成[J]. 计算机工程,2012,38(2):256-258. Zhang Yupei,Kong Min,ZhaiSulan,Luo Bin.Video summary generation based on shot mark and dynamic sliding window[J]. Computer Engineering,2012,38(2):256-258.(in Chinese)
[8] 彭帝超,刘琳,陈广宇,陈海东,左伍衡,陈为.一种新的视频摘要可视化算法[J]. 计算机研究与发展,2013,50(2):371-378. Peng Dichao,Liu Lin,Chen Guangyu,Zuo Wuheng,Cheng Wei.A novel approach for abstractive video visualization[J]. Journal of Computer Research and Development,2013,50(2):371-378.(in Chinese)
[9] 王璐.一种基于关键帧技术的监控视频摘要提取方法[J]. 北京劳动保障职业学院学报,2015,9(1):47-50. Wang Lu.A surveillance video abstraction method based on key frame technology[J]. A Journal of Beijing Vocational College of Labour and Social Security,2015,9(1):47-50.(in Chinese)
[10] 顾益军,解易,夏天.基于内容代表性评价的关键帧抽取[J]. 计算机科学,2014,41(8):286-288. Gu Yijun,Xie Yi,Xia Tian.Key frame extraction based on representative evaluation of contents[J]. Computer Science,2014,41(8):286-288.(in Chinese)
[11] SIMAKOV,D,CASPI,Y,SHECHTMAN,E,AND IRANI,M.Summarizing visual data using bidirectional similarity[A]. Computer Vision and Pattern Recognition[C]. Alaska,USA:IEEE.2008.1-8.
[12] Nie Y,Sun H,Li P,et al.Object movements synopsis via part assembling and stitching[J]. IEEE Trans,2014,TVCG-20(9):1303-1315.
[13] 张运超,陈靖,王涌天.基于城市级位置服务的移动增强现实[J]. 电子学报,2014,42(8):1503-1508. Zhang Yunchao,Chen Jing,WangYongtian.City-scale location services based on mobile augmented reality[J]. Acta Electronica Sinica,2014,42(8):1503-1508.(in Chinese)
[14] 鲁敏,郁文贤,鲍虎军,等.基于机电跟踪的三维虚拟演播室系统[J]. 电子学报,2003,31(S1):2035-2039. Lu Min,Yu Wenxian,BaoHujun,3D virtual studio system based on electromechanical tracking[J]. Acta Electronica Sinica,2003,31(S1):2035-2039.(in Chinese)
[15] Germann M,Popa T,Keiser R,et al.Novel-view synthesis of outdoor sport events using an adaptive view-dependent geometry[J]. Computer Graphics Forum,2012,31(21):325-333.
[16] 张春晓,赵剡.一种基于三角形重投影的虚拟视角合成[J]. 北京航空航天大学学报,2012,38(1):86-90. Zhang Chunxiao Zhao Yan.Virtual view synthesis based on triangular re-projection[J]. Journal of Beijing University of Aeronautics and Astronautics,2012,38(1):86-90.(in Chinese)
[17] 张志佳,王博实,李雅红,等.基于双视角的可见外壳三维重建研究[J]. 计算机技术与发展,2015,(3):50-53. Zhang Zhijia,Wang Boshi,Li Yahongetc,Research on 3D Reconstruction of visible hull based on double perspectives[J]. Computer Technology and Development,2015,(3):50-53.(in Chinese)
[18] Saxena A,Sun M,Ng A Y.Make3D:learning 3D scene structure from a single still image[J]. IEEE Trans,2009,PAMI-31(5):824-840.
[19] HARTLEY,R,ZISSERMAN,A.Multiple view geometry in computer vision[M]. Cambridge,UK:Cambridge University Press.2003:23-64.
[20] Maki A,Watanabe M,Wiles C.Geotensity:Combining motion and lighting for 3d surface reconstruction[J]. International Journal of Computer Vision,2002,48(2):75-90.
[21] Malik J,Rosenholtz R.Computing local surface orientation and shape from texture for curved surfaces[J]. International Journal of Computer Vision,1997,23(2):149-168.
[22] Hassner T,Basri R.Example based 3D reconstruction from single 2D images[A]. Computer Vision and Pattern Recognition Workshop[C]. New York,USA:IEEE.2006.15-15.
[23] Nagai T,Ikehara M,Kurematsu A.HMM-based surface reconstruction from single images[A]. International Conference on Image Processing[C]. New York,USA:IEEE.2002.2(2):561-564.
[24] 郝宁波,廖海斌.真实感3D人脸快速建模研究[J]. 计算机应用研究,2011,28(1):352-355. HaoNing-bo,LiaoHai-bin.3Rapid 3D modeling of personalized face[J]. Application Research of Computers,2011,28(1):352-355.(in Chinese)
[25] Hoiem D,Efros A A,Hebert M.Recovering surface layout from an image[J]. International Journal of Computer Vision,2007,75(1):151-172.
[26] Fiorini P,Shillert Z.Motion planning in dynamic environments using velocity obstacles[J]. The International Journal of Robotics Research,1998,17(7):760-772.
[27] 黄永龙,仲训昱.基于改进速度障碍法的多机器人避碰规划算法[J]. 计算机工程与应用,2012,48(32):47-51. Huang Yonglong,Zhong Xunyu.Improved velocity obstacles-based collision avoidance algorithm for mul-tiple mobile robots[J]. Computer Engineering and Applications,2012,48(32):47-51.(in Chinese)
[28] Alonso-Mora J,Breitenmoser A,Beardsley P,et al.Reciprocal collision avoidance for multiple car-like robots[A]. IEEE International Conference on Robotics and Automation[C]. Saint Paul,USA:2012.360-366.
[29] 朱琳,周杰,宋靖雁,等.一种新的基于在线采样的多物体跟踪方法[J]. 计算机学报,2008,31(1):151-160. Zhu Lin,ZhouJie Song Jingyan.Multiple objects tracking based on online sampling[J]. Chinese Journal of Computers,2008,31(1):151-160.(in Chinese)
[30] Floater M S.Mean value coordinates[J]. Computer Aided Geometric Design,2003,20(1):19-27.