An Approach to Progressive Description of Video Streaming Based on Visual Saliency Computation
LIANG Yong-sheng1,2, LIU Wei1, ZHOU Ying2, WEI Ze-feng2, ZHANG Ji-hong2
1. Shenzhen Key Lab of Visual Media Processing and Transmission, Shenzhen Institute of Information Technology, Shenzhen, Guangdong 518029, China;
2. School of Information Engineering, Shenzhen University, Shenzhen, Guangdong 518060, China
In order to balance the contradiction among network bandwidth,video quality and subscriber's real-time access,a new approach to progressive description of video streaming based on visual saliency computation is proposed in this paper.On the basis of video content analysis and comprehension,scene classification and Visual Sensitive Region(VSR) extraction are performed firstly.Secondly,frame importance according to coding information and slice data importance are determined.Finally,based on visual saliency computation,a new approach to progressive description of video streaming adapted to network bandwidth and quality scalability is proposed in this paper.Applying MGS Coding,experimental study is performed on video sequence with salient region and cluttered regions in network simulation platform,the experimental results show that the progressive description based on visual saliency computation proposed in this paper is accurate and effective.
[1] Abdelhamid Nafaa,Manon Gaucher.Implementation and analysis of a peer-to-peer retransmissions system for live video services[J].IEEE Multimedia,2011,18(2):60-71.
[2] Yan Li,Athina Markopoulou,John Apostolopoulos,Nicholas Bambos.Content-aware playout and packet scheduling for video streaming over wireless links[J].IEEE Transactions on Multimedia,2008,10(5):885-895.
[3] Jung-Hwan Lee,Chuck Yoo.Scalable ROI agorithm for H.264/SVC-based video streaming[J],IEEE Transactions on Consumer Electronics,2011,57(2):882-887.
[4] Dan Grois,Ofer Hadar.Complexity-aware adaptive preprocessing scheme for region-of-interest spatial scalable video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(6):1025-1039.
[5] 李晓峰,周宁,刘洪盛,张敏.一种基于缩减栅格算法的SVC联合信源/信道编码方法[J].电子学报,2011,39(4):859-864. LI Xiao-feng,ZHOU Ning,LIU Hong-sheng,ZHANG Min.A joint source/channel coding with reduced trellis algorithm for the scalable extension of H.264/AVC[J].Acta Electronica Sinica,2011,39(4):859-864.(in Chinese)
[6] 陈旭,张基宏,柳伟,梁永生.基于视觉注意的的视频可伸缩ROI算法[J].山东大学学报(工学版),2013,43(1):15-21. Chen Xu,Zhang Jihong,Liu Wei,Liang Yongsheng.New scalable ROI algorithm based on visual attention[J].Journal of Shandong University (Engineering and Technology Edition),2013,43(1):15-31.(in Chinese)
[7] 刘家瑛,郭宗明,Yongjin CHO.面向H.264/SVC空域-质量域可伸缩编码的码率分配算法[J].电子学报,2010,28(9):2112-2117. LIU Jia-ying,GUO Zong-ming,Yongjin CHO,Bit allocation algorithm in H.264/SVC spatial-quality with dependent R-D modeling[J].Acta Electronica Sinica,2010,28(9):2112-2117.(in Chinese)
[8] Ching-Lung Su,Tse-Min Chen,Chih-Yang Huang.Cluster-based motion estimation algorithm with low memory and bandwidth requirements for H.264/AVC scalable extension.IEEE transactions on circuits and systems for video technology,2014,24(6):1016-1024.
[9] 崔子冠,朱昌秀,干宗良,唐贵进,刘峰.H.264视频编码率失真优化和码率控制技术研究进展[J].电子学报,2013,41(12):2443-2450. CUI Zi-guan,ZHU Xiu-chang,GAN Zong-liang,TANG Gui-jin,LIU Feng.Advance in rate distortion optimization and rate control techniques for H.264 video coding[J].Acta Electronica Sinica,2013,41(12):2443-2450.(in Chinese)
[10] Wang J,Wang F,Zhang C,Shen HC,Quan L.Linear neighborhood propagation and its applications[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(9):1600-1615.
[11] Christian Siagian,Laurent Itti.Rapid biologically-inspired scene classification using features shared with visual attention[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(2):300-312.
[12] Jun Huang,Xiao Yang,Xiang-zhong Fang,Wei-yao Lin,Rui Zhang.Integrating visual saliency and consistency for re-ranking image search results[J].IEEE Transactions on Multimedia,2011,13(4):653-661.
[13] Heiko Schwarz,Thomas Wiegand.Implementation and performance of FGS,MGS,and CGS[S].Joint Video Team,Doc.JVT-V126,2007.2-3.
[14] Rohan Gupta,Akshay Pulipaka,Patrick Seeling,etc.H.264 coarse grain scalable(CGS) and medium grain scalable(MGS) encoded video:A trace based traffic and quality evaluation[J].IEEE Transactions on Broadcasting,2012,58(3):428-439.
[15] 汪大勇,孙世新.可伸缩视频编码研究现况综述[J].电子测量与仪器学报,2009,(8):78-84. WANG Da-yong,SUN Shi-xin.Summary of research on scalable video coding[J].Journal of Electronic Measurement and Instrument,2009,(8):78-84.(in Chinese)
[16] 纪超,刘慧英,孙景峰,贺胜,黄民主.基于空域和频域的图像显著区域检测[J].吉林大学学报(工学版),2014,44(1):177-183. JI Chao,LIU Hui-ying,SUN Jing-feng,HE Sheng,HUANG Min-zhu.Image salient region detection based on spatial and frequency domains.Journal of Jilin University (Engineering and Technology Edition),2014,44(1):177-183.(in Chinese)
[17] Edmund Y.Lam,Joseph W.Goodman.A mathematical analysis of the DCT coefficient distributions for images[J].IEEE Transactions on Image Processing,2000,9(10):1661-1666.
[18] Truong Cong Thang,Jung Won Kang,Jeong-Ju Yoo,Jae-Gon Kim.Multilayer adaptation for MGS-based SVC bitstream[C].Proceeding of the 16th ACM International Conference on Multimedia,Vancouver,British Columbia,Canada,2008.689-692.
[19] S.Wenger,Y.K.Wang,T.Schierl,A.Eleftheriadis.RTP payload format for SVC video[S].IETF Internet Draft draft-ietf-avt-rtp-svc-06,2010.47-95.
[20] Chan-Won Seo,Jong-Ki Han.Rate control scheme for consistent video quality in scalable video codec[J].IEEE Transactions on Image Processing,2011,20(8):2166-2176.
[21] Michal Ries,Olivia Nemethova,Markus Rupp.Video quality estimation for mobile H.264/AVC video streaming[J].Journal of Communications,2008,3(1):41-50.
[22] Kalpana Seshadrinathan,Rajiv Soundararajan,Aaln Contrad Bovik,Lawrcncc K.Cormack.Study of subjective and objective quality assessment of video[J].IEEE Transactions on Image Processing,2010,19(16):1427-1441.
2017, Vol. 45 
