结合显著性的MVD视频整帧丢失错误隐藏
Error Concealment for Frame Losses in MVD Video Using Saliency
- 电子学报 2022年50卷第9期 页码：2146-2154
- 作者机构：
杭州电子科技大学通信工程学院, 浙江杭州 310018
- 作者简介：
[ "崔金鹏 男，1996年9月出生于山东省潍坊市.现为杭州电子科技大学通信工程学院硕士生.主要研究方向为3D视频编解码与差错控制.E‑mail：reader@163.com" ]
[ "周 洋 男，1979年1月出生于浙江省台州市.现为杭州电子科技大学通信工程学院副教授，硕士生导师，主要研究方向为3D视频编解码、视频图像处理." ]
殷海兵 男，1974年6月出生于湖北黄梅.教授，博士生导师.主要研究方向为智能信息处理.E‑mail：yhb@hdu.edu.cn
黄晓峰 男，1988年1月出生于江西婺源.主要研究方向为视频芯片架构.E‑mail：xfhuang@hdu.edu.cn
陆 宇 男，1977年5月出生于广西博白.主要研究方向视频编解码.E‑mail：y_lu@hdu.edu.cn
- 基金信息：浙江省自然科学基金(Y21F020076);国家自然科学基金(61972123);浙江省影视媒体技术研究重点实验室开放基金(CM2021001)
DOI：10.12263/DZXB.20210393
中图分类号： TP391;TP943收稿：2021-03-25，
修回：2022-02-08，
纸质出版：2022-09-25
- 稿件说明：
移动端阅览
崔金鹏,周洋,殷海兵等.结合显著性的MVD视频整帧丢失错误隐藏[J].电子学报,2022,50(09):2146-2154.
CUI Jin-peng,ZHOU Yang,YIN Hai-bing,et al.Error Concealment for Frame Losses in MVD Video Using Saliency[J].ACTA ELECTRONICA SINICA,2022,50(09):2146-2154.
崔金鹏,周洋,殷海兵等.结合显著性的MVD视频整帧丢失错误隐藏[J].电子学报,2022,50(09):2146-2154. DOI： 10.12263/DZXB.20210393.
CUI Jin-peng,ZHOU Yang,YIN Hai-bing,et al.Error Concealment for Frame Losses in MVD Video Using Saliency[J].ACTA ELECTRONICA SINICA,2022,50(09):2146-2154. DOI： 10.12263/DZXB.20210393.
摘要
基于多视点视频加深度（Multiview Video plus Depth， MVD）格式的三维视频能给用户提供多视角的沉浸式视觉体验.针对MVD视频流传输中彩色帧发生整帧丢失的情况，提出了一种结合视觉显著性的三维视频整帧丢失错误隐藏算法.先将丢失帧各区域划分为三种不同的显著等级，然后对低、中、高显著区域，分别采用时域直接拷贝、时域和视点间像素自适应搜索法和新型的运动矢量补偿法进行恢复.相较于目前的对比算法，经提出算法重建后的丢失帧的峰值信噪比值（Peak Signal to Noise Ratio， PSNR）和结构相似度值（Structural Similarity Index Measure， SSIM）能分别提高0.99~2.61 dB和0.005~0.012，并且重建图像主观视觉质量更佳.
Abstract
The 3D video based on multiview video plus depth(MVD) format can provide users with immersive experiences from different viewpoints. To solve the whole color frame losses in MVD video stream transmission
we propose a novel error concealment algorithm of whole-frames loss using visual saliency. Firstly
each region of the lost frame is classified into three different saliency levels. The low
medium and high saliency regions are recovered by direct temporal copy
adaptive pixel matching search between temporally and inter-view domains
and a new motion vector compensation
respectively. Compared with the recent algorithms
the lost frame concealed by the proposed algorithm achieves significant improvements in the average PSNR and SSIM with approximate gains of 0.99~2.61 dB and 0.005~0.012
respectively. In addition
our concealed results have better visual quality than compared methods.
关键词
Keywords
references
WANG T , LI F , QIAO X , et al . Low-complexity error resilient HEVC video coding: a deep learning approach [J]. IEEE Transactions on Image Processing , 2021 , 30 ( 2 ): 1245 - 1260 .
XIANG X , CHENG Y . A frame loss error concealment scheme for multi-view video coding [C]// Proceedings of 2014 International Conference on Instrumentation, Measurement, Computer, Communication and Control . Harbin, China : IEEE Computer Society , 2014 : 857 - 861 .
KUAN Y K , LI G L , CHEN M J , et al . Error concealment algorithm using inter-view correlation for multi-view video [J]. EURASIP Journal on Image and Video Processing , 2014 , 38 ( 1 ): 1 - 11 .
ZHOU Y , XIANG W , WANG G . Frame loss concealment for multiview video transmission over wireless multimedia sensor networks [J]. IEEE Sensors Journal , 2015 , 15 ( 3 ): 1892 - 1901 .
YU H , RAN M , LI P , et al . A new hybrid error concealment for multi-view video [C]// Proceedings of 2015 IEEE China Summit and International Conference on Signal and Information Processing . Chengdu, China : IEEE , 2015 : 176 - 180 .
AHAMED Z G , SHARMA V K , JAIN A . Image error concealment using robust data hiding technique [C]// Proceedings of 2020 International Conference on Innovative Mechanisms for Industry Applications . Bangalore, India : IEEE , 2020 : 470 - 474 .
EL-SHAFAI W , EL-RABAIE S , EL-HALAWANY M M , et al . Recursive Bayesian filtering-based error concealment scheme for 3D video communication over severely lossy wireless channels [J]. Circuits, Systems, and Signal Processing , 2018 , 37 ( 11 ): 4810 - 4841 .
YANG M , LAN X , ZHENG N , et al . Depth-assisted temporal error concealment for Intra frame slices in 3-D video [J]. IEEE Transactions on Broadcasting , 2014 , 60 ( 2 ): 385 - 393 .
EL-SHAFAI W , EL-RABAIE S , EL-HALAWANY M , et al . Encoder-independent decoder-dependent depth-assisted error concealment algorithm for wireless 3D video communication [J]. Multimedia Tools and Applications , 2018 , 77 ( 11 ): 13145 - 13172 .
HUANG P C , LIN J R , LI G L , et al . Improved depth- asisted error concealment algorithm for 3D video transmission [J]. IEEE Transactions on Multimedia , 2017 , 19 ( 11 ): 2625 - 2632 .
TAI S C , WANG C C , HONG C S , et al . An effiicient full frame algorithm for object-based error concealment in 3D depth-based video [J]. Multimedia Tools and Applications , 2016 , 75 ( 16 ): 9927 - 9947 .
KHATTAK S , MAUGEY T , HAMZAOUI R , et al . Temporal and inter-view consistent error concealment technique for multiview plus depth video [J]. IEEE Transactions on Circuits and Systems for Video Technology , 2016 , 26 ( 5 ): 829 - 840 .
LIN T L , WANG C J , DING T L , et al . Recovery of lost color and depth frames in multiview videos [J]. IEEE Transactions on Image Processing , 2018 , 27 ( 11 ): 5449 - 5463 .
LEE P J , KUO K T . An adaptive error concealment method for depth map in 3D video coding [J]. International Journal of Fuzzy Systems , 2016 , 18 ( 1 ): 62 - 71 .
ASSUNCAO P A , MARCELINO S , SOARES S , et al . Spatial error concealment for intra-coded depth maps in multiview video-plus-depth [J]. Multimedia Tools and Applications , 2017 , 76 ( 12 ): 13835 - 13858 .
LEE G W , HAN J K . Hole concealment for depth image using pixel classification in multiview system [C]// Proceedings of 2021 IEEE International Conference on Consumer Electronics . Las Vegas, USA : IEEE Computer Society , 2021 : 1 - 5 .
ZHANG X , ZHAO Y , LIN C , et al . Warping-driven mode selection for depth error concealment [C]// Proceedings of 2014 IEEE Global Conference on Signal and Information Processing . Atlanta, USA : IEEE , 2014 : 302 - 306 .
丁颖 , 刘延伟 , 等 . 虚拟现实全景图像显著性检测研究进展综述 [J]. 电子学报 , 2019 , 47 ( 7 ): 1575 - 1583 .
DING Ying , LIU Yan-wei , et al . An overview of research progress on saliency detection of panoramicVR images [J]. Acta Electronica Sinica , 2019 , 47 ( 7 ): 1575 - 1583 . (in Chinese)
CONG R , LEI J , FU H , et al . Going from RGB to RGBD saliency: a depth-guided transformation model [J]. IEEE Transactions on Cybernetics , 2020 , 50 ( 8 ): 3627 - 3639 .
CHENG M , MITRA N J , HUANG X , et al . Global contrast based salient region detection [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence , 2011 , 37 ( 3 ): 409 - 416 .
WANG J , DASILVA M P , LECALLET P , et al . Computational model of stereoscopic 3D visual saliency [J]. IEEE Transactions on Image Processing , 2013 , 22 ( 6 ): 2151 - 2165 .
JEON B , HUU T , CANH T N . A frame loss concealment for spatial scalable HEVC using base layer motion [C]// Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting . Jeju, Korea : IEEE , 2019 : 1 - 4 .
SONG K , CHUNG T , OH Y , et al . Error concealment of multi-view video sequences using inter-view and intra-view correlations [J]. Journal of Visual Communication and Image Representation , 2009 , 20 ( 4 ): 281 - 292 .
LIU X , PENG Q , FAN X . Frame loss concealment for multi-view video plus depth [C]// Proceedings of the 15th IEEE International Symposium on Consumer Electronics . Singapore : IEEE , 2011 : 208 - 211 .
0
浏览量
14
下载量
0
CSCD
- 网络PDF下载
- Meta-XML下载
- BibTeX下载
- Endnote下载
- RefMan 下载
- NoteExpress下载
- NoteFirst下载
关联资源
相关文章
相关作者
相关机构
- 技术支持由北京北大方正电子有限公司提供 京ICP备09064830号-19
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies