Error Concealment for Frame Losses in MVD Video Using Saliency

CUI Jin-peng; ZHOU Yang; YIN Hai-bing; HUANG Xiao-feng; LU Yu

doi:10.12263/DZXB.20210393

您当前的位置：

首页 >

文章列表页 >

Error Concealment for Frame Losses in MVD Video Using Saliency

PAPERS | 更新时间：2025-12-08

- Error Concealment for Frame Losses in MVD Video Using Saliency
- ACTA ELECTRONICA SINICA Vol. 50, Issue 9, Pages: 2146-2154(2022)
- 作者机构：
  
  杭州电子科技大学通信工程学院, 浙江杭州 310018
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20210393
  CLC： TP391;TP943
- Received：25 March 2021，
  
  Revised：2022-02-08，
  
  Published：25 September 2022
- 稿件说明：
移动端阅览
崔金鹏,周洋,殷海兵等.结合显著性的MVD视频整帧丢失错误隐藏[J].电子学报,2022,50(09):2146-2154.

CUI Jin-peng,ZHOU Yang,YIN Hai-bing,et al.Error Concealment for Frame Losses in MVD Video Using Saliency[J].ACTA ELECTRONICA SINICA,2022,50(09):2146-2154.
崔金鹏,周洋,殷海兵等.结合显著性的MVD视频整帧丢失错误隐藏[J].电子学报,2022,50(09):2146-2154. DOI： 10.12263/DZXB.20210393.

CUI Jin-peng,ZHOU Yang,YIN Hai-bing,et al.Error Concealment for Frame Losses in MVD Video Using Saliency[J].ACTA ELECTRONICA SINICA,2022,50(09):2146-2154. DOI： 10.12263/DZXB.20210393.

摘要

基于多视点视频加深度（Multiview Video plus Depth， MVD）格式的三维视频能给用户提供多视角的沉浸式视觉体验.针对MVD视频流传输中彩色帧发生整帧丢失的情况，提出了一种结合视觉显著性的三维视频整帧丢失错误隐藏算法.先将丢失帧各区域划分为三种不同的显著等级，然后对低、中、高显著区域，分别采用时域直接拷贝、时域和视点间像素自适应搜索法和新型的运动矢量补偿法进行恢复.相较于目前的对比算法，经提出算法重建后的丢失帧的峰值信噪比值（Peak Signal to Noise Ratio， PSNR）和结构相似度值（Structural Similarity Index Measure， SSIM）能分别提高0.99~2.61 dB和0.005~0.012，并且重建图像主观视觉质量更佳.

Abstract

The 3D video based on multiview video plus depth(MVD) format can provide users with immersive experiences from different viewpoints. To solve the whole color frame losses in MVD video stream transmission

we propose a novel error concealment algorithm of whole-frames loss using visual saliency. Firstly

each region of the lost frame is classified into three different saliency levels. The low

medium and high saliency regions are recovered by direct temporal copy

adaptive pixel matching search between temporally and inter-view domains

and a new motion vector compensation

respectively. Compared with the recent algorithms

the lost frame concealed by the proposed algorithm achieves significant improvements in the average PSNR and SSIM with approximate gains of 0.99~2.61 dB and 0.005~0.012

respectively. In addition

our concealed results have better visual quality than compared methods.

关键词

Keywords

references

WANG T , LI F , QIAO X , et al . Low-complexity error resilient HEVC video coding: a deep learning approach [J]. IEEE Transactions on Image Processing , 2021 , 30 ( 2 ): 1245 - 1260 .

XIANG X , CHENG Y . A frame loss error concealment scheme for multi-view video coding [C]// Proceedings of 2014 International Conference on Instrumentation, Measurement, Computer, Communication and Control . Harbin, China : IEEE Computer Society , 2014 : 857 - 861 .

KUAN Y K , LI G L , CHEN M J , et al . Error concealment algorithm using inter-view correlation for multi-view video [J]. EURASIP Journal on Image and Video Processing , 2014 , 38 ( 1 ): 1 - 11 .

ZHOU Y , XIANG W , WANG G . Frame loss concealment for multiview video transmission over wireless multimedia sensor networks [J]. IEEE Sensors Journal , 2015 , 15 ( 3 ): 1892 - 1901 .

YU H , RAN M , LI P , et al . A new hybrid error concealment for multi-view video [C]// Proceedings of 2015 IEEE China Summit and International Conference on Signal and Information Processing . Chengdu, China : IEEE , 2015 : 176 - 180 .

AHAMED Z G , SHARMA V K , JAIN A . Image error concealment using robust data hiding technique [C]// Proceedings of 2020 International Conference on Innovative Mechanisms for Industry Applications . Bangalore, India : IEEE , 2020 : 470 - 474 .

EL-SHAFAI W , EL-RABAIE S , EL-HALAWANY M M , et al . Recursive Bayesian filtering-based error concealment scheme for 3D video communication over severely lossy wireless channels [J]. Circuits, Systems, and Signal Processing , 2018 , 37 ( 11 ): 4810 - 4841 .

YANG M , LAN X , ZHENG N , et al . Depth-assisted temporal error concealment for Intra frame slices in 3-D video [J]. IEEE Transactions on Broadcasting , 2014 , 60 ( 2 ): 385 - 393 .

EL-SHAFAI W , EL-RABAIE S , EL-HALAWANY M , et al . Encoder-independent decoder-dependent depth-assisted error concealment algorithm for wireless 3D video communication [J]. Multimedia Tools and Applications , 2018 , 77 ( 11 ): 13145 - 13172 .

HUANG P C , LIN J R , LI G L , et al . Improved depth- asisted error concealment algorithm for 3D video transmission [J]. IEEE Transactions on Multimedia , 2017 , 19 ( 11 ): 2625 - 2632 .

TAI S C , WANG C C , HONG C S , et al . An effiicient full frame algorithm for object-based error concealment in 3D depth-based video [J]. Multimedia Tools and Applications , 2016 , 75 ( 16 ): 9927 - 9947 .

KHATTAK S , MAUGEY T , HAMZAOUI R , et al . Temporal and inter-view consistent error concealment technique for multiview plus depth video [J]. IEEE Transactions on Circuits and Systems for Video Technology , 2016 , 26 ( 5 ): 829 - 840 .

LIN T L , WANG C J , DING T L , et al . Recovery of lost color and depth frames in multiview videos [J]. IEEE Transactions on Image Processing , 2018 , 27 ( 11 ): 5449 - 5463 .

LEE P J , KUO K T . An adaptive error concealment method for depth map in 3D video coding [J]. International Journal of Fuzzy Systems , 2016 , 18 ( 1 ): 62 - 71 .

ASSUNCAO P A , MARCELINO S , SOARES S , et al . Spatial error concealment for intra-coded depth maps in multiview video-plus-depth [J]. Multimedia Tools and Applications , 2017 , 76 ( 12 ): 13835 - 13858 .

LEE G W , HAN J K . Hole concealment for depth image using pixel classification in multiview system [C]// Proceedings of 2021 IEEE International Conference on Consumer Electronics . Las Vegas, USA : IEEE Computer Society , 2021 : 1 - 5 .

ZHANG X , ZHAO Y , LIN C , et al . Warping-driven mode selection for depth error concealment [C]// Proceedings of 2014 IEEE Global Conference on Signal and Information Processing . Atlanta, USA : IEEE , 2014 : 302 - 306 .

丁颖 , 刘延伟 , 等 . 虚拟现实全景图像显著性检测研究进展综述 [J]. 电子学报 , 2019 , 47 ( 7 ): 1575 - 1583 .

DING Ying , LIU Yan-wei , et al . An overview of research progress on saliency detection of panoramicVR images [J]. Acta Electronica Sinica , 2019 , 47 ( 7 ): 1575 - 1583 . (in Chinese)

CONG R , LEI J , FU H , et al . Going from RGB to RGBD saliency: a depth-guided transformation model [J]. IEEE Transactions on Cybernetics , 2020 , 50 ( 8 ): 3627 - 3639 .

CHENG M , MITRA N J , HUANG X , et al . Global contrast based salient region detection [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence , 2011 , 37 ( 3 ): 409 - 416 .

WANG J , DASILVA M P , LECALLET P , et al . Computational model of stereoscopic 3D visual saliency [J]. IEEE Transactions on Image Processing , 2013 , 22 ( 6 ): 2151 - 2165 .

JEON B , HUU T , CANH T N . A frame loss concealment for spatial scalable HEVC using base layer motion [C]// Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting . Jeju, Korea : IEEE , 2019 : 1 - 4 .

SONG K , CHUNG T , OH Y , et al . Error concealment of multi-view video sequences using inter-view and intra-view correlations [J]. Journal of Visual Communication and Image Representation , 2009 , 20 ( 4 ): 281 - 292 .

LIU X , PENG Q , FAN X . Frame loss concealment for multi-view video plus depth [C]// Proceedings of the 15th IEEE International Symposium on Consumer Electronics . Singapore : IEEE , 2011 : 208 - 211 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Motion Compensation Optimization Method for 3D Multi-Object Tracking

Video Compressed Sensing Reconstruction Network Based on Temporal-Attention Feature Alignment

Review of the Methods for Salient Object Detection Based on Deep Learning

Extracting Attention Information Algorithm Based onContrast Sensitivity and Markov Chain

Related Author

Jin-peng CUI

Yang ZHOU

Hai-bing YIN

Xiao-feng HUANG

Yu LU

WANG Shun-hong

ZHANG Yu

SHEN Jiang-nan

Related Institution

School of Telecommunication Engineering， Hangzhou Dianzi University

School of Computer Science and Technology， University of Science and Technology of China

School of Electronic and Information Engineering, South China University of Technology

College of Electronic and Information Engineering， Nanjing University of Aeronautics and Astronautics

College of Information Engineering， Jiangxi University of Science and Technology

⁰