电子学报 ›› 2020, Vol. 48 ›› Issue (5): 1018-1029.DOI: 10.3969/j.issn.0372-2112.2020.05.024
郭红伟1,2, 朱策1, 刘宇洋1
收稿日期:
2019-04-23
修回日期:
2019-11-21
出版日期:
2020-05-25
通讯作者:
作者简介:
基金资助:
GUO Hong-wei1,2, ZHU Ce1, LIU Yu-yang1
Received:
2019-04-23
Revised:
2019-11-21
Online:
2020-05-25
Published:
2020-05-25
摘要: 率失真优化技术能有效地提升编码器的压缩性能,是视频编码领域重要的研究内容.本文综合论述了基于拉格朗日乘子的率失真优化方法,从独立率失真优化、依赖率失真优化和码率控制中的比特分配等几方面归纳和分析最新研究进展.首先,通过形象的举例解释了约束优化问题通过拉格朗日乘子法转换为非约束优化问题,并根据相关文献分析拉格朗日乘子与量化参数的关系.其次,分析了编码过程中存在的率失真依赖性,并分类讨论近年提出的依赖率失真优化方法.然后,简要介绍了率失真依赖关系在码率控制比特分配中的影响.最后通过分析比较不同率失真优化技术的特征和性能,提出当前面临的挑战和值得进一步探索的工作.
中图分类号:
郭红伟, 朱策, 刘宇洋. 视频编码率失真优化技术研究综述[J]. 电子学报, 2020, 48(5): 1018-1029.
GUO Hong-wei, ZHU Ce, LIU Yu-yang. Overview of Rate-Distortion Optimization for Video Coding[J]. Acta Electronica Sinica, 2020, 48(5): 1018-1029.
[1] SULLIVAN G J,WIEGAND T.Rate-distortion optimization for video compression[J].IEEE Signal Processing Magazine,1998,15(6):74-90. [2] SHANNON C E.Coding theorems for a discrete source with a fidelity criterion[J].Institute of Radio Engineers,International Convention Record,1959,7(7):142-163. [3] 杨天武,彭强,邓云.一种基于率失真优化的关键参考帧选择算法[J].电子学报,2006,34(7):1241-1245. YANG T-W,PENG Q,DENG Y.A novel key frame reference picture selection algorithm based on rate-distortion optimization[J].Acta Electronica Sinica,2006,34(7):1241-1245.(in Chinese) [4] LI H Q,LI B,XU J Z.Rate-distortion optimized reference picture management for high efficiency video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2012,22(12):1844-1857. [5] 陆寄远,朝红阳,黄承慧.计算能力可伸缩的运动估计率失真优化[J].电子学报,2014,42(8):1495-1502. LU J-Y,CHAO H-Y,HUANG C-H.Rate distortion optimization of complexity scalable motion estimation[J].Acta Electronica Sinica,2014,42(8):1495-1502.(in Chinese) [6] ORTEGA A,RAMCHANDRAN K.Rate-distortion methods for image and video compression[J].IEEE Signal Processing Magazine,1998,15(6):23-50. [7] WIEGAND T,GIROD B.Lagrange multiplier selection in hybrid video coder control[A].Proceedings of International Conference on Image Processing (ICIP)[C].Piscataway:IEEE,2001.542-545. [8] LI X,OERTEL N,KAUP A.Adaptive Lagrange multiplier selection for intra-frame video coding[A].Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS)[C].Piscataway:IEEE,2007.3643-3646. [9] LI X,OERTEL N,HUTTER A,et al.Laplace distribution based lagrangian rate distortion optimization for hybrid video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2009,19(2):193-205. [10] ZHANG J,YI X Q,LING N,et al.Context adaptive lagrange multiplier (CALM) for rate-distortion optimal motion estimation in video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2010,20(6):820-828. [11] GONZALEZ-DE-SUSO J L,JIMENEZ-MORENO A,et al.Improved method to select the Lagrange multiplier for rate-distortion based motion estimation in video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(3):452-464. [12] ZHANG F,BULL D R.Rate-distortion optimization using adaptive Lagrange multipliers[J].IEEE Transactions on Circuits and Systems for Video Technology,2019,29(10):3121-3131. [13] AN C H,NGUYEN T Q.Adaptive Lagrange multiplier selection using classification-maximization and its application to chroma qp offset decision[J].IEEE Transactions on Circuits and Systems for Video Technology,2011,21(6):783-791. [14] LI B,XU J Z,ZHANG D,et al.QP refinement according to lagrange multiplier for high efficiency video coding[A].Proceedings of International Symposium on Circuits and Systems[C].Piscataway:IEEE,2013.477-480. [15] LI B,LI H Q,LI L,et al. λ-domain rate control algorithm for high efficiency video coding[J].IEEE Transactions on Image Processing,2014,23(9):3841-3854. [16] WANG M H,NGAN K N,LI H L.An efficient frame-content based intra frame rate control for high efficiency video coding[J].IEEE Signal Processing Letters,2015,22(7):896-900. [17] LI L,LI B,LIU D,et al.λ-domain rate control algorithm for HEVC scalable extension[J].IEEE Transactions on Multimedia,2016,18(10):2023-2039. [18] LI B,XU J,LI H.Refining QP to improve coding efficiency in AVS[A].Proceedings of IEEE International Conference on Image Processing[C].Piscataway:IEEE,2013.1528-1531. [19] 张方,吴成柯,肖嵩.一种基于MPEG-4的感兴趣区域视频编码新方法[J].电子学报,2005,33(4):634-637. ZHANG F,WU C K,XIAO S.A new regions of interest video coding method based on MPEG-4[J].Acta Electronica Sinica,2005,33(4):634-637.(in Chinese) [20] LI Y,TAO X M,LU J.Hybrid model-and-object-based real-time conversational video coding[J].Signal Processing-Image Communication,2015,35(2015):9-19. [21] HU P,SHUAI B,LIU J,et al.Deep level sets for salient object detection[A].Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition[C].Piscataway:IEEE,2017.540-549. [22] XU M,JIANG L,SUN X Y,et al.Learning to detect video saliency with HEVC features[J].IEEE Transactions on Image Processing,2017,26(1):369-385. [23] XIONG B,FAN X J,ZHU C,et al.Face region based conversational video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2011,21(7):917-931. [24] YANG X K,LIN W S,LU Z K,et al.Rate control for videophone using local perceptual cues[J].IEEE Transactions on Circuits and Systems for Video Technology,2005,15(4):496-507. [25] LIU Y,LI Z G,SOH Y C.Region-of-interest based resource allocation for conversational video communication of H.264/AVC[J].IEEE Transactions on Circuits and Systems for Video Technology,2008,18(1):134-139. [26] XU M,DENG X,LI S X,et al.Region-of-Interest based conversational HEVC coding with hierarchical perception model of face[J].IEEE Journal of Selected Topics in Signal Processing,2014,8(3):475-489. [27] ZHANG Z W,JING T,HAN J N,et al.A new rate control scheme for video coding based on region of interest[J].IEEE Access,2017,5(0):13677-13688. [28] HADIZADEH H,BAJIC I V.Saliency-aware video compression[J].IEEE Transactions on Image Processing,2014,23(1):19-33. [29] LI S,XU M,DENG X,et al.Weight-based R-λ rate control for perceptual HEVC coding on conversational videos[J].Signal Processing-Image Communication,2015,38(2015):127-140. [30] ZENG H Q,YANG A S,NGAN K N,et al.Perceptual sensitivity-based rate control method for high efficiency video coding[J].Multimedia Tools and Applications,2016,75(17):10383-10396. [31] LI S X,XU M,et al.Closed-form optimization on saliency-guided image compression for HEVC-MSP[J].IEEE Transactions on Multimedia,2018,20(1):155-170. [32] ZENG H,NGAN K N,WANG M.Perceptual adaptive Lagrangian multiplier for high efficiency video coding[A].Proceedings of Picture Coding Symposium[C].Piscataway:IEEE,2013:69-72. [33] LAINEMA J,BOSSEN F,et al.Intra coding of the HEVC standard[J].IEEE Transactions on Circuits and Systems for Video Technology,2012,22(12):1792-1801. [34] LIN J L,CHEN Y W,HUANG Y W,et al.Motion vector coding in the HEVC standard[J].IEEE Journal of Selected Topics in Signal Processing,2013,7(6):957-968. [35] BICHON M,TANOU J L,et al.Inter-block dependencies consideration for intra coding in H.264/AVC and HEVC standards[A].Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing[C].Piscataway:IEEE,2017.1537-1541. [36] BICHON M,LE TANOU J,ROPERT M,et al.Low complexity joint RDO of prediction units couples for HEVC intra coding[A].Proceedings of IEEE International Conference on Acoustics,Speech and Signal Processing[C].Piscataway:IEEE,2018.1733-1737. [37] YOU J,CHOI C,JEONG J.Modified rate distortion optimization using inter-block dependence for H.264/AVC intra coding[J].IEEE Transactions on Consumer Electronics,2008,54(3):1383-1388. [38] SUN L,AU O C,DAI W,et al.Modified distortion redistribution problem for high efficiency video coding (HEVC)[A].Proceedings of IEEE 14th International Workshop on Multimedia Signal Processing (MMSP)[C].Piscataway:IEEE,2012.278-282. [39] PANG C,AU O C,ZOU F,et al.Optimal distortion redistribution in block-based image coding using successive convex optimization[A].Proceedings of IEEE International Conference on Multimedia and Expo (ICME)[C].Piscataway:IEEE,2011.1-5. [40] WU Q B,XIONG J,LUO B,et al.A novel joint rate distortion optimization scheme for intra prediction coding in H.264/AVC[J].IEICE Transactions on Information and Systems,2014,E97-D(4):989-992. [41] KIM I-K,MCCANN K,SUGIMOTO K,et al.High Efficiency Video Coding (HEVC) Test Model 11(HM11) Encoder Description[R].Document JCTVC-M1002,Incheon,KR,2013. [42] SCHWARZ H,MARPE D,WIEGAND T.Overview of the scalable video coding extension of the H.264/AVC standard[J].IEEE Transactions on Circuits and Systems for Video Technology,2007,17(9):1103-1120. [43] LI X A,AMON P,HUTTER A,et al.Model based analysis for quantization parameter cascading in hierarchical video coding[A].Proceedings of IEEE International Conference on Image Processing (ICIP)[C].Piscataway:IEEE,2009.3765-3768. [44] WAN W X,CHEN Y,WANG Y K,et al.Efficient hierarchical inter picture coding for H.264/AVC baseline profile[A].Proceedings of Picture Coding Symposium (PCS)[C].Piscataway:IEEE,2009.1-4. [45] LI X,AMON P,HUTTER A,et al.Adaptive quantization parameter cascading for hierarchical video coding[A].Proceedings of IEEE International Symposium on Circuits and Systems[C].Piscataway:IEEE,2010.4197-4200. [46] XU Y,LI Q,LI X,et al.Efficient QP cascading in H.265/HEVC low-delay prediction[A].Proceedings of IEEE International Conference on Multimedia & Expo Workshops[C].Piscataway:IEEE,2017.1-6. [47] AMER H,YANG E H.Low-delay HEVC adaptive quantization parameter selection through temporal propagation length estimation[A].Proceedings of IEEE International Conference on Image Processing (ICIP)[C].Piscataway:IEEE,2018.211-215. [48] PAPADOPOULOS M A,ZHANG F,et al.An adaptive QP offset determination method for HEVC[A].Proceedings of IEEE International Conference on Image Processing[C].Piscataway:IEEE,2016.4220-4224. [49] YANG Y,WAN S,GONG Y,et al.Adaptive quantization parameter cascading for random-access prediction in H.265/HEVC based on dependent R-D models[A].Proceedings of IEEE International Conference on Image Processing[C].Piscataway:IEEE,2016.4235-4239. [50] ZHAO T S,WANG Z,CHEN C W.Adaptive quantization parameter cascading in HEVC hierarchical coding[J].IEEE Transactions on Image Processing,2016,25(7):2997-3009. [51] GONG Y C,WAN S,YANG K F,et al.Rate-distortion-optimization-based quantization parameter cascading technique for random-access configuration in H.265/HEVC[J].IEEE Transactions on Circuits and Systems for Video Technology,2017,27(6):1304-1312. [52] YANG K,WAN S,GONG Y,et al.Content adaptive quantization parameter cascading for random-access structure in HEVC[A].Proceedings of IEEE International Conference on Image Processing (ICIP)[C].Piscataway:IEEE,2017.2498-2502. [53] ZHOU Y M,WANG H Y,TIAN L,et al.Temporal corelation based hierarchical quantization parameter determination for HEVC video coding[A].Proceedings of IEEE International Conference on Image Processing (ICIP)[C].Piscataway:IEEE,2017.2478-2482. [54] HE J,YANG E H,et al.Adaptive quantization parameter selection for H.265/HEVC by employing inter-frame dependency[J].IEEE Transactions on Circuits and Systems for Video Technology,2018,28(12):3424-3436. [55] WANG M H,NGAN K N,et al.Improved block level adaptive quantization for high efficiency video coding[A].Proceedings of IEEE International Symposium on Circuits and Systems[C].Piscataway:IEEE,2015.509-512. [56] YIN H B,CAI H,FAN M T,et al.Quantization parameter cascading for video coding:leveraging a new temporal distortion propagation model[J].Signal Image and Video Processing,2017,11(5):801-808. [57] XIANG G Q,JIA H Z,YANG M Y,et al.A novel adaptive quantization method for video coding[J].Multimedia Tools and Applications,2018,77(12):14817-14840. [58] ROPERT M,TANOU J L,BICHON M,et al.R-D spatio-temporal adaptive quantization based on temporal distortion backpropagation in HEVC[A].Proceedings of IEEE International Workshop on Multimedia Signal Processing (MMSP)[C].Piscataway:IEEE,2017.1-6. [59] BICHON M,TANOU J L,ROPERT M,et al.Temporal adaptive quantization using accurate estimations of inter and skip probabilities[A].Proceedings of Picture Coding Symposium[C].Piscataway:IEEE,2018.81-85. [60] YANG T,ZHU C,FAN X,et al.Source distortion temporal propagation model for motion compensated video coding optimization[A].Proceedings of International Conference on Multimedia & Expo (ICME)[C].Piscataway:IEEE,2012.85-90. [61] GAO Y,ZHU C,LI S,et al.Temporal dependent rate-distortion optimization for low-delay hierarchical video coding[J].IEEE Transactions on Image Processing,2017,26(9):4457-4470. [62] GAO Y,ZHU C,et al.Source distortion temporal propagation analysis for random-access hierarchical video coding optimization[J].IEEE Transactions on Circuits and Systems for Video Technology,2019,29(2):546-559. [63] LI S,ZHU C,GAO Y B,et al.Lagrangian multiplier adaptation for rate-distortion optimization with inter-frame dependency[J].IEEE Transactions on Circuits and Systems for Video Technology,2016,26(1):117-129. [64] XU J Z,JOSHI R,COHEN R A.Overview of the emerging HEVC screen content coding extension[J].IEEE Transactions on Circuits and Systems for Video Technology,2016,26(1):50-62. [65] XIAO W,LI B,XU J,et al.Weighted rate-distortion optimization for screen content coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2018,28(2):499-512. [66] GONZALEZ-DE-SUSO J L,MARTINEZ-ENRIQUEZ E,DIAZ-DE-MARIA F.Adaptive Lagrange multiplier estimation algorithm in HEVC[J].Signal Processing-Image Communication,2017,56(2017):40-51. [67] YANG K F,WAN S,GONG Y C,et al.An efficient Lagrangian multiplier selection method based on temporal dependency for rate-distortion optimization in H.265/HEVC[J].Signal Processing-Image Communication,2017,57(2017):68-75. [68] ZHANG F,BULL D R.An adaptive Lagrange multiplier determination method for rate-distortion optimisation in hybrid video codecs[A].Proceedings of IEEE International Conference on Image Processing (ICIP)[C].Piscataway:IEEE,2015.671-675. [69] WANG X,SONG L,LUO Z,et al.Lagrangian method based rate-distortion optimization revisited for dependent video coding[A].Proceedings of IEEE International Conference on Image Processing (ICIP)[C].Piscataway:IEEE,2017.3021-3025. [70] DE ABREU A,CHEUNG G,FROSSARD P,et al.Optimal Lagrange multipliers for dependent rate allocation in video coding[J].Signal Processing-Image Communication,2018,63(2018):113-124. [71] 崔子冠,朱秀昌,干宗良.H.264视频编码率失真优化和码率控制技术研究进展[J].电子学报,2013,41(12):2443-2450. CUI Z G,ZHU X C,GAN Z L.Advances in rate distortion optimization and rate control techniques for h.264 video coding[J].Acta Electronica Sinica,2013,41(12):2443-2450.(in Chinese) [72] CHEN Z Z,NGAN K N.Recent advances in rate control for video coding[J].Signal Processing-Image Communication,2007,22(2007):19-38. [73] CHOI H,YOO J,NAM J,et al.Pixel-wise unified rate-quantization model for multi-level rate control[J].IEEE Journal of Selected Topics in Signal Processing,2013,7(6):1112-1123. [74] LEE B,KIM M,NGUYEN T Q.A frame-level rate control scheme based on texture and nontexture rate models for high efficiency video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(3):465-479. [75] WANG S,REHMAN A,ZENG K,et al.SSIM-motivated two-pass VBR coding for HEVC[J].IEEE Transactions on Circuits and Systems for Video Technology,2017,27(10):2189-2203. [76] ZUPANCIC I,NACCARI M,MRAK M,et al.Two-pass rate control for improved quality of experience in UHDTV delivery[J].IEEE Journal of Selected Topics in Signal Processing,2017,11(1):167-179. [77] WEN J T,FANG M Y,TANG M H,et al.R-lambda model based improved rate control for HEVC with pre-encoding[A].Proceedings of Data Compression Conference (DCC)[C].Piscataway:IEEE,2015.53-62. [78] LIU M H,REN P,XIANG Z.Frame-level bit allocation for hierarchical coding of H.265/HEVC considering dependent rate-distortion characteristics[J].Signal Image and Video Processing,2016,10(8):1457-1463. [79] LI L,LI B,LI H,et al. λ-domain optimal bit allocation algorithm for high efficiency video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2018,28(1):130-142. [80] FIENGO A,CHIERCHIA G,CAGNAZZO M,et al.Rate allocation in predictive video coding using a convex optimization framework[J].IEEE Transactions on Image Processing,2017,26(1):479-489. [81] WANG S S,MA S W,et al.Rate-GOP based rate control for high efficiency video coding[J].IEEE Journal of Selected Topics in Signal Processing,2013,7(6):1101-1111. [82] Gao W,Kwong S,Yuan H,et al.DCT coefficient distribution modeling and quality dependency analysis based frame-level bit allocation for HEVC[J].IEEE Transactions on Circuits and Systems for Video Technology,2016,26(1):139-153. [83] GUO H W,ZHU C,LI S X,et al.Optimal bit allocation at frame level for rate control in HEVC[J].IEEE Transactions on Broadcasting,2019,65(2):270-281. [84] LI S,XU M,WANG Z,et al.Optimal bit allocation for CTU level rate control in HEVC[J].IEEE Transactions on Circuits and Systems for Video Technology,2017,27(11):2409-2424. [85] GUO H W,ZHU C,XU M,et al.Inter-block dependency based CTU level rate controlfor HEVC[J].IEEE Transactions on Broadcasting,2020,66(1):113-126. [86] WIEN M,BOYCE J M,STOCKHAMMER T,et al.Standardization status of immersive video coding[J].IEEE Journal on Emerging and Selected Topics in Circuits and Systems,2019,9(1):5-17. [87] GARRIDO M J,PESCADOR F,CHAVARRÍAS M,et al.A 2-D multiple transform processor for the versatile video coding standard[J].IEEE Transactions on Consumer Electronics,2019,65(3):274-283. [88] YANG H,SHEN L,DONG X,et al.Low complexity CTU partition structure decision and fast intra mode decision for versatile video coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2019,doi:10.1109/TCSVT.2019.2904198. |
[1] | 周作成, 贾克斌. 基于3D-HEVC标准的相邻块视差矢量获取算法质量优化的研究[J]. 电子学报, 2017, 45(8): 1931-1936. |
[2] | 曹倩, 李辉勇, 左敏, 姜同强, 蔡强, 王瑜. 任务敏感的多模式视频编码系统功耗控制方法[J]. 电子学报, 2016, 44(7): 1592-1598. |
[3] | 唐振华, 梁祥严, 覃团发, 常侃. 分布式视频编码中基于多概率混合分布的相关噪声建模方法[J]. 电子学报, 2015, 43(2): 365-370. |
[4] | 陆寄远, 朝红阳, 黄承慧, 侯昉. 计算能力可伸缩的运动估计率失真优化[J]. 电子学报, 2014, 42(8): 1495-1502. |
[5] | 夏北吨, 杨春玲. 无反馈分布式视频编码中码率分配算法研究[J]. 电子学报, 2014, 42(10): 1938-1943. |
[6] | 崔子冠, 朱秀昌, 干宗良, 唐贵进, 刘峰. H.264视频编码率失真优化和码率控制技术研究进展[J]. 电子学报, 2013, 41(12): 2443-2450. |
[7] | 陈胜刚;陈书明;谷会涛;刘尧. 一种用于并行H.264编码器的语法元素级分组并行算术编码器体系结构的评估[J]. 电子学报, 2012, 40(2): 400-405. |
[8] | 高攀, 彭强, 王琼华. 基于多视点视频编码的差错控制算法[J]. 电子学报, 2012, 40(12): 2544-2548. |
[9] | 李晓峰;周宁;刘洪盛;张敏. 一种基于缩减栅格算法的SVC联合信源/信道编码方法[J]. 电子学报, 2011, 39(4): 859-864. |
[10] | 李海燕;张春元;付剑. 基于流体系结构的帧内预测算法优化设计[J]. 电子学报, 2010, 38(5): 1014-1020. |
[11] | 封 颖;李云松;吴成柯;宋 锐. 分布式视频解码器端的码率估计算法[J]. 电子学报, 2009, 37(6): 1232-1236. |
[12] | 宋立锋;戴青云. H.264实时编码的指令Cache优化[J]. 电子学报, 2008, 36(8): 1615-1619. |
[13] | 宋建斌, 李 波, 李 炜, 吴 波. 适用于H.264/AVC的快速帧内预测算法[J]. 电子学报, 2007, 35(4): 668-672. |
[14] | 黎洪松, 许保华. 一种用于视频对象编码的运动模式识别算法[J]. 电子学报, 2007, 35(12): 2324-2328. |
[15] | 干宗良, 齐丽娜, 朱秀昌. 一种空间域Wyner-Ziv视频编码系统的性能改进算法[J]. 电子学报, 2007, 35(10): 2014-2018. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||