面向全息视频通信的自适应分块传输方法

朱原玮; 黄亚坤; 乔秀全

doi:10.12263/DZXB.20230788

您当前的位置：

首页 >

文章列表页 >

面向全息视频通信的自适应分块传输方法

中国电子学会科学技术奖特约专栏 | 更新时间：2025-12-08

- 面向全息视频通信的自适应分块传输方法
- Towards Holographic Video Communications: An Adaptive Tiling Solution
- 电子学报 2024年52卷第4期页码：1144-1154
- 作者机构：
  
  北京邮电大学网络与交换技术全国重点实验室，北京 100876
- 作者简介：
  
  朱原玮男，1997年1月出生于安徽省亳州市.现为北京邮电大学计算机学院（国家示范性软件学院）、网络与交换技术全国重点实验室博士研究生.主要研究方向为体积视频、沉浸式视频传输等.
  黄亚坤男，1992年9月出生于安徽省合肥市.现为北京邮电大学计算机学院（国家示范性软件学院）副研究员.主要研究方向为体积视频、移动计算、增强现实等.在国内外期刊及会议上发表学术论文20余篇.
  乔秀全男，1978年6月出生于山西省汾阳市.现为北京邮电大学网络与交换技术全国重点实验室教授，博士生导师.北京邮电大学信息化技术中心副主任（主持工作）.北京市科技新星，牵头获中国电子学会技术发明一等奖.主持了6G重点研发计划课题、国家重点基础研究发展计划（973计划）课题等10多项国家纵向项目.主要研究方向为未来互联网、服务计算、计算机视觉、分布式深度学习、增强现实、虚拟现实和5G网络等.在国内外期刊及会议上合作发表学术论文60余篇.中国电子学会会员编号：E190156601M.E-mail: qiaoxq@bupt.edu.cn
- 基金信息：
  
  国家重点研发计划(2022YFB2902900);国家自然科学基金(62202065)
- DOI：10.12263/DZXB.20230788
  中图分类号： TP37;TP393.0
- 收稿：2023-08-16，
  
  修回：2023-12-05，
  
  纸质出版：2024-04-25
- 稿件说明：
移动端阅览
朱原玮,黄亚坤,乔秀全.面向全息视频通信的自适应分块传输方法[J].电子学报,2024,52(04):1144-1154.

ZHU Yuan-wei, HUANG Ya-kun, QIAO Xiu-quan.Towards Holographic Video Communications: An Adaptive Tiling Solution[J].Acta Electronica Sinica, 2024, 52(04): 1144-1154.
朱原玮,黄亚坤,乔秀全.面向全息视频通信的自适应分块传输方法[J].电子学报,2024,52(04):1144-1154. DOI：10.12263/DZXB.20230788

ZHU Yuan-wei, HUANG Ya-kun, QIAO Xiu-quan.Towards Holographic Video Communications: An Adaptive Tiling Solution[J].Acta Electronica Sinica, 2024, 52(04): 1144-1154. DOI：10.12263/DZXB.20230788

摘要

基于分治和按需传输思想的分块传输技术是解决三维全息视频流传输的有效手段.然而，现有的分块方案要么缺乏自适应机制，要么不适用于移动实时通信场景.为此，本文提出了VVSTiler（Volumetric Video Streaming Tiling selector），一种面向全息视频通信的自适应分块传输方法，能够在动态且有限的计算和带宽资源下最大化视频的观感质量.具体而言，本文对不同粒度的分块方案带来的影响进行了初步研究，发现细粒度的分块方案可提高动态网络资源的利用率，粗粒度的分块方案可保证视频编解码效率和鲁棒性.基于此，本文构建了考虑预测视口、可用计算资源以及网络带宽等上下文信息的视频观感质量优化问题，并设计了一个高效的求解方案以支持在线的分块粒度决策.本文在8iVFB（8i Voxelized Full Bodies）标准数据集上将VVSTiler与当前主流的分块传输方法进行了比较.实验结果表明，VVSTiler在有偏差的视口预测情况下实现了高达60.4%的视频观感质量提升，在较准确的视口预测情况下平均每帧视频节省了27%的带宽资源.

Abstract

Tile-based methods that use the divide-and-conquer and on-demand transmission techniques are promising to handle 3D holographic video streaming. However

the current solutions either lack an adaptive tiling scheme or cannot apply to mobile real-time scenarios. In this paper

we propose VVSTiler (Volumetric Video Streaming Tiling selector)

an adaptive tiling selector for holographic video communications

which can adaptively maximize perceived video quality under dynamic and limited computing and bandwidth resources. To be specific

we first conduct a preliminary study on the impacts of different tiling schemes and find that fine-grained tiles improve the rational utilization of dynamic network resources and coarse tiles ensure coding efficiency and robustness

which stimulates us to construct an adaptive tiling optimization based on the predicted viewport

available computing resources

and network bandwidth; and then devise a fast algorithm to enable online tiling decisions. Rich experiments on the 8iVFB (8i Voxelized Full Bodies) datasets are conducted to compare VVSTiler with state-of-the-art tiling-based baselines. The results exhibit that VVSTiler can achieve up to 60.4% video quality improvements and save on average 27% bandwidth per frame against the closest competitor

in cases of terrible and accurate viewport predictions

respectively.

关键词

Keywords

references

QIAN F , HAN B , PAIR J , et al . Toward practical volumetric video streaming on commodity smartphones [C ] // Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications . New York : ACM , 2019 : 135 - 140 .

LIU Z , LI Q Y , CHEN X F , et al . Point cloud video streaming: Challenges and solutions [J ] . IEEE Network , 2021 , 35 ( 5 ): 202 - 209 .

RUSU R B , COUSINS S . 3D is here: Point cloud library (PCL) [C ] // 2011 IEEE International Conference on Robotics and Automation . Piscataway : IEEE , 2011 : 1 - 4 .

GOOGLE . Draco [EB/OL ] . ( 2017-04-15 )[ 2023-07-14 ] . https://github.com/google/draco https://github.com/google/draco .

GRAZIOSI D , NAKAGAMI O , KUMA S , et al . An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC) [J ] . APSIPA Transactions on Signal and Information Processing , 2020 , 9 ( 1 ): e13 .

HOSSEINI M , TIMMERER C . Dynamic adaptive point cloud streaming [C ] // Proceedings of the 23rd Packet Video Workshop . New York : ACM , 2018 : 25 - 30 .

VAN DER HOOFT J , WAUTERS T , DE TURCK F , et al . Towards 6DoF HTTP adaptive streaming through point cloud compression [C ] // Proceedings of the 27th ACM International Conference on Multimedia . New York : ACM , 2019 : 2405 - 2413 .

WANG L S , LI C L , DAI W R , et al . QoE-driven adaptive streaming for point clouds [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 2543 - 2558 .

HAN B , LIU Y , QIAN F . ViVo: Visibility-aware mobile volumetric video streaming [C ] // Proceedings of the 26th Annual International Conference on Mobile Computing and Networking . New York : ACM , 2020 : 1 - 13 .

LEE K , YI J , LEE Y , et al . GROOT: A real-time streaming system of high-fidelity volumetric videos [C ] // Proceedings of the 26th Annual International Conference on Mobile Computing and Networking . New York : ACM , 2020 : 1 - 14 .

LI J , ZHANG C , LIU Z , et al . Joint communication and computational resource allocation for QoE-driven point cloud video streaming [C ] // ICC 2020 - 2020 IEEE International Conference on Communications (ICC) . Piscataway : IEEE , 2020 : 1 - 6 .

PARK J , CHOU P A , HWANG J N . Volumetric media streaming for augmented reality [C ] // 2018 IEEE Global Communications Conference (GLOBECOM) . Piscataway : IEEE , 2018 : 1 - 6 .

PARK J , CHOU P A , HWANG J N . Rate-utility optimized streaming of volumetric media for augmented reality [J ] . IEEE Journal on Emerging and Selected Topics in Circuits and Systems , 2019 , 9 ( 1 ): 149 - 162 .

SUBRAMANYAM S , VIOLA I , HANJALIC A , et al . User centered adaptive streaming of dynamic point clouds with low complexity tiling [C ] // Proceedings of the 28th ACM International Conference on Multimedia . New York : ACM , 2020 : 3669 - 3677 .

LI J , ZHANG C , LIU Z , et al . Optimal volumetric video streaming with hybrid saliency based tiling [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 2939 - 2953 .

D'EON E , HARRISON B , MYERS T , et al . 8i voxelized full bodies—A voxelized point cloud dataset [J ] . ISO/IEC JTC 1 /SC29 Joint WG11/WG1 (MPEG/JPEG) Input Document WG11M40059/WG1M74006, 2017, 7 ( 8 ): 11 .

ESRI . Limited error point cloud compression [EB/OL ] . ( 2018 )[ 2023-07-14 ] . https://github.com/Esri/lepcc/ https://github.com/Esri/lepcc/ .

QUE Z Z , LU G , XU D . VoxelContext-Net: An Octree based Framework for Point Cloud Compression [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 6038 - 6047 .

HUBO E , MERTENS T , HABER T , et al . The quantized kd-tree: Efficient ray tracing of compressed point clouds [C ] // 2006 IEEE Symposium on Interactive Ray Tracing . Piscataway : IEEE , 2006 : 105 - 113 .

ZHANG L , SUO Y Y , WU X M , et al . TBRA: Tiling and bitrate adaptation for mobile 360-degree video streaming [C ] // Proceedings of the 29th ACM International Conference on Multimedia . New York : ACM , 2021 : 4007 - 4015 .

XIE L , XU Z M , BAN Y X , et al . 360ProbDASH: Improving QoE of 360 video streaming using tile-based HTTP adaptive streaming [C ] // Proceedings of the 25th ACM international conference on Multimedia . New York : ACM , 2017 : 315 - 323 .

YADAV P K , OOI W T . Tile rate allocation for 360-degree tiled adaptive video streaming [C ] // Proceedings of the 28th ACM International Conference on Multimedia . New York : ACM , 2020 : 3724 - 3733 .

XIAO M B , ZHOU C , LIU Y , et al . OpTile: Toward optimal tiling in 360-degree video streaming [C ] // Proceedings of the 25th ACM international conference on Multimedia . New York : ACM , 2017 : 708 - 716 .

KELLERER H , PFERSCHY U , PISINGER D . Introduction to NP-completeness of knapsack problems [M ] // Knapsack Problems . Berlin : Springer , 2004 : 483 - 493 .

POULARAKIS K , IOSIFIDIS G , ARGYRIOU A , et al . Caching and operator cooperation policies for layered video content delivery [C ] // IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications . Piscataway : IEEE , 2016 : 1 - 9 .

ASSARSSON U , MÖLLER T . Optimized view frustum culling algorithms for bounding boxes [J ] . Journal of Graphics Tools , 2000 , 5 ( 1 ): 9 - 22 .

Lighthouse 3 d . View frustum culling [EB/OL ] .( 2017 )[ 2023-07-14 ] . http://www.lighthouse3d.com/tutorials/view-frustum-culling/index/ http://www.lighthouse3d.com/tutorials/view-frustum-culling/index/ .

VAN DER HOOFT J , PETRANGELI S , WAUTERS T , et al . HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks [J ] . IEEE Communications Letters , 2016 , 20 ( 11 ): 2177 - 2180 .

HOU X S , ZHANG J Z , BUDAGAVI M , et al . Head and body motion prediction to enable mobile VR experiences with low latency [C ] // 2019 IEEE Global Communications Conference (GLOBECOM) . Piscataway : IEEE , 2019 : 1 - 7 .

JAMALI M , COULOMBE S , VAKILI A , et al . LSTM-based viewpoint prediction for multi-quality tiled video coding in virtual reality streaming [C ] // 2020 IEEE International Symposium on Circuits and Systems (ISCAS) . Piscataway : IEEE , 2020 : 1 - 5 .

HORÉ A , ZIOU D . Image quality metrics: PSNR vs. SSIM [C ] // 2010 20th International Conference on Pattern Recognition . Piscataway : IEEE , 2010 : 2366 - 2369 .

HUANG Y K , ZHU Y W , QIAO X Q , et al . Toward holographic video communications: A promising AI-driven solution [J ] . IEEE Communications Magazine , 2022 , 60 ( 11 ): 82 - 88 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

AIOS综述：安全可信的人工智能原生操作系统

面向医疗缺失数据的选择性学习

一种基于自偏置技术的低功耗低抖动SerDes时钟源锁相环设计

高集成度多模式单级AC-DC BUCK转换器

泛在计算环境下模型推理输出长度预测