Towards Holographic Video Communications: An Adaptive Tiling Solution

ZHU Yuan-wei; HUANG Ya-kun; QIAO Xiu-quan

doi:10.12263/DZXB.20230788

您当前的位置：

首页 >

文章列表页 >

Towards Holographic Video Communications: An Adaptive Tiling Solution

Special Issue: Recipients of CIE Science and Technology Awards | 更新时间：2025-12-08

- Towards Holographic Video Communications: An Adaptive Tiling Solution
- ACTA ELECTRONICA SINICA Vol. 52, Issue 4, Pages: 1144-1154(2024)
- 作者机构：
  
  北京邮电大学网络与交换技术全国重点实验室，北京 100876
- 作者简介：
- 基金信息：
  
  National Key R&D Program of China(2022YFB2902900);National Natural Science Foundation of China(62202065)
- DOI：10.12263/DZXB.20230788
  CLC： TP37;TP393.0
- Received：16 August 2023，
  
  Revised：2023-12-05，
  
  Published：25 April 2024
- 稿件说明：
移动端阅览
朱原玮,黄亚坤,乔秀全.面向全息视频通信的自适应分块传输方法[J].电子学报,2024,52(04):1144-1154.

ZHU Yuan-wei, HUANG Ya-kun, QIAO Xiu-quan.Towards Holographic Video Communications: An Adaptive Tiling Solution[J].Acta Electronica Sinica, 2024, 52(04): 1144-1154.
朱原玮,黄亚坤,乔秀全.面向全息视频通信的自适应分块传输方法[J].电子学报,2024,52(04):1144-1154. DOI：10.12263/DZXB.20230788

ZHU Yuan-wei, HUANG Ya-kun, QIAO Xiu-quan.Towards Holographic Video Communications: An Adaptive Tiling Solution[J].Acta Electronica Sinica, 2024, 52(04): 1144-1154. DOI：10.12263/DZXB.20230788

摘要

基于分治和按需传输思想的分块传输技术是解决三维全息视频流传输的有效手段.然而，现有的分块方案要么缺乏自适应机制，要么不适用于移动实时通信场景.为此，本文提出了VVSTiler（Volumetric Video Streaming Tiling selector），一种面向全息视频通信的自适应分块传输方法，能够在动态且有限的计算和带宽资源下最大化视频的观感质量.具体而言，本文对不同粒度的分块方案带来的影响进行了初步研究，发现细粒度的分块方案可提高动态网络资源的利用率，粗粒度的分块方案可保证视频编解码效率和鲁棒性.基于此，本文构建了考虑预测视口、可用计算资源以及网络带宽等上下文信息的视频观感质量优化问题，并设计了一个高效的求解方案以支持在线的分块粒度决策.本文在8iVFB（8i Voxelized Full Bodies）标准数据集上将VVSTiler与当前主流的分块传输方法进行了比较.实验结果表明，VVSTiler在有偏差的视口预测情况下实现了高达60.4%的视频观感质量提升，在较准确的视口预测情况下平均每帧视频节省了27%的带宽资源.

Abstract

Tile-based methods that use the divide-and-conquer and on-demand transmission techniques are promising to handle 3D holographic video streaming. However

the current solutions either lack an adaptive tiling scheme or cannot apply to mobile real-time scenarios. In this paper

we propose VVSTiler (Volumetric Video Streaming Tiling selector)

an adaptive tiling selector for holographic video communications

which can adaptively maximize perceived video quality under dynamic and limited computing and bandwidth resources. To be specific

we first conduct a preliminary study on the impacts of different tiling schemes and find that fine-grained tiles improve the rational utilization of dynamic network resources and coarse tiles ensure coding efficiency and robustness

which stimulates us to construct an adaptive tiling optimization based on the predicted viewport

available computing resources

and network bandwidth; and then devise a fast algorithm to enable online tiling decisions. Rich experiments on the 8iVFB (8i Voxelized Full Bodies) datasets are conducted to compare VVSTiler with state-of-the-art tiling-based baselines. The results exhibit that VVSTiler can achieve up to 60.4% video quality improvements and save on average 27% bandwidth per frame against the closest competitor

in cases of terrible and accurate viewport predictions

respectively.

关键词

Keywords

references

QIAN F , HAN B , PAIR J , et al . Toward practical volumetric video streaming on commodity smartphones [C ] // Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications . New York : ACM , 2019 : 135 - 140 .

LIU Z , LI Q Y , CHEN X F , et al . Point cloud video streaming: Challenges and solutions [J ] . IEEE Network , 2021 , 35 ( 5 ): 202 - 209 .

RUSU R B , COUSINS S . 3D is here: Point cloud library (PCL) [C ] // 2011 IEEE International Conference on Robotics and Automation . Piscataway : IEEE , 2011 : 1 - 4 .

GOOGLE . Draco [EB/OL ] . ( 2017-04-15 )[ 2023-07-14 ] . https://github.com/google/draco https://github.com/google/draco .

GRAZIOSI D , NAKAGAMI O , KUMA S , et al . An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC) [J ] . APSIPA Transactions on Signal and Information Processing , 2020 , 9 ( 1 ): e13 .

HOSSEINI M , TIMMERER C . Dynamic adaptive point cloud streaming [C ] // Proceedings of the 23rd Packet Video Workshop . New York : ACM , 2018 : 25 - 30 .

VAN DER HOOFT J , WAUTERS T , DE TURCK F , et al . Towards 6DoF HTTP adaptive streaming through point cloud compression [C ] // Proceedings of the 27th ACM International Conference on Multimedia . New York : ACM , 2019 : 2405 - 2413 .

WANG L S , LI C L , DAI W R , et al . QoE-driven adaptive streaming for point clouds [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 2543 - 2558 .

HAN B , LIU Y , QIAN F . ViVo: Visibility-aware mobile volumetric video streaming [C ] // Proceedings of the 26th Annual International Conference on Mobile Computing and Networking . New York : ACM , 2020 : 1 - 13 .

LEE K , YI J , LEE Y , et al . GROOT: A real-time streaming system of high-fidelity volumetric videos [C ] // Proceedings of the 26th Annual International Conference on Mobile Computing and Networking . New York : ACM , 2020 : 1 - 14 .

LI J , ZHANG C , LIU Z , et al . Joint communication and computational resource allocation for QoE-driven point cloud video streaming [C ] // ICC 2020 - 2020 IEEE International Conference on Communications (ICC) . Piscataway : IEEE , 2020 : 1 - 6 .

PARK J , CHOU P A , HWANG J N . Volumetric media streaming for augmented reality [C ] // 2018 IEEE Global Communications Conference (GLOBECOM) . Piscataway : IEEE , 2018 : 1 - 6 .

PARK J , CHOU P A , HWANG J N . Rate-utility optimized streaming of volumetric media for augmented reality [J ] . IEEE Journal on Emerging and Selected Topics in Circuits and Systems , 2019 , 9 ( 1 ): 149 - 162 .

SUBRAMANYAM S , VIOLA I , HANJALIC A , et al . User centered adaptive streaming of dynamic point clouds with low complexity tiling [C ] // Proceedings of the 28th ACM International Conference on Multimedia . New York : ACM , 2020 : 3669 - 3677 .

LI J , ZHANG C , LIU Z , et al . Optimal volumetric video streaming with hybrid saliency based tiling [J ] . IEEE Transactions on Multimedia , 2022 , 25 : 2939 - 2953 .

D'EON E , HARRISON B , MYERS T , et al . 8i voxelized full bodies—A voxelized point cloud dataset [J ] . ISO/IEC JTC 1 /SC29 Joint WG11/WG1 (MPEG/JPEG) Input Document WG11M40059/WG1M74006, 2017, 7 ( 8 ): 11 .

ESRI . Limited error point cloud compression [EB/OL ] . ( 2018 )[ 2023-07-14 ] . https://github.com/Esri/lepcc/ https://github.com/Esri/lepcc/ .

QUE Z Z , LU G , XU D . VoxelContext-Net: An Octree based Framework for Point Cloud Compression [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 6038 - 6047 .

HUBO E , MERTENS T , HABER T , et al . The quantized kd-tree: Efficient ray tracing of compressed point clouds [C ] // 2006 IEEE Symposium on Interactive Ray Tracing . Piscataway : IEEE , 2006 : 105 - 113 .

ZHANG L , SUO Y Y , WU X M , et al . TBRA: Tiling and bitrate adaptation for mobile 360-degree video streaming [C ] // Proceedings of the 29th ACM International Conference on Multimedia . New York : ACM , 2021 : 4007 - 4015 .

XIE L , XU Z M , BAN Y X , et al . 360ProbDASH: Improving QoE of 360 video streaming using tile-based HTTP adaptive streaming [C ] // Proceedings of the 25th ACM international conference on Multimedia . New York : ACM , 2017 : 315 - 323 .

YADAV P K , OOI W T . Tile rate allocation for 360-degree tiled adaptive video streaming [C ] // Proceedings of the 28th ACM International Conference on Multimedia . New York : ACM , 2020 : 3724 - 3733 .

XIAO M B , ZHOU C , LIU Y , et al . OpTile: Toward optimal tiling in 360-degree video streaming [C ] // Proceedings of the 25th ACM international conference on Multimedia . New York : ACM , 2017 : 708 - 716 .

KELLERER H , PFERSCHY U , PISINGER D . Introduction to NP-completeness of knapsack problems [M ] // Knapsack Problems . Berlin : Springer , 2004 : 483 - 493 .

POULARAKIS K , IOSIFIDIS G , ARGYRIOU A , et al . Caching and operator cooperation policies for layered video content delivery [C ] // IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications . Piscataway : IEEE , 2016 : 1 - 9 .

ASSARSSON U , MÖLLER T . Optimized view frustum culling algorithms for bounding boxes [J ] . Journal of Graphics Tools , 2000 , 5 ( 1 ): 9 - 22 .

Lighthouse 3 d . View frustum culling [EB/OL ] .( 2017 )[ 2023-07-14 ] . http://www.lighthouse3d.com/tutorials/view-frustum-culling/index/ http://www.lighthouse3d.com/tutorials/view-frustum-culling/index/ .

VAN DER HOOFT J , PETRANGELI S , WAUTERS T , et al . HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks [J ] . IEEE Communications Letters , 2016 , 20 ( 11 ): 2177 - 2180 .

HOU X S , ZHANG J Z , BUDAGAVI M , et al . Head and body motion prediction to enable mobile VR experiences with low latency [C ] // 2019 IEEE Global Communications Conference (GLOBECOM) . Piscataway : IEEE , 2019 : 1 - 7 .

JAMALI M , COULOMBE S , VAKILI A , et al . LSTM-based viewpoint prediction for multi-quality tiled video coding in virtual reality streaming [C ] // 2020 IEEE International Symposium on Circuits and Systems (ISCAS) . Piscataway : IEEE , 2020 : 1 - 5 .

HORÉ A , ZIOU D . Image quality metrics: PSNR vs. SSIM [C ] // 2010 20th International Conference on Pattern Recognition . Piscataway : IEEE , 2010 : 2366 - 2369 .

HUANG Y K , ZHU Y W , QIAO X Q , et al . Toward holographic video communications: A promising AI-driven solution [J ] . IEEE Communications Magazine , 2022 , 60 ( 11 ): 82 - 88 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Highly Integrated and Multi-Mode Single-Stage AC-DC BUCK Converter

Remote Traffic Fingerprinting Attack Based on Cell Sequence Reconstruction

Exploring Freshness Modeling for Encrypted Traffic Analysis: From “Data Retention” to “Value Accumulation”

“面向人机物融合场景的泛在操作系统与环境研究进展”专栏

Test-Time Adaptation of Graph Neural Networks under Distribution Shifts

Related Author

QIAO Xiu-quan

Related Institution

No data

⁰