基于深度增强学习与子流耦合感知的多路传输控制机制

秦久人; 许长桥; 杨树杰; 高楷; 张宏科

doi:10.12263/DZXB.20201414

您当前的位置：

首页 >

文章列表页 >

基于深度增强学习与子流耦合感知的多路传输控制机制

学术论文 | 更新时间：2025-12-08

- 基于深度增强学习与子流耦合感知的多路传输控制机制
- Multipath Transmission Control Mechanism Based on Deep Reinforcement Learning and Sub-flow Coupling Perception
- 电子学报 2022年50卷第2期页码：346-357
- 作者机构：
  
  1.北京邮电大学网络与交换技术国家重点实验室，北京 100876
  2.北京交通大学电子信息工程学院，北京 100044
- 作者简介：
  
  [ "秦久人女，1992年生于河北省廊坊市.北京邮电大学计算机学院网络与交换技术国家重点实验室博士研究生.主要研究方向为多路径传输、人工智能、多媒体通信等. E-mail: jrqin@bupt.edu.cn" ]
  [ "许长桥（通讯作者）男，1977年生于江西省吉安市.博士，教授、博士生导师.北京邮电大学网络与交换技术国家重点实验室网络体系结构研究中心主任.主要研究方向为移动互联网、未来网络、网络多媒体、网络安全. E-mail: cqxu@bupt.edu.cn" ]
  [ "杨树杰男，1989年生于山东省烟台市.博士，讲师、硕士生导师.主要研究方向为移动互联网、网络空间安全、网络资源优化等. E-mail: sjyang@bupt.edu.cn" ]
  [ "高楷男，1992年生于河北省邯郸市.北京邮电大学计算机学院网络与交换技术国家重点实验室博士研究生.主要研究方向为多路径传输、软件定义网络、博弈论等. E-mail: gaokai@bupt.edu.cn" ]
  [ "张宏科男，1957年生于山西省大同市.博士，教授、博士生导师.北京交通大学下一代互联网络互联设备国家工程实验室主任.国家973项目“一体化可信网络与普适服务体系基础研究”与“智慧协同网络理论基础研究”首席科学家.主要研究方向为下一代信息网络关键理论与技术.E-mail: hkzhang@bjtu.edu.cn" ]
- 基金信息：
  
  国家自然科学基金资助项目(61871048;61872253;62001057);国家重点研发项目(2018YFE0205502)
- DOI：10.12263/DZXB.20201414
  中图分类号： TP393;
- 收稿：2020-12-09，
  
  修回：2021-03-04，
  
  纸质出版：2022-02-25
- 稿件说明：
移动端阅览
秦久人,许长桥,杨树杰等.基于深度增强学习与子流耦合感知的多路传输控制机制[J].电子学报,2022,50(02):346-357.

QIN Jiu-ren,XU Chang-qiao,YANG Shu-jie,et al.Multipath Transmission Control Mechanism Based on Deep Reinforcement Learning and Sub-flow Coupling Perception[J].ACTA ELECTRONICA SINICA,2022,50(02):346-357.
秦久人,许长桥,杨树杰等.基于深度增强学习与子流耦合感知的多路传输控制机制[J].电子学报,2022,50(02):346-357. DOI： 10.12263/DZXB.20201414.

QIN Jiu-ren,XU Chang-qiao,YANG Shu-jie,et al.Multipath Transmission Control Mechanism Based on Deep Reinforcement Learning and Sub-flow Coupling Perception[J].ACTA ELECTRONICA SINICA,2022,50(02):346-357. DOI： 10.12263/DZXB.20201414.

摘要

为了解决多路传输中子流耦合感知缺乏与传输控制效率低下等的问题，针对未来异构、动态的网络环境，提出了一种基于耦合感知与深度Q网络的多路传输控制机制（WaveLet and Deep Q Network based multipath transmission control mechanism，WL-DQN）.利用小波去噪技术，消除子流单向传输时延中由非耦合路段及系统随机产生的噪声，并基于子流互相关系数对子流耦合特性进行提取；在此基础上，依据深度增强学习理论对多路传输控制进行建模，并提出多路DQN拥塞控制算法，实现了异构、动态网络环境下的智能多路拥塞控制.仿真结果表明，所提算法在传输吞吐量、传输时延、数据包重传避免等方面均优于标准及相似的代表性解决方案.

Abstract

To solve the problems of lacking coupled sub-flow perception and low transmission control efficiency for multipath transmission

a multipath transmission control mechanism based on wavelet and deep Q network(WL-DQN) is proposed for future heterogeneous and dynamic networks. Wavelet techniques are adopted to eliminate the noise caused by uncoupled queueing and system random delay in the one-way transmission delay

and the coupling characteristics are extracted based on the path correlation factors. On this basis

the deep reinforcement learning theories are applied to model the multipath transmission control

and the multipath DQN control algorithm is developed to realize the intelligent multipath congestion control under the heterogeneous and dynamic networks. Simulation results demonstrate that the proposed algorithms outperform the standard and similar representative solutions in terms of transmission throughput

transmission delay

packet retransmission avoidance

etc.

关键词

Keywords

references

LIU Y , YUAN X , XIONG Z , et al . Federated learning for 6G communications: Challenges, methods, and future directions [J]. China Communications , 2020 , 17 ( 9 ): 105 - 118 .

Ford A . TCP extensions for multipath operation with multiple addresses [EB/OL]. ( 2020-03 )[ 2020-08 ]. https://datatracker.ietf.org/doc/html/rfc8684.html https://datatracker.ietf.org/doc/html/rfc8684.html .

CAO Y , GUO H , LIU J , et al . Optimal satellite gateway placement in space-ground integrated networks [J]. IEEE Network , 2018 , 32 ( 5 ): 32 - 37 .

CHOO K R , GRITZALIS S , PARK J H . Cryptographic solutions for industrial internet-of-things: Research challenges and opportunities [J]. IEEE Transactions on Industrial Informatics , 2018 , 14 ( 8 ): 3567 - 3569 .

KHABBAZ M , ANTOUN J , ASSI C . Modeling and performance analysis of UAV-assisted vehicular networks [J]. IEEE Transactions on Vehicular Technology , 2019 , 68 ( 9 ): 8384 - 8396 .

FERLIN S , ALAY Ö , DREIBHOLZ T , et al . Revisiting congestion control for Multipath TCP with shared bottleneck Detection [C]// IEEE International Conference on Computer Communications(INFOCOM) . San Francisco, CA, USA : IEEE , 2016 : 1 - 9 .

Raiciu C . Coupled Congestion Control for Multipath Transport Protocols [EB/OL]. ( 2011-10 )[ 2020-08 ]. https://datatracker.ietf.org/doc/html/rfc6356.html https://datatracker.ietf.org/doc/html/rfc6356.html .

OH B H , LEE J . Feedback-based path failure detection and buffer blocking protection for MPTCP [J]. IEEE/ACM Transactions on Networking , 2016 , 24 ( 6 ): 3450 - 3461 .

ZHAO J , LIU J , WANG H , et al . Measurement, analysis, and enhancement of multipath TCP energy efficiency for datacenters [J]. IEEE/ACM Transactions on Networking , 2020 , 28 ( 1 ): 57 - 70 .

XU C , LI Z , LI J , et al . Cross-layer fairness-driven concurrent multipath video delivery over heterogeneous wireless networks [J]. IEEE Transactions on Circuits & Systems for Video Technology , 2015 , 25 ( 7 ): 1175 - 1189 .

XU C , WANG P , XIONG C , et al . Pipeline network coding-based multipath data transfer in heterogeneous wireless networks [J]. IEEE Transactions on Broadcasting , 2017 , 63 ( 2 ): 376 - 390 .

XU C , JIA S , WANG M , et al . Performance-aware mobile community-based VoD streaming over vehicular Ad Hoc networks [J]. IEEE Transactions on Vehicular Technology , 2015 , 64 ( 3 ): 1201 - 1217 .

DONOHO D L , JOHNSTONE I M . Ideal spatial adaptation by wavelet shrinkage [J]. Biometrika , 1994 , 81 ( 3 ): 425 - 455 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J]. Nature , 2015 , 518 ( 7540 ): 529 - 533 .

SADIO O , NGOM I , LISHOU C . Design and prototyping of a software defined vehicular networking [J]. IEEE Transactions on Vehicular Technology , 2020 , 69 ( 1 ): 842 - 850 .

GE X . Ultra-reliable low-latency communications in autonomous vehicular networks [J]. IEEE Transactions on Vehicular Technology , 2019 , 68 ( 5 ): 5005 - 5016 .

李红艳 , 张焘 , 张靖乾 , 等 . 基于时变图的天地一体化网络时间确定性路由算法与协议 [J]. 通信学报 , 2020 , 41 ( 10 ): 116 - 129 .

LI H Y , ZHANG T , ZHANG J Q , et al . Time deterministic routing algorithm and protocol based on time-varying graph over the space-ground integrated network [J]. Journal on Communications , 2020 , 41 ( 10 ): 116 - 129 . (in Chinese)

HUI H , ZHOU C , XU S , et al . A novel secure data transmission scheme in industrial internet of things [J]. China Communications , 2020 , 17 ( 1 ): 73 - 88 .

LI X , LI D , WAN J , et al . Adaptive transmission optimization in SDN-based industrial internet of things with edge computing [J]. IEEE Internet of Things Journal , 2018 , 5 ( 3 ): 1351 - 1360 .

孙鹏浩 , 兰巨龙 , 申涓 , 等 . 一种基于深度增强学习的智能路由技术 [J]. 电子学报 , 2020 , 48 ( 11 ): 92 - 99 .

SUN P H , LAN J L , SHEN J , et al . An intelligent routing technology based on deep reinforcement learning [J]. Acta Electronica Sinica , 2020 , 48 ( 11 ): 92 - 99 . (in Chinese)

POKHREL S R , CHOI J . Low-delay scheduling for internet of vehicles: Load-balanced multipath communication with FEC [J]. IEEE Transactions on Communications , 2019 , 67 ( 12 ): 8489 - 8501 .

SINGH P K , SHARMA S , NANDI S K , et al . Multipath TCP for V2I communication in SDN controlled small cell deployment of smart city [J]. Vehicular Communications , 2019 , 15 ( 1 ): 1 - 15 .

NING Z , FENG Y , COLLOTTA M , et al . Deep learning in edge of vehicles: Exploring trirelationship for data transmission [J]. IEEE Transactions on Industrial Informatics , 2019 , 15 ( 10 ): 5737 - 5746 .

LI W , ZHANG H , GAO S , et al . SmartCC: A reinforcement learning approach for multipath TCP congestion control in heterogeneous networks [J]. IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 11 ): 2621 - 2633 .

XU Z , TANG J , YIN C , et al . Experience-driven congestion control: When multi-path TCP meets deep reinforcement learning [J]. IEEE Journal on Selected Areas in Communications , 2019 , 37 ( 6 ): 1325 - 1336 .

WINSTEIN K , BALAKRISHNAN H . TCP ex machina: Computer-generated congestion control [C]// Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM . New York, NY, USA : ACM , 2013 : 123 - 134 .

KASHIF N . Mptcp implementation in ns3 [EB/OL]. ( 2019-04-05 )[ 2021-12-30 ]. https://github.com/Kashif-Nadeem/ns-3-dev-git https://github.com/Kashif-Nadeem/ns-3-dev-git .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于无线传感器网络的跨层拥塞控制协议

一种基于预测PI控制器的自相似网络主动队列管理算法