基于Q学习的蜂窝车联网边缘计算系统PC-5/Uu接口联合卸载策略

冯伟杨; 林思雨; 冯婧涛; 李赟; 孔繁鹏; 艾渤

doi:10.12263/DZXB.20220922

您当前的位置：

首页 >

文章列表页 >

基于Q学习的蜂窝车联网边缘计算系统PC-5/Uu接口联合卸载策略

学术论文 | 更新时间：2025-12-11

- 基于Q学习的蜂窝车联网边缘计算系统PC-5/Uu接口联合卸载策略
- Q-Learning Based Joint PC-5/Uu Offloading Strategy for C-V2X Based Vehicular Edge Computing System
- 电子学报 2024年52卷第2期页码：385-395
- 作者机构：
  
  1.北京交通大学电子信息工程学院,北京 100044
  2.轨道交通安全协同创新中心,北京 100044
  3.中国铁路信息科技集团有限公司信息调度中心,北京 100089
  4.中铁信(北京)网络技术研究院有限公司信息技术研究室,北京 100089
  5.智慧高铁系统前沿科学中心,北京 100044
- 作者简介：
  
  [ "冯伟杨男，1996年11月出生，山东临沂人．现为北京交通大学电子信息工程学院博士研究生．主要研究方向为车联网与边缘计算． E-mail: wyfeng@bjtu.edu.cn" ]
  [ "林思雨男，1984年12月出生，北京人．现为北京交通大学电子信息工程学院教授、博士生导师．主要研究方向为无线通信、车联网与轨道交通专用通信．中国电子学会会员编号：E190014444S． E-mail: sylin@bjtu.edu.cn" ]
  [ "冯婧涛女，1997年12月出生，河北邢台人．2022年获得北京交通大学电子信息工程学院硕士学位．主要研究方向为车联网与边缘计算． E-mail: 857382473@qq.com" ]
  [ "李赟男，1982年3月出生，山西太原人．现为中国铁路信息科技集团有限公司信息调度中心副主任、高级工程师．主要研究方向为计算机网络技术． E-mail: liyun@sinorail.com" ]
  [ "孔繁鹏男，1982年4月出生，北京人.现为中铁信（北京）网络技术研究院有限公司信息技术研究室主任．主要研究方向为信息网络技术、通信技术及网络运维技术． E-mail: kongfanpeng@sinorail.com" ]
  [ "艾渤男，1974年2月出生，陕西西安人．现为北京交通大学教授、博士生导师，电子信息工程学院院长.主要研究方向为宽带移动通信系统与专用移动通信． E-mail: boai@bjtu.edu.cn" ]
- 基金信息：
  
  国家重点研发计划(2022YFB3207400);国家自然科学基金(62221001;61971030);中国国家铁路集团有限公司科技研究开发计划资助项目(P2021S005)
- DOI：10.12263/DZXB.20220922
  中图分类号： TN929.5;
- 收稿：2022-08-05，
  
  修回：2022-12-18，
  
  纸质出版：2024-02-25
- 稿件说明：
移动端阅览
冯伟杨,林思雨,冯婧涛,等.基于Q学习的蜂窝车联网边缘计算系统PC-5/Uu接口联合卸载策略[J].电子学报,2024,52(02):385-395.

FENG Wei-yang, LIN Si-yu, FENG Jing-tao, et al.Q-Learning Based Joint PC-5/Uu Offloading Strategy for C-V2X Based Vehicular Edge Computing System[J].Acta Electronica Sinica, 2024, 52(02): 385-395.
冯伟杨,林思雨,冯婧涛,等.基于Q学习的蜂窝车联网边缘计算系统PC-5/Uu接口联合卸载策略[J].电子学报,2024,52(02):385-395. DOI：10.12263/DZXB.20220922

FENG Wei-yang, LIN Si-yu, FENG Jing-tao, et al.Q-Learning Based Joint PC-5/Uu Offloading Strategy for C-V2X Based Vehicular Edge Computing System[J].Acta Electronica Sinica, 2024, 52(02): 385-395. DOI：10.12263/DZXB.20220922

摘要

智能驾驶等智能交通服务对时延要求高，在车辆本身算力不足的情况下，车辆需要周围车辆和路旁边缘计算单元帮助其一起完成任务的计算处理．本文在既有车联网边缘计算卸载策略基础上，考虑了蜂窝车联网系统5G-NR接口与PC-5接口链路的特征差异，提出了一种基于Q学习的PC-5/Uu接口联合边缘计算卸载策略．在对蜂窝车联网PC-5链路传输成功率进行建模的基础上，推导了PC-5链路的传输速率表征方法．以最小化蜂窝车联网任务处理时延为目标，以任务车辆发射功率与边缘计算车辆的计算能量损耗为约束，构建了系统时延最小化的有约束马尔科夫决策过程．通过拉格朗日方法，将有约束马尔科夫决策过程问题转化为一个等价的极小极大的无约束马尔科夫决策过程，引入Q学习设计卸载策略，进而提出基于Q学习的蜂窝车联网边缘计算系统卸载策略．仿真结果表明，与其他基线方案相比，本文提出的算法可以降低系统时延27.3%以上．

Abstract

Intelligent transportation services

such as smart driving

put forward high requirements for latency. When the vehicle itself has insufficient computing power

the vehicle needs the surrounding vehicles and roadside edge computing units to help it complete the task computation. In this paper

based on the existing vehicular edge computing (VEC) offloading strategy

considering the differences between the 5G-NR interface and PC-5 interface link of cellular-V2X (C-V2X) system

we propose a Q-Learning based joint PC-5/Uu interface edge computing offloading strategy. The successful transmission probability of PC-5 link in C-V2X system is modeled

and then the transmission rate characterization method of PC-5 link is deduced. We formulate a constrained Markov decision process (CMDP) to minimize the system latency

where the objective function is the task processing latency in C-V2X system

and constraints are transmission power at task vehicle and energy consumption of computation at vehicles with edge computing unit. By Lagrangian approach

the CMDP problem is transformed into an equivalent min-max non-constrained MDP problem

and Q-Learning is introduced to design the offloading strategy

and then the offloading strategy of C-V2X based VEC system based on Q-Learning is proposed. Simulation results show that compared with other baseline schemes

the proposed algorithm can significantly improve the system latency performance by at least 27.3%.

关键词

Keywords

references

ZHANG N , ZHANG S , YANG P , et al . Software defined space-air-ground integrated vehicular networks: Challenges and solutions [J]. IEEE Communications Magazine , 2017 , 55 ( 7 ): 101 - 109 .

崔玉亚 , 张德干 , 张婷 , 等 . 一种面向移动边缘计算的多用户细粒度任务卸载调度方法 [J]. 电子学报 , 2021 , 49 ( 11 ): 2202 - 2207 .

CUI Y Y , ZHANG D G , ZHANG T , et al . A multi-user fine-grained task offloading scheduling approach of mobile edge computing [J]. Acta Electronica Sinica , 2021 , 49 ( 11 ): 2202 - 2207 . (in Chinese)

乐光学 , 戴亚盛 , 杨晓慧 , 等 . 海上边缘计算云边智能协同服务建模 [J]. 电子学报 , 2021 , 49 ( 12 ): 2407 - 2418 .

YUE G X , DAI Y S , YANG X H , et al . Model of cloud-edge cooperative service for maritime edge computing [J]. Acta Electronica Sinica , 2021 , 49 ( 12 ): 2407 - 2418 . (in Chinese)

ZHANG Y X , DUAN S J , ZHANG D Y , et al . Transparent computing: Development and current status [J]. Chinese Journal of Electronics , 2020 , 29 ( 5 ): 793 - 811 .

HUANG C , LU R X , CHOO K K R . Vehicular fog computing: Architecture, use case, and security and forensic challenges [J]. IEEE Communications Magazine , 2017 , 55 ( 11 ): 105 - 111 .

廖勇 , 田肖懿 , 蔡志镕 , 等 . 面向C-V2I的基于边缘计算的智能信道估计 [J]. 电子学报 , 2021 , 49 ( 5 ): 833 - 842 .

LIAO Y , TIAN X Y , CAI Z R , et al . Intelligent channel estimation based on edge computing for C-V2I [J]. Acta Electronica Sinica , 2021 , 49 ( 5 ): 833 - 842 . (in Chinese)

FENG W Y , YANG S Z , GAO Y , et al . Reverse offloading for latency minimization in vehicular edge computing [C]// ICC 2021 - IEEE International Conference on Communications . Piscataway : IEEE , 2021 : 1 - 6 .

FENG W Y , LIN S Y , ZHANG N , et al . Joint C-V2X based offloading and resource allocation in multi-tier vehicular edge computing system [J]. IEEE Journal on Selected Areas in Communications , 2023 , 41 ( 2 ): 432 - 445 .

AHSAN KAZMI S M , OTOUM S , HUSSAIN R , et al . A novel deep reinforcement learning-based approach for task-offloading in vehicular networks [C]// 2021 IEEE Global Communications Conference (GLOBECOM) . Piscataway : IEEE , 2021 : 1 - 6 .

FENG W Y , ZHANG N , LI S C , et al . Latency minimization of reverse offloading in vehicular edge computing [J]. IEEE Transactions on Vehicular Technology , 2022 , 71 ( 5 ): 5343 - 5357 .

陈思光 , 陈佳民 , 赵传信 . 基于深度强化学习的云边协同计算迁移研究 [J]. 电子学报 , 2021 , 49 ( 1 ): 157 - 166 .

CHEN S G , CHEN J M , ZHAO C X . Deep reinforcement learning based cloud-edge collaborative computation offloading mechanism [J]. Acta Electronica Sinica , 2021 , 49 ( 1 ): 157 - 166 . (in Chinese)

SHINDE S , BOZORGCHENANI A , TARCHI D , et al . On the design of federated learning in latency and energy constrained computation offloading operations in vehicular edge computing systems [J]. IEEE Transactions on Vehicular Technology , 2022 , 71 ( 2 ): 2041 - 2057 .

HE Y , ZHAO N , YIN H X . Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach [J]. IEEE Transactions on Vehicular Technology , 2018 , 67 ( 1 ): 44 - 55 .

XIE R C , TANG Q Q , WANG Q N , et al . Collaborative vehicular edge computing networks: Architecture design and research challenges [J]. IEEE Access , 2019 , 7 : 178942 - 178952 .

LIU Y , YU H M , XIE S L , et al . Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks [J]. IEEE Transactions on Vehicular Technology , 2019 , 68 ( 11 ): 11158 - 11168 .

XIE R C , TANG Q Q , LIANG C H , et al . Dynamic computation offloading in IoT fog systems with imperfect channel-state information: A POMDP approach [J]. IEEE Internet of Things Journal , 2021 , 8 ( 1 ): 345 - 356 .

ZHANG Y M , LAN X L , REN J , et al . Efficient computing resource sharing for mobile edge-cloud computing networks [J]. IEEE/ACM Transactions on Networking , 28 ( 3 ): 1227 - 1240 .

LAN X L , ZHANG Y M , CAI L , et al . Energy efficient buffer-aided transmission scheme in wireless powered cooperative NOMA relay network [J]. IEEE Transactions on Communications , 2019 , 68 ( 3 ): 1432 - 1447 .

LIU Y , YANG C , JIANG L , et al . Intelligent edge computing for IoT-based energy management in smart cities [J]. IEEE Network , 2019 , 33 ( 2 ): 111 - 117 .

ZHAN W H , LUO C B , WANG J , et al . Deep-reinforcement-learning-based offloading scheduling for vehicular edge computing [J]. IEEE Internet of Things Journal , 2020 , 7 ( 6 ): 5449 - 5465 .

LIU Y J , WANG S G , ZHAO Q L , et al . Dependency-aware task scheduling in vehicular edge computing [J]. IEEE Internet of Things Journal , 2020 , 7 ( 6 ): 4961 - 4971 .

3GPP. Release 14: Technical Specifications Group Services and System Aspects: 3GPP TR21.94v0.8.0 [S]. Sophia Antipolis Cedex: 3GPP TR, 2017 .

GARCIA M H C , MOLINA-GALAN A , BOBAN M T , et al . A tutorial on 5G NR V2X communications [J]. IEEE Communications Surveys & Tutorials , 2021 , 23 ( 3 ): 1972 - 2026 .

GONZALEZ-MARTIN M , SEPULCRE M , MOLINA-MASEGOSA R , et al . Analytical models of the performance of C-V2X mode 4 vehicular communications [J]. IEEE Transactions on Vehicular Technology , 2019 , 68 ( 2 ): 1155 - 1166 .

MAO Y Y , YOU C S , ZHANG J , et al . A survey on mobile edge computing: The communication perspective [J]. IEEE Communications Surveys & Tutorials , 2017 , 19 ( 4 ): 2322 - 2358 .

SHORTLE J F , THOMPSON J M , GROSS D , et al . Fundamentals of Queueing Theory [M]. New York : Wiley , 2018 : 399 .

MENG X L , WANG W , WANG Y T , et al . Closed-form delay-optimal computation offloading in mobile edge computing systems [J]. IEEE Transactions on Wireless Communications , 2019 , 18 ( 10 ): 4653 - 4667 .

ALTMAN E . Constrained Markov Decision Processes [M]. Boca Raton : Chapman & Hall/CRC , 1999 .

ABOUNADI J , BERTSEKAS D , BORKAR V S . Learning algorithms for Markov decision processes with average cost [J]. SIAM Journal on Control and Optimization , 2001 , 40 ( 3 ): 681 - 698 .

FENG J T , FENG W Y , LIN S Y . Reverse computing offloading for enhanced computing capacity in cooperative vehicle infrastructure system [C]// 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) . New York : ACM , 2021 : 1011 - 1016 .

BORKAR V S . Stochastic approximation with two time scales [J]. Systems & Control Letters , 1997 , 29 ( 5 ): 291 - 294 .

MARTIN-VEGA F J , SORET B , AGUAYO-TORRES M C , et al . Geolocation-based access for vehicular communications: Analysis and optimization via stochastic geometry [J]. IEEE Transactions on Vehicular Technology , 2018 , 67 ( 4 ): 3069 - 3084 .

DAI Y Y , XU D , MAHARJAN S , et al . Joint load balancing and offloading in vehicular edge computing and networks [J]. IEEE Internet of Things Journal , 2019 , 6 ( 3 ): 4377 - 4387 .

WANG Z , ZHONG Z D , ZHAO D M , et al . Vehicle-based cloudlet relaying for mobile computation offloading [J]. IEEE Transactions on Vehicular Technology , 2018 , 67 ( 11 ): 11181 - 11191 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的云边协同计算迁移研究

基于轻量自蒸馏的低成本联邦学习

边缘计算环境下基于PUF的多接收者匿名签密方案

基于双层联邦学习的高动态车联网业务边缘协作计算机制