电子学报 ›› 2021, Vol. 49 ›› Issue (1): 157-166.DOI: 10.12263/DZXB.20200131

• 学术论文 • 上一篇    下一篇

基于深度强化学习的云边协同计算迁移研究

陈思光1,2, 陈佳民1,3, 赵传信2   

  1. 1. 南京邮电大学江苏省通信与网络技术工程研究中心, 江苏南京 210003;
    2. 安徽师范大学网络与信息安全安徽省重点实验室, 安徽芜湖 241002;
    3. 南京邮电大学江苏省宽带无线通信和物联网重点实验室, 江苏南京 210003
  • 收稿日期:2020-01-20 修回日期:2020-03-19 出版日期:2021-01-25
    • 通讯作者:
    • 陈思光
    • 作者简介:
    • 陈佳民 男,1995年12月出生于江苏省南通市.南京邮电大学在读硕士研究生.主要研究方向为边缘计算和机器学习.E-mail:18252010292@163.com;赵传信 男,1977年生于安徽凤阳.现为安徽师范大学教授、博士生导师.主要研究方向为物联网、边缘计算及网络资源分配优化等.E-mail:zhaocx@ahnu.edu.cn
    • 基金资助:
    • 国家自然科学基金 (No.61971235,No.61771258,No.61871412); 江苏省"六大人才高峰"高层次人才项目 (No.XYDXXJS-044); 江苏省"333高层次人才培养工程"; 南京邮电大学"1311"人才计划; 中国博士后科学基金 (面上一等) (No.2018M630590); 南京邮电大学国家自然科学基金孵化项目 (No.NY217057,No.NY218058); 网络与信息安全安徽省重点实验室开放课题 (No.AHNIS2020001); 江苏省通信与网络技术工程研究中心开放课题重点项目 (No.JSGCZX17011); 赛尔网络下一代互联网技术创新项目 (No.NGII20190702)

Deep Reinforcement Learning Based Cloud-Edge Collaborative Computation Offloading Mechanism

CHEN Si-guang1,2, CHEN Jia-min1,3, ZHAO Chuan-xin2   

  1. 1. Jiangsu Engineering Research Center of Communications and Network Technology, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu 210003, China;
    2. Anhui Provincial Key Laboratory of Network and Information Security, Anhui Normal University, Wuhu, Anhui 241002, China;
    3. Jiangsu Key Lab of Broadband Wireless Communication and Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu 210003, China
  • Received:2020-01-20 Revised:2020-03-19 Online:2021-01-25 Published:2021-01-25

摘要: 基于单一边缘节点计算、存储资源的有限性及大数据场景对高效计算服务的需求,本文提出了一种基于深度强化学习的云边协同计算迁移机制.具体地,基于计算资源、带宽和迁移决策的综合性考量,构建了一个最小化所有用户任务执行延迟与能耗权重和的优化问题.基于该优化问题提出了一个异步云边协同的深度强化学习算法,该算法充分利用了云边双方的计算能力,可有效满足大数据场景对高效计算服务的需求;同时,面向边缘云中边缘节点所处环境的多样及动态变化性,该算法能自适应地调整迁移策略以实现系统总成本的最小化.最后,大量的仿真结果表明本文所提出的算法具有收敛速度快、鲁棒性高等特点,并能够以最低的计算成本获得近似贪心算法的最优迁移决策.

 

关键词: 深度强化学习, 边缘计算, 计算迁移, 资源分配, 能量消耗

Abstract: Based on the computation and storage resources limitation of single edge node and the demand for efficient computing services in big data scenarios,this paper proposes a deep reinforcement learning based cloud-edge collaborative computation offloading mechanism.Specifically,based on a comprehensive consideration of computing resources,bandwidth and offloading policy,an optimization problem is formulated to minimize the weight sum of execution delay and energy consumption of all user tasks.An asynchronous cloud-edge collaborative deep reinforcement learning (ACEC-DRL) algorithm is proposed to solve such optimization problem.This algorithm can effectively satisfy the demand of efficient computing services in big data scenario by jointly leveraging the computation capabilities of cloud and edge nodes.Meanwhile,under the various and dynamic environments of edge nodes in the edge cloud,this algorithm can adaptively adjust offloading policy to achieve the minimization of system cost.Finally,the extensive simulation results show that the proposed ACEC-DRL algorithm has the characteristics of fast convergence rate and high robustness,and its optimal offloading policy closely approximates to the solution of greedy algorithm with the lowest computation cost.

 

Key words: deep reinforcement learning, edge computing, computation offloading, resource allocation, energy consumption

中图分类号: