Directed Graph and Convolutional Network Reinforcement Learning for Terminal-Side Collaborative Computing Resource Allocation Scheme

GU Jian-hua; FENG Jian-hua; XU Hui-yang; LIU Tong-tong; ZHOU Ting

doi:10.12263/DZXB.20251106

您当前的位置：

首页 >

文章列表页 >

Directed Graph and Convolutional Network Reinforcement Learning for Terminal-Side Collaborative Computing Resource Allocation Scheme

Large-Scale Models and the Internet | 更新时间：2025-10-16

- Directed Graph and Convolutional Network Reinforcement Learning for Terminal-Side Collaborative Computing Resource Allocation Scheme
- ACTA ELECTRONICA SINICA Vol. 53, Issue 6, Pages: 1771-1783(2025)
- 作者机构：
  
  1.清华大学计算机科学与技术系，北京 100084
  2.中国移动通信集团终端有限公司，北京 100053
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20251106
  CLC： TN915.07;
- Received：09 December 2024，
  
  Revised：2025-05-30，
  
  Published：25 June 2025
- 稿件说明：
移动端阅览
顾健华, 冯建华, 许辉阳, 等. 基于有向图与卷积网络强化学习的端侧协同算力资源分配方法[J]. 电子学报, 2025, 53(06): 1771-1783.

GU Jian-hua, FENG Jian-hua, XU Hui-yang, et al. Directed Graph and Convolutional Network Reinforcement Learning for Terminal-Side Collaborative Computing Resource Allocation Scheme[J]. Acta Electronica Sinica, 2025, 53(06): 1771-1783.
顾健华, 冯建华, 许辉阳, 等. 基于有向图与卷积网络强化学习的端侧协同算力资源分配方法[J]. 电子学报, 2025, 53(06): 1771-1783. DOI：10.12263/DZXB.20251106

GU Jian-hua, FENG Jian-hua, XU Hui-yang, et al. Directed Graph and Convolutional Network Reinforcement Learning for Terminal-Side Collaborative Computing Resource Allocation Scheme[J]. Acta Electronica Sinica, 2025, 53(06): 1771-1783. DOI：10.12263/DZXB.20251106

摘要

随着人工智能应用场景的集中式爆发，移动应用对数据通信和计算的需求日益增长，位于远端的传统云计算处理方法难以满足快速响应的要求.因此，整合利用海量的用户侧终端设备算力（包括计算、存储、通信等）的端侧算力网络，通过分布式协作合理地利用终端算力完成计算任务成为一种新的处理方法.鉴于单台终端设备的资源受限，高企的通信开销限制任务协同效果，导致终端很难高效协同完成高度复杂的计算任务.本文提出利用点对点（Device-to-Device，D2D）通信辅助终端节点协同计算，并设计了基于有向图卷积网络（Directed Graph Convolutional Network，DGCN）的协作拓扑和资源分配决策算法（Multi-Agent Soft Actor-Critic，MA-SAC），将有向无环图（Directed Acyclic Graph，DAG）任务中包含的子任务部署到多个终端进行协同计算，满足DAG子任务部署在多个不同节点间的跨节点传输需求，降低子任务间数据传输在基站侧的网络通信开销.仿真结果显示，所提算法能够在保证业务时延要求下，降低38.2%的网络通信开销，有效提升31.9%的端侧资源利用率.

Abstract

Driven by the concentrated surge of AI application scenarios

the increasing requirements on data communication and computation in mobile applications is growing

the traditional cloud computing which relies on remote processing

often fails to meet low-latency requirements. Therefore

a new paradigm has emerged: terminal-side computing power that aggregate the vast terminal devices (including computing

storage

communication

etc) through distributed collaboration to efficiently execute computational tasks. However

constrained by the limited resource of standalone device and prohibitive communication overhead that impairs task coordination

such terminals still face significant challenges in achieving efficient collaboration for highly complex computing tasks.This paper presents device-to-device (D2D) communication assisted terminal devices collaborative computing

and a multi-agent soft actor-critic (MA-SAC) based on directed graph convolutional network (DGCN) is designed to solve this problem.The subtasks included in directed acyclic graph (DAG) tasks were deployed to multiple terminals for collaborative computing

it is introduced to cater to the exigencies of task transmission between disparate nodes within the DAG

and reduces the communication overhead when data transmission in the network. Through the simulations

the efficacy of the proposed scheme is demonstrated. The proposed scheme reduces network communication overhead by 38.2% and effectively improve resource utilization by 31.9%.

关键词

Keywords

references

GU J H , FENG J H , XU H Y , et al . Research on terminal-side computing force network based on massive terminals [J ] . Electronics , 2022 , 11 ( 13 ): 2108 .

LI X N , LU J G , ZHANG P M . A GNN-based routing and scheduling mechanism for multi-domain computing first network [C ] // 2023 IEEE 9th International Conference on Cloud Computing and Intelligent Systems (CCIS) . Piscataway : IEEE , 2023 : 153 - 163 .

JIN H , LUO R K , HE Q , et al . Cost-effective data placement in edge storage systems with erasure code [J ] . IEEE Transactions on Services Computing , 2023 , 16 ( 2 ): 1039 - 1050 .

LI J C , ZHOU F Q , LI W J , et al . Componentized task scheduling in cloud-edge cooperative scenarios based on GNN-enhanced DRL [C ] // NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium . Piscataway : IEEE , 2023 : 1 - 4 .

ZENG Q , PENG B , LI Q , et al . Container-based task scheduling for edge computing using A multi-strategy hybrid of genetic beetles [C ] // 2022 IEEE 8th International Conference on Computer and Communications (ICCC) . Piscataway : IEEE , 2022 : 2198 - 2203 .

ZHU L L , HUANG K , HU Y F , et al . A self-adapting task scheduling algorithm for container cloud using learning automata [J ] . IEEE Access , 2021 , 9 : 81236 - 81252 .

HU Y , DE LAAT C , ZHAO Z M . Multi-objective container deployment on heterogeneous clusters [C ] // 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) . Piscataway : IEEE , 2019 : 1 .

LAI W K , WANG Y C , WEI S C . Delay-aware container scheduling in Kubernetes [J ] . IEEE Internet of Things Journal , 2023 , 10 ( 13 ): 11813 - 11824 .

MENG C Y , SONG S J , TONG H G , et al . DeepScaler: Holistic autoscaling for microservices based on spatiotemporal GNN with adaptive graph learning [C ] // 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) . Piscataway : IEEE , 2023 : 53 - 65 .

FATICANTI F , DE PELLEGRINI F , SIRACUSA D , et al . Cutting throughput with the edge: App-aware placement in fog computing [C ] // 2019 6th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)/2019 5th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom) . Piscataway : IEEE , 2019 : 196 - 203 .

HUANG Y K , QIAO X Q , LAI W H , et al . Enabling DNN acceleration with data and model parallelization over ubiquitous end devices [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 16 ): 15053 - 15065 .

DUAN S J , WANG D , REN J , et al . Distributed artificial intelligence empowered by end-edge-cloud computing: A survey [J ] . IEEE Communications Surveys Tutorials , 2023 , 25 ( 1 ): 591 - 624 .

NING Z L , DONG P R , WANG X J , et al . Distributed and dynamic service placement in pervasive edge computing networks [J ] . IEEE Transactions on Parallel and Distributed Systems , 2021 , 32 ( 6 ): 1277 - 1292 .

MA L L , YI S H , CARTER N , et al . Efficient live migration of edge services leveraging container layered storage [J ] . IEEE Transactions on Mobile Computing , 2019 , 18 ( 9 ): 2020 - 2033 .

ZHANG X F , ZHANG J , LIU Z T , et al . MDP-based task offloading for vehicular edge computing under certain and uncertain transition probabilities [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 3 ): 3296 - 3309 .

3GPP. Proximity-based services (ProSe); Stage 2: TS 23.303 v17.7.0 [S ] .

3GPP. Proximity based Services (ProSe) in the 5G System (5GS): TS 23.304 v17.7.0 [S ] .

DAS A , DAS N , DAS BARMAN A . Multi-hop D2D communication in cellular networks to minimize EMR [J ] . IEEE Transactions on Green Communications and Networking , 2022 , 6 ( 2 ): 713 - 722 .

TEJA P R , MISHRA P K . Path selection and resource allocation for 5G multi-hop D2D networks [J ] . Computer Communications , 2022 , 195 : 292 - 302 .

MUY S , LEE J R . Spectrum efficiency maximization for multi-hop D2D communication underlaying cellular networks: Machine learning-based methods [J ] . Expert Systems with Applications , 2023 , 213 : 118167 .

SUN H , NARAGHI-POUR M , SHENG W X , et al . A hop-by-hop relay selection strategy in multi-hop cognitive relay networks [J ] . IEEE Access , 2020 , 8 : 21117 - 21126 .

叶梓峰 , 王永华 , 万频 , 等 . 基于优先记忆库结合竞争深度Q网络的动态功率控制 [J ] . 电讯技术 , 2019 , 59 ( 10 ): 1132 - 1139 .

YE Z F , WANG Y H , WAN P , et al . A dynamic power control strategy based on dueling deep Q network with prioritized experience replay [J ] . Telecommunication Engineering , 2019 , 59 ( 10 ): 1132 - 1139 . (in Chinese)

WANG J S , LIU Y , ZHANG W T , et al . ReLFA: Resist link flooding attacks via renyi entropy and deep reinforcement learning in SDN-IoT [J ] . China Communications , 2022 , 19 ( 7 ): 157 - 171 .

BAO R M , ZHOU Y L , JIANG W R . FL-CNN-LSTM: Indoor air quality prediction using fuzzy logic and CNN-LSTM model [C ] // 2022 2nd International Conference on Electrical Engineering and Control Science (IC2ECS) . Piscataway : IEEE , 2022 : 986 - 989 .

AWASTHI A K , GAROV A K , SHARMA M , et al . GNN model based on node classification forecasting in social network [C ] // 2023 International Conference on Artificial Intelligence and Smart Communication (AISC) . Piscataway : IEEE , 2023 : 1039 - 1043 .

FERRIOL-GALMÉS M , PAILLISSE J , SUÁREZ-VARELA J , et al . RouteNet-Fermi: Network modeling with graph neural networks [J ] . IEEE/ACM Transactions on Networking , 2023 , 31 ( 6 ): 3080 - 3095 .

HAN X Y , XIE M X , YU K , et al . Combining graph neural network with deep reinforcement learning for resource allocation in computing force networks [J ] . Frontiers of Information Technology Electronic Engineering , 2024 , 25 ( 5 ): 701 - 712 .

ANAMURO C V , VARSIER N , SCHWOERER J , et al . Distance-aware relay selection in an energy-efficient discovery protocol for 5G D2D communication [J ] . IEEE Transactions on Wireless Communications , 2021 , 20 ( 7 ): 4379 - 4391 .

肖硕 , 黄珍珍 , 张国鹏 , 等 . 基于SAC的多智能体深度强化学习算法 [J ] . 电子学报 , 2021 , 49 ( 9 ): 1675 - 1681 .

XIAO S , HUANG Z Z , ZHANG G P , et al . Deep reinforcement learning algorithm of multi-agent based on SAC [J ] . Acta Electronica Sinica , 2021 , 49 ( 9 ): 1675 - 1681 . (in Chinese)

秦鹏 , 王硕 , 付民 , 等 . 基于MATD3的空地网络资源优化 [J ] . 中国科学: 信息科学 , 2024 , 54 ( 6 ): 1474 - 1486 .

QIN P , WANG S , FU M , et al . Air-ground integrated network resource optimization based on MATD3 [J ] . Scientia Sinica (Informationis) , 2024 , 54 ( 6 ): 1474 - 1486 . (in Chinese)

TONG Z , DENG X M , YE F , et al . Adaptive computation offloading and resource allocation strategy in a mobile edge computing environment [J ] . Information Sciences , 2020 , 537 : 116 - 131 .

MO C T , CHEN J H , LIAO W . Graph convolutional network augmented deep reinforcement learning for dependent task offloading in mobile edge computing [C ] // 2023 IEEE Wireless Communications and Networking Conference (WCNC) . Piscataway : IEEE , 2023 : 1 - 6 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰