A Review of Joint Trajectory and Phase-Shift Optimization in RIS-Assisted UAV Communication Based on Deep Reinforcement Learning

WANG Enliang; LI Kun; ZHANG Weiying; SUN Zhixin

doi:10.12263/DZXB.20260377

您当前的位置：

首页 >

文章列表页 >

A Review of Joint Trajectory and Phase-Shift Optimization in RIS-Assisted UAV Communication Based on Deep Reinforcement Learning

SURVEY AND REVIEW | 更新时间：2026-10-16

- A Review of Joint Trajectory and Phase-Shift Optimization in RIS-Assisted UAV Communication Based on Deep Reinforcement Learning
- ACTA ELECTRONICA SINICA Vol. 54, Issue 6, Pages: 2879-2898(2026)
- 作者机构：
  
  1.南京邮电大学江苏省邮政大数据技术与应用工程研究中心，江苏南京 210023
  2.南京邮电大学国家邮政局邮政行业技术研发中心（物联网技术），江苏南京 210023
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62272239);Guizhou Provincial Key Technology R&D Program(［2023］272)
- DOI：10.12263/DZXB.20260377
  CLC： TN929.5;TP18
- Received：22 April 2026，
  
  Accepted：27 May 2026，
  
  Published：25 June 2026
- 稿件说明：
移动端阅览
王恩良, 李琨, 张炜荥, 等. 基于深度强化学习的RIS辅助UAV通信轨迹与相移联合优化研究综述[J]. 电子学报, 2026, 54(06): 2879-2898.

WANG Enliang, LI Kun, ZHANG Weiying, et al. A Review of Joint Trajectory and Phase-Shift Optimization in RIS-Assisted UAV Communication Based on Deep Reinforcement Learning[J]. Acta Electronica Sinica, 2026, 54(06): 2879-2898.
王恩良, 李琨, 张炜荥, 等. 基于深度强化学习的RIS辅助UAV通信轨迹与相移联合优化研究综述[J]. 电子学报, 2026, 54(06): 2879-2898. DOI：10.12263/DZXB.20260377

WANG Enliang, LI Kun, ZHANG Weiying, et al. A Review of Joint Trajectory and Phase-Shift Optimization in RIS-Assisted UAV Communication Based on Deep Reinforcement Learning[J]. Acta Electronica Sinica, 2026, 54(06): 2879-2898. DOI：10.12263/DZXB.20260377

摘要

可重构智能表面（Reconfigurable Intelligent Surface，RIS）与无人机（Unmanned Aerial Vehicle，UAV）的协同部署为提升未来无线网络的覆盖范围、传输容量和服务灵活性提供了新的技术路径，其核心在于UAV轨迹、RIS相移矩阵、主动波束成形及资源分配变量的联合优化。该问题具有跨时隙耦合、动作维度高、约束非凸和环境动态变化等特征，传统交替优化、逐次凸近似和半正定松弛方法在大规模或快速时变场景中面临迭代开销较高、在线适应性不足以及对精确信道状态信息依赖较强等问题。深度强化学习（Deep Reinforcement Learning，DRL）能够将上述联合优化表述为序贯决策过程，为复杂动态环境下的在线策略生成提供新的求解思路。本文围绕基于DRL的RIS辅助UAV通信轨迹—相移联合优化开展系统综述。首先，建立包含轨迹、相移和波束成形变量的统一优化框架，并从动作空间、状态空间和奖励函数三个维度分析马尔可夫决策过程建模的关键选择。随后，依据联合动作的表示与分解方式，将现有研究归纳为离散决策、连续决策和分布式协同三类技术路线，比较其算法机制、适用场景、样本效率、收敛稳定性、可扩展性与工程代价。进一步，从可扩展性、信道状态信息假设、仿真—部署迁移和公平比较基准四个方面提炼该领域的共性瓶颈。综合分析表明，离散方法适用于中小规模RIS和低比特相移控制；连续方法具有较高控制精度，但易受高维策略学习与样本效率限制；多智能体方法与分布式系统架构较为匹配，但会引入环境非平稳性和信用分配问题。在大规模场景下，现有方法普遍依赖结构化降维或DRL与传统优化相结合的混合分解，严格意义上的端到端联合求解仍较少。最后，从结构化DRL、离线强化学习与数字孪生、标准化评测平台、近场与太赫兹通信以及通感算一体化等方面讨论未来研究方向，为相关算法设计、性能评估与系统部署提供参考。

Abstract

The collaborative deployment of reconfigurable intelligent surfaces (RISs) and unmanned aerial vehicles (UAVs) provides a promising technological approach to improving the coverage

transmission capacity

and service flexibility of future wireless networks. Its central challenge lies in the joint optimization of UAV trajectories

RIS phase-shift matrices

active beamforming

and resource allocation variables. This problem is characterized by inter-slot coupling

high-dimensional action spaces

nonconvex constraints

and dynamically varying environments. In large-scale or rapidly time-varying scenarios

conventional methods based on alternating optimization

successive convex approximation

and semidefinite relaxation generally suffer from high iterative complexity

limited online adaptability

and a strong dependence on accurate channel state information. Deep reinforcement learning (DRL) can formulate the joint optimization problem as a sequential decision-making process

thereby providing a new approach to online policy generation in complex dynamic environments. This paper presents a systematic review of DRL-based joint trajectory and phase-shift optimization in RIS-assisted UAV communications. First

a unified optimization framework incorporating trajectory

phase-shift

and beamforming variables is established

and the key choices in Markov decision process (MDP) modeling are analyzed from the perspectives of action-space design

state-space construction

and reward-function formulation. Existing studies are then classified into three technical paradigms according to the representation and decomposition of joint actions: discrete decision-making

continuous decision-making

and distributed cooperation. Their algorithmic mechanisms

applicable scenarios

sample efficiency

convergence stability

scalability

and implementation costs are comparatively examined. Four common bottlenecks are further identified

namely limited scalability

idealized channel state information (CSI) assumptions

the simulation-to-deployment gap

and the lack of fair benchmarking protocols. The comparative analysis shows that discrete methods are suitable for small- and medium-scale RISs with low-bit phase-shift control

whereas continuous methods provide higher control precision but are constrained by high-dimensional policy learning and limited sample efficiency. Multi-agent methods are well matched to distributed system architectures but introduce environmental nonstationarity and credit-assignment difficulties. In large-scale scenarios

most existing approaches rely on structured dimensionality reduction or hybrid decomposition combining DRL with conventional optimization

while strictly end-to-end joint optimization remains relatively uncommon. Finally

future research directions are discussed

including structured DRL architectures

offline reinforcement learning and digital twins

standardized benchmarking platforms

near-field and terahertz communications

and integrated communication

sensing

and computing. This review provides a reference for subsequent algorithm design

performance evaluation

and system deployment.

关键词

Keywords

references

Wu Qingqing , Zhang Rui . Intelligent reflecting surface enhanced wireless network via joint active and passive beamforming [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 11 ): 5394 - 5409 . DOI: 10.1109/twc.2019.2936025 http://dx.doi.org/10.1109/twc.2019.2936025

Zeng Yong , Zhang Rui , Lim T J . Wireless communications with unmanned aerial vehicles: Opportunities and challenges [J ] . IEEE Communications Magazine , 2016 , 54 ( 5 ): 36 - 42 . DOI: 10.1109/mcom.2016.7470933 http://dx.doi.org/10.1109/mcom.2016.7470933

陈新颖 , 盛敏 , 李博 , 等 . 面向6G的无人机通信综述 [J ] . 电子与信息学报 , 2022 , 44 ( 3 ): 781 - 789 . DOI: 10.11999/JEIT210789 http://dx.doi.org/10.11999/JEIT210789

Chen Xinying , Sheng Min , Li Bo , et al . Survey on unmanned aerial vehicle communications for 6G [J ] . Journal of Electronics & Information Technology , 2022 , 44 ( 3 ): 781 - 789 . (in Chinese) . DOI: 10.11999/JEIT210789 http://dx.doi.org/10.11999/JEIT210789

Li Sixian , Duo Bin , Yuan Xiaojun , et al . Reconfigurable intelligent surface assisted UAV communication: Joint trajectory design and passive beamforming [J ] . IEEE Wireless Communications Letters , 2020 , 9 ( 5 ): 716 - 720 . DOI: 10.1109/lwc.2020.2966705 http://dx.doi.org/10.1109/lwc.2020.2966705

Wu Qingqing , Zhang Shuowen , Zheng Beixiong , et al . Intelligent reflecting surface-aided wireless communications: A tutorial [J ] . IEEE Transactions on Communications , 2021 , 69 ( 5 ): 3313 - 3351 . DOI: 10.1109/tcomm.2021.3051897 http://dx.doi.org/10.1109/tcomm.2021.3051897

Zhou Gui , Pan Cunhua , Ren Hong , et al . A framework of robust transmission design for IRS-aided MISO communications with imperfect cascaded channels [J ] . IEEE Transactions on Signal Processing , 2020 , 68 : 5092 - 5106 . DOI: 10.1109/tsp.2020.3019666 http://dx.doi.org/10.1109/tsp.2020.3019666

Pan Cunhua , Zhou Gui , Zhi Kangda , et al . An overview of signal processing techniques for RIS/IRS-aided wireless systems [J ] . IEEE Journal of Selected Topics in Signal Processing , 2022 , 16 ( 5 ): 883 - 917 . DOI: 10.1109/jstsp.2022.3195671 http://dx.doi.org/10.1109/jstsp.2022.3195671

Luong N C , Hoang D T , Gong Shimin , et al . Applications of deep reinforcement learning in communications and networking: A survey [J ] . IEEE Communications Surveys & Tutorials , 2019 , 21 ( 4 ): 3133 - 3174 . DOI: 10.1109/comst.2019.2916583 http://dx.doi.org/10.1109/comst.2019.2916583

Bayerlein H , De Kerret P , Gesbert D . Trajectory optimization for autonomous flying base station via reinforcement learning [C ] // 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC) . Piscataway : IEEE , 2018 : 1 - 5 . DOI: 10.1109/spawc.2018.8445768 http://dx.doi.org/10.1109/spawc.2018.8445768

Liu Xiao , Liu Yuanwei , Chen Yue . Reinforcement learning in multiple-UAV networks: Deployment and movement design [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 8 ): 8036 - 8049 . DOI: 10.1109/tvt.2019.2922849 http://dx.doi.org/10.1109/tvt.2019.2922849

Huang Chongwen , Mo Ronghong , Yuen C . Reconfigurable intelligent surface assisted multiuser MISO systems exploiting deep reinforcement learning [J ] . IEEE Journal on Selected Areas in Communications , 2020 , 38 ( 8 ): 1839 - 1850 . DOI: 10.1109/jsac.2020.3000835 http://dx.doi.org/10.1109/jsac.2020.3000835

Feng Keming , Wang Qisheng , Li Xiao , et al . Deep reinforcement learning based intelligent reflecting surface optimization for MISO communication systems [J ] . IEEE Wireless Communications Letters , 2020 , 9 ( 5 ): 745 - 749 . DOI: 10.1109/lwc.2020.2969167 http://dx.doi.org/10.1109/lwc.2020.2969167

Liu Xiao , Liu Yuanwei , Chen Yue . Machine learning empowered trajectory and passive beamforming design in UAV-RIS wireless networks [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 7 ): 2042 - 2055 . DOI: 10.1109/jsac.2020.3041401 http://dx.doi.org/10.1109/jsac.2020.3041401

Mei Haibo , Yang Kun , Liu Qiang , et al . 3D-trajectory and phase-shift design for RIS-assisted UAV systems using deep reinforcement learning [J ] . IEEE Transactions on Vehicular Technology , 2022 , 71 ( 3 ): 3020 - 3029 . DOI: 10.1109/tvt.2022.3143839 http://dx.doi.org/10.1109/tvt.2022.3143839

Zhang Haijun , Huang Miaolin , Zhou Huan , et al . Capacity maximization in RIS-UAV networks: A DDQN-based trajectory and phase shift optimization approach [J ] . IEEE Transactions on Wireless Communications , 2023 , 22 ( 4 ): 2583 - 2591 . DOI: 10.1109/twc.2022.3212830 http://dx.doi.org/10.1109/twc.2022.3212830

Puspitasari A A , Lee B M . A survey on reinforcement learning for reconfigurable intelligent surfaces in wireless communications [J ] . Sensors , 2023 , 23 ( 5 ): 2554 . DOI: 10.3390/s23052554 http://dx.doi.org/10.3390/s23052554

Chen Hengsheng , Lin Yuanguo , Fu Mingjian , et al . A survey on reinforcement learning methods for UAV systems [J ] . ACM Computing Surveys , 2026 , 58 ( 4 ): 1 - 37 . DOI: 10.1145/3769426 http://dx.doi.org/10.1145/3769426

王侃 , 曹铁林 , 李旭杰 , 等 . 无人机辅助边缘计算网络轨迹规划与资源分配研究综述 [J ] . 电子与信息学报 , 2025 , 47 ( 5 ): 1266 - 1281 .

Wang Kan , Cao Tielin , Li Xujie , et al . A survey on trajectory planning and resource allocation in unmanned aerial vehicle-assisted edge computing networks [J ] . Journal of Electronics & Information Technology , 2025 , 47 ( 5 ): 1266 - 1281 . (in Chinese)

张海霞 , 马晓燕 . 可重构智能表面辅助的通信信号处理技术 [J ] . 信号处理 , 2024 , 40 ( 6 ): 993 - 1006 .

Zhang Haixia , Ma Xiaoyan . Signal processing technologies for RIS-assisted wireless communications [J ] . Journal of Signal Processing , 2024 , 40 ( 6 ): 993 - 1006 . (in Chinese)

Adam A B M , Ouamri M A , Wan Xiaoyu , et al . Secure communication in UAV-RIS-empowered multiuser networks: Joint beamforming, phase shift, and UAV trajectory optimization [J ] . IEEE Systems Journal , 2024 , 18 ( 2 ): 1009 - 1019 . DOI: 10.1109/jsyst.2024.3379456 http://dx.doi.org/10.1109/jsyst.2024.3379456

刘志新 , 赵松晗 , 杨毅 , 等 . 智能反射面辅助的无人机无线携能通信网络吞吐量最大化算法研究 [J ] . 电子与信息学报 , 2022 , 44 ( 7 ): 2325 - 2331 .

Liu Zhixin , Zhao Songhan , Yang Yi , et al . Throughput maximization algorithm for intelligent reflecting surface-aided unmanned aerial vehicle communication networks with wireless energy transfer [J ] . Journal of Electronics & Information Technology , 2022 , 44 ( 7 ): 2325 - 2331 . (in Chinese)

Zeng Yong , Xu Jie , Zhang Rui . Energy minimization for wireless communication with rotary-wing UAV [J ] . IEEE Transactions on Wireless Communications , 2019 , 18 ( 4 ): 2329 - 2345 . DOI: 10.1109/twc.2019.2902559 http://dx.doi.org/10.1109/twc.2019.2902559

Wu Qingqing , Zhang Rui . Beamforming optimization for wireless network aided by intelligent reflecting surface with discrete phase shifts [J ] . IEEE Transactions on Communications , 2020 , 68 ( 3 ): 1838 - 1851 . DOI: 10.1109/tcomm.2019.2958916 http://dx.doi.org/10.1109/tcomm.2019.2958916

Lee G , Jung M , Kasgari A T Z , et al . Deep reinforcement learning for energy-efficient networking with reconfigurable intelligent surfaces [C ] // Proceedings of IEEE International Conference on Communications (ICC) . Piscataway : IEEE , 2020 : 1 - 6 . DOI: 10.1109/icc40277.2020.9149380 http://dx.doi.org/10.1109/icc40277.2020.9149380

杨青青 , 陈剑 , 彭艺 . 基于DDPG的无人机轨迹规划及功率控制算法 [J ] . 北京邮电大学学报 , 2023 , 46 ( 3 ): 43 - 48 .

Yang Qingqing , Chen Jian , Peng Yi . Unmanned aerial vehicle trajectory planning and power control algorithm based on deep deterministic policy gradient [J ] . Journal of Beijing University of Posts and Telecommunications , 2023 , 46 ( 3 ): 43 - 48 . (in Chinese)

Xu Jianpeng , Ai Bo , Quek T Q S . Toward interference suppression: RIS-aided high-speed railway networks via deep reinforcement learning [J ] . IEEE Transactions on Wireless Communications , 2023 , 22 ( 6 ): 4188 - 4201 . DOI: 10.1109/twc.2022.3224009 http://dx.doi.org/10.1109/twc.2022.3224009

Umer M , Mohsin M A , Mahmood A , et al . Deep reinforcement learning for trajectory and phase shift optimization of aerial RIS in CoMP-NOMA networks [C ] // Proceedings of the IEEE Global Communications Conference . Piscataway : IEEE , 2024 : 79 - 84 . DOI: 10.1109/globecom52923.2024.10901709 http://dx.doi.org/10.1109/globecom52923.2024.10901709

Tang Runzhi , Wang Junxuan , Zhang Yanyan , et al . Throughput maximization in NOMA enhanced RIS-assisted multi-UAV networks: A deep reinforcement learning approach [J ] . IEEE Transactions on Vehicular Technology , 2025 , 74 ( 1 ): 730 - 745 . DOI: 10.1109/tvt.2024.3452979 http://dx.doi.org/10.1109/tvt.2024.3452979

Wang Wei , Zhang Wei . Intelligent reflecting surface configurations for smart radio using deep reinforcement learning [J ] . IEEE Journal on Selected Areas in Communications , 2022 , 40 ( 8 ): 2335 - 2346 . DOI: 10.1109/jsac.2022.3180787 http://dx.doi.org/10.1109/jsac.2022.3180787

Tang Runzhi , Wang Junxuan , Jiang Fan , et al . Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems [J ] . Physical Communication , 2024 , 66 : 102456 . DOI: 10.1016/j.phycom.2024.102456 http://dx.doi.org/10.1016/j.phycom.2024.102456

Adhikari B , Khwaja A S , Jaseemuddin M , et al . Energy efficient RIS-assisted UAV networks using twin delayed DDPG technique [J ] . IEEE Transactions on Wireless Communications , 2024 , 23 ( 12 ): 18423 - 18439 . DOI: 10.1109/twc.2024.3468162 http://dx.doi.org/10.1109/twc.2024.3468162

张涛 , 张迁 , 朱颖雯 , 等 . 基于能量感知的智能反射面辅助无人机时效数据收集策略 [J ] . 电子与信息学报 , 2025 , 47 ( 2 ): 427 - 438 .

Zhang Tao , Zhang Qian , Zhu Yingwen , et al . Energy aware reconfigurable intelligent surface assisted unmanned aerial vehicle age of information enabled data collection policies [J ] . Journal of Electronics & Information Technology , 2025 , 47 ( 2 ): 427 - 438 . (in Chinese)

陈宫紫钰 , 贾向东 , 张鑫 . 智能反射面辅助UAV通信系统的信息年龄优化方案 [J ] . 信号处理 , 2024 , 40 ( 8 ): 1480 - 1488 .

Chen Gongziyu , Jia Xiangdong , Zhang Xin . Age of information optimization scheme for an intelligent reflecting surface assisted UAV communication system [J ] . Journal of Signal Processing , 2024 , 40 ( 8 ): 1480 - 1488 . (in Chinese)

Zhou Gui , Pan Cunhua , Ren Hong , et al . Robust beamforming design for intelligent reflecting surface aided MISO communication systems [J ] . IEEE Wireless Communications Letters , 2020 , 9 ( 10 ): 1658 - 1662 . DOI: 10.1109/lwc.2020.3000490 http://dx.doi.org/10.1109/lwc.2020.3000490

Bansal A , Agrawal N , Singh K , et al . RIS selection scheme for UAV-based multi-RIS-aided multiuser downlink network with imperfect and outdated CSI [J ] . IEEE Transactions on Communications , 2023 , 71 ( 8 ): 4650 - 4664 . DOI: 10.1109/tcomm.2023.3277540 http://dx.doi.org/10.1109/tcomm.2023.3277540

张馨予 , 孔祥鲲 , 刘子庆 , 等 . 加载可重构智能超表面的无人机通信系统设计与验证 [J ] . 电波科学学报 , 2024 , 39 ( 4 ): 741 - 747 .

Zhang Xinyu , Kong Xiangkun , Liu Ziqing , et al . Design and verification of a communication system based on UAV with airborne reconfigurable intelligent surface [J ] . Chinese Journal of Radio Science , 2024 , 39 ( 4 ): 741 - 747 . (in Chinese)

Levine S , Kumar A , Tucker G , et al . Offline reinforcement learning: Tutorial, review, and perspectives on open problems [PP/OL ] . V3. arXiv ( 2020-11-01 )[ 2026-04-24 ] . https://arxiv.org/abs/2005.01643 https://arxiv.org/abs/2005.01643 .

Wang Yi , Deng Yu , Kang Ling , et al . Reinforcement learning-based energy efficiency optimization for RIS-Assisted UAV hybrid uplink and downlink system [J ] . Computer Networks , 2024 , 245 : 110390 . DOI: 10.1016/j.comnet.2024.110390 http://dx.doi.org/10.1016/j.comnet.2024.110390

Nguyen K K , Khosravirad S R , da Costa D B , et al . Reconfigurable intelligent surface-assisted multi-UAV networks: Efficient resource allocation with deep reinforcement learning [J ] . IEEE Journal of Selected Topics in Signal Processing , 2022 , 16 ( 3 ): 358 - 368 . DOI: 10.1109/jstsp.2021.3134162 http://dx.doi.org/10.1109/jstsp.2021.3134162

Lin Na , Liu Chunxiao , Wu Tianxiong , et al . Green communications: RIS-assisted fixed-wing UAV coverage scheme based on deep reinforcement learning [J ] . IEEE Internet of Things Journal , 2025 , 12 ( 4 ): 4115 - 4127 . DOI: 10.1109/jiot.2024.3483778 http://dx.doi.org/10.1109/jiot.2024.3483778

Yu Yingfeng , Liu Xin , Liu Zechen , et al . Joint trajectory and resource optimization for RIS assisted UAV cognitive radio [J ] . IEEE Transactions on Vehicular Technology , 2023 , 72 ( 10 ): 13643 - 13648 . DOI: 10.1109/tvt.2023.3270313 http://dx.doi.org/10.1109/tvt.2023.3270313

万诗晴 , 仲伟志 , 何艺 , 等 . 基于深度强化学习的可重构智能超表面辅助无人机通信联合波束成形与轨迹优化 [J ] . 电波科学学报 , 2024 , 39 ( 4 ): 722 - 731 .

Wan Shiqing , Zhong Weizhi , He Yi , et al . The optimization of beamforming and trajectory for reconfigurable intelligent surface assisted UAV communication system based on deep reinforcement learning [J ] . Chinese Journal of Radio Science , 2024 , 39 ( 4 ): 722 - 731 . (in Chinese)

Jiao Shiyu , Xie Ximing , Ding Zhiguo . Deep reinforcement learning-based optimization for RIS-based UAV-NOMA downlink networks (invited paper) [J ] . Frontiers in Signal Processing , 2022 , 2 : 915567 . DOI: 10.3389/frsip.2022.915567 http://dx.doi.org/10.3389/frsip.2022.915567

Chen Shuo , Liu Zhongda , Wang Jianwei , et al . RIS assisted UAV trajectory planning based on deep reinforcement learning [C ] // Proceedings of the 2nd International Conference on Networks, Communications and Intelligent Computing (NCIC 2024) . Singapore : Springer Nature Singapore , 2025 : 1079 - 1090 . DOI: 10.1007/978-981-96-5006-4_94 http://dx.doi.org/10.1007/978-981-96-5006-4_94

Dong Runze , Wang Buhong , Cao Kunrui , et al . Secure transmission design of RIS enabled UAV communication networks exploiting deep reinforcement learning [J ] . IEEE Transactions on Vehicular Technology , 2024 , 73 ( 6 ): 8404 - 8419 . DOI: 10.1109/tvt.2024.3357821 http://dx.doi.org/10.1109/tvt.2024.3357821

Yang Helin , Liu Shuai , Xiao Liang , et al . Learning-based reliable and secure transmission for UAV-RIS-assisted communication systems [J ] . IEEE Transactions on Wireless Communications , 2024 , 23 ( 7 ): 6954 - 6967 . DOI: 10.1109/twc.2023.3336535 http://dx.doi.org/10.1109/twc.2023.3336535

Haarnoja T , Zhou A , Abbeel P , et al . Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor [C ] // Proceedings of the 35th International Conference on Machine Learning . PMLR , 2018 , 80 : 1861 - 1870 .

Adam A B M , Diallo E M , Elhassan M A M . Adaptive soft actor-critic framework for RIS-assisted and UAV-aided communication [PP/OL ] . arXiv ( 2024-11-16 )[ 2026-04-24 ] . https://arxiv.org/abs/2411.10882 https://arxiv.org/abs/2411.10882 .

Nguyen K K , Masaracchia A , Sharma V , et al . RIS-assisted UAV communications for IoT with wireless power transfer using deep reinforcement learning [J ] . IEEE Journal of Selected Topics in Signal Processing , 2022 , 16 ( 5 ): 1086 - 1096 . DOI: 10.1109/jstsp.2022.3172587 http://dx.doi.org/10.1109/jstsp.2022.3172587

Hong Zheng , Zhao Sai , Huang Gaofei , et al . RIS-assisted UAV NOMA secure communication based on deep reinforcement learning [J ] . Physical Communication , 2025 , 72 : 102713 . DOI: 10.1016/j.phycom.2025.102713 http://dx.doi.org/10.1016/j.phycom.2025.102713

Pang Xiaowei , Zhao Nan , Tang Jie , et al . IRS-assisted secure UAV transmission via joint trajectory and beamforming design [J ] . IEEE Transactions on Communications , 2022 , 70 ( 2 ): 1140 - 1152 . DOI: 10.1109/tcomm.2021.3136563 http://dx.doi.org/10.1109/tcomm.2021.3136563

胡浪涛 , 毕松姣 , 刘全金 , 等 . 基于强化学习的智能超表面辅助无人机通信系统物理层安全算法 [J ] . 电子与信息学报 , 2022 , 44 ( 7 ): 2407 - 2415 . DOI: 10.11999/JEIT211613 http://dx.doi.org/10.11999/JEIT211613

Hu Langtao , Bi Songjiao , Liu Quanjin , et al . Physical layer security algorithm of reconfigurable intelligent surface-assisted unmanned aerial vehicle communication system based on reinforcement learning [J ] . Journal of Electronics & Information Technology , 2022 , 44 ( 7 ): 2407 - 2415 . (in Chinese) . DOI: 10.11999/JEIT211613 http://dx.doi.org/10.11999/JEIT211613

Xu Qian , You Qian , Gong Yanyun , et al . RIS-assisted UAV-enabled green communications for industrial IoT exploiting deep learning [J ] . IEEE Internet of Things Journal , 2024 , 11 ( 16 ): 26595 - 26609 . DOI: 10.1109/jiot.2024.3369687 http://dx.doi.org/10.1109/jiot.2024.3369687

Feng Wanmei , Tang Jie , Wu Qingqing , et al . Resource allocation for power minimization in RIS-assisted multi-UAV networks with NOMA [J ] . IEEE Transactions on Communications , 2023 , 71 ( 11 ): 6662 - 6676 . DOI: 10.1109/tcomm.2023.3298984 http://dx.doi.org/10.1109/tcomm.2023.3298984

Moon S , Lee C G , Liu Huaping , et al . Deep reinforcement learning-based sum rate maximization for RIS-assisted ISAC-UAV network [J ] . ICT Express , 2024 , 10 ( 5 ): 1174 - 1178 . DOI: 10.1016/j.icte.2024.09.002 http://dx.doi.org/10.1016/j.icte.2024.09.002

Zhao Binglin , Zhai Linbo , Sun Jiande , et al . Beamforming design and trajectory optimization for learning-based multi-UAV-assisted integrated sensing and communication systems [J ] . Computer Networks , 2025 , 273 : 111751 . DOI: 10.1016/j.comnet.2025.111751 http://dx.doi.org/10.1016/j.comnet.2025.111751

Wang Wei , Li Peizheng , Doufexi A , et al . A heuristic-integrated DRL approach for phase optimization in large-scale RISs [J ] . IEEE Communications Letters , 2025 , 29 ( 7 ): 1579 - 1583 . DOI: 10.1109/lcomm.2025.3568652 http://dx.doi.org/10.1109/lcomm.2025.3568652

Tang Xiao , Xiong Zhihao , Dong Limeng , et al . UAV-enabled aerial active RIS with learning deployment for secured wireless communications [J ] . Chinese Journal of Aeronautics , 2025 , 38 ( 10 ): 103383 . DOI: 10.1016/j.cja.2024.103383 http://dx.doi.org/10.1016/j.cja.2024.103383

Wang Jianwei , Chen Shuo . Deep reinforcement learning-based secrecy rate optimization for simultaneously transmitting and reflecting reconfigurable intelligent surface-assisted unmanned aerial vehicle-integrated sensing and communication systems [J ] . Sensors , 2025 , 25 ( 5 ): 1541 . DOI: 10.3390/s25051541 http://dx.doi.org/10.3390/s25051541

Tesfaw B A , Juang R T , Lin H P , et al . Multi-agent deep reinforcement learning based optimizing joint 3D trajectories and phase shifts in RIS-assisted UAV-enabled wireless communications [J ] . IEEE Open Journal of Vehicular Technology , 2024 , 5 : 1712 - 1726 . DOI: 10.1109/OJVT.2024.3486197 http://dx.doi.org/10.1109/OJVT.2024.3486197

Huroon A M , Tarekegn G B , Abdo A M A , et al . Multiagent deep reinforcement learning for AAV-RIS-assisted integrated sensing and communication [J ] . IEEE Internet of Things Journal , 2025 , 12 ( 19 ): 40083 - 40097 . DOI: 10.1109/jiot.2025.3588482 http://dx.doi.org/10.1109/jiot.2025.3588482

Wang Liang , Wang Kezhi , Pan Cunhua , et al . Multi-agent deep reinforcement learning-based trajectory planning for multi-UAV assisted mobile edge computing [J ] . IEEE Transactions on Cognitive Communications and Networking , 2021 , 7 ( 1 ): 73 - 84 . DOI: 10.1109/TCCN.2020.3027695 http://dx.doi.org/10.1109/TCCN.2020.3027695

Zhao Jingjing , Yu Lanchenhui , Cai Kaiquan , et al . RIS-aided ground-aerial NOMA communications: A distributionally robust DRL approach [J ] . IEEE Journal on Selected Areas in Communications , 2022 , 40 ( 4 ): 1287 - 1301 . DOI: 10.1109/jsac.2022.3143230 http://dx.doi.org/10.1109/jsac.2022.3143230

Qin Zhenquan , Liu Zhonghao , Han Guangjie , et al . Distributed UAV-BSs trajectory optimization for user-level fair communication service with multi-agent deep reinforcement learning [J ] . IEEE Transactions on Vehicular Technology , 2021 , 70 ( 12 ): 12290 - 12301 . DOI: 10.1109/tvt.2021.3117792 http://dx.doi.org/10.1109/tvt.2021.3117792

Li Linpei , Guan Wanqing , Zhao Chuan , et al . Trajectory planning, phase shift design, and IoT devices association in flying-RIS-assisted mobile edge computing [J ] . IEEE Internet of Things Journal , 2024 , 11 ( 1 ): 147 - 157 . DOI: 10.1109/jiot.2023.3300700 http://dx.doi.org/10.1109/jiot.2023.3300700

常宽 , 张雷 , 王玉 , 等 . 辅助车辆通信的双智能反射面相移优化与无人机轨迹控制 [J ] . 电子与信息学报 , 2025 , 47 ( 10 ): 3670 - 3679 .

Chang Kuan , Zhang Lei , Wang Yu , et al . Dual-reconfigurable intelligent surface phase shift optimization and unmanned aerial vehicle trajectory control for vehicle communication [J ] . Journal of Electronics & Information Technology , 2025 , 47 ( 10 ): 3670 - 3679 . (in Chinese)

周毅 , 晋占齐 , 石华光 , 等 . 空中智能反射面辅助的无线供能通信网络轨迹优化研究 [J ] . 电子与信息学报 , 2024 , 46 ( 7 ): 2812 - 2820 .

Zhou Yi , Jin Zhanqi , Shi Huaguang , et al . Trajectory optimization research of wireless power communication networks assisted by aerial intelligent reflecting surface [J ] . Journal of Electronics & Information Technology , 2024 , 46 ( 7 ): 2812 - 2820 . (in Chinese)

贾向东 , 张鑫 , 原帅前 , 等 . IRS辅助的UAV无线传感网络数据采集优化方案 [J ] . 信号处理 , 2024 , 40 ( 6 ): 1041 - 1051 .

Jia Xiangdong , Zhang Xin , Yuan Shuaiqian , et al . IRS-assisted UAV wireless sensor network data collection optimization scheme [J ] . Journal of Signal Processing , 2024 , 40 ( 6 ): 1041 - 1051 . (in Chinese)

Dai Linglong , Wang Bichai , Wang Min , et al . Reconfigurable intelligent surface-based wireless communications: Antenna design, prototyping, and experimental results [J ] . IEEE Access , 2020 , 8 : 45913 - 45923 . DOI: 10.1109/access.2020.2977772 http://dx.doi.org/10.1109/access.2020.2977772

Wei Xiuhong , Dai Linglong , Zhao Yajun , et al . Codebook design and beam training for extremely large-scale RIS: Far-field or near-field? [J ] . China Communications , 2022 , 19 ( 6 ): 193 - 204 . DOI: 10.23919/jcc.2022.06.015 http://dx.doi.org/10.23919/jcc.2022.06.015

Wei Xiuhong , Shen Decai , Dai Linglong . Channel estimation for RIS assisted wireless communications - Part I: Fundamentals, solutions, and future opportunities [J ] . IEEE Communications Letters , 2021 , 25 ( 5 ): 1398 - 1402 . DOI: 10.1109/lcomm.2021.3052822 http://dx.doi.org/10.1109/lcomm.2021.3052822

Dai Linglong , Wei Xiuhong . Distributed machine learning based downlink channel estimation for RIS assisted wireless communications [J ] . IEEE Transactions on Communications , 2022 , 70 ( 7 ): 4900 - 4909 . DOI: 10.1109/tcomm.2022.3175175 http://dx.doi.org/10.1109/tcomm.2022.3175175

Di Renzo M , Zappone A , Debbah M , et al . Smart radio environments empowered by reconfigurable intelligent surfaces: How it works, state of research, and the road ahead [J ] . IEEE Journal on Selected Areas in Communications , 2020 , 38 ( 11 ): 2450 - 2525 . DOI: 10.1109/jsac.2020.3007211 http://dx.doi.org/10.1109/jsac.2020.3007211

Liu Yuanwei , Liu Xiao , Mu Xidong , et al . Reconfigurable intelligent surfaces: Principles and opportunities [J ] . IEEE Communications Surveys & Tutorials , 2021 , 23 ( 3 ): 1546 - 1577 . DOI: 10.1109/comst.2021.3077737 http://dx.doi.org/10.1109/comst.2021.3077737

Abeywickrama S , Zhang Rui , Wu Qingqing , et al . Intelligent reflecting surface: Practical phase shift model and beamforming optimization [J ] . IEEE Transactions on Communications , 2020 , 68 ( 9 ): 5849 - 5863 . DOI: 10.1109/tcomm.2020.3001125 http://dx.doi.org/10.1109/tcomm.2020.3001125

徐俊杰 , 李斌 , 杨敬松 . 禁飞区约束下的无人机可重构智能表面辅助通信网络性能优化 [J ] . 电子与信息学报 , 2026 , 48 ( 2 ): 743 - 751 .

Xu Junjie , Li Bin , Yang Jingsong . Performance optimization of UAV-RIS-assisted communication networks under No-fly zone constraints [J ] . Journal of Electronics & Information Technology , 2026 , 48 ( 2 ): 743 - 751 . (in Chinese)

Zhang Zhengming , Huang Yongming , Zhang Cheng , et al . Digital twin-enhanced deep reinforcement learning for resource management in networks slicing [J ] . IEEE Transactions on Communications , 2024 , 72 ( 10 ): 6209 - 6224 . DOI: 10.1109/tcomm.2024.3395698 http://dx.doi.org/10.1109/tcomm.2024.3395698

Kurma S , Singh K , Sharma P K , et al . DRL approach for spectral-energy trade-off in RIS-assisted full-duplex multi-user MIMO systems [C ] // Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC) . Piscataway : IEEE , 2023 : 1 - 6 . DOI: 10.1109/wcnc55385.2023.10118961 http://dx.doi.org/10.1109/wcnc55385.2023.10118961

Shang Bodong , Poor H V , Liu Lingjia . Aerial reconfigurable intelligent surfaces meet mobile edge computing [J ] . IEEE Wireless Communications , 2022 , 29 ( 6 ): 104 - 111 . DOI: 10.1109/mwc.001.2200009 http://dx.doi.org/10.1109/mwc.001.2200009

Ning Zhaolong , Li Tengfeng , Wu Yu , et al . 6G communication new paradigm: The integration of autonomous aerial vehicles and intelligent reflecting surfaces [J ] . IEEE Communications Surveys & Tutorials , 2025 , 27 ( 6 ): 3382 - 3416 . DOI: 10.1109/comst.2025.3526251 http://dx.doi.org/10.1109/comst.2025.3526251

Huang Chongwen , Yang Zhaohui , Alexandropoulos G C , et al . Multi-hop RIS-empowered terahertz communications: A DRL-based hybrid beamforming design [J ] . IEEE Journal on Selected Areas in Communications , 2021 , 39 ( 6 ): 1663 - 1677 . DOI: 10.1109/jsac.2021.3071836 http://dx.doi.org/10.1109/jsac.2021.3071836

Cui Mingyao , Wu Zidong , Lu Yu , et al . Near-field MIMO communications for 6G: Fundamentals, challenges, potentials, and future directions [J ] . IEEE Communications Magazine , 2023 , 61 ( 1 ): 40 - 46 . DOI: 10.1109/MCOM.004.2200136 http://dx.doi.org/10.1109/MCOM.004.2200136

Liu Yuanwei , Ouyang Chongjun , Wang Zhaolin , et al . Near-field communications: A comprehensive survey [J ] . IEEE Communications Surveys & Tutorials , 2025 , 27 ( 3 ): 1687 - 1728 . DOI: 10.1109/comst.2024.3475884 http://dx.doi.org/10.1109/comst.2024.3475884

Le Ha an , Van Chien T , Choi W . Graph neural network-based active and passive beamforming for distributed STAR-RIS-assisted multi-user MISO systems [J ] . IEEE Transactions on Communications , 2025 , 73 ( 10 ): 9299 - 9312 . DOI: 10.1109/tcomm.2025.3558504 http://dx.doi.org/10.1109/tcomm.2025.3558504

Chen Lili , Lu K , Rajeswaran A , et al . Decision transformer: Reinforcement learning via sequence modeling [C ] // Proceedings of the 35th International Conference on Neural Information Processing Systems . Red Hook : Curran Associates Inc. , 2021 : 15084 - 15097 .

陈真 , 杜晓宇 , 唐杰 , 等 . 基于深度强化学习的RIS辅助通感融合网络: 挑战与机遇 [J ] . 电子与信息学报 , 2024 , 46 ( 9 ): 3467 - 3473 .

Chen Zhen , Du Xiaoyu , Tang Jie , et al . DRL-based RIS-assisted ISAC network: Challenges and opportunities [J ] . Journal of Electronics & Information Technology , 2024 , 46 ( 9 ): 3467 - 3473 . (in Chinese)

张广驰 , 何梓楠 , 崔苗 . 基于深度强化学习的无人机辅助移动边缘计算系统能耗优化 [J ] . 电子与信息学报 , 2023 , 45 ( 5 ): 1635 - 1643 .

Zhang Guangchi , He Zinan , Cui Miao . Energy consumption optimization of unmanned aerial vehicle assisted mobile edge computing systems based on deep reinforcement learning [J ] . Journal of Electronics & Information Technology , 2023 , 45 ( 5 ): 1635 - 1643 . (in Chinese)

Li Hongyu , Shen Shanpu , Nerini M , et al . Reconfigurable intelligent surfaces 2.0: Beyond diagonal phase shift matrices [J ] . IEEE Communications Magazine , 2024 , 62 ( 3 ): 102 - 108 . DOI: 10.1109/mcom.001.2300019 http://dx.doi.org/10.1109/mcom.001.2300019

李国权 , 胡航 , 王玥涛 , 等 . STAR-RIS辅助的CR-SWIPT系统安全波束成形算法 [J ] . 电子学报 , 2024 , 52 ( 12 ): 4002 - 4008 . DOI: 10.12263/DZXB.20231098 http://dx.doi.org/10.12263/DZXB.20231098

Li Guoquan , Hu Hang , Wang Yuetao , et al . Secure beamforming algorithm for STAR-RIS assisted cognitive radio systems with SWIPT [J ] . Acta Electronica Sinica , 2024 , 52 ( 12 ): 4002 - 4008 . (in Chinese) . DOI: 10.12263/DZXB.20231098 http://dx.doi.org/10.12263/DZXB.20231098

Views

187

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Resource Allocation of IRS-Assisted UAV Covert Communication Systems with User Fairness Consideration

Reconfigurable Intelligent Surface Assisted Robust Secrecy Transmission for Wideband Terahertz Integrated Sensing and Communication Systems

Topology-Aware Dual-Graph Reinforcement Learning for Spectrum Management in Low-Altitude Internet of Things

An Efficient BiCore-Index Maintenance Algorithm on Dynamic Bipartite Graphs

Impulsive Noise Suppression Based on Sparse Bayesian Learning for OTFS Systems

Related Author

WANG Zheng-qiang

LI Chun

REN Xin-zhi

XU Yong-jun

HAO Wanming

WAN Xiangyang

ZHU Zhengyu

LI Xingwang

Related Institution

School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications

School of Electrical Information Engineering, Zhengzhou University

School of Physical and Electronic Information Engineering, Henan Polytechnic University

School of Business, Central South University

Xiangjiang Laboratory

⁰