无蜂窝毫米波大规模MIMO系统基于深度强化学习的节能睡眠策略
Energy-Efficient Sleep-Mode Based on Deep Reinforcement Learning for Cell-Free mmWave Massive MIMO Systems
- 电子学报 2023年51卷第10期 页码：2831-2843
- 作者机构：
1.重庆邮电大学通信与信息工程学院，重庆 400065
2.重庆邮电大学通信核心芯片、协议及系统应用团队，重庆 400065
- 作者简介：
[ "何云 女，1979年生，湖北武汉人，现为重庆邮电大学博士生研究生.主要研究方向为协作通信、大规模MIMO 系统能效优化.E-mail: heyun@cqupt.edu.cn" ]
[ "申敏（通讯作者） 女，1963年生，贵州湄潭人.现为重庆邮电大学教授、博士生导师.主要研究方向为通信核心芯片、协议与系统应用技术." ]
[ "王蕊 女，1988年生，云南玉溪人.现为重庆邮电大学博士生研究生.主要研究方向为Cell-Free大规模MIMO系统的资源分配、动态协作和预编码.E-mail: d190101012@stu.cqupt.edu.cn" ]
[ "张梦 女，1988年生，重庆人.现为浙江大学博士后.主要研究方向为物理层安全，物联网安全. E-mail: zhangmengyang@zju.edu.cn" ]
- 基金信息：国家科技重大专项基金(2018ZX03001026-002)
DOI：10.12263/DZXB.20220247
中图分类号： TN929.5;收稿：2021-10-08，
修回：2023-09-01，
纸质出版：2023-10-25
- 稿件说明：
移动端阅览
何云,申敏,王蕊等.无蜂窝毫米波大规模MIMO系统基于深度强化学习的节能睡眠策略[J].电子学报,2023,51(10):2831-2843.
HE Yun,SHEN Min,WANG Rui,et al.Energy-Efficient Sleep-Mode Based on Deep Reinforcement Learning for Cell-Free mmWave Massive MIMO Systems[J].ACTA ELECTRONICA SINICA,2023,51(10):2831-2843.
何云,申敏,王蕊等.无蜂窝毫米波大规模MIMO系统基于深度强化学习的节能睡眠策略[J].电子学报,2023,51(10):2831-2843. DOI： 10.12263/DZXB.20220247.
HE Yun,SHEN Min,WANG Rui,et al.Energy-Efficient Sleep-Mode Based on Deep Reinforcement Learning for Cell-Free mmWave Massive MIMO Systems[J].ACTA ELECTRONICA SINICA,2023,51(10):2831-2843. DOI： 10.12263/DZXB.20220247.
摘要
为了提升无蜂窝毫米波大规模MIMO （Cell-Free millimeter-Wave massive MIMO，CF mmWave mMIMO）系统总能量效率，本文研究时变信道环境中接入点（Access Point，AP）睡眠节能机制.将AP开关切换（AP Switch ON-OFF，ASO）策略看作一个马尔可夫决策过程，使用深度强化学习（Deep Reinforcement Learning，DRL）工具解决AP开关问题.引入干扰感知技术和局部敏感哈希检索方法减少代理与复杂环境的交互以及样本偏差，构造了一个新的效用函数，在严格用户服务质量（Quality of Service，QoS）约束下更好地权衡总能效和可达速率性能.通过对效用函数离散化分级处理，将状态空间映射为更小的分级状态空间，以加快决斗深度Q网络（Dueling Deep Q-Network，Dueling DQN）的收敛速度.仿真结果证明了该方案的稳定性、收敛性和严格QoS约束下的总能效性能优势.
Abstract
To improve the global energy-efficiency (GEE) performance in cell-free millimeter-wave massive MIMO (CF mmWave mMIMO) systems
the access points (APs) sleep-mode techniques in dynamic time-varying channels are investigated. The AP switch ON-OFF (ASO) strategy is formulated as a Markov decision process. Thus
a deep reinforcement learning (DRL) model can be used to solve the AP activation problem. The interference-aware method and the locality-sensitive hashing method are introduced to reduce sample bias and interaction between agents and complex environments. A novel cost function is constructed to achieve a better balance between GEE and achievable rate under the strict quality of service (QoS) constraints. In order to accelerate the convergence of the dueling deep Q-Network (DQN)
the state space is mapped to the smaller hierarchical state space by discretizing the cost function. Simulation results have demonstrated the performance advantage of the convergence of deep reinforcement learning and GEE under the strict QoS constraint.
关键词
Keywords
references
KIM S , SHIM B . Energy-efficient millimeter-wave cell-free systems under limited feedback [J]. IEEE Transactions on Communications , 2021 , 69 ( 6 ): 4067 - 4082 .
VAN CHIEN T , BJöRNSON E , LARSSON E G . Joint power allocation and load balancing optimization for energy-efficient cell-free massive MIMO networks [J]. IEEE Transactions on Wireless Communications , 2020 , 19 ( 10 ): 6798 - 6812 .
FEMENIAS G , LASSOUED N , RIERA-PALOU F . Access point switch ON/OFF strategies for green cell-free massive MIMO networking [J]. IEEE Access , 2020 , 8 : 21788 - 21803 .
USAMA M , EROL-KANTARCI M . A survey on recent trends and open issues in energy efficiency of 5G [J]. Sensors , 2019 , 19 ( 14 ): 3126 .
ZHUANG B , GUO D , HONIG M L . Energy-efficient cell activation, user association, and spectrum allocation in heterogeneous networks [J]. IEEE Journal on Selected Areas in Communications , 2016 , 34 ( 4 ): 823 - 831 .
GARCíA-MORALES J , FEMENIAS G , RIERA-PALOU F . Energy-efficient access-point sleep-mode techniques for cell-free mmWave massive MIMO networks with non-uniform spatial traffic density [J]. IEEE Access , 2020 , 8 : 137587 - 137605 .
HE H , JIN S , WEN C-K , et al . Model-driven deep learning for physical layer communications [J]. IEEE Wireless Communications , 2019 , 26 ( 5 ): 77 - 83 .
PHAM Q V , MIRJALILI S , KUMAR N , et al . Whale optimization algorithm with applications to resource allocation in wireless networks [J]. IEEE Transactions on Vehicular Technology , 2020 , 69 ( 4 ): 4285 - 4297 .
MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J]. Nature , 2015 , 518 ( 7540 ): 529 - 533 .
HASHMI U S , ZAIDI S A R , IMRAN A , et al . Enhancing downlink QoS and energy efficiency through a user-centric Stienen cell architecture for mmWave networks [J]. IEEE Transactions on Green Communications and Networking , 2020 , 4 ( 2 ): 387 - 403 .
SUN G L , ZHAN T , OWUSU B G , et al . Revised reinforcement learning based on anchor graph hashing for autonomous cell activation in cloud-RANs [J]. Future Generation Computer Systems , 2020 , 104 : 60 - 73 .
SUN G L , AYEPAH-MENSAH D , XU R , et al . Transfer learning for autonomous cell activation based on relational reinforcement learning with adaptive reward [J]. IEEE Systems Journal , 2021 , 16 ( 1 ): 1044 - 1055 .
VAN HUYNH N , HOANG D T , NGUYEN D N , et al . DeepFake: Deep dueling-based deception strategy to defeat reactive jammers [J]. IEEE Transactions on Wireless Communications , 2021 , 20 ( 10 ): 6898 - 6914 .
DATAR M , IMMORLICA N , INDYK P , et al . Locality-sensitive hashing scheme based on p-stable distributions [C]// Proceedings of the twentieth annual symposium on Computational geometry . New York : ACM , 2004 : 253 - 262 .
CHAFIK S , YACOUBI M A EL , DAOUDI I , et al . Unsupervised deep neuron-per-neuron hashing [J]. Applied Intelligence , 2019 , 49 ( 6 ): 2218 - 2232 .
ALONZO M , BUZZI S , ZAPPONE A , et al . Energy-efficient power control in cell-free and user-centric massive MIMO at millimeter wave [J]. IEEE Transactions on Green Communications and Networking , 2019 , 3 ( 3 ): 651 - 663 .
RIBEIRO L N , SCHWARZ S , RUPP M , et al . Energy efficiency of mmWave massive MIMO precoding with low-resolution DACs [J]. IEEE Journal of Selected Topics in Signal Processing , 2018 , 12 ( 2 ): 298 - 312 .
WANG Z , SCHAUL T , HESSEL M , et al . Dueling network architectures for deep reinforcement learning [C]// International Conference on Machine Learning . New York : ICML , 2016 : 1995 - 2003 .
SARWAR S S , SRINIVASAN G , HAN B , et al . Energy efficient neural computing: A study of cross-layer approximations [J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems , 2018 , 8 ( 4 ): 796 - 809 .
ZHANG T , ZHU K , WANG J . Energy-efficient mode selection and resource allocation for D2D-enabled heterogeneous networks: A deep reinforcement learning approach [J]. IEEE Transactions on Wireless Communications , 2020 , 20 ( 2 ): 1175 - 1187 .
ZHANG J , HUANG Y , WANG J , et al . Intelligent interactive beam training for millimeter wave communications [J]. IEEE Transactions on Wireless Communications , 2020 , 20 ( 3 ): 2034 - 2048 .
0
浏览量
28
下载量
2
CSCD
- 网络PDF下载
- Meta-XML下载
- BibTeX下载
- Endnote下载
- RefMan 下载
- NoteExpress下载
- NoteFirst下载
关联资源
相关文章
相关作者
相关机构
- 技术支持由北京北大方正电子有限公司提供 京ICP备09064830号-19
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies