基于多智能体强化学习的源荷分离方法

华星源; 段思婧; 崔文朋; 陈玉哲; 杨清辰; 乔楠; 任炬

doi:10.12263/DZXB.20260338

您当前的位置：

首页 >

文章列表页 >

基于多智能体强化学习的源荷分离方法

学术论文 | 更新时间：2026-10-16

- 基于多智能体强化学习的源荷分离方法
- Source-Load Separation Method Based on Multi-Agent Reinforcement Learning
- 电子学报 2026年54卷第6期页码：2763-2775
- 作者机构：
  
  1.清华大学计算机科学与技术系，北京 100084
  2.北京智芯微电子科技有限公司，北京 102200
  3.中南大学计算机学院，湖南长沙 410083
- 作者简介：
  
  华星源男，2002年5月出生于北京市。现为清华大学计算机科学与技术系博士研究生。主要研究方向为强化学习、智能体后训练。中国电子学会会员编号：E190183325A。E-mail: huaxy24@mails.tsinghua.edu.cn
  段思婧女，1994年7月出生于湖南省娄底市。现为清华大学计算机系助理研究员。主要研究方向为移动计算、边缘智能、具身智能。E-mail: duansj@tsinghua.edu.cn
  崔文朋男，1987年5月出生于山东省济宁市。现为国家电网北京智芯微电子科技有限公司人工智能事业部总经理助理。主要研究方向为人工智能芯片及其应用。E-mail: cuiwenpeng@sgchip.sgcc.com.cn
  陈玉哲男，1999年3月出生于山东省青岛市。现为国家电网北京智芯微电子科技有限公司大模型应用开发组副组长。主要研究方向为人工智能芯片及其应用。E-mail: chenyuzhe1@sgchip.sgcc.com.cn
  杨清辰男，1998年5月出生于山东省东营市。现为国家电网北京智芯微电子科技有限公司工程师。主要研究方向为人工智能芯片及其应用。E-mail: qingchen.yang98@gmail.com
  乔楠男，1999年11月出生于山西省阳泉市。现为中南大学计算机学院博士研究生。主要研究方向为强化学习、大模型训练和端侧智能决策。E-mail: nanqiao.ai@gmail.com
  任炬男，1987年12月出生于湖南省汨罗市。现为清华大学计算机系副教授。主要研究方向为端侧智能与边缘智能。中国电子学会会员编号：E190018924S。E-mail: renju@tsinghua.edu.cn
- 基金信息：
  
  中国电子学会智芯科研专项(2024-02);国家自然科学基金(62432004;62402279);教育部“基础与交叉学科突破计划”(JYB2025XDXM122);国家资助博士后研究人员计划(BX20250375);中国博士后科学基金(2025M771503)
- DOI：10.12263/DZXB.20260338
  中图分类号： TP39;TM715
- 收稿：2026-04-28，
  
  录用：2026-05-21，
  
  纸质出版：2026-06-25
- 稿件说明：
移动端阅览
华星源, 段思婧, 崔文朋, 等. 基于多智能体强化学习的源荷分离方法[J]. 电子学报, 2026, 54(06): 2763-2775.

HUA Xingyuan, DUAN Sijing, CUI Wenpeng, et al. Source-Load Separation Method Based on Multi-Agent Reinforcement Learning[J]. Acta Electronica Sinica, 2026, 54(06): 2763-2775.
华星源, 段思婧, 崔文朋, 等. 基于多智能体强化学习的源荷分离方法[J]. 电子学报, 2026, 54(06): 2763-2775. DOI：10.12263/DZXB.20260338

HUA Xingyuan, DUAN Sijing, CUI Wenpeng, et al. Source-Load Separation Method Based on Multi-Agent Reinforcement Learning[J]. Acta Electronica Sinica, 2026, 54(06): 2763-2775. DOI：10.12263/DZXB.20260338

摘要

近年来，全球可持续能源发展的需求日益增长，推动了光伏发电系统在居民、商业及工业场景中的应用。为了保障电网运行安全和提升能源调度效率，如何准确预测用户侧的电力负荷变化尤为关键。然而现有方法忽略了实际用户侧接入电表所采集的数据中往往同时包含光伏发电（源）和用电负荷（荷）信息，二者未被有效分离，导致预测结果存在偏差。本文提出基于多智能体强化学习的源荷分离方法SoLED，解决了用户之间复杂时空依赖关系建模能力不足，以及负荷与光伏发电混合数据未分离导致的预测偏差的问题。具体地，本文首先设计了一种特征提取模块，能够捕捉用户负荷短期与长期尺度上的变化规律，从而提升对电网时序特征的表达能力。然后，基于物理世界的电网结构构建仿真电网环境，通过在仿真环境中模拟增强时序特征对空间关系的敏感性。最后，提出了基于多智能体强化学习的源荷分离方法，利用仿真环境生成反馈来训练解耦模型，解决了数据分离困难导致的解耦不准确问题。通过仿真实验和实际系统验证，本文提出方法的准确率在两个城镇电网真实数据集中的不同天气条件下相较现有方法分别提升5.7%~24.6%和6.6%~24.2%。该实验结果表明，提出的方法能够准确解耦功率，有效地解决了用户之间复杂时空依赖关系建模能力不足，以及负荷与光伏发电混合数据未分离导致的预测偏差的问题。

Abstract

In recent years

the growing demand for global sustainable energy development has accelerated the deployment of photovoltaic (PV) systems across residential

commercial

and industrial sectors. To ensure the safe operation of power grids and improve energy dispatch efficiency

accurately forecasting user-side power load variations has become increasingly critical. However

existing methods often overlook that the data collected from user-side smart meters typically contain both PV generation (source) and power consumption (load) components

which are not effectively separated

leading to biased prediction results. To address this issue

this paper proposes a multi-agent reinforcement learning-based source-load decoupling method

namely SoLED

that mitigates prediction bias caused by the unseparated mixed data of load and PV generation

as well as insufficient modeling of complex spatiotemporal dependencies among users. Specifically

we first design a feature extraction module to capture both short-term and long-term variations in user load

thereby enhancing the representation of spatiotemporal characteristics within the power grid. Then

based on the physical topology of the grid

we construct a simulation environment that models voltage responses under different load and PV power conditions. Finally

leveraging the feedback generated from this simulation

we train a decoupling model within a multi-agent reinforcement learning framework to achieve accurate source-load separation. Experimental results demonstrate that the proposed method improves prediction accuracy by 5.7%~24.6% and 6.6%~24.2% under different weather conditions on two real-world urban power grid datasets. These results confirm the effectiveness of the proposed approach in accurately decoupling user-side mixed power data and enhancing the modeling capability of complex spatiotemporal dependencies

thereby reducing the prediction bias in user-side load forecasting.

关键词

Keywords

references

Detchon R , Van Leeuwen R . Policy: Bring sustainable energy to the developing world [J ] . Nature , 2014 , 508 ( 7496 ): 309 - 311 . DOI: 10.1038/508309a http://dx.doi.org/10.1038/508309a

Batzelis E , Rather Z H , Barton J , et al . Solar integration in the UK and India: Technical barriers and future directions [PP/OL ] . V1. arXiv ( 2021-04-01 )[ 2023-06-15 ] . https://doi.org/10.17028/rd.lboro.14453133.v1 https://doi.org/10.17028/rd.lboro.14453133.v1 .

杨峰 , 刘心松 , 邱元杰 , 等 . 电网调度自动化的任务服务模型和调度算法研究 [J ] . 电子学报 , 2002 , 30 ( 12 ): 1811 - 1815 .

Yang Feng , Liu Xinsong , Qiu Yuanjie , et al . Study on task service model and scheduling algorithm in electric power automatization [J ] . Acta Electronica Sinica , 2002 , 30 ( 12 ): 1811 - 1815 . (in Chinese)

于昕妍 , 沈艳霞 , 陈杰 , 等 . 考虑概率区间的微电网短期负荷多目标预测方法 [J ] . 电子学报 , 2017 , 45 ( 4 ): 930 - 936 .

Yu Xinyan , Shen Yanxia , Chen Jie , et al . A multi-objective prediction method for short-term microgrid load considering interval probability [J ] . Acta Electronica Sinica , 2017 , 45 ( 4 ): 930 - 936 . (in Chinese)

王后能 , 殷劲骁 , 廖小兵 , 等 . 微电网并离网切换系统混合樽海鞘优化控制策略 [J ] . 电子学报 , 2025 , 53 ( 11 ): 4091 - 4103 .

Wang Houneng , Yin Jinxiao , Liao Xiaobing , et al . Hybrid salp swarm optimization control strategy for microgrid grid-connected and islanded mode switching system [J ] . Acta Electronica Sinica , 2025 , 53 ( 11 ): 4091 - 4103 . (in Chinese)

Chen Xin , Qu Guannan , Tang Yujie , et al . Reinforcement learning for selective key applications in power systems: Recent advances and future challenges [J ] . IEEE Transactions on Smart Grid , 2022 , 13 ( 4 ): 2935 - 2958 . DOI: 10.1109/tsg.2022.3154718 http://dx.doi.org/10.1109/tsg.2022.3154718

吴忠强 , 程洪强 . 网络攻击下考虑状态受限的微电网安全运行与控制 [J ] . 电子学报 , 2024 , 52 ( 9 ): 3240 - 3250 . DOI: 10.12263/DZXB.20221092 http://dx.doi.org/10.12263/DZXB.20221092

Wu Zhongqiang , Cheng Hongqiang . Safe operation and control of microgrid considering state constraints under network attacks [J ] . Acta Electronica Sinica , 2024 , 52 ( 9 ): 3240 - 3250 . (in Chinese) . DOI: 10.12263/DZXB.20221092 http://dx.doi.org/10.12263/DZXB.20221092

Xu Feiyang , Liu Shunyu , Qing Yunpeng , et al . Temporal prototype-aware learning for active voltage control on power distribution networks [C ] // Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . New York : ACM , 2024 : 3598 - 3609 . DOI: 10.1145/3637528.3671790 http://dx.doi.org/10.1145/3637528.3671790

Liu Shunyu , Luo Wei , Zhou Yanzhen , et al . Transmission interface power flow adjustment: A deep reinforcement learning approach based on multi-task attribution map [J ] . IEEE Transactions on Power Systems , 2024 , 39 ( 2 ): 3324 - 3335 . DOI: 10.1109/tpwrs.2023.3298007 http://dx.doi.org/10.1109/tpwrs.2023.3298007

Shi Xingjian , Chen Zhourong , Wang Hao , et al . Convolutional LSTM Network: A machine learning approach for precipitation nowcasting [C ] // Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1 . New York : ACM , 2015 : 802 - 810 .

Vaswani A , Shazeer N , Parmar N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 6000 - 6010 .

Wu Zonghan , Pan Shirui , Chen Fengwen , et al . A comprehensive survey on graph neural networks [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2021 , 32 ( 1 ): 4 - 24 . DOI: 10.1109/tnnls.2020.2978386 http://dx.doi.org/10.1109/tnnls.2020.2978386

Masters C L . Voltage rise: The big issue when connecting embedded generation to long 11 kV overhead lines [J ] . Power Engineering Journal , 2002 , 16 ( 1 ): 5 - 12 . DOI: 10.1049/pe:20020101 http://dx.doi.org/10.1049/pe:20020101

Yang Guangya , Marra F , Juamperez M , et al . Voltage rise mitigation for solar PV integration at LV grids [J ] . Journal of Modern Power Systems and Clean Energy , 2015 , 3 ( 3 ): 411 - 421 . DOI: 10.1007/s40565-015-0132-0 http://dx.doi.org/10.1007/s40565-015-0132-0

Agalgaonkar Y P , Pal B C , Jabr R A . Distribution voltage control considering the impact of PV generation on tap changers and autonomous regulators [J ] . IEEE Transactions on Power Systems , 2014 , 29 ( 1 ): 182 - 192 . DOI: 10.1109/tpwrs.2013.2279721 http://dx.doi.org/10.1109/tpwrs.2013.2279721

Huang Jia , Wei Qing , Wang Tiankuo , et al . Spatio-temporal feature fusion-based hybrid GAT-CNN-LSTM model for enhanced short-term power load forecasting [J ] . Energies , 2025 , 18 ( 21 ): 5686 . DOI: 10.3390/en18215686 http://dx.doi.org/10.3390/en18215686

Su Zhonge , Zheng Guoqiang , Hu Miaosen , et al . Short-term load forecasting of regional integrated energy system based on spatio-temporal convolutional graph neural network [J ] . Electric Power Systems Research , 2024 , 232 : 110427 . DOI: 10.1016/j.epsr.2024.110427 http://dx.doi.org/10.1016/j.epsr.2024.110427

Cheng Min , Yu Jiawei , Wu Mingkang , et al . A spatiotemporal deep learning framework for joint load and renewable energy forecasting in stability-constrained power systems [J ] . Information , 2025 , 16 ( 8 ): 662 . DOI: 10.3390/info16080662 http://dx.doi.org/10.3390/info16080662

Chung J , Gulcehre C , Cho K , et al . Empirical evaluation of gated recurrent neural networks on sequence modeling [PP/OL ] . V1.arXiv ( 2014-12-11 )[ 2023-06-15 ] . https://doi.org/10.48550/arXiv.1412.3555 https://doi.org/10.48550/arXiv.1412.3555 .

LeCun Y , Bottou L , Bengio Y , et al . Gradient-based learning applied to document recognition [J ] . Proceedings of the IEEE , 1998 , 86 ( 11 ): 2278 - 2324 . DOI: 10.1109/5.726791 http://dx.doi.org/10.1109/5.726791

Liu Yicheng , Zhang Jinghuai , Fang Liangji , et al . Multimodal motion prediction with stacked transformers [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2021 : 7573 - 7582 . DOI: 10.48550/arXiv.2103.11624 http://dx.doi.org/10.48550/arXiv.2103.11624

Kipf T N , Welling M . Semi-supervised classification with graph convolutional networks [C ] // International Conference on Learning Representation . Washington DC : ICLR , 2017 .

Javed F , Arshad N , Wallin F , et al . Forecasting for demand response in smart grids: An analysis on use of anthropologic and structural data and short term multiple loads forecasting [J ] . Applied Energy , 2012 , 96 : 150 - 160 . DOI: 10.1016/j.apenergy.2012.02.027 http://dx.doi.org/10.1016/j.apenergy.2012.02.027

Yuan Ye , Yuan Xinping , Wang Haiyan , et al . Net load forecasting method in distribution grid planning based on LSTM network [J ] . Science and Technology for Energy Transition , 2024 , 79 : 57 . DOI: 10.2516/stet/2024062 http://dx.doi.org/10.2516/stet/2024062

Kumar N . Optimized power distribution using machine learning for load forecasting, fault detection, and voltage regulation [J ] . International Journal of Research and Review in Applied Science, Humanities, and Technology , 2024 : 94 - 102 . DOI: 10.71143/0m5nm797 http://dx.doi.org/10.71143/0m5nm797

Awad M , Khanna R . Support vector regression [M ] // Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers . Berkeley, CA : Apress , 2015 : 67 - 80 . DOI: 10.1007/978-1-4302-5990-9_4 http://dx.doi.org/10.1007/978-1-4302-5990-9_4

Wang Guang , Xie Xiaoyang , Zhang Fan , et al . bCharge: Data-driven real-time charging scheduling for large-scale electric bus fleets [C ] // 2018 IEEE Real-Time Systems Symposium . Piscataway : IEEE , 2018 : 45 - 55 . DOI: 10.1109/rtss.2018.00015 http://dx.doi.org/10.1109/rtss.2018.00015

Zaremba W , Sutskever I , Vinyals O . Recurrent neural network regularization [PP/OL ] . V5. arXiv ( 2015-02-19 )[ 2023-06-15 ] . https://doi.org/10.48550/arXiv.1409.2329 https://doi.org/10.48550/arXiv.1409.2329 .

Zhou Haoyi , Zhang Shanghang , Peng Jieqi , et al . Informer: Beyond efficient transformer for long sequence time-series forecasting [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 12 ): 11106 - 11115 . DOI: 10.1609/aaai.v35i12.17325 http://dx.doi.org/10.1609/aaai.v35i12.17325

Lu Shuai , Bao Taotao . Short-term electricity load forecasting based on NeuralProphet and CNN-LSTM [J ] . IEEE Access , 2024 , 12 : 76870 - 76879 . DOI: 10.1109/access.2024.3407094 http://dx.doi.org/10.1109/access.2024.3407094

Jia Yilong , Wang Jun , Hosseini M R , et al . Temporal graph attention network for building thermal load prediction [J ] . Energy and Buildings , 2024 , 321 : 113507 . DOI: 10.1016/j.enbuild.2023.113507 http://dx.doi.org/10.1016/j.enbuild.2023.113507

Orji U , Güven Ç , Stowell D . Enhanced load forecasting with GAT-LSTM: Leveraging grid and temporal features [PP/OL ] . V1. arXiv ( 2025-02-12 )[ 2023-06-15 ] . https://doi.org/10.48550/arXiv.2502. 08376 https://doi.org/10.48550/arXiv.2502.08376 .

Yarats D , Fergus R , Lazaric A , et al . Reinforcement learning with prototypical representations [C ] // Proceedings of the 38th International Conference on Machine Learning . New York : PMLR , 2021 : 11920 - 11931 .

Zhang Zaixi , Liu Qi , Wang Hao , et al . ProtGNN: Towards self-explaining graph neural networks [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 8 ): 9127 - 9135 . DOI: 10.1609/aaai.v36i8.20898 http://dx.doi.org/10.1609/aaai.v36i8.20898

Mnih V , Kavukcuoglu K , Silver D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 . DOI: 10.1038/nature14236 http://dx.doi.org/10.1038/nature14236

Schulman J , Moritz P , Levine S , et al . High-dimensional continuous control using generalized advantage estimation [PP/OL ] . V6. arXiv ( 2018-10-20 )[ 2023-06-15 ] . https://doi.org/10.48550/arXiv.1506.02438 https://doi.org/10.48550/arXiv.1506.02438 .

Fujimoto S , Van Hoof H , Meger D . Addressing function approximation error in actor-critic methods [C ] // Proceedings of the 35th International Conference on Machine Learning . New York : PMLR , 2018 : 1587 - 1596 . DOI: 10.48550/arXiv.1802.09477 http://dx.doi.org/10.48550/arXiv.1802.09477

Schulman J , Wolski F , Dhariwal P , et al . Proximal policy optimization algorithms [PP/OL ] . V2. arXiv ( 2017-08-28 )[ 2023-06-15 ] . https://doi.org/10.48550/arXiv.1707.06347 https://doi.org/10.48550/arXiv.1707.06347 .

Bilal M , Lopez L C . Temporal graph MLP mixer for spatio-temporal forecasting [PP/OL ] . V1. arXiv ( 2025-01-17 )[ 2023-06-15 ] . https://doi.org/10.48550/arXiv.2501.10214 https://doi.org/10.48550/arXiv.2501.10214 .

Wang Xing , Wang Xiaojun , Huang Faliang , et al . DSTF: A Diversified spatio-temporal feature extraction model for traffic flow prediction [J ] . Neurocomputing , 2025 , 621 : 129280 . DOI: 10.1016/j.neucom.2024.129280 http://dx.doi.org/10.1016/j.neucom.2024.129280

Gupta N , Hare J Z , Milzman J , et al . Deep meta coordination graphs for multi-agent reinforcement learning [PP/OL ] . V3. arXiv ( 2026-02-10 )[ 2023-06-15 ] . https://doi.org/10.48550/arXiv.2502.04028 https://doi.org/10.48550/arXiv.2502.04028 .

Wu Jia , Huang ZiXiao . Promoting diversity in mixed complex cooperative and competitive multi-agent environment [C ] // Proceedings of the 32nd ACM International Conference on Information and Knowledge Management . New York : ACM , 2023 : 4355 - 4359 . DOI: 10.1145/3583780.3615217 http://dx.doi.org/10.1145/3583780.3615217

Yu Chao , Velu A , Vinitsky E , et al . The surprising effectiveness of PPO in cooperative multi-agent games [C ] // Advances in Neural Information Processing Systems 35 . Neural Information Processing Systems Foundation, Inc. (NeurIPS) , 2022 : 24611 - 24624 . DOI: 10.52202/068431-1787 http://dx.doi.org/10.52202/068431-1787

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

物理信息增强的工业物联网轻量跨层认证技术

通用知识与专用信息共驱动的具身智能抗干扰通信策略生成方法

多环境选择策略引导的约束多目标进化优化算法

电磁频谱具身智能概念、研究现状及发展趋势

应急场景无人机辅助实时视频传输联合优化策略