Balance Control Strategy of Redundant Battery in Satellite Power Supply Based on Deep Reinforcement Learning

YE Ze-yu; YIN Jing-yuan; JIA Hai-peng; SHI Chang-li; WEI Tong-zhen; LUO Yan

doi:10.12263/DZXB.20211267

您当前的位置：

首页 >

文章列表页 >

Balance Control Strategy of Redundant Battery in Satellite Power Supply Based on Deep Reinforcement Learning

PAPERS | 更新时间：2025-12-08

- Balance Control Strategy of Redundant Battery in Satellite Power Supply Based on Deep Reinforcement Learning
- ACTA ELECTRONICA SINICA Vol. 51, Issue 9, Pages: 2419-2427(2023)
- 作者机构：
  
  1.中国科学院电工研究所,北京 100190
  2.中国科学院大学电子电气与通信工程学院,北京 100049
  3.中国空间技术研究院钱学森空间技术实验室,北京 100094
- 作者简介：
- 基金信息：
  
  National Defense Basic Scientific Research program of China(JCKY2019130C002)
- DOI：10.12263/DZXB.20211267
  CLC： TM912;TN710
- Received：15 September 2021，
  
  Revised：2021-12-30，
  
  Published：25 September 2023
- 稿件说明：
移动端阅览
叶泽雨,尹靖元,贾海鹏等.基于深度强化学习的卫星电源冗余电池均衡控制策略[J].电子学报,2023,51(09):2419-2427.

YE Ze-yu,YIN Jing-yuan,JIA Hai-peng,et al.Balance Control Strategy of Redundant Battery in Satellite Power Supply Based on Deep Reinforcement Learning[J].ACTA ELECTRONICA SINICA,2023,51(09):2419-2427.
叶泽雨,尹靖元,贾海鹏等.基于深度强化学习的卫星电源冗余电池均衡控制策略[J].电子学报,2023,51(09):2419-2427. DOI： 10.12263/DZXB.20211267.

YE Ze-yu,YIN Jing-yuan,JIA Hai-peng,et al.Balance Control Strategy of Redundant Battery in Satellite Power Supply Based on Deep Reinforcement Learning[J].ACTA ELECTRONICA SINICA,2023,51(09):2419-2427. DOI： 10.12263/DZXB.20211267.

摘要

为提高卫星蓄电池组的智能化管理能力水平，提出了一种基于深度强化学习的卫星冗余电池均衡控制策略.训练智能体根据蓄电池组当前的运行状态给出动作，改变单体电池的投入状态和数量，实现单体之间容量均衡，减小母线电压变化范围并减少开关调节次数.在MATLAB\Simulink和OpenAI的gym环境中分别搭建了电池组仿真环境对智能体进行了训练，通过算例检验了该策略的可行性并与基于阈值的控制方法进行了比较，证明了方法可以有效的实现电池间均衡并减小母线电压的变化范围.

Abstract

In order to improve the intelligent management ability of satellite battery pack

this paper proposes a satellite redundant battery balancing control strategy based on deep reinforcement learning. The method can train the agent to extract the characteristics of the current operation state of the battery group and judge it

change the input state and quantity of the battery unit

realize the capacity balance between the battery units

reduce the range of bus voltage change and reduce the switching adjustment times. The battery pack simulation environment is built in MATLAB \ Simulink and OpenAI gym environment respectively to train the reinforcement learning agent. The feasibility of this method is verified by an example

and compared with the traditional sorting method

it is proved that this method can effectively achieve the capacity balance between batteries and reduce the variation range of bus voltage.

关键词

Keywords

references

李朋 , 周军 , 于晓洲 , 等 . 翱翔之星电源系统的设计与在轨验证 [J]. 太阳能学报 , 2018 , 39 ( 4 ): 1002 - 1007 .

LI P , ZHOU J , YU X Z , et al . Design and on-orbit validation of electric power system for aoxiang-sat [J]. Acta Energiae Solaris Sinica , 2018 , 39 ( 4 ): 1002 - 1007 . (in Chinese)

张怀峰 , 江婧 , 张香燕 , 等 . 面向卫星电源系统的一种新颖异常检测方法 [J]. 宇航学报 , 2019 , 40 ( 12 ): 1468 - 1477 .

ZHANG H F , JIANG J , ZHANG X Y , et al . Novel anomaly detection method for satellite power system [J]. Journal of Astronautics , 2019 , 40 ( 12 ): 1468 - 1477 . (in Chinese)

赵汉宾 , 马锋 , 叶晓东 , 等 . 基于改进型飞渡电容的电池组动态均衡控制方法研究 [J]. 电子学报 , 2018 , 46 ( 4 ): 819 - 826 .

ZHAO H B , MA F , YE X D , et al . Battery dynamic balance control algorithm based on modified flying-capacitor [J]. Acta Electronica Sinica , 2018 , 46 ( 4 ): 819 - 826 . (in Chinese)

刘月峰 , 赵光权 , 彭喜元 . 多核相关向量机优化模型的锂电池剩余寿命预测方法 [J]. 电子学报 , 2019 , 47 ( 6 ): 1285 - 1292 .

LIU Y F , ZHAO G Q , PENG X Y . A lithium-ion battery remaining using life prediction method based on multi-kernel relevance vector machine optimized model [J]. Acta Electronica Sinica , 2019 , 47 ( 6 ): 1285 - 1292 . (in Chinese)

刘红锐 , 李园专 , 陈仕龙 , 等 . 锂离子蓄电池组充放电均衡器及均衡策略 [J]. 电子学报 , 2016 , 44 ( 8 ): 1981 - 1987 .

LIU H R , LI Y Z , CHEN S L , et al . Lithium-ion battery charging and discharging equalizer and balancing strategies [J]. Acta Electronica Sinica , 2016 , 44 ( 8 ): 1981 - 1987 . (in Chinese)

朱立颖 , 刘治钢 , 张晓峰 . 国外小天体探测器电源系统设计分析 [J]. 航天器工程 , 2018 , 27 ( 1 ): 130 - 136 .

ZHU L Y , LIU Z G , ZHANG X F . Analysis on power system design of probe for small celestial body [J]. Spacecraft Engineering , 2018 , 27 ( 1 ): 130 - 136 . (in Chinese)

廖力 , 纪锋 , 吴铁洲 , 等 . 采用附加电源的均衡电路与容量自均衡方法 [J]. 电测与仪表 , 2020 , 57 ( 7 ): 48 - 53 .

LIAO L , JI F , WU T Z , et al . Equalization circuit with additional power supply and capacity self-balancing method [J]. Electrical Measurement & Instrumentation , 2020 , 57 ( 7 ): 48 - 53 . (in Chinese)

MANENTI A , ABBA A , MERATI A , et al . A new BMS architecture based on cell redundancy [J]. IEEE Transactions on Industrial Electronics , 2011 , 58 ( 9 ): 4314 - 4322 .

王莹 , 严晓 , 林文魁 , 等 . 具备冗余单元的电池组放电效率研究 [J]. 电源技术 , 2018 , 42 ( 12 ): 1832 - 1834, 1926 .

WANG Y , YAN X , LIN W K , et al . Research on discharge efficiency of battery pack with redundant cells [J]. Chinese Journal of Power Sources , 2018 , 42 ( 12 ): 1832 - 1834, 1926 . (in Chinese)

王友仁 , 黄薛 , 耿星 , 等 . 容错航空蓄电池电源及其均衡管理 [J]. 航空学报 , 2018 , 39 ( 5 ): 147 - 157 .

WANG Y R , HUANG X , GENG X , et al . Fault-tolerant battery power supply for aircraft and its active equalization management [J]. Acta Aeronautica et Astronautica Sinica , 2018 , 39 ( 5 ): 147 - 157 . (in Chinese)

李结胜 . 高效高精度电池管理系统的设计和开发 [D]. 武汉 : 华中科技大学 , 2019 .

LI J S . Design and Development of High-Efficiency and High-Precision Battery Management System [D]. Wuhan : Huazhong University of Science and Technology , 2019 . (in Chinese)

张自东 , 邱才明 , 张东霞 , 等 . 基于深度强化学习的微电网复合储能协调控制方法 [J]. 电网技术 , 2019 , 43 ( 6 ): 1914 - 1921 .

ZHANG Z D , QIU C M , ZHANG D X , et al . A coordinated control method for hybrid energy storage system in microgrid based on deep reinforcement learning [J]. Power System Technology , 2019 , 43 ( 6 ): 1914 - 1921 . (in Chinese)

雷英俊 , 马亮 , 李海津 . 一种小型SAR卫星电源系统设计 [J]. 航天器工程 , 2019 , 28 ( 5 ): 101 - 104 .

LEI Y J , MA L , LI H J . Design of power supply system for small SAR satellite [J]. Spacecraft Engineering , 2019 , 28 ( 5 ): 101 - 104 . (in Chinese)

TREMBLAY O , DESSAINT L A . Experimental validation of a battery dynamic model for EV applications [J]. World Electric Vehicle Journal , 2009 , 3 ( 2 ): 289 - 298 .

赵冬斌 , 邵坤 , 朱圆恒 , 等 . 深度强化学习综述: 兼论计算机围棋的发展 [J]. 控制理论与应用 , 2016 , 33 ( 6 ): 701 - 717 .

ZHAO D B , SHAO K , ZHU Y H , et al . Review of deep reinforcement learning and discussions on the development of computer Go [J]. Control Theory & Applications , 2016 , 33 ( 6 ): 701 - 717 . (in Chinese)

杨启萌 , 禹龙 , 田生伟 , 等 . 基于深度强化学习的维吾尔语人称代词指代消解 [J]. 电子学报 , 2020 , 48 ( 6 ): 1077 - 1083 .

YANG Q M , YU L , TIAN S W , et al . Anaphora resolution of uyghur personal pronouns based on deep reinforcement learning [J]. Acta Electronica Sinica , 2020 , 48 ( 6 ): 1077 - 1083 . (in Chinese)

冀大雄 , 方文巍 , 朱华 , 等 . 基于相对测量的水下机器人主动定位方法研究 [J]. 电子学报 , 2021 , 49 ( 7 ): 1249 - 1256 .

JI D X , FANG W W , ZHU H , et al . Active localization of autonomous underwater vehicle using noisy relative measurement [J]. Acta Electronica Sinica , 2021 , 49 ( 7 ): 1249 - 1256 . (in Chinese)

VAN HASSELT H , GUEZ A , SILVER D . Deep reinforcement learning with double Q -learning [C]// Proceedings of the the Thirtieth AAAI Conference on Artificial Intelligence . Phoenix : AAAI Press , 2016 : 2094 - 2

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Causal Tree-of-Thought-Based Model for Battery State-of-Charge Prediction in Electric Vehicles

Efficient Task Offloading Based on Traffic Prediction in IoV-Enabled Edge Computing

Research on UAV Path Planning Algorithm for Fairness Data Collection and Energy Supplement

Multi-Channel Dynamic Spectrum Access Based on Multi-Agent Proximal Policy Optimization

Related Author

YIN Jing-yuan

PENG Zi-ran

YANG Xiao-yang

LI Xue-yong

ZHOU Yu

XU Xiao-long

YANG Wei

YANG Chen-yi

Related Institution

School of Transportation and Electrical Engineering, Hunan University of Technology

School of Software, Nanjing University of Information Science and Technology

Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology

State Key Laboratory for Novel Software Technology, Nanjing University

College of Computer Science and Technology, China University of Petroleum (East China)

⁰