基于TD3算法的电动汽车智能充/放电调度策略

doi:10.16088/j.issn.1001-6600.2025072401

摘要/Abstract

摘要： 伴随电动汽车(electric vehicle, EV)规模化发展,其作为“移动储能单元”不容忽视的调控潜力,正深刻影响电力系统运行范式。在EV入网的背景下,充分考虑EV的可控负荷和移动储能的双重特性,本文构建一个综合EV的充电需求、动态电价、储能时间耦合约束及电池寿命损耗等多重关键因素的EV动态充/放电调度模型。针对EV接入充电时间和初始状态随机性,以及传统强化学习方法在决策变量连续控制场景中的维数灾难与收敛困难问题,本文提出一种基于TD3的智能充/放电控制与优化调度算法,该算法通过智能体与环境持续交互、设计奖励反馈机制,能够根据电价波动做出最佳充/放电决策,并保证在充电结束后达到预期充电量,从而实现对EV充/放电行为的智能控制和优化调度,最小化EV的充电成本。基于真实场景数据进行仿真,结果表明所提算法能够有效适应智能电网中电价的动态变化,有效降低EV用户的充电成本。与DDPG(deep deterministic policy gradient)、DQN(deep Q-network)、PSO(particle swarm optimization)等一系列主流算法相比,本文所提算法将充电成本降低4.41%至24.23%,充分验证了其性能与经济性优势。

关键词: 智能电网, 电动汽车, 车网互动, 充/放电调度, 深度强化学习

Abstract: With the large-scale development of Electric Vehicle (EV), their regulatory potential as "mobile energy storage units" cannot be overlooked, which profoundly influence the operational paradigm of power systems. In the context of EV grid integration, fully considering the dual characteristics of EV as controllable loads and mobile energy storage, a comprehensive dynamic charging/discharging scheduling model for EV is constructed, incorporating multiple key factors such as EV charging demand, dynamic electricity prices, time-coupling constraints of energy storage, and battery degradation. To address the randomness of EV charging start times and initial states, as well as the curse of dimensionality and convergence difficulties of traditional reinforcement learning methods in scenarios with continuous decision variables, an intelligent charging/discharging control and optimal scheduling algorithm based on Twin Delayed Deep Deterministic Policy Gradient (TD3) is proposed. Through continuous interaction between the agent and the environment and the design of a reward feedback mechanism, this algorithm can make optimal charging/discharging decisions based on electricity price fluctuations, ensuring that the expected charging capacity is achieved after the charging process, thereby realizing intelligent control and optimal scheduling of EV charging/discharging behavior to minimize charging costs. Simulations based on real-world scenario data demonstrate that the proposed algorithm effectively adapts to dynamic electricity price changes in smart grids and significantly reduces charging costs for EV users. Compared with a series of mainstream algorithms (such as DDPG, DQN, PSO, etc.), the proposed algorithm reduces charging costs by 4.41% to 24.23%, fully validating its performance and economic advantages.

Key words: smart grid, electric vehicle, vehicle-to-grid, charge/discharge scheduling, deep reinforcement learning

中图分类号: TM734

张旭, 刘迪迪. 基于TD3算法的电动汽车智能充/放电调度策略[J]. 广西师范大学学报（自然科学版）, 2026, 44(4): 46-55.

Zhang Xu, Liu Didi. Intelligent charging/discharging scheduling strategy for electric vehicles based on TD3 algorithm[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 46-55.

参考文献

[1] 陈江滢, 葛顺奇. 中国新能源汽车国际化: 特征、挑战及对策[J]. 国际经济评论, 2025(5): 29-63, 4-5.
[2] 周原冰, 龚乃玮, 王皓界, 等. 中国电动汽车发展及车网互动对新型储能配置的影响[J]. 中国电力, 2024, 57(10): 1-11. DOI: 10.11930/j.issn.1004-9649.202405058.
[3] Woo S, Strobel L, Yuan Y H, et al. Exploring bidirectional charging strategies for an electric vehicle population[J]. Applied Energy, 2025, 397: 126361. DOI: 10.1016/j.apenergy.2025.126361.
[4] 刘迪迪, 杨益菲, 杨玉荟, 等. 随机环境下电动汽车充电实时管理与优化控制算法[J]. 上海交通大学学报, 2023, 57(1): 1-9. DOI: 10.16183/j.cnki.jsjtu.2021.499.
[5] 胡梦锴, 陈胜, 卫志农, 等. 考虑V2G参与意愿与充电需求的配电网-电动汽车协同优化研究[J]. 浙江电力, 2025, 44(8): 3-14. DOI: 10.19585/j.zjdl.202508001.
[6] 黄元清, 刘迪迪, 覃光锋, 等. 计及车主需求的电动汽车聚合商能量调度策略[J]. 南方电网技术, 2024, 18(10): 161-170. DOI: 10.13648/j.cnki.issn1674-0629.2024.10.016.
[7] 贾东梨, 刘佳静, 詹惠瑜, 等. 考虑运行风险的新型配电系统多元时空灵活性资源协同配置[J]. 中国电力, 2025, 58(12): 73-85.
[8] 汪颖, 钟啟明, 刘育权, 等. 新型电力系统的电能质量扰动治理关键技术综述及展望[J]. 中国电机工程学报, 2025, 45(22): 8670-8691. DOI: 10.13334/j.0258-8013.pcsee.250424.
[9] Then J, Agalgaonkar A P, Muttaqi K M. Coordinated charging of spatially distributed electric vehicles for mitigating voltage rise and voltage unbalance in modern distribution networks[J]. IEEE Transactions on Industry Applications, 2023, 59(4): 5149-5157. DOI: 10.1109/TIA.2023.3273186.
[10] Liu J Y, Lin G, Huang S H, et al. Optimal EV charging scheduling by considering the limited number of chargers[J]. IEEE Transactions on Transportation Electrification, 2021, 7(3): 1112-1122. DOI: 10.1109/TTE.2020.3033995.
[11] 陈海鹏, 唐俊敏, 吴昊, 等. 计及电动汽车需求响应与碳配额收益的配电网优化调度[J]. 电网技术, 2025, 49(7): 2841-2851. DOI: 10.13335/j.1000-3673.pst.2024.1365.
[12] 朱永胜, 杨振涛, 丁同奎, 等. 考虑用户动态充电需求的电动汽车充电站规划[J]. 郑州大学学报(工学版), 2023, 44(2): 82-90. DOI: 10.13705/j.issn.1671-6833.2023.02.001.
[13] Li Y T, Su H, Chen X X, et al. A V2G scheduling strategy based on electric vehicle users' willingness model[C]//2021 IEEE 5th Conference on Energy Internet and Energy System Integration (EI2). Piscataway NJ: IEEE, 2022: 237-243. DOI: 10.1109/EI252483.2021.9712928.
[14] Sohet B, Hayel Y, Beaude O, et al. Hierarchical coupled driving-and-charging model of electric vehicles, stations and grid operators[J]. IEEE Transactions on Smart Grid, 2021, 12(6): 5146-5157. DOI: 10.1109/TSG.2021.3107896.
[15] Yi T, Cheng X B, Chen Y X, et al. Joint optimization of charging station and energy storage economic capacity based on the effect of alternative energy storage of electric vehicle[J]. Energy, 2020, 208: 118357. DOI: 10.1016/j.energy.2020.118357.
[16] 李咸善, 周晓岚, 姚俊伟, 等. 考虑车主多模式需求响应模糊意愿的优化调度策略[J]. 电力系统保护与控制, 2023, 51(2): 89-101. DOI: 10.19783/j.cnki.pspc.220697.
[17] Wang N, Li B, Duan Y, et al. A multi-energy scheduling strategy for orderly charging and discharging of electric vehicles based on multi-objective particle swarm optimization[J]. Sustainable Energy Technologies and Assessments, 2021, 44: 101037. DOI: 10.1016/j.seta.2021.101037.
[18] 张良, 孙成龙, 蔡国伟, 等. 基于PSO算法的电动汽车有序充放电两阶段优化策略[J]. 中国电机工程学报, 2022, 42(5): 1837-1852. DOI: 10.13334/j.0258-8013.pcsee.211150.
[19] Mignoni N, Carli R, Dotoli M. Distributed noncooperative MPC for energy scheduling of charging and trading electric vehicles in energy communities[J]. IEEE Transactions on Control Systems Technology, 2023, 31(5): 2159-2172. DOI: 10.1109/TCST.2023.3291549.
[20] Wei Z, Li Y, Cai L. Electric vehicle charging scheme for a park-and-charge system considering battery degradation costs[J]. IEEE Transactions on Intelligent Vehicles, 2018, 3(3): 361-373. DOI: 10.1109/TIV.2018.2843126.
[21] Zhang R Q, Cheng X, Yang L Q. Flexible energy management protocol for cooperative EV-to-EV charging[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(1): 172-184. DOI: 10.1109/TITS.2018.2807184.
[22] Xiao Q, Zhang R T, Wang Y C, et al. A deep reinforcement learning based charging and discharging scheduling strategy for electric vehicles[J]. Energy Reports, 2024, 12: 4854-4863. DOI: 10.1016/j.egyr.2024.10.056.
[23] 黄媛艳, 路旋, 詹凯杰, 等. 基于DQN的智能电动汽车动态充电调度算法研究[J]. 广西师范大学学报(自然科学版), 2025, 43(5): 52-63. DOI: 10.16088/j.issn.1001-6600.2024092101.
[24] 黄小庆, 于慎仟, 朱彬, 等. 移动充放电设施技术及其规划与运营研究综述[J]. 电力自动化设备, 2024, 44(7): 246-254. DOI: 10.16081/j.epae.202405005.
[25] Luo C, Huang Y F, Gupta V. Stochastic dynamic pricing for EV charging stations with renewable integration and energy storage[J]. IEEE Transactions on Smart Grid, 2018, 9(2): 1494-1505. DOI: 10.1109/TSG.2017.2696493.
[26] Cao J, Harrold D, Fan Z, et al. Deep reinforcement learning-based energy storage arbitrage with accurate lithium-ion battery degradation model[J]. IEEE Transactions on Smart Grid, 2020, 11(5): 4513-4521. DOI: 10.1109/TSG.2020.2986333.
[27] Nord Pool As. Historical market data[DS/OL].[2026-03-02]. http://www.nordpoolgroup.com.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed