Journal of Guangxi Normal University(Natural Science Edition) ›› 2025, Vol. 43 ›› Issue (5): 52-63.doi: 10.16088/j.issn.1001-6600.2024092101

• Physics and Electronic Engineering • Previous Articles     Next Articles

Study on Intelligent EV Dynamic Charging Scheduling Algorithm Based on DQN

HUANG Yuanyan1,3, LU Xuan1,3, ZHAN Kaijie1,3, ZENG Haiyong1,2,3*   

  1. 1. Guangxi Key Laboratory of Brain-inspired Computing and Intelligent Chips (Guangxi Normal University), Guilin Guangxi 541004, China;
    2. Key Laboratory of Integrated Circuits and Microsystems in Guangxi Universities (Guangxi Normal University), Guilin Guangxi 541004, China;
    3. School of Electronics and Information Engineering/School of Integrated Circuits, Guangxi Normal University, Guilin Guangxi 541004, China
  • Received:2024-09-21 Revised:2024-11-05 Online:2025-09-05 Published:2025-08-05

Abstract: With the widespread adoption of electric vehicles (EVs) and the implementation of environmental policies, efficiently scheduling EV charging behaviors has become increasingly important for meeting user demands and can ensure grid stability. Addressing the different charging needs of EV owners in smart grids and the curse of dimensionality problem in traditional machine learning algorithms, this paper proposes an EV charging scheduling algorithm based on Deep Q-Network (DQN). This algorithm leverages the advantages of deep reinforcement learning, combining real-time electricity prices and vehicle status information to dynamically adjust EV charging and discharging behaviors, aiming to maximize economic benefits and optimize grid load. Experimental results show that compared with uncontrolled charging strategies, the proposed algorithm can reduce charging costs by approximately 65.6% over a 30-day test period. It effectively adapts to real-time grid price fluctuations and changes in user demands, reduces peak charging requirements, effectively lowers charging costs, and improves grid stability.

Key words: electric vehicle charging scheduling, deep reinforcement learning, energy optimization management, vehicle-to-grid, smart grid, information interactive

CLC Number:  U491.8;TM73
[1] 詹华,江昌旭, 苏庆列. 基于分层强化学习的电动汽车充电引导方法[J]. 电力自动化设备, 2022, 42(10): 264-272. DOI: 10.16081/j.epae.202208022.
[2] MANRÍQUEZ F, SAUMA E, AGUADO J, et al. The impact of electric vehicle charging schemes in power system expansion planning[J]. Applied Energy, 2020, 262: 114527. DOI: 10.1016/j.apenergy.2020.114527.
[3] HU J J, YANG G Y, ZIRAS C, et al. Aggregator operation in the balancing market through network-constrained transactive energy[J]. IEEE Transactions on Power Systems, 2019, 34(5): 4071-4080. DOI: 10.1109/TPWRS.2018.2874255.
[4] 田梦秋, 承楠, 李长乐. 6G无线网络场景知识研究综述[J]. 无线电通信技术, 2024, 50(3): 484-495.
[5] 简富俊, 黄黎明, 汪洋, 等. 智能电网通信系统技术的研究[J]. 自动化应用, 2024(S1): 464-466. DOI: 10.19769/j.zdhy.2024.S1.153.
[6] 温昕. 车联网技术在新能源汽车控制系统设计中的应用研究[J]. 装备维修技术, 2024(1): 70-73. DOI: 10.16648/j.cnki.1005-2917.2024.01.017.
[7] 高爽, 戴如鑫. 电动汽车集群参与调频辅助服务市场的充电调控策略[J]. 电力系统自动化, 2023, 47(18): 60-67.
[8] 郭永强, 张靖, 何宇, 等. 考虑电动汽车充放电响应的微电网混合储能配置[J]. 电网与清洁能源, 2022, 38(8): 82-93. DOI: 10.3969/j.issn.1674-3814.2022.08.011.
[9] 吴洲洋, 艾欣, 胡俊杰. 需求侧灵活性资源参与调频辅助服务的备用优化与实时调度[J]. 电力系统自动化, 2021, 45(6): 148-157. DOI: 10.7500/AEPS20200629008.
[10] 付学林. 电动汽车混合模式充电调度策略研究[D]. 成都: 电子科技大学, 2020. DOI: 10.27005/d.cnki.gdzku.2020.002099.
[11] 傅质馨, 朱韦翰, 朱俊澎, 等. 动态路-电耦合网络下电动出租车快速充电引导及其定价策略[J]. 电力自动化设备, 2022, 42(4): 9-17. DOI: 10.16081/j.epae.202201013.
[12] SHEIK M S, IMTHIAS A T P, DEVARAJ D. Optimized charge scheduling of plug-In electric vehicles using modified placement algorithm[C]// 2019 International Conference on Computer Communication and Informatics (ICCCI). January 23-25, 2019. Coimbatore, Tamil Nadu, India. IEEE, 2019: 1-5.. DOI: 10.1109/iccci.2019.8821960.
[13] REN J M, WANG H, WEI Y, et al. A novel genetic algorithm-based emergent electric vehicle charging scheduling scheme[C]// IECON 2019 - 45th Annual Conference of the IEEE Industrial Electronics Society. October 14-17, 2019, Lisbon, Portugal. IEEE, 2019: 4289-4292. DOI: 10.1109/IECON.2019.8926746.
[14] ZHANG J, WANG W, WANG R H, et al. An electric vehicle scheduling strategy for the power grid peak regulation based on V2G[C]// 2023 3rd International Conference on New Energy and Power Engineering (ICNEPE). November 24-26, 2023, Huzhou, China. IEEE, 2023: 1181-1187. DOI: 10.1109/ICNEPE60694.2023.10429591.
[15] CHIŞ A, LUNDÉN J, KOIVUNEN V. Reinforcement learning-based plug-in electric vehicle charging with forecasted price[J]. IEEE Transactions on Vehicular Technology, 2017, 66(5): 3674-3684. DOI: 10.1109/TVT.2016.2603536.
[16] WAN Z Q, LI H P, HE H B, et al. Model-free real-time EV charging scheduling based on deep reinforcement learning[J]. IEEE Transactions on Smart Grid, 2019, 10(5): 5246-5257. DOI: 10.1109/TSG.2018.2879572.
[17] 杜明秋, 李妍, 王标, 等. 电动汽车充电控制的深度增强学习优化方法[J]. 中国电机工程学报, 2019, 39(14): 4042-4049. DOI: 10.13334/j.0258-8013.pcsee.181187.
[18] 李航, 李国杰, 汪可友. 基于深度强化学习的电动汽车实时调度策略[J]. 电力系统自动化, 2020, 44(22): 161-167.
[19] 黄元清, 刘迪迪, 覃光锋, 等. 计及车主需求的电动汽车聚合商能量调度策略[J]. 南方电网技术, 2024, 18(10): 161-170. DOI: 10.13648/j.cnki.issn1674-0629.2024.10.016.
[20] 柴雅琪, 海立卫, 韩颖, 等. 电动汽车站内充电调度优化方法研究[J]. 电工技术, 2024(14): 180-183. DOI: 10.19768/j.cnki.dgjs.2024.14.051.
[21] 彭可兴. 基于强化学习的多智能体协作通信策略研究[D]. 南京: 南京信息工程大学, 2024. DOI: 10.27248/d.cnki.gnjqc.2024.000016.
[22] 王娟, 陈明,支刚, 等. 电动汽车充电负荷的蒙特卡洛预测方法[J]. 云南电力技术, 2024, 52(4): 10-14.
[23] 李培建, 闫锡桐. 一种应用动态规划的通信链路推荐方法[C]// 第三十八届中国(天津)2024'IT、网络、信息技术、电子、仪器仪表创新学术会议论文集. 天津:天津市电子学会, 2024: 235-238. DOI: 10.26914/c.cnkihy.2024.014526.
[24] NORD POOL. Historical market data[DB/OL]. (2018-09-17)[2024-09-22]. http://www.nordpoolgroup.com/historical-market-data/.
[25] MHAISEN N, FETAIS N, MASSOUD A. Real-time scheduling for electric vehicles charging/discharging using reinforcement learning[C]// 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT). Piscataway, NJ: IEEE, 2020: 1-6. DOI: 10.1109/iciot48696.2020.9089471.
[1] HUANG Renhui, ZHANG Ruifeng, WEN Xiaohao, BI Jinjie, HUANG Shoulin, LI Tinghui. Complex-value Covariance-based Convolutional Neural Network for Decoding Motor Imagery-based EEG Signals [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 43-56.
[2] TANG Liang, CHEN Bowen, NIU Yisen, MA Ronggeng. A YOLOv8-based Real-time Object Detection Method for Vehicles and Pedestrians in Foggy Weather [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 72-83.
[3] LIANG Yinjie, NAN Xinyuan, CAI Xin, LI Yunpeng, GOU Haiguang. Bridge Defect Detection Based on Data Augmentation and Improved YOLOv8 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 84-97.
[4] TIAN Sheng, CHEN Dong. A Joint Eco-driving Optimization Research for Connected Fuel Cell Hybrid Vehicle via Deep Reinforcement Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 67-80.
[5] CHEN Xiufeng, WANG Chengxin, ZHAO Fengyang, YANG Kai, GU Kexin. A Single Intersection Signal Control Method Based on Improved DQN Algorithm [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 81-88.
[6] TANG Fengzhu, TANG Xin, LI Chunhai, LI Xiaohuan. Dynamic Task Allocation Method for UAVs Based on Deep Reinforcement Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(6): 63-71.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] ZHONG Qiao, CHEN Shenglong, TANG Congcong. Hydrogel Technology for Microalgae Collection: Status Overview, Challenges and Development Analysis[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 16 -29 .
[2] SHI Huilu, MO Yanhua, LUO Haiyu, MA Jiangming. Inhibitory Activity of Ethyl Acetate Extracts of Loropetalum chinense against Pathogens[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(1): 1 -8 .
[3] HE Qing, LI Dong, LUO Siyuan, HE Yudong, LI Biao, WANG Qiang. Research Progress in Ultra-wideband Rydberg Atomic Antenna Technology[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(2): 1 -19 .
[4] HUANG Renhui, ZHANG Ruifeng, WEN Xiaohao, BI Jinjie, HUANG Shoulin, LI Tinghui. Complex-value Covariance-based Convolutional Neural Network for Decoding Motor Imagery-based EEG Signals[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 43 -56 .
[5] TIAN Sheng, XIONG Chenyin, LONG Anyang. Point Cloud Classification Method of Urban Roads Based on Improved PointNet++[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(4): 1 -14 .
[6] LI Zongxiao, ZHANG Jian, LUO Xinyue, ZHAO Yifei, LU Fei. Research on Arrival Trajectory Prediction Based on K-means and Adam-LSTM[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(4): 15 -23 .
[7] SONG Mingkai, ZHU Chengjie. Research on Fault Location of Distribution Network Based on H-WOA-GWO and Region Correction Strategies[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(4): 24 -37 .
[8] HAN Shuo, JIANG Linfeng, YANG Jianbin. Attention-based PINNs Method for Solving Saint-Venant Equations[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(4): 58 -68 .
[9] LI Zhixin, KUANG Wenlan. Fine-grained Image Classification Combining Adaptive Spatial Mutual Attention and Feature Pair Integration Discrimination[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(4): 69 -82 .
[10] SHI Tianyi, NAN Xinyuan, GUO Xiangyu, ZHAO Pu, CAI Xin. Improved ConvNeXt-based Algorithm for Apple Leaf Disease Classification[J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(4): 83 -96 .