Journal of Guangxi Normal University(Natural Science Edition) ›› 2026, Vol. 44 ›› Issue (4): 28-45.doi: 10.16088/j.issn.1001-6600.2025120503

• Physical and Electronic Engineering • Previous Articles     Next Articles

Energy management strategy for fuel cell vehicles based on improved deep reinforcement learning

Tian Sheng*, Xie Hualin, Chen Dong   

  1. School of Civil Engineering and Transportation, South China University of Technology, Guangzhou Guangdong 510641, China
  • Received:2025-12-05 Revised:2026-02-03 Online:2026-07-05 Published:2026-07-01

Abstract: In the field of energy management for fuel cell vehicles, energy management strategies based on deep reinforcement learning have become a research hotspot for hybrid powertrains. However, insufficient learning capability, low learning efficiency, and difficulties in model convergence have still been observed under various complex driving conditions. To address these problems, an energy management strategy based on an improved deep reinforcement learning algorithm is proposed. The traditional TD3 algorithm is modified, and prioritized experience replay together with a reward-oriented adaptive noise scale is adopted to obtain an improved AE-TD3 algorithm. The performance of the AE-TD3 algorithm under standard driving conditions is evaluated on the CLTC, UDDS, FTP75 and RCDC cycles, and its performance under real urban road driving conditions is also tested. Simulation studies are carried out, and the AE-TD3 algorithm is found to converge faster and to possess stronger exploration capability than the TD3 algorithm. Regarding the output performance of the energy storage unit, the fluctuation range of the battery SOC relative to its target value obtained by the AE-TD3-based energy management strategy is significantly smaller than that obtained by the TD3-based strategy under all four training cycles. In addition, when the vehicle operats at high speed, the fuel cell output power under the AE-TD3-based strategy is significantly higher than that under the TD3 algorithm. In terms of fuel cell hydrogen consumption, the hydrogen usage is reduced by 2.7%, 1.1%, 5.7% and 7.3%, respectively, compared with the TD3 algorithm. By applying the energy management strategy trained under standard driving cycles to real-world road conditions, it is shown that the AE-TD3-based energy management strategy has good online applicability and strong adaptability to different driving conditions.

Key words: fuel cell hybrid electric vehicle, energy management strategy, deep reinforcement learning, prioritized experience replay, AE-TD3 algorithm

CLC Number:  U469.7
[1] 马建, 刘晓东, 陈轶嵩, 等. 中国新能源汽车产业与技术发展现状及对策[J]. 中国公路学报, 2018, 31(8): 1-19. DOI: 10.3969/j.issn.1001-7372.2018.08.001.
[2] 欧阳明高. 中国新能源汽车的研发及展望[J]. 科技导报, 2016, 34(6): 13-20. DOI: 10.3981/j.issn.1000-7857.2016.06.001.
[3] 邵志刚, 衣宝廉. 氢能与燃料电池发展现状及展望[J]. 中国科学院院刊, 2019, 34(4): 469-477. DOI: 10.16418/j.issn.1000-3045.2019.04.012.
[4] 尤国贵, 卢剑伟, 张国军, 等. 插电式混合动力汽车能量管理策略研究综述[J]. 机械工程学报, 2025, 61(4): 195-218.
[5] 孙晓军, 宋恩哲, 姚崇, 等. 基于自适应扰动鲸鱼优化算法的混合动力能量管理策略研究[J]. 哈尔滨工程大学学报, 2024, 45(10): 1991-2000. DOI: 10.11990/jheu.202207034.
[6] 王亚雄, 余庆港, 王薛超, 等. 考虑性能衰退的燃料电池汽车自适应优化能量管理策略[J]. 交通运输工程学报, 2022, 22(1): 190-204. DOI: 10.19818/j.cnki.1671-1637.2022.01.016.
[7] 许晨阳. 基于自适应动态规划的仿射非线性系统无模型最优控制研究[D]. 北京: 北京邮电大学, 2024.
[8] Ma Z, Luan Y X, Zhang F Q, et al. A data-driven energy management strategy for plug-in hybrid electric buses considering vehicle mass uncertainty[J]. Journal of Energy Storage, 2024, 77: 109963. DOI: 10.1016/j.est.2023.109963.
[9] 林歆悠, 夏玉田, 李雪凡, 等. 燃料电池汽车行驶里程自适应ECMS策略[J]. 汽车工程, 2019, 41(7): 750-756. DOI: 10.19562/j.chinasae.qcgc.2019.07.004.
[10] 徐鑫, 武小花, 杨佳珞, 等. 燃料电池客车模型预测控制能量管理策略[J]. 电源学报, 2025, 23(6): 309-319. DOI: 10.13234/j.issn.2095-2805.2025.6.309.
[11] 王亚楠, 韩雪冰, 卢兰光, 等. 电动汽车动力电池研究展望: 智能电池、智能管理与智慧能源[J]. 汽车工程, 2022, 44(4): 617-637. DOI: 10.19562/j.chinasae.qcgc.2022.04.017.
[12] 崔明阳, 黄荷叶, 许庆, 等. 智能网联汽车架构、功能与应用关键技术[J]. 清华大学学报(自然科学版), 2022, 62(3): 493-508. DOI: 10.16511/j.cnki.qhdxxb.2021.26.026.
[13] GÓMez-Barroso Á, Makazaga I V, Zulueta E. Optimizing hybrid electric vehicle performance: a detailed overview of energy management strategies[J]. Energies, 2024, 18(1): 1-32. DOI: 10.3390/en18010010.
[14] 齐春阳, 宋传学, 宋世欣, 等. 基于逆强化学习的混合动力汽车能量管理策略研究[J]. 汽车工程, 2023, 45(10): 1954-1964, 1974. DOI: 10.19562/j.chinasae.qcgc.2023.10.016.
[15] Min D H, Song Z, Chen H C, et al. Genetic algorithm optimized neural network based fuel cell hybrid electric vehicle energy management strategy under start-stop condition[J]. Applied Energy, 2022, 306: 118036.
[16] Liu B, Wei X D, Sun C, et al. A controllable neural network-based method for optimal energy management of fuel cell hybrid electric vehicles[J]. International Journal of Hydrogen Energy, 2024, 55: 1371-1382. DOI: 10.1016/j.ijhydene.2023.10.215.
[17] 周奔滔. 基于深度学习的燃料电池/锂离子电池混合动力汽车能量管理系统[D]. 赣州: 江西理工大学, 2024.
[18] Xu B, Malmir F, Rathod D, et al. Real-time reinforcement learning optimized energy management for a 48V mild hybrid electric vehicle[J]. SAE Technical Paper Series, 2019, 1: 2019-1-1208. DOI: 10.4271/2019-01-1208.
[19] Tang X L, Zhou H T, Wang F, et al. Longevity-conscious energy management strategy of fuel cell hybrid electric vehicle based on deep reinforcement learning[J]. Energy, 2022, 238: 121593. DOI: 10.1016/j.energy.2021.121593.
[20] Lian R Z, Peng J K, Wu Y K, et al. Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle[J]. Energy, 2020, 197: 117297. DOI: 10.1016/j.energy.2020.117297.
[21] Zhou J H, Xue S W, Xue Y, et al. A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning[J]. Energy, 2021, 224: 120118.
[22] Ren K Z, Liu J, Wu Z Y, et al. A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters[J]. Applied Energy, 2024, 355: 122258.
[23] Liu W R, Yao P F, Wu Y, et al. Imitation reinforcement learning energy management for electric vehicles with hybrid energy storage system[J]. Applied Energy, 2025, 378: 124832. DOI: 10.1016/j.apenergy.2024.124832.
[24] 李晓之. 混合动力车辆预见性能量管理策略研究[D]. 长春: 吉林大学, 2023.
[25] 李亚鹏. 基于凸优化的车辆动力系统参数快速优化与预测型能量管理策略研究[D]. 重庆: 重庆大学, 2023.
[26] 胡晓松, 唐小林. 电动车辆锂离子动力电池建模方法综述[J]. 机械工程学报, 2017, 53(16): 20-31. DOI: 10.3901/JME.2017.16.020.
[27] 侯明, 衣宝廉. 燃料电池技术发展现状与展望[J]. 电化学, 2012, 18(1): 1-13. DOI: 10.13208/j.electrochem.2012.01.003.
[28] 刘宗巍, 史天泽, 郝瀚, 等. 中国燃料电池汽车发展问题研究[J]. 汽车技术, 2018(1): 1-9. DOI: 10.19620/j.cnki.1000-3703.20172154.
[29] Chen D F, Pei P C, Meng Y N, et al. Novel extraction method of working condition spectrum for the lifetime prediction and energy management strategy evaluation of automotive fuel cells[J]. Energy, 2022, 255: 124523. DOI: 10.1016/j.energy.2022.124523.
[30] 李卫. 基于深度强化学习的燃料电池混合动力汽车能量管理策略[D]. 深圳: 中国科学院大学(中国科学院深圳先进技术研究院), 2021.
[31] Pei P, Chang Q, Tang T. A quick evaluating method for automotive fuel cell lifetime[J]. International Journal of Hydrogen Energy, 2008, 33(14): 3829-3836. DOI: 10.1016/j.ijhydene.2008.04.048.
[32] 李畅. 电动汽车动力电池建模与SOC估计研究[D]. 西安: 西安工业大学, 2023.
[33] 王勇辉. 质子交换膜燃料电池电动汽车建模与控制研究[D]. 西安: 长安大学, 2023.
[1] Zhang Xu, Liu Didi. Intelligent charging/discharging scheduling strategy for electric vehicles based on TD3 algorithm [J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 46-55.
[2] HUANG Yuanyan, LU Xuan, ZHAN Kaijie, ZENG Haiyong. Study on Intelligent EV Dynamic Charging Scheduling Algorithm Based on DQN [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(5): 52-63.
[3] TIAN Sheng, CHEN Dong. A Joint Eco-driving Optimization Research for Connected Fuel Cell Hybrid Vehicle via Deep Reinforcement Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 67-80.
[4] CHEN Xiufeng, WANG Chengxin, ZHAO Fengyang, YANG Kai, GU Kexin. A Single Intersection Signal Control Method Based on Improved DQN Algorithm [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 81-88.
[5] TANG Fengzhu, TANG Xin, LI Chunhai, LI Xiaohuan. Dynamic Task Allocation Method for UAVs Based on Deep Reinforcement Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(6): 63-71.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] TIAN Sheng, ZHAO Kailong, MIAO Jialin. Research on Automatic Driving Road Traffic Detection Algorithm Based on Improved YOLO11n Model[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(1): 1 -9 .
[2] Tang Chenghua, Yi Jianbing, Wu Xin, Xiong Wenwu, Wang Jingyong. A review of cross-domain few-shot image semantic segmentation methods[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 1 -27 .
[3] Zhang Xu, Liu Didi. Intelligent charging/discharging scheduling strategy for electric vehicles based on TD3 algorithm[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 46 -55 .
[4] Yan Yuanyang, Xie Lirong, Zhang Longjun, Ren Juan, Huang Chenchen, Hu Chao. Ultra-short-term wind power prediction model based on multi-objective optimization[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 56 -70 .
[5] Tao Zhenzhuo, Wei Duqu. Adaptive chaos synchronization of PMSM with unknown parameters[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 71 -78 .
[6] Wang Chenglong, Song Qiang, Li Wenfeng, Zhang Shimin. PAM-DETR: a small-object defect detection algorithm for medical gloves based on improved RT-DETR[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 79 -95 .
[7] Wei Wujie, Chen Qingfeng. Fair graph learning via view disentanglement and counterfactual augmentation[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 96 -106 .
[8] Cao Feng, Wu Shukang, Zhu Weizhen, Yi Jianbing. Evaluation of contradiction separation clause based on multi-criteria decision making[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 107 -120 .
[9] Luo Cheng, Huang Wentao, He Dongping, Zhang Yue. Weakly persistent centers for a ten-parameter family of complex planar cubic polynomial differential systems[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 121 -129 .
[10] Wang Zhanxin, Wei Yuming. Analysis of a deterministic and stochastic SIS-SIRS epidemic model with saturated incidence rates[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 130 -146 .