基于改进深度强化学习的燃料电池汽车能量管理策略

doi:10.16088/j.issn.1001-6600.2025120503

摘要/Abstract

摘要： 在燃料电池汽车能量管理技术领域,基于深度强化学习算法的能量管理策略目前成为混合动力总成技术的研究热点,但仍然存在面向不同的复杂驾驶工况时出现学习能力不足、学习效率低、模型收敛困难等问题。为解决该问题,本文提出一种基于改进深度强化学习的能量管理策略。该策略对传统的TD3算法进行改进,使用优先经验回放、奖励导向的自适应噪声尺度等方法得到改进后的AE-TD3算法。采用CLTC、UDDS、FTP75以及RCDC工况分别验证AE-TD3算法在标准驾驶工况以及真实城市道路驾驶工况下的性能表现。仿真结果显示,AE-TD3算法相比于TD3算法在收敛速度上更快,拥有更强的探索能力。在储能单元输出性能方面,与TD3算法相比,在4个训练工况下,基于AE-TD3算法的能量管理策略得到的电池SOC相对目标值的波动范围明显比基于TD3算法的能量管理策略低。除此之外,当车辆速度处于高速工况时,基于AE-TD3算法的能量管理策略的燃料电池输出功率明显高于TD3算法。在燃料电池氢气消耗方面,相较于TD3算法分别提升2.7%、1.1%、5.7%和7.3%。同时,通过将标准工况下训练好的能量管理策略应用于实际道路工况,由实验结果可知,基于AE-TD3算法的能量管理策略具有较好的在线应用能力和工况适应能力。

关键词: 燃料电池混合动力汽车, 能量管理策略, 深度强化学习, 优先经验回放, AE-TD3算法

Abstract: In the field of energy management for fuel cell vehicles, energy management strategies based on deep reinforcement learning have become a research hotspot for hybrid powertrains. However, insufficient learning capability, low learning efficiency, and difficulties in model convergence have still been observed under various complex driving conditions. To address these problems, an energy management strategy based on an improved deep reinforcement learning algorithm is proposed. The traditional TD3 algorithm is modified, and prioritized experience replay together with a reward-oriented adaptive noise scale is adopted to obtain an improved AE-TD3 algorithm. The performance of the AE-TD3 algorithm under standard driving conditions is evaluated on the CLTC, UDDS, FTP75 and RCDC cycles, and its performance under real urban road driving conditions is also tested. Simulation studies are carried out, and the AE-TD3 algorithm is found to converge faster and to possess stronger exploration capability than the TD3 algorithm. Regarding the output performance of the energy storage unit, the fluctuation range of the battery SOC relative to its target value obtained by the AE-TD3-based energy management strategy is significantly smaller than that obtained by the TD3-based strategy under all four training cycles. In addition, when the vehicle operats at high speed, the fuel cell output power under the AE-TD3-based strategy is significantly higher than that under the TD3 algorithm. In terms of fuel cell hydrogen consumption, the hydrogen usage is reduced by 2.7%, 1.1%, 5.7% and 7.3%, respectively, compared with the TD3 algorithm. By applying the energy management strategy trained under standard driving cycles to real-world road conditions, it is shown that the AE-TD3-based energy management strategy has good online applicability and strong adaptability to different driving conditions.

Key words: fuel cell hybrid electric vehicle, energy management strategy, deep reinforcement learning, prioritized experience replay, AE-TD3 algorithm

中图分类号: U469.7

田晟, 谢华林, 陈东. 基于改进深度强化学习的燃料电池汽车能量管理策略[J]. 广西师范大学学报（自然科学版）, 2026, 44(4): 28-45.

Tian Sheng, Xie Hualin, Chen Dong. Energy management strategy for fuel cell vehicles based on improved deep reinforcement learning[J]. Journal of Guangxi Normal University(Natural Science Edition), 2026, 44(4): 28-45.

参考文献

[1] 马建, 刘晓东, 陈轶嵩, 等. 中国新能源汽车产业与技术发展现状及对策[J]. 中国公路学报, 2018, 31(8): 1-19. DOI: 10.3969/j.issn.1001-7372.2018.08.001.
[2] 欧阳明高. 中国新能源汽车的研发及展望[J]. 科技导报, 2016, 34(6): 13-20. DOI: 10.3981/j.issn.1000-7857.2016.06.001.
[3] 邵志刚, 衣宝廉. 氢能与燃料电池发展现状及展望[J]. 中国科学院院刊, 2019, 34(4): 469-477. DOI: 10.16418/j.issn.1000-3045.2019.04.012.
[4] 尤国贵, 卢剑伟, 张国军, 等. 插电式混合动力汽车能量管理策略研究综述[J]. 机械工程学报, 2025, 61(4): 195-218.
[5] 孙晓军, 宋恩哲, 姚崇, 等. 基于自适应扰动鲸鱼优化算法的混合动力能量管理策略研究[J]. 哈尔滨工程大学学报, 2024, 45(10): 1991-2000. DOI: 10.11990/jheu.202207034.
[6] 王亚雄, 余庆港, 王薛超, 等. 考虑性能衰退的燃料电池汽车自适应优化能量管理策略[J]. 交通运输工程学报, 2022, 22(1): 190-204. DOI: 10.19818/j.cnki.1671-1637.2022.01.016.
[7] 许晨阳. 基于自适应动态规划的仿射非线性系统无模型最优控制研究[D]. 北京: 北京邮电大学, 2024.
[8] Ma Z, Luan Y X, Zhang F Q, et al. A data-driven energy management strategy for plug-in hybrid electric buses considering vehicle mass uncertainty[J]. Journal of Energy Storage, 2024, 77: 109963. DOI: 10.1016/j.est.2023.109963.
[9] 林歆悠, 夏玉田, 李雪凡, 等. 燃料电池汽车行驶里程自适应ECMS策略[J]. 汽车工程, 2019, 41(7): 750-756. DOI: 10.19562/j.chinasae.qcgc.2019.07.004.
[10] 徐鑫, 武小花, 杨佳珞, 等. 燃料电池客车模型预测控制能量管理策略[J]. 电源学报, 2025, 23(6): 309-319. DOI: 10.13234/j.issn.2095-2805.2025.6.309.
[11] 王亚楠, 韩雪冰, 卢兰光, 等. 电动汽车动力电池研究展望: 智能电池、智能管理与智慧能源[J]. 汽车工程, 2022, 44(4): 617-637. DOI: 10.19562/j.chinasae.qcgc.2022.04.017.
[12] 崔明阳, 黄荷叶, 许庆, 等. 智能网联汽车架构、功能与应用关键技术[J]. 清华大学学报(自然科学版), 2022, 62(3): 493-508. DOI: 10.16511/j.cnki.qhdxxb.2021.26.026.
[13] GÓMez-Barroso Á, Makazaga I V, Zulueta E. Optimizing hybrid electric vehicle performance: a detailed overview of energy management strategies[J]. Energies, 2024, 18(1): 1-32. DOI: 10.3390/en18010010.
[14] 齐春阳, 宋传学, 宋世欣, 等. 基于逆强化学习的混合动力汽车能量管理策略研究[J]. 汽车工程, 2023, 45(10): 1954-1964, 1974. DOI: 10.19562/j.chinasae.qcgc.2023.10.016.
[15] Min D H, Song Z, Chen H C, et al. Genetic algorithm optimized neural network based fuel cell hybrid electric vehicle energy management strategy under start-stop condition[J]. Applied Energy, 2022, 306: 118036.
[16] Liu B, Wei X D, Sun C, et al. A controllable neural network-based method for optimal energy management of fuel cell hybrid electric vehicles[J]. International Journal of Hydrogen Energy, 2024, 55: 1371-1382. DOI: 10.1016/j.ijhydene.2023.10.215.
[17] 周奔滔. 基于深度学习的燃料电池/锂离子电池混合动力汽车能量管理系统[D]. 赣州: 江西理工大学, 2024.
[18] Xu B, Malmir F, Rathod D, et al. Real-time reinforcement learning optimized energy management for a 48V mild hybrid electric vehicle[J]. SAE Technical Paper Series, 2019, 1: 2019-1-1208. DOI: 10.4271/2019-01-1208.
[19] Tang X L, Zhou H T, Wang F, et al. Longevity-conscious energy management strategy of fuel cell hybrid electric vehicle based on deep reinforcement learning[J]. Energy, 2022, 238: 121593. DOI: 10.1016/j.energy.2021.121593.
[20] Lian R Z, Peng J K, Wu Y K, et al. Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle[J]. Energy, 2020, 197: 117297. DOI: 10.1016/j.energy.2020.117297.
[21] Zhou J H, Xue S W, Xue Y, et al. A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning[J]. Energy, 2021, 224: 120118.
[22] Ren K Z, Liu J, Wu Z Y, et al. A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters[J]. Applied Energy, 2024, 355: 122258.
[23] Liu W R, Yao P F, Wu Y, et al. Imitation reinforcement learning energy management for electric vehicles with hybrid energy storage system[J]. Applied Energy, 2025, 378: 124832. DOI: 10.1016/j.apenergy.2024.124832.
[24] 李晓之. 混合动力车辆预见性能量管理策略研究[D]. 长春: 吉林大学, 2023.
[25] 李亚鹏. 基于凸优化的车辆动力系统参数快速优化与预测型能量管理策略研究[D]. 重庆: 重庆大学, 2023.
[26] 胡晓松, 唐小林. 电动车辆锂离子动力电池建模方法综述[J]. 机械工程学报, 2017, 53(16): 20-31. DOI: 10.3901/JME.2017.16.020.
[27] 侯明, 衣宝廉. 燃料电池技术发展现状与展望[J]. 电化学, 2012, 18(1): 1-13. DOI: 10.13208/j.electrochem.2012.01.003.
[28] 刘宗巍, 史天泽, 郝瀚, 等. 中国燃料电池汽车发展问题研究[J]. 汽车技术, 2018(1): 1-9. DOI: 10.19620/j.cnki.1000-3703.20172154.
[29] Chen D F, Pei P C, Meng Y N, et al. Novel extraction method of working condition spectrum for the lifetime prediction and energy management strategy evaluation of automotive fuel cells[J]. Energy, 2022, 255: 124523. DOI: 10.1016/j.energy.2022.124523.
[30] 李卫. 基于深度强化学习的燃料电池混合动力汽车能量管理策略[D]. 深圳: 中国科学院大学(中国科学院深圳先进技术研究院), 2021.
[31] Pei P, Chang Q, Tang T. A quick evaluating method for automotive fuel cell lifetime[J]. International Journal of Hydrogen Energy, 2008, 33(14): 3829-3836. DOI: 10.1016/j.ijhydene.2008.04.048.
[32] 李畅. 电动汽车动力电池建模与SOC估计研究[D]. 西安: 西安工业大学, 2023.
[33] 王勇辉. 质子交换膜燃料电池电动汽车建模与控制研究[D]. 西安: 长安大学, 2023.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed