2025年04月23日 星期三

广西师范大学学报（自然科学版） ›› 2024, Vol. 42 ›› Issue (6): 67-80.doi: 10.16088/j.issn.1001-6600.2023120802

• “污水处理”专栏 • 上一篇    下一篇

基于深度强化学习的网联燃料电池混合动力汽车生态驾驶联合优化方法

田晟*, 陈东   

  1. 华南理工大学 土木与交通学院, 广东 广州 510640
  • 收稿日期:2023-12-08 修回日期:2024-01-08 出版日期:2024-12-30 发布日期:2024-12-30
  • 通讯作者: 田晟(1969—),男,江西九江人,华南理工大学副教授,工学博士。E-mail:shitian1@scut.edu.cn
  • 基金资助:
    广东省自然科学基金(2021A1515011587,2020A1515010382)

A Joint Eco-driving Optimization Research for Connected Fuel Cell Hybrid Vehicle via Deep Reinforcement Learning

TIAN Sheng*, CHEN Dong   

  1. School of Civil Engineering and Transportation, South China University of Technology, Guangzhou Guangdong 510640, China
  • Received:2023-12-08 Revised:2024-01-08 Online:2024-12-30 Published:2024-12-30

PDF (PC)

120

摘要： 随着物联网、无人驾驶等新技术的快速发展,基于网联交通驾驶环境为混合动力车辆节能驾驶与能量管理优化注入了新的研究思路。针对燃料电池混合动力汽车在多信号灯城市道路的驾驶场景,本文提出一种基于深度强化学习算法的车速与能量管理的多目标分层联合优化方法(DDPG-DP)。在上层节能速度规划方面采用DDPG算法,同时设计多目标奖励值函数和加入优先经验回放机制,在提高算法速度和稳定性的基础上,进行节能、驾驶舒适性以及通行效率的多目标速度规划。在下层能量管理方面采用动态规划算法(DP),以氢气消耗最小化为目标实现混合动力系统的最优节能控制。结果表明:在本文设定的2种场景中,DDPG-DP算法比IDM-DP算法在通行效率上分别提高15.25%、20.18%,氢气燃料消耗分别降低25.66%、17.86%;同时在本文设定的2种场景中DDPG-DP算法相比于全局最优算法(DP-DP)在通行时间上只存在5 s左右差距,氢气燃料消耗比最优算法仅相差2.84%、4.7%。在通行平稳性上DDPG-DP算法比另外2种算法(IDM-DP、DP-DP)速度波动更小且未出现急加减速情况,能够较好地保证乘坐的舒适性。本文通过速度规划和能量管理双层主动式架构,能够实现混合动力车辆主动式节能优化,将为混合动力汽车日常驾驶提供更大节能潜力,同时对于网联燃料电池混合动力汽车的多目标生态驾驶优化奠定了研究基础。

关键词: 能量管理, 燃料电池, 混合动力汽车, 深度强化学习, 联合优化, 智能网联车辆

Abstract: With the rapid development of the new technologies about Internet of Things (IoT) and automatic driving, an advanced research target has been injected into the optimization of eco-driving and energy management of hybrid vehicles based on the connected driving environment. Aiming at the fuel cell hybrid vehicles driving on multi-signalized urban roads, this paper proposes a hierarchical multi-objective optimization method combined deep deterministic policy gradient and dynamic planning (DDPG-DP) for speed planning and energy management. The DDPG algorithm is used in the upper layer of energy-saving speed planning, while the multi-objective reward value function and the priority experience replay mechanism are designed to carry out the multi-objective speed planning for energy saving, driving comfort, and passage efficiency on the basis of improving the algorithm’s speed and stability, and the dynamic planning algorithm is used in the lower layer of energy management to achieve the optimal energy-saving of the hybrid system with the goal of minimizing the hydrogen consumption. In scenarios 1 and 2, the results show that the DDPG-DP algorithm improves the traveling efficiency by 15.25% and 20.18% than the IDM-DP algorithm, and reduces the hydrogen fuel consumption by 25.66% and 17.86%, respectively. Meanwhile, there is a gap of only about 5 s in the passing time of the DDPG-DP algorithm compared with the global optimal algorithm (DP-DP) in Scenarios 1 and 2, and the hydrogen fuel consumption is lower than the optimal algorithm. Meanwhile, there is only a difference of about 5 s between the DDPG-DP algorithm and the global optimal algorithm (DP-DP) in traveling time, and there is only a difference of 2.84% and 4.7% in the hydrogen fuel consumption compared with the DP-DP algorithm. In field of driving smoothness, the DDPG-DP algorithm has less speed fluctuation than the other two algorithms (IDM-DP and DP-DP) and doesn’t have large acceleration/deceleration. It will provide greater energy-saving potential for daily driving of hybrid vehicles and support the further research for multi-objective eco-driving optimization of connected fuel cell hybrid vehicles.

Key words: energy management, fuel cell, hybrid vehicle, deep reinforcement learning, co-optimization, connected and autonomous vehicles

中图分类号:  TP18;U469.7

[1] 陈晓龙,焦晓红.网联混合动力汽车跟驰场景的预测能量管理控制[J].燕山大学学报,2023,47(1):43-53.DOI: 10.3969/j.issn.1007-791X.2023.01.005.
[2] 陈峥,张玉果,沈世全,等.城市郊区道路跟车条件下智能网联汽车速度规划[J].中国公路学报,2023,36(6):298-310.DOI: 10.19721/j.cnki.1001-7372.2023.06.023.
[3] LV Z H, LOU R R, SINGH A K. AI empowered communication systems for intelligent transportation systems[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(7): 4579-4587. DOI: 10.1109/TITS.2020.3017183.
[4] 魏小栋,孙超,刘波,等.燃料电池汽车车速与能量联合优化[J].机械工程学报,2023,59(8):204-212.DOI: 10.3901/JME.2023.08.204.
[5] 周健豪,顾诚,刘军,等.基于IGWO的燃料电池汽车模糊控制能量管理策略[J].重庆理工大学学报(自然科学版),2021,35(5):33-41.DOI: 10.3969/j.issn.1674-8425(z).2021.05.005.
[6] 于瀛霄,孙闫,夏长高,等.燃料电池汽车双层模糊控制能量管理策略[J].重庆理工大学学报(自然科学版),2022,36(8):21-28.DOI: 10.3969/j.issn.1674-8425(z).2022.08.003.
[7] ZOU W T, LI J W, YANG Q Q, et al. An improved max-min game theory control of fuel cell and battery hybrid energy system against system uncertainty[J]. IEEE Journal of Emerging and Selected Topics in Power Electronics, 2023, 11(1): 78-87. DOI: 10.1109/JESTPE.2022.3168374.
[8] 赵天宇,陈东,霍为炜,等.氢燃料电池汽车能量管理系统模糊控制仿真研究[J].重庆理工大学学报(自然科学版),2022,36(3):36-40.DOI: 10.3969/j.issn.1674-8425(z).2022.03.005.
[9] 王春生,张佳男,王吉全.基于驾驶工况辨识的插电式混合动力汽车保电能量管理策略[J].汽车工程学报,2023,13(4):539-547.
[10] 周晓华,朱佳龙,冯雨辰.基于双启发式动态规划的PHEV能量管理策略[J].工业仪表与自动化装置,2023(3):99-105,133.DOI: 10.19950/j.cnki.cn61-1121/th.2023.03.020.
[11] SONG Z Y, HOFMANN H, LI J Q, et al. Energy management strategies comparison for electric vehicles with hybrid energy storage system[J]. Applied Energy, 2014, 134: 321-331. DOI: 10.1016/j.apenergy.2014.08.035.
[12] 杜常清,陈磊,杨贤诚,等.混合动力重卡自适应等效燃油消耗最小化能量管理策略[J].内燃机工程,2023,44(1):1-8.DOI: 10.13949/j.cnki.nrjgc.2023.01.001.
[13] SUN C, SUN F C, HE H W. Investigating adaptive-ECMS with velocity forecast ability for hybrid electric vehicles[J]. Applied Energy, 2017, 185(Part 2): 1644-1653. DOI: 10.1016/j.apenergy.2016.02.026.
[14] 付主木,龚慧贤,宋书中,等.燃料电池电动汽车改进深度强化学习能量管理[J].河南科技大学学报(自然科学版),2023,44(4):41-48.DOI: 10.15926/j.cnki.issn1672-6871.2023.04.006.
[15] 李家曦,孙友长,庞玉涵,等.基于并行深度强化学习的混合动力汽车能量管理策略优化[J].重庆理工大学学报(自然科学版),2020,34(9):62-72.DOI: 10.3969/j.issn.1674-8425(z).2020.09.007.
[16] CHEN Z, MI C C, XU J, et al. Energy management for a power-split plug-in hybrid electric vehicle based on dynamic programming and neural networks[J]. IEEE Transactions on Vehicular Technology, 2014, 63(4): 1567-1580. DOI: 10.1109/TVT.2013.2287102.
[17] TANG X L, ZHOU H T, WANG F, et al. Longevity-conscious energy management strategy of fuel cell hybrid electric vehicle based on deep reinforcement learning[J]. Energy, 2022, 238(Part A): 121593. DOI: 10.1016/j.energy.2021.121593.
[18] ZHANG Q, JU F, ZHANG S M, et al. Power management for hybrid energy storage system of electric vehicles considering inaccurate terrain information[J]. IEEE Transactions on Automation Science and Engineering, 2017, 14(2): 608-618. DOI: 10.1109/TASE.2016.2645780.
[19] YANG Y, SU L, QIN D T, et al. Energy management strategy for hybrid electric vehicle based on system efficiency and battery life optimization[J]. Wuhan University Journal of Natural Sciences, 2014, 19(3): 269-276. DOI: 10.1007/s11859-014-1012-6.
[20] EBBESEN S, ELBERT P, GUZZELLA L. Battery state-of-health perceptive energy management for hybrid electric vehicles[J]. IEEE Transactions on Vehicular Technology, 2012, 61(7): 2893-2900. DOI: 10.1109/TVT.2012.2203836.
[21] DU R H, HU X S, XIE S B, et al. Battery aging- and temperature-aware predictive energy management for hybrid electric vehicles[J]. Journal of Power Sources, 2020, 473: 228568. DOI: 10.1016/j.jpowsour.2020.228568.
[22] BAI Z W, HAO P, SHANGGUAN W, et al. Hybrid reinforcement learning-based eco-driving strategy for connected and automated vehicles at signalized intersections[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(9): 15850-15863. DOI: 10.1109/TITS.2022.3145798.
[23] WANG Q Z, GONG Y B, YANG X F. Connected automated vehicle trajectory optimization along signalized arterial: A decentralized approach under mixed traffic environment[J]. Transportation Research Part C: Emerging Technologies, 2022, 145: 103918. DOI: 10.1016/j.trc.2022.103918.
[24] 庄伟超,丁昊楠,董昊轩,等.信号交叉口网联电动汽车自适应学习生态驾驶策略[J].吉林大学学报(工学版),2023,53(1):82-93.DOI: 10.13229/j.cnki.jdxbgxb20210598.
[25] WANG Z R, WU G Y, BARTH M J. Cooperative eco-driving at signalized intersections in a partially connected and automated vehicle environment[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(11): 2029-2038. DOI: 10.1109/TITS.2019.2911607.
[26] 王金鑫,刘显贵,李运富,等.网联环境下连续交叉口车速引导及优化[J].厦门理工学院学报,2023,31(1):9-16.DOI: 10.19697/j.cnki.1673-4432.202301002.
[27] 杨超,杜雪龙,王伟达,等.智能网联环境下的PHEV实时优化能量管理策略法[J].汽车安全与节能学报,2021,12(2):210-218.DOI: 10.3969/j.issn.1674-8484.2021.01.009.
[28] 张扬,梁栋,张鹏飞,等.网联环境下混合动力汽车分层能量管理策略[J].重庆理工大学学报,2023,37(1):47-55.DOI: 10.3969/j.issn.1674-8425(z).2023.01.006.
[29] LIU Y G, HUANG Z Z, LI J, et al. Cooperative optimization of velocity planning and energy management for connected plug-in hybrid electric vehicles[J]. Applied Mathematical Modelling, 2021, 95: 715-733. DOI: 10.1016/j.apm.2021.02.033.
[30] LIU B, SUN C, WANG B, et al. Bi-level convex optimization of eco-driving for connected Fuel Cell Hybrid Electric Vehicles through signalized intersections[J]. Energy, 2022, 252: 123956. DOI: 10.1016/j.energy.2022.123956.
[31] DONG H X, ZHUANG W C, CHEN B L, et al. Predictive energy-efficient driving strategy design of connected electric vehicle among multiple signalized intersections[J]. Transportation Research Part C: Emerging Technologies, 2022, 137: 103595. DOI: 10.1016/j.trc.2022.103595.
[32] PANDE S S, NEERAJA B, KUMAR K K, et al. Off-policy reinforcement based on a safe model eco-driving education for fully-automated, connected hybrid vehicles[C] // 2023 Second International Conference on Electronics and Renewable Systems (ICEARS), Piscataway,NJ: IEEE, 2023: 10-95. DOI: 10.1109/ICEARS56392.2023.10085149.
[1] 陈秀锋, 王成鑫, 赵凤阳, 杨凯, 谷可鑫. 改进DQN算法的单点交叉口信号控制方法[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 81-88.
[2] 唐峯竹, 唐欣, 李春海, 李晓欢. 基于深度强化学习的多无人机任务动态分配[J]. 广西师范大学学报（自然科学版）, 2021, 39(6): 63-71.
[3] 赵东江, 马松艳, 田喜强. CoSe2/C催化剂在电催化氧还原中的应用研究进展[J]. 广西师范大学学报（自然科学版）, 2021, 39(5): 30-43.
Viewed
Full text
120
HTML PDF
Just accepted Online first Issue Just accepted Online first Issue
0 0 0 0 0 120

  From local
  Times 120
  Rate 100%

Abstract
72
Just accepted Online first Issue
0 0 72
  From Others local
  Times 63 9
  Rate 88% 12%

Cited
Web of Science  Crossref   ScienceDirect  Search for Citations in Google Scholar >>
 
This page requires you have already subscribed to WoS.
  Shared   
  Discussed   
[1] 朱格格, 黄安书, 覃盈盈. 基于Web of Science的国际红树林研究发展态势分析[J]. 广西师范大学学报（自然科学版）, 2024, 42(5): 1 -12 .
[2] 何静, 冯元柳, 邵靖雯. 基于CiteSpace的多源数据融合研究进展[J]. 广西师范大学学报（自然科学版）, 2024, 42(5): 13 -27 .
[3] 王淑颖, 卢宇翔, 董淑彤, 陈默, 康秉娅, 蒋长兰, 宿程远. 污水中抗生素抗性基因传播过程及控制技术研究进展[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 1 -15 .
[4] 钟俏, 陈生龙, 唐聪聪. 水凝胶技术在微藻采收中的应用:现状、挑战与发展分析[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 16 -29 .
[5] 翟思琪, 蔡文君, 朱苏, 李韩龙, 宋海亮, 杨小丽, 杨玉立. 汲取液溶质反向扩散与正渗透中膜污染的相互关系研究[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 30 -39 .
[6] 郑国权, 秦永丽, 汪晨祥, 葛仕佳, 闻倩敏, 蒋永荣. ABR硫酸盐还原体系分级沉淀酸性矿山废水中重金属及矿物形成[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 40 -52 .
[7] 刘洋, 张毅杰, 章延, 李玲, 孔祥铭, 李红. 饮用水处理中藻类混凝消除技术的现状与趋势——基于CiteSpace的可视化分析[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 53 -66 .
[8] 陈秀锋, 王成鑫, 赵凤阳, 杨凯, 谷可鑫. 改进DQN算法的单点交叉口信号控制方法[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 81 -88 .
[9] 李欣, 宁静. 基于时空特征融合的电力系统暂态稳定评估[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 89 -100 .
[10] 段沁宇, 薛贵军, 谭全伟, 谢文举. 基于SVMD的改进BWO-TimesNet短期热负荷预测模型[J]. 广西师范大学学报（自然科学版）, 2024, 42(6): 101 -116 .
版权所有 © 广西师范大学学报（自然科学版）编辑部
地址：广西桂林市三里店育才路15号　邮编：541004
电话：0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发