基于深度强化学习的网联燃料电池混合动力汽车生态驾驶联合优化方法

doi:10.16088/j.issn.1001-6600.2023120802

Abstract

Abstract: With the rapid development of the new technologies about Internet of Things (IoT) and automatic driving, an advanced research target has been injected into the optimization of eco-driving and energy management of hybrid vehicles based on the connected driving environment. Aiming at the fuel cell hybrid vehicles driving on multi-signalized urban roads, this paper proposes a hierarchical multi-objective optimization method combined deep deterministic policy gradient and dynamic planning (DDPG-DP) for speed planning and energy management. The DDPG algorithm is used in the upper layer of energy-saving speed planning, while the multi-objective reward value function and the priority experience replay mechanism are designed to carry out the multi-objective speed planning for energy saving, driving comfort, and passage efficiency on the basis of improving the algorithm’s speed and stability, and the dynamic planning algorithm is used in the lower layer of energy management to achieve the optimal energy-saving of the hybrid system with the goal of minimizing the hydrogen consumption. In scenarios 1 and 2, the results show that the DDPG-DP algorithm improves the traveling efficiency by 15.25% and 20.18% than the IDM-DP algorithm, and reduces the hydrogen fuel consumption by 25.66% and 17.86%, respectively. Meanwhile, there is a gap of only about 5 s in the passing time of the DDPG-DP algorithm compared with the global optimal algorithm (DP-DP) in Scenarios 1 and 2, and the hydrogen fuel consumption is lower than the optimal algorithm. Meanwhile, there is only a difference of about 5 s between the DDPG-DP algorithm and the global optimal algorithm (DP-DP) in traveling time, and there is only a difference of 2.84% and 4.7% in the hydrogen fuel consumption compared with the DP-DP algorithm. In field of driving smoothness, the DDPG-DP algorithm has less speed fluctuation than the other two algorithms (IDM-DP and DP-DP) and doesn’t have large acceleration/deceleration. It will provide greater energy-saving potential for daily driving of hybrid vehicles and support the further research for multi-objective eco-driving optimization of connected fuel cell hybrid vehicles.

Key words: energy management, fuel cell, hybrid vehicle, deep reinforcement learning, co-optimization, connected and autonomous vehicles

CLC Number: TP18;U469.7

TIAN Sheng, CHEN Dong. A Joint Eco-driving Optimization Research for Connected Fuel Cell Hybrid Vehicle via Deep Reinforcement Learning[J].Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 67-80.

References

[1] 陈晓龙,焦晓红.网联混合动力汽车跟驰场景的预测能量管理控制[J].燕山大学学报,2023,47(1):43-53.DOI: 10.3969/j.issn.1007-791X.2023.01.005.
[2] 陈峥,张玉果,沈世全,等.城市郊区道路跟车条件下智能网联汽车速度规划[J].中国公路学报,2023,36(6):298-310.DOI: 10.19721/j.cnki.1001-7372.2023.06.023.
[3] LV Z H, LOU R R, SINGH A K. AI empowered communication systems for intelligent transportation systems[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(7): 4579-4587. DOI: 10.1109/TITS.2020.3017183.
[4] 魏小栋,孙超,刘波,等.燃料电池汽车车速与能量联合优化[J].机械工程学报,2023,59(8):204-212.DOI: 10.3901/JME.2023.08.204.
[5] 周健豪,顾诚,刘军,等.基于IGWO的燃料电池汽车模糊控制能量管理策略[J].重庆理工大学学报(自然科学版),2021,35(5):33-41.DOI: 10.3969/j.issn.1674-8425(z).2021.05.005.
[6] 于瀛霄,孙闫,夏长高,等.燃料电池汽车双层模糊控制能量管理策略[J].重庆理工大学学报(自然科学版),2022,36(8):21-28.DOI: 10.3969/j.issn.1674-8425(z).2022.08.003.
[7] ZOU W T, LI J W, YANG Q Q, et al. An improved max-min game theory control of fuel cell and battery hybrid energy system against system uncertainty[J]. IEEE Journal of Emerging and Selected Topics in Power Electronics, 2023, 11(1): 78-87. DOI: 10.1109/JESTPE.2022.3168374.
[8] 赵天宇,陈东,霍为炜,等.氢燃料电池汽车能量管理系统模糊控制仿真研究[J].重庆理工大学学报(自然科学版),2022,36(3):36-40.DOI: 10.3969/j.issn.1674-8425(z).2022.03.005.
[9] 王春生,张佳男,王吉全.基于驾驶工况辨识的插电式混合动力汽车保电能量管理策略[J].汽车工程学报,2023,13(4):539-547.
[10] 周晓华,朱佳龙,冯雨辰.基于双启发式动态规划的PHEV能量管理策略[J].工业仪表与自动化装置,2023(3):99-105,133.DOI: 10.19950/j.cnki.cn61-1121/th.2023.03.020.
[11] SONG Z Y, HOFMANN H, LI J Q, et al. Energy management strategies comparison for electric vehicles with hybrid energy storage system[J]. Applied Energy, 2014, 134: 321-331. DOI: 10.1016/j.apenergy.2014.08.035.
[12] 杜常清,陈磊,杨贤诚,等.混合动力重卡自适应等效燃油消耗最小化能量管理策略[J].内燃机工程,2023,44(1):1-8.DOI: 10.13949/j.cnki.nrjgc.2023.01.001.
[13] SUN C, SUN F C, HE H W. Investigating adaptive-ECMS with velocity forecast ability for hybrid electric vehicles[J]. Applied Energy, 2017, 185(Part 2): 1644-1653. DOI: 10.1016/j.apenergy.2016.02.026.
[14] 付主木,龚慧贤,宋书中,等.燃料电池电动汽车改进深度强化学习能量管理[J].河南科技大学学报(自然科学版),2023,44(4):41-48.DOI: 10.15926/j.cnki.issn1672-6871.2023.04.006.
[15] 李家曦,孙友长,庞玉涵,等.基于并行深度强化学习的混合动力汽车能量管理策略优化[J].重庆理工大学学报(自然科学版),2020,34(9):62-72.DOI: 10.3969/j.issn.1674-8425(z).2020.09.007.
[16] CHEN Z, MI C C, XU J, et al. Energy management for a power-split plug-in hybrid electric vehicle based on dynamic programming and neural networks[J]. IEEE Transactions on Vehicular Technology, 2014, 63(4): 1567-1580. DOI: 10.1109/TVT.2013.2287102.
[17] TANG X L, ZHOU H T, WANG F, et al. Longevity-conscious energy management strategy of fuel cell hybrid electric vehicle based on deep reinforcement learning[J]. Energy, 2022, 238(Part A): 121593. DOI: 10.1016/j.energy.2021.121593.
[18] ZHANG Q, JU F, ZHANG S M, et al. Power management for hybrid energy storage system of electric vehicles considering inaccurate terrain information[J]. IEEE Transactions on Automation Science and Engineering, 2017, 14(2): 608-618. DOI: 10.1109/TASE.2016.2645780.
[19] YANG Y, SU L, QIN D T, et al. Energy management strategy for hybrid electric vehicle based on system efficiency and battery life optimization[J]. Wuhan University Journal of Natural Sciences, 2014, 19(3): 269-276. DOI: 10.1007/s11859-014-1012-6.
[20] EBBESEN S, ELBERT P, GUZZELLA L. Battery state-of-health perceptive energy management for hybrid electric vehicles[J]. IEEE Transactions on Vehicular Technology, 2012, 61(7): 2893-2900. DOI: 10.1109/TVT.2012.2203836.
[21] DU R H, HU X S, XIE S B, et al. Battery aging- and temperature-aware predictive energy management for hybrid electric vehicles[J]. Journal of Power Sources, 2020, 473: 228568. DOI: 10.1016/j.jpowsour.2020.228568.
[22] BAI Z W, HAO P, SHANGGUAN W, et al. Hybrid reinforcement learning-based eco-driving strategy for connected and automated vehicles at signalized intersections[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(9): 15850-15863. DOI: 10.1109/TITS.2022.3145798.
[23] WANG Q Z, GONG Y B, YANG X F. Connected automated vehicle trajectory optimization along signalized arterial: A decentralized approach under mixed traffic environment[J]. Transportation Research Part C: Emerging Technologies, 2022, 145: 103918. DOI: 10.1016/j.trc.2022.103918.
[24] 庄伟超,丁昊楠,董昊轩,等.信号交叉口网联电动汽车自适应学习生态驾驶策略[J].吉林大学学报(工学版),2023,53(1):82-93.DOI: 10.13229/j.cnki.jdxbgxb20210598.
[25] WANG Z R, WU G Y, BARTH M J. Cooperative eco-driving at signalized intersections in a partially connected and automated vehicle environment[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(11): 2029-2038. DOI: 10.1109/TITS.2019.2911607.
[26] 王金鑫,刘显贵,李运富,等.网联环境下连续交叉口车速引导及优化[J].厦门理工学院学报,2023,31(1):9-16.DOI: 10.19697/j.cnki.1673-4432.202301002.
[27] 杨超,杜雪龙,王伟达,等.智能网联环境下的PHEV实时优化能量管理策略法[J].汽车安全与节能学报,2021,12(2):210-218.DOI: 10.3969/j.issn.1674-8484.2021.01.009.
[28] 张扬,梁栋,张鹏飞,等.网联环境下混合动力汽车分层能量管理策略[J].重庆理工大学学报,2023,37(1):47-55.DOI: 10.3969/j.issn.1674-8425(z).2023.01.006.
[29] LIU Y G, HUANG Z Z, LI J, et al. Cooperative optimization of velocity planning and energy management for connected plug-in hybrid electric vehicles[J]. Applied Mathematical Modelling, 2021, 95: 715-733. DOI: 10.1016/j.apm.2021.02.033.
[30] LIU B, SUN C, WANG B, et al. Bi-level convex optimization of eco-driving for connected Fuel Cell Hybrid Electric Vehicles through signalized intersections[J]. Energy, 2022, 252: 123956. DOI: 10.1016/j.energy.2022.123956.
[31] DONG H X, ZHUANG W C, CHEN B L, et al. Predictive energy-efficient driving strategy design of connected electric vehicle among multiple signalized intersections[J]. Transportation Research Part C: Emerging Technologies, 2022, 137: 103595. DOI: 10.1016/j.trc.2022.103595.
[32] PANDE S S, NEERAJA B, KUMAR K K, et al. Off-policy reinforcement based on a safe model eco-driving education for fully-automated, connected hybrid vehicles[C] // 2023 Second International Conference on Electronics and Renewable Systems (ICEARS), Piscataway,NJ: IEEE, 2023: 10-95. DOI: 10.1109/ICEARS56392.2023.10085149.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	ZHU Gege, HUANG Anshu, QIN Yingying. Analysis of Development Trend of International Mangrove Research Based on Web of Science[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(5): 1 -12 .
[2]	HE Jing, FENG Yuanliu, SHAO Jingwen. Research Progress on Multi-source Data Fusion Based on CiteSpace[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(5): 13 -27 .
[3]	WANG Shuying, LU Yuxiang, DONG Shutong, CHEN Mo, KANG Bingya, JIANG Zhanglan, SU Chengyuan. Research Progress on the Propagation Process and Control Technology of ARGs in Wastewater[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 1 -15 .
[4]	ZHONG Qiao, CHEN Shenglong, TANG Congcong. Hydrogel Technology for Microalgae Collection: Status Overview, Challenges and Development Analysis[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 16 -29 .
[5]	ZHAI Siqi, CAI Wenjun, ZHU Su, LI Hanlong, SONG Hailiang, YANG Xiaoli, YANG Yuli. Dynamic Relationship Between Reverse Solute Flux and Membrane Fouling in Forward Osmosis[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 30 -39 .
[6]	ZHENG Guoquan, QIN Yongli, WANG Chenxiang, GE Shijia, WEN Qianmin, JIANG Yongrong. Stepwise Precipitation of Heavy Metals from Acid Mine Drainage and Mineral Formation in Sulfate-Reducing Anaerobic Baffled Reactor System[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 40 -52 .
[7]	LIU Yang, ZHANG Yijie, ZHANG Yan, LI Ling, KONG Xiangming, LI Hong. Current Status and Trends of Algal Coagulation Elimination Technology in Drinking Water Treatment: a Visual Analysis Based on CiteSpace[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 53 -66 .
[8]	CHEN Xiufeng, WANG Chengxin, ZHAO Fengyang, YANG Kai, GU Kexin. A Single Intersection Signal Control Method Based on Improved DQN Algorithm[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 81 -88 .
[9]	LI Xin, NING Jing. Online Assessment of Transient Stability in Power Systems Based on Spatiotemporal Feature Fusion[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 89 -100 .
[10]	DUAN Qinyu, XUE Guijun, TAN Quanwei, XIE Wenju. Improved BWO-TimesNet Short-term Heat Load Forecasting Model Based onSVMD[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 101 -116 .

A Joint Eco-driving Optimization Research for Connected Fuel Cell Hybrid Vehicle via Deep Reinforcement Learning

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 3

Metrics

Comments

Recommended 10

[1]	CHEN Xiufeng, WANG Chengxin, ZHAO Fengyang, YANG Kai, GU Kexin. A Single Intersection Signal Control Method Based on Improved DQN Algorithm [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 81-88.
[2]	TANG Fengzhu, TANG Xin, LI Chunhai, LI Xiaohuan. Dynamic Task Allocation Method for UAVs Based on Deep Reinforcement Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(6): 63-71.
[3]	ZHAO Dongjiang, MA Songyan, TIAN Xiqiang. Applications of CoSe₂/C Catalyst in Electrocatalytic Oxygen Reduction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 30-43.