Journal of Guangxi Normal University(Natural Science Edition) ›› 2022, Vol. 40 ›› Issue (2): 37-48.doi: 10.16088/j.issn.1001-6600.2021072002

Previous Articles     Next Articles

Vehicle Detection for Autonomous Vehicle System Based on Multi-modal Feature Fusion

XUE Qiwei1,2, WU Xiru1,2*   

  1. 1. School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin Guangxi 541004, China;
    2. Guangxi Key Laboratory for Nonlinear Circuit and Optical Communication (Guangxi Normal University), Guilin Guangxi 541004, China
  • Received:2021-07-20 Revised:2021-09-26 Published:2022-05-31

PDF (PC)

136

Abstract: Aiming at the low accuracy of vehicle detection in unmanned system environment perception, a three-dimensional vehicle detection algorithm based on multi-modal feature fusion is proposed. Through the joint calibration of millimeter wave radar and camera, the coordinate relationship between the two sensors is matched and the sampling error is reduced. Statistical filtering is used to eliminate the redundant points of millimeter wave radar data and reduce the interference of outliers. The multi-modal feature fusion module is constructed, and the point cloud and image information are fused by pixel average. Adding the feature pyramid to extract the fused high-level feature information to improve the detection accuracy in complex road scenes, a feature fusion region recommendation structure is established, and the region recommendation is generated according to the advanced feature information. After removing the redundant detection frame, the vehicle detection results are output through the vertex matching of the detection frame. The experimental results on KITTI data set show that the proposed method can realize vehicle detection quickly and accurately. The average detection time is 0.14 s and the average detection accuracy is 84.71%. The algorithm has important theoretical and practical value, and can provide a powerful means for vehicle detection in unmanned system.

Key words: millimeter wave radar, environment perception, multi-modal feature fusion, vehicle detection, autonomous vehicle system

CLC Number: 

  • TP391.41
[1] CAI P D, WANG S K, SUN Y X, et al. Probabilistic end-to-end vehicle navigation in complex dynamic environments with multimodal sensor fusion[J]. IEEE Robotics and Automation Letters, 2020, 5(3): 4218-4224. DOI: 10.1109/LRA.2020.2994027.
[2] 刘英璇, 伍锡如, 雪刚刚. 基于深度学习的道路交通标志多目标实时检测[J]. 广西师范大学学报(自然科学版), 2020, 38(2): 96-106. DOI: 10.16088/j.issn.1001-6600.2020.02.011.
[3] ZHANG Y, SONG B, DU X J, et al. Vehicle tracking using surveillance with multimodal data fusion[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(7): 2353-2361. DOI: 10.1109/TITS.2017.2787101.
[4] STANISLAS L, DUNBABIN M. Multimodal sensor fusion for robust obstacle detection and classification in the maritime RobotX challenge[J]. IEEE Journal of Oceanic Engineering, 2019, 44(2): 343-351. DOI: 10.1109/JOE.2018.2868488.
[5] 谢德胜, 徐友春, 王任栋, 等. 基于三维激光雷达的无人车障碍物检测与跟踪[J]. 汽车工程, 2018, 40(8): 952-959. DOI: 10.19562/j.chinasae.qcgc.2018.08.013.
[6] 薛培林, 吴愿, 殷国栋, 等. 基于信息融合的城市自主车辆实时目标识别[J]. 机械工程学报, 2020, 56(12): 165-173. DOI: 10.3901/JME.2020.12.165.
[7] 郑少武, 李巍华, 胡坚耀. 基于激光点云与图像信息融合的交通环境车辆检测[J]. 仪器仪表学报, 2019, 40(12): 143-151. DOI: 10.19650/j.cnki.cjsi.J1905607.
[8] WANG G J, WU J, XU T, et al. 3D vehicle detection with RSU LiDAR for autonomous mine[J]. IEEE Transactions on Vehicular Technology, 2021, 70(1): 344-355. DOI: 10.1109/TVT.2020.3048985.
[9] DAI D Y, WANG J K, CHEN Z H, et al. Image guidance based 3D vehicle detection in traffic scene[J]. Neurocomputing, 2021, 428: 1-11. DOI: 10.1016/j.neucom.2020.11.060.
[10] 陈龙, 司译文, 田滨, 等. 基于3D LiDAR的矿山无人驾驶车行驶边界检测[J]. 煤炭学报, 2020, 45(6): 2140-2146. DOI: 10.13225/j.cnki.jccs.zn 20.0093.
[11] CHOE J S, JOO K D, IMTIAZ T, et al. Volumetric propagation network: stereo-LiDAR fusion for long-range depth estimation[J]. IEEE Robotics and Automation Letters, 2021, 6(3): 4672-4679. DOI: 10.1109/LRA.2021.3068712.
[12] 张灿龙, 李燕茹, 李志欣, 等. 基于核相关滤波与特征融合的分块跟踪算法[J]. 广西师范大学学报(自然科学版), 2020, 38(5): 12-23. DOI: 10.16088/j.issn.1001-6600.2020.05.002.
[13] NIE J, YAN J, YIN H L, et al. A multimodality fusion deep neural network and safety test strategy for intelligent vehicles[J]. IEEE Transactions on Intelligent Vehicles, 2021, 6(2): 310-322. DOI: 10.1109/TIV.2020.3027319.
[14] ZHANG X Y, LI Z W, GAO X. Channel attention in LiDAR-camera fusion for lane line segmentation[J]. Pattern Recognition, 2021, 118: 108020. DOI: 10.1016/J.PATCOG.2021.108020.
[15] 王肖, 李克强, 王建强, 等. 基于三维激光雷达的智能车辆目标参数辨识[J]. 汽车工程, 2016, 38(9): 1146-1152. DOI: 10.19562/j.chinasae.qcgc.2016.09.017.
[16] 李明磊, 王力, 宗文鹏, 等. 采用八叉树体素生长的点云平面提取[J]. 光学精密工程, 2018, 26(1): 172-183. DOI: 10.3788/OPE.20182601.0172.
[17] 吴毅华, 梁华为, 王智灵, 等. 基于激光雷达回波信号的自适应阈值车道线检测[J]. 机器人, 2015, 37(4): 451-458. DOI: 10.13973/j.cnki.robot.2015.0451.
[18] 陈紫强, 张雅琼. 一种基于YOLOv4的改进DeepSort目标跟踪算法[J]. 桂林电子科技大学学报, 2021, 41(2): 140-145. DOI: 10.16725/j.cnki.cn45-1351/tn.2021.02.009.
[19] 丁萌, 姜欣言. 先进驾驶辅助系统中基于单目视觉的场景深度估计方法[J]. 光学学报, 2020, 40(17): 1715001. DOI: 10.3788/AOS202040.1715001.
[20] 彭博, 蔡晓禹, 唐聚, 等. 基于形态检测与深度学习的高空视频车辆识别[J]. 交通运输系统工程与信息, 2019, 19(6): 45-51. DOI: 10.16097/j.cnki.1009-6744.2019.06.008.
[21] 程海博, 熊显名. 基于GIoU的YOLOv3车辆识别方法[J]. 桂林电子科技大学学报,2020, 40(5):429-433. DOI: 10.16725/j.cnki.cn45-1351/tn.2020.05.010.
[22] ZHAO X M, SUN P P, XU Z G, et al. Fusion of 3D LIDAR and camera data for object detection in autonomous vehicle applications[J]. IEEE Sensors Journal, 2020, 20(9): 4901-4913. DOI: 10.1109/JSEN.2020.2966034.
[23] ZHE T, HUANG L Q, WU Q, et al. Inter-vehicle distance estimation method based on monocular vision using 3D detection[J]. IEEE Transactions on Vehicular Technology, 2020, 69(5): 4907-4919. DOI: 10.1109/TVT.2020.2977623.
[24] POURMOHAMAD T, LEE H K H. The statistical filter approach to constrained optimization[J]. Technometrics, 2020, 62(3): 303-312. DOI: 10.1080/00401706.2019.1638304.
[25] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2016: 770-778. DOI: 10.1109/CVPR.2016.90.
[26] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2017: 936-944. DOI: 10.1109/CVPR.2017.106.
[27] GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: the KITTI dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237. DOI: 10.1177/0278364913491297.
[1] LI Ziyan,LIU Weiming. New Method of Moving Vehicle Detection Based on Partial HOG Feature [J]. Journal of Guangxi Normal University(Natural Science Edition), 2017, 35(3): 1-13.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] AI Yan, JIA Nan, WANG Yuan, GUO Jing, PAN Dongdong. Review of Statistical Methods and Applications of Genetic Association Analysis for Multiple Traits and Multiple Locus[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 1 -14 .
[2] BAI Defa, XU Xin, WANG Guochang. Review of Generalized Linear Models and Classification for Functional Data[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 15 -29 .
[3] ZENG Qingfan, QIN Yongsong, LI Yufang. Empirical Likelihood Inference for a Class of Spatial Panel Data Models[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 30 -42 .
[4] ZHANG Zhifei, DUAN Qian, LIU Naijia, HUANG Lei. High-dimensional Nonlinear Regression Model Based on JMI[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 43 -56 .
[5] YANG Di, FANG Yangxin, ZHOU Yan. New Category Classification Research Based on MEB and SVM Methods[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 57 -67 .
[6] CHEN Zhongxiu, ZHANG Xingfa, XIONG Qiang, SONG Zefang. Estimation and Test for Asymmetric DAR Model[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 68 -81 .
[7] DU Jinfeng, WANG Hairong, LIANG Huan, WANG Dong. Progress of Cross-modal Retrieval Methods Based on Representation Learning[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 1 -12 .
[8] LI Muhang, HAN Meng, CHEN Zhiqiang, WU Hongxin, ZHANG Xilong. Survey of Algorithms Oriented to Complex High Utility Pattern Mining[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 13 -30 .
[9] CHAO Rui, ZHANG Kunli, WANG Jiajia, HU Bin, ZHANG Weicong, HAN Yingjie, ZAN Hongying. Construction of Chinese Multimodal Knowledge Base[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 31 -39 .
[10] LI Zhengguang, CHEN Heng, LIN Hongfei. Identification of Adverse Drug Reaction on Social Media Using Bi-directional Language Model[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 40 -48 .
Copyright © Journal of Guangxi Normal University(Natural Science Edition), All Rights Reserved.
Tel: 0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
Powered by Beijing Magtech Co. Ltd