Journal of Guangxi Normal University(Natural Science Edition) ›› 2021, Vol. 39 ›› Issue (5): 134-146.doi: 10.16088/j.issn.1001-6600.2020111404

Previous Articles     Next Articles

Real-time Citrus Recognition under Orchard Environment by Improved YOLOv4

CHEN Wenkang1, LU Shenglian1,2*, LIU Binghao3, LI Guo1, LIU Xiaoyu1, CHEN Ming1   

  1. 1. College of Computer Science and Engineering, Guangxi Normal University, Guilin Guangxi 541004, China;
    2. Guangxi Key Lab of Multisource Information Mining and Security, Guangxi Normal University, Guilin Guangxi 541004, China;
    3. Guangxi Academy of Specialty Crops/Guangxi Citrus Breeding and Cultivation Engineering Technology Center,Guilin Guangxi 541004, China
  • Received:2020-11-14 Revised:2021-01-29 Online:2021-09-25 Published:2021-10-19

Abstract: The automatic detection of fruits is a key technology in agricultural applications such as automatic picking, orchard spraying, and post-harvesting sorting. Aiming at the problems of small citrus targets, many noises, and serious occlusion in the orchard environment, this paper proposes an improved fast identification method for citrus in the orchard environment based on the YOLOv4 algorithm. The main improvement include: one is to use the Canopy algorithm and K-Means++ algorithm to automatically select the number and size of the priori boxes in the training phase; the other is to add an adjustment layer before each output layer of different scale features in the YOLOv4 network, where the residual network structure is combined with densely connected network, and the loss function of the regression box is modified to detect small citrus in a complex background; third, on the premise of ensuring that a large amount of detection accuracy is not lost, the unimportant channels and networks in the network Layers are pruned. The experimental results of comparison with the three commonly used target detection algorithms show that the improved YOLOv4 detection method in this paper has better detection results for citrus in different growth periods in the orchard environment, with an average accuracy rate of 96.04% and a real-time detection speed of 0.06 s per image, are better than the above three mainstream target detection algorithms. The method proposed in this paper can provide technical and methodological guidance for citrus harvesting and yield evaluation in orchards under natural conditions.

Key words: citrus recognition, small target detection, deep learning, improved YOLOv4, convolutional neural networks

CLC Number: 

  • TP391.41
[1] 王刘坤, 祁春节. 中国柑橘主产区的区域比较优势及其影响因素研究: 基于省级面板数据的实证分析[J]. 中国农业资源与区划, 2018, 39(11): 121-128.
[2] 刘英璇, 伍锡如, 雪刚刚. 基于深度学习的道路交通标志多目标实时检测[J]. 广西师范大学学报(自然科学版), 2020, 38(2): 96-106. DOI:10.16088/j.issn.1001-6600.2020.02.011.
[3] 翟治芬, 徐哲, 周新群, 等. 基于朴素贝叶斯分类器的棉花盲椿象危害等级识别[J]. 农业工程学报, 2015, 31(1): 204-211. DOI:10.3969/j.issn.1002-6819.2015.01.028.
[4] 黄小玉, 李光林, 马驰, 等. 基于改进判别区域特征融合算法的近色背景绿色桃子识别[J]. 农业工程学报,2018, 34(23): 142-148. DOI:10.11975/j.issn.1002-6819.2018.23.017.
[5] 汪成龙, 李小昱, 武振中, 等. 基于流形学习算法的马铃薯机械损伤机器视觉检测方法[J]. 农业工程学报, 2014, 30(1): 245-252. DOI:10.3969/j.issn.1002-6819.2014.01.031.
[6] GONGAL A, AMATYA S, KARKEE M, et al. Sensors and systems for fruit detection and localization: a review[J]. Computers and Electronics in Agriculture, 2015, 116: 8-19.DOI:10.1016/j.compag.2015.05.021.
[7] LIN G C, TANG Y C, ZOU X J, et al. In-field citrus detection and localisation based on RGB-D image analysis[J]. Biosystems Engineering, 2019, 186:34-44. DOI:10.1016/j.biosystemseng.2019.06.019.
[8] 唐熔钗, 伍锡如. 基于改进YOLO-V3网络的百香果实时检测[J]. 广西师范大学学报(自然科学版), 2020, 38(6): 32-39. DOI:10.16088/j.issn.1001-6600.2020.06.004.
[9] 黄豪杰, 段先华, 黄欣辰. 基于深度学习水果检测的研究与改进[J]. 计算机工程与应用, 2020, 56(3): 127-133. DOI:10.3778/j.issn.1002-8331.1906-0384.
[10] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444. DOI:10.1038/nature14539.
[11] 岑冠军, 华俊达, 潘怡颖, 等. 基于深度学习芒果图像在线识别与计数方法研究[J]. 热带作物学报, 2020, 41(3): 425-432. DOI:10.3969/j.issn.1000-2561.2020.03.002.
[12] 闫建伟, 赵源, 张乐伟, 等. 改进Faster-RCNN自然环境下识别刺梨果实[J]. 农业工程学报, 2019, 35(18): 143-150. DOI:10.11975/j.issn.1002-6819.2019.18.018.
[13] KOIRALA A, WALSH K B, WANG Z, et al. Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO’[J]. Precision Agriculture, 2019, 20(6): 1107-1135. DOI:10.1007/s11119-019-09642-0.
[14] 熊俊涛, 郑镇辉, 梁嘉恩, 等. 基于改进YOLO v3网络的夜间环境柑橘识别方法[J]. 农业机械学报, 2020, 51(4): 199-206. DOI:10.6041/j.issn.1000-1298.2020.04.023.
[15] KANG H W, CHEN C. Fruit detection and segmentation for apple harvesting using visual sensor in orchards[J]. Sensors, 2019, 19(20): 4599. DOI:10.3390/s19204599.
[16] FONT D, PALLEIÀ T, TRESANCHEZ M, et al. A proposal for automatic fruit harvesting by combining a low cost stereovision camera and a robotic arm[J]. Sensors, 2014, 14(7): 11557-11579. DOI:10.3390/s140711557.
[17] CHEN M Y, TANG Y C, ZOU X J, et al. Three-dimensional perception of orchard banana central stock enhanced by adaptive multi-vision technology[J]. Computers and Electronics in Agriculture, 2020, 174: 105508. DOI:10.1016/j.compag.2020.105508.
[18] HAAS T, SCHUBERT C, EICKHOFF M, et al. BubCNN: bubble detection using faster RCNN and shape regression network[J]. Chemical Engineering Science, 2020, 216: 115467. DOI:10.1016/j.ces.2019.115467.
[19] CHEN P, LI Y, ZHOU H, et al. Detection of small ship objects using anchor boxes cluster and feature pyramid network model for SAR imagery[J]. Journal of Marine Science and Engineering, 2020, 8(2): 112. DOI:10.3390/jmse8020112.
[20] TIAN Y N, YANG G D, WANG Z, et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019, 157: 417-426. DOI:10.1016/j.compag.2019.01. 012.
[21] WU D H, WU Q, YIN X Q, et al. Lameness detection of dairy cows based on the YOLOv3 deep learning algorithm and a relative step size characteristic vector[J]. Biosystems Engineering, 2020, 189: 150-163. DOI:10.1016/j.biosystemseng. 2019.11.017.
[22] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. (2020-04-23)[2020-11-14]. https:// arxiv.org/pdf/2004.10934v1.
[23] MISRA D. Mish: a self regularized non-monotonic neural activation function[EB/OL]. (2019-08-23)[2020-11-14]. https:// arxiv.org/pdf/1908.08681v2.
[24] ZHANG Y, ZHOU W J, WANG Y J, et al. A real-time recognition method of static gesture based on DSSD[J]. Multimedia Tools and Applications, 2020, 79(25/26): 17445-17461. DOI:10.1007/s11042-020-08725-9.
[25] 鞠默然, 罗海波, 王仲博, 等. 改进的YOLO V3算法及其在小目标检测中的应用[J]. 光学学报, 2019, 39(7): 0715004. DOI:10.3788/AOS201939.0715004.
[26] 戴加明,佟继周.基于深度残差网络的星系形态分类[J].天文学进展,2018,36(4):384-397. DOI:10.3969/j.issn.1000-8349.2018.04.03.
[27] LIU Z, LI J G, SHEN Z Q, et al. Learning efficient convolutional networks through network slimming[C] // 2017 IEEE International Conference on Computer Vision (ICCV). Los Alamitos, CA: IEEE Computer Society, 2017: 2755-2763. DOI:10.1109/ICCV.2017.298.
[28] MEHTA S S, TON C, ASUNDI S, et al. Multiple camera fruit localization using a particle filter[J]. Computers and Electronics in Agriculture, 2017, 142(Part A): 139-154. DOI:10.1016/j.compag.2017.08.007.
[29] JIANG B, WU Q, YIN X Q, et al. FLYOLOv3 deep learning for key parts of dairy cow body detection[J]. Computers and Electronics in Agriculture, 2019, 166: 104982. DOI:10.1016/j.compag.2019.104982.
[30] CAI Z W, VASCONCELOS N. Cascade R-CNN: high quality object detection and instance segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(5): 1483-1498. DOI:10.1109/TPAMI.2019.2956516.
[31] WANG H, WANG Q L, GAO M Q, et al. Multi-scale location-aware kernel representation for object detection[C] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 1248-1257. DOI:10.1109/CVPR.2018.00136.
[1] WU Lingyu, LAN Yang, XIA Haiying. Retinal Image Registration Using Convolutional Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 122-133.
[2] YANG Zhou, FAN Yixing, ZHU Xiaofei, GUO Jiafeng, WANG Yue. Survey on Modeling Factors of Neural Information Retrieval Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(2): 1-12.
[3] DENG Wenxuan, YANG Hang, JIN Ting. A Dimensionality-reduction Method Based on Attention Mechanismon Image Classification [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(2): 32-40.
[4] XUE Tao, QIU Senhui, LU Hao, QIN Xingsheng. Exchange Rate Prediction Based on Empirical Mode Decomposition and Multi-branch LSTM Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(2): 41-50.
[5] TANG Rongchai, WU Xiru. Real-time Detection of Passion Fruit Based on Improved YOLO-V3 Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(6): 32-39.
[6] ZHANG Mingyu,ZHAO Meng,CAI Fuhong,LIANG Yu,WANG Xinhong. Wave Power Prediction Based on Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(3): 25-32.
[7] LI Weiyong, LIU Bin, ZHANG Wei, CHEN Yunfang. An Automatic Summarization Model Based on Deep Learning for Chinese [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 51-63.
[8] LIU Yingxuan, WU Xiru, XUE Ganggang. Multi-target Real-time Detection for Road Traffic SignsBased on Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 96-106.
[9] ZHANG Jinlei, LUO Yuling, FU Qiang. Predicting Financial Time Series Based on Gated Recurrent Unit Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(2): 82-89.
[10] HUANG Liming,CHEN Weizheng,YAN Hongfei,CHEN Chong. A Stock Prediction Method Based on Recurrent Neural Network and Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 13-22.
[11] WU Wenya,CHEN Yufeng,XU Jin’an,ZHANG Yujie. High-level Semantic Attention-based Convolutional Neural Networks for Chinese Relation Extraction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 32-41.
[12] YUE Tianchi, ZHANG Shaowu, YANG Liang, LIN Hongfei, YU Kai. Stance Detection Method Based on Two-Stage Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 42-49.
[13] YU Chuanming,LI Haonan,AN Lu. Analysis of Text Emotion Cause Based on Multi-task Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 50-61.
[14] WANG Qi,QIU Jiahui,RUAN Tong,GAO Daqi,GAO Ju. Recurrent Capsule Network for Clinical Relation Extraction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 80-88.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] CHEN Chun-qiang, DENG Hua, HUANG Fang-fang. Concentration and Speciation Analysis of Heavy Metals in Soils in Guiping Manganese Mine, Guangxi, China[J]. Journal of Guangxi Normal University(Natural Science Edition), 2014, 32(4): 108 -114 .
[2] LI Yufeng, QIN Jiashuang, MA Jiangming, YANG Zhangqi, LI Mingjin, LU Shaohao, SONG Zunrong. Establishment of Above-ground Biomass Model and Distribution Characteristics of Pinus massoniana Plantations in Southern Subtropical[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 170 -180 .
[3] ZHAN Xin, CHEN Lijing, LIAO Guangfeng, LI Bing, LU Rumei. Research Progress of New C21-Steroids in Medicinal Plant of Asclepiadaceae (Ⅰ)[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 1 -29 .
[4] ZHAO Dongjiang, MA Songyan, TIAN Xiqiang. Applications of CoSe2/C Catalyst in Electrocatalytic Oxygen Reduction[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 30 -43 .
[5] GUAN Xiaojin, ZHAO Keyi, LIU Shiling, LI Yi, YU Fangming, LI Chunming, LIU Kehui. Global Trends and Hot Topics in the Field of Manganese Phytoremediation over the Past Three Decades: A Review Based on Citespace Visualization[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 44 -57 .
[6] LIANG Qiufang, DONG Xiaoyan, FENG Ping. Advanced in CYP2D Subfamily Genes and Evolutionary Mechanisms[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 58 -63 .
[7] XU Lunhui, SU Nan, PIAN Yuzhuang, LIN Peiqun. Bus Travel Time Prediction Based on Extreme Learning Machine Optimized by Artificial Bee Colony Algorithm[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 64 -77 .
[8] WENG Xiaoxiong, XIE Zhipeng. Study on Freeway Nodes Importance Based on Multilayer Complex Network[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 78 -88 .
[9] WU Kangkang, ZHU Xufei, LU Ye, ZHOU Peng, DONG Cui, DAI Qinxuan, ZHOU Runchang. LS-FIR Filter Based on Least Square Method[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 89 -99 .
[10] HAI Tao, LI Nana, ZHOU Wenjie, CHEN Juan, SONG Min. Design of Intelligent Monitoring System for Photovoltaic Greenhouse Based on LPWAN Internet of Things[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 100 -109 .