Journal of Guangxi Normal University(Natural Science Edition) ›› 2021, Vol. 39 ›› Issue (6): 63-71.doi: 10.16088/j.issn.1001-6600.2021032402

Previous Articles     Next Articles

Dynamic Task Allocation Method for UAVs Based on Deep Reinforcement Learning

TANG Fengzhu1, TANG Xin2, LI Chunhai1*, LI Xiaohuan1   

  1. 1. School of Information and Communication, Guilin University of Electronic Technology, Guilin Guangxi 541004, China;
    2. Institute of Information Technology of GUET, Guilin Guangxi 541004, China
  • Received:2021-03-24 Revised:2021-05-06 Online:2021-11-25 Published:2021-12-08

Abstract: Aiming at the problem of low task completion caused by task completion time constraints in the scenario where tasks are randomly assigned, a distributed and dynamic multi-UAV task allocation method based on deep reinforcement learning is proposed. The method uses the interaction between UAVs to quantify the time constraints, task size, task priority and other characteristics of the tasks being performed and new tasks in real time. At the same time, new task priority features are generated dynamically during task execution. Then, the task features after UAV interaction are regarded as the global task shared by UAV, and constantly updated to form a dynamic decision-making basis. Finally, according to the real-time task completion and the time constraints, the behavioral decision-making of the UAV is made based on the deep reinforcement learning, so as to improve the task completion by achieving the dynamic allocation of new tasks and ongoing tasks. This behavioral decision is to realize the dynamic assignment of tasks to improve the task completion. Simulation results show that this method can improve the overall task completion of the system under time constraints.

Key words: UAV, task allocation, deep reinforcement learning, dynamics, task completion

CLC Number: 

  • TP181
[1] 庞强伟, 胡永江, 李文广, 等.多无人机协同侦察任务规划方法研究综述[J]. 电讯技术, 2019, 59(6): 741-748. DOI:10.3969/j.issn.1001-893x.2019.06.020.
[2] MAHMUD I, CHO Y Z. Detection avoidance and priority-aware target tracking for UAV group reconnaissance operations[J]. Journal of Intelligent & Robotic Systems, 2018, 92(2): 381-392. DOI:10.1007/s10846-017-0745-9.
[3] ZHENG Y J, DU Y C, LING H F, et al. Evolutionary collaborative human-UAV search for escaped criminals[J]. IEEE Transactions on Evolutionary Computation, 2020, 24(2): 217-231. DOI:10.1109/TEVC.2019.2925175.
[4] CHEN X, LIU Y T, YIN L Y, et al. Cooperative task assignment and track planning for multi-UAV attack mobile targets[J]. Journal of Intelligent & Robotic Systems, 2020, 100(3/4): 1383-1400. DOI:10.1007/s10846-020-01241-w.
[5] 牛轶峰, 沈林成, 李杰, 等. 无人-有人机协同控制关键问题[J]. 中国科学: 信息科学, 2019, 49(5): 538-554. DOI:10.1360/N112019-00008.
[6] ZHOU S L, YIN G Y, WU Q P. UAV cooperative multiple task assignment based on discrete particle swarm optimization[C]//2015 7th International Conference on Intelligent Human-Machine Systems and Cybernetics. Los Alamitos, CA: IEEE Computer Society, 2015: 81-86. DOI:10.1109/IHMSC.2015.206.
[7] 赵晓林, 张可为, 李宗哲, 等. 多无人机动态侦察资源分配问题研究[J].电光与控制, 2020, 27(6): 11-15, 31. DOI:10.3969/j.issn.1671-637X.2020.06.003.
[8] ZHENG X C, WANG F, LI Z H. A multi-UAV cooperative route planning methodology for 3D fine-resolution building model reconstruction[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 146: 483-494. DOI:10.1016/j.isprsjprs.2018.11.004.
[9] MÁTHÉ K, BUŞNIU L. Vision and control for UAVs: a survey of general methods and of inexpensive platforms for infrastructure inspection[J]. Sensors, 2015, 15(7): 14887-14916. DOI:10.3390/s150714887.
[10] CHEN Y B, YANG D, YU J Q. Multi-UAV task assignment with parameter and time-sensitive uncertainties using modified two-part wolf pack search algorithm[J]. IEEE Transactions on Aerospace and Electronic Systems, 2018, 54(6): 2853-2872. DOI:10.1109/TAES.2018.2831138.
[11] LUAN H Y, XU Y T, LIU D X, et al. Energy efficient task cooperation for multi-UAV networks: a coalition formation game approach[J]. IEEE Access, 2020, 8: 149372-149384. DOI:10.1109/ACCESS.2020.3016009.
[12] GHAMRY K A, KAMEL M A, ZHANG Y M. Multiple UAVs in forest fire fighting mission using particle swarm optimization[C]//2017 International Conference on Unmanned Aircraft Systems (ICUAS). Piscataway, NJ: IEEE, 2017: 1404-1409. DOI:10.1109/ICUAS.2017.7991527.
[13] 田震,王晓芳. 基于多基因遗传算法的异构多无人机协同任务分配[J]. 飞行力学,2019,37(1): 39-44. DOI:10.13645/j.cnki.f.d.20181106.015.
[14] 邓可, 连振江, 周德云, 等. 基于改进量子粒子群算法的多无人机任务分配[J]. 指挥控制与仿真, 2018, 40(5): 32-36. DOI:10.3969/j.issn.1673-3819.2018.05.007.
[15] 许可, 宫华, 秦新立, 等. 基于分布式拍卖算法的多无人机分组任务分配[J]. 信息与控制, 2018, 47(3): 341-346. DOI:10.13976/j.cnki.xk.2018.8013.
[16] 王庆贺, 万刚, 柴峥, 等. 基于改进遗传算法的多机协同多目标分配方法[J]. 计算机应用研究, 2018, 35(9): 2597-2601. DOI:10.3969/j.issn.1001-3695.2018.09.008.
[17] 刘广瑞, 王庆海, 姚冬艳. 基于改进人工蜂群算法的多无人机协同任务规划[J]. 郑州大学学报(工学版), 2018, 39(3): 51-55. DOI:10.13705/j.issn.1671-6833.2017.06.026.
[18] WEI Y, BLAKE M B, MADEY G R. An operation-time simulation framework for UAV swarm configuration and mission planning[J]. Procedia Computer Science, 2013, 18: 1949-1958. DOI:10.1016/j.procs.2013.05.364.
[19] KIM K, CAMPBELL J, DUONG W, et al.DisCoF+: asynchronous DisCoF with flexible decoupling for cooperative pathfinding in distributed systems[C]//2015 IEEE International Conference on Automation Science and Engineering (CASE). Piscataway, NJ: IEEE, 2015: 369-376. DOI:10.1109/CoASE.2015.7294106.
[20] ZHEN Z Y, XING D J, GAO C. Cooperative search-attack mission planning for multi-UAV based on intelligent self-organized algorithm[J]. Aerospace Science and Technology, 2018, 76: 402-411. DOI:10.1016/j.ast.2018.01.035.
[21] 李相民, 唐嘉钰, 代进进, 等. 异构多智能体联盟动态任务分配[J]. 西北工业大学学报, 2020, 38(5): 1094-1104. DOI:10.1051/jnwpu/20203851094.
[22] 孙鹏, 陈冠宇, 张杰勇, 等. 基于突发事件的任务计划动态调整模型及算法[J]. 控制与决策, 2020, 35(5): 1052-1062. DOI:10.13195/j.kzyjc.2018.1096.
[23] ATENCIA C R, SER J D, CAMACHO D. Weighted strategies to guide a multi-objective evolutionary algorithm for multi-UAV mission planning[J]. Swarm and Evolutionary Computation, 2019, 44: 480-495. DOI:10.1016/j.swevo.2018.06.005.
[24] 王然然, 魏文领, 杨铭超, 等. 考虑协同航路规划的多无人机任务分配[J]. 航空学报, 2020, 41(S2): 724234. DOI:10.7527/S1000-6893.2020.24234.
[25] YAO W R, QI N M, WAN N, et al. An iterative strategy for task assignment and path planning of distributed multiple unmanned aerial vehicles[J]. Aerospace Science and Technology, 2019, 86: 455-464. DOI:10.1016/j.ast.2019.01.061.
[26] 吴蔚楠, 关英姿, 郭继峰, 等. 基于SEAD任务特性约束的协同任务分配方法[J]. 控制与决策, 2017, 32(9): 1574-1582. DOI:10.13195/j.kzyjc.2016.0858.
[27] LUO Y S, HU Q Q, WANG Y F, et al. Revenue optimization of a UAV-fog collaborative framework for remote data collection services[J]. IEEE Access, 2020, 8: 150599-150610. DOI:10.1109/ACCESS.2020.3016779.
[28] ZHOU Z Y, FENG J H, GU B, et al. When mobile crowd sensing meets UAV: energy-efficient task assignment and route planning[J]. IEEE Transactions on Communications, 2018, 66(11): 5526-5538. DOI:10.1109/TCOMM.2018.2857461.
[1] RUAN Wenjing, YANG Qigui. Research on Complex Dynamics of a New Four-dimensional Hyperchaotic System with Finite and Infinite Isolated Singularities [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 173-181.
[2] SHE Lianbing, GAO Yunlong. Backward-compact Dynamics for Non-autonomous Navier-Stokes Equations on Unbounded Domains [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(1): 41-46.
[3] HONG Lingling, YANG Qigui. Research on Complex Dynamics of a New 4D Hyperchaotic System [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(3): 96-105.
[4] ZHANG Yijin. Random Dynamics for Stochastic Delay Lattice Systems in Xρ Space [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(3): 106-110.
[5] WANG Yi, ZOU Yanli, LI Ke, HUANG Li. The Influence of the Distributed Power Station Connection Modeson the Power Grid Synchronization [J]. Journal of Guangxi Normal University(Natural Science Edition), 2017, 35(4): 24-31.
[6] LIU Huimin, GUAN Dongjie, ZHANG Mengjie. Stress Factorsand Stress Mechanism on Subsequent Developmentof Ecological Security in the Three Gorges Reservoir Area [J]. Journal of Guangxi Normal University(Natural Science Edition), 2016, 34(3): 150-158.
[7] QIN Jian, CHEN Shan-zhen, BAI Ke-zhao, KONG Ling-jiang. Subconscious Effect on Pedestrian Emergency Evacuation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2014, 32(3): 27-32.
[8] LIAO Zhi-xian, LUO Xiao-shu, HUANG Guo-xian. Non-linear Dynamics Model and Chaotic Behavior of the Single-phase Full Bridge Photovoltaic Grid-connected Inverter [J]. Journal of Guangxi Normal University(Natural Science Edition), 2013, 31(4): 1-6.
[9] XIE Guang-qiang, ZHANG Yun, LI Yang, ZENG Qi-jie. Research of Krause's Multi-Agent Consensus Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2013, 31(3): 106-113.
[10] GAO Jun-fen, HU Wei-ping. Recognition and Study of Pathological Voices Based on NonlinearDynamics Using GMM [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(3): 5-8.
[11] XIONG Fei-bing, YANG Feng. Fluorescence Decay Dynamics of Praseodymium Ions Doped in Lead Tungstate Crystal [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(2): 71-75.
[12] SU Cheng, CHEN Wen-na, ZHOU Ling, HUANG Dong-mei. Mechanism of Multi-agent Task Allocation for Ocean Spatial Data Integration [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(2): 205-209.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] HU Jinming, WEI Duqu. Hybrid Projective Synchronization of Fractional-order PMSM with Different Orders[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 1 -8 .
[2] WU Kangkang, ZHOU Peng, LU Ye, JIANG Dan, YAN Jianghong, QIAN Zhengcheng, GONG Chuang. FIR Equalizer Based on Mini-batch Gradient Descent Method[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 9 -20 .
[3] LIU Dong, ZHOU Li, ZHENG Xiaoliang. A Very Short-term Electric Load Forecasting Based on SA-DBN[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 21 -33 .
[4] ZHANG Weibin, WU Jun, YI Jianbing. Research on Feature Fusion Controlled Items Detection Algorithm Based on RFB Network[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 34 -46 .
[5] WANG Jinyan, HU Chun, GAO Jian. An OBDD Construction Method for Knowledge Compilation[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 47 -54 .
[6] LU Miao, HE Dengxu, QU Liangdong. Grey Wolf Optimization Algorithm Based on Elite Learning for Nonlinear Parameters[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 55 -67 .
[7] LI Lili, ZHANG Xingfa, LI Yuan, DENG Chunliang. Daily GARCH Model Estimation Using High Frequency Data[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 68 -78 .
[8] LI Songtao, LI Qunhong, ZHANG Wen. Co-dimension-two Grazing Bifurcation and Chaos Control of Three-degree-of-freedom Vibro-impact Systems[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 79 -92 .
[9] ZHAO Hongtao, LIU Zhiwei. Decompositions of λ-fold Complete Bipartite 3-uniform Hypergraphs λK(3)n,n into Hypergraph Triangular Bipyramid[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 93 -98 .
[10] LI Meng, CAO Qingxian, HU Baoqing. Spatial-temporal Analysis of Continental Coastline Migration from 1960 to 2018 in Guangxi, China[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 99 -108 .