基于深度强化学习的多无人机任务动态分配

doi:10.16088/j.issn.1001-6600.2021032402

广西师范大学学报（自然科学版） ›› 2021, Vol. 39 ›› Issue (6): 63-71.doi: 10.16088/j.issn.1001-6600.2021032402

基于深度强化学习的多无人机任务动态分配

唐峯竹¹, 唐欣², 李春海^1*, 李晓欢¹

1.桂林电子科技大学信息与通信学院, 广西桂林 541004;
2.桂林电子科技大学信息科技学院, 广西桂林 541004

收稿日期:2021-03-24 修回日期:2021-05-06 出版日期:2021-11-25 发布日期:2021-12-08
通讯作者: 李春海(1982—), 男, 山东沂南人, 桂林电子科技大学副研究员, 博士。E-mail: chunhaili@guet.edu.cn
基金资助:
国家自然科学基金(61762030); 广西自然科学基金(2019GXNSFFA245007, 2018GXNSFDA281013); 广西科技计划项目(AA18242021, AB19110050, AA19110044, ZY19183005, AB20238033); 桂林市科技计划项目(20190214-3); 广西高校中青年教师基础能力提升项目(2021KY1654)

Dynamic Task Allocation Method for UAVs Based on Deep Reinforcement Learning

TANG Fengzhu¹, TANG Xin², LI Chunhai^1*, LI Xiaohuan¹

1. School of Information and Communication, Guilin University of Electronic Technology, Guilin Guangxi 541004, China;
2. Institute of Information Technology of GUET, Guilin Guangxi 541004, China

Received:2021-03-24 Revised:2021-05-06 Online:2021-11-25 Published:2021-12-08

摘要/Abstract

摘要： 针对无人机任务随机下发场景中由于任务完成时间约束带来的任务完成度低的问题,本文提出一种基于深度强化学习的分布式多无人机任务动态分配方法。该方法通过无人机之间的实时交互,首先,对正在执行的任务和新到任务的时间约束、任务量、优先级等特征进行实时量化,同时在任务执行过程中动态生成新的任务优先级特征;然后,将无人机交互后形成的全局任务特征视为无人机共享的任务情况,并不断进行更新形成动态决策依据;最后,在时间约束下,根据实时任务完成情况通过深度强化学习方法进行无人机的行为决策,达到新任务与正在执行任务的动态分配以提高任务完成度。仿真结果表明,该方法能提高时间约束下的系统整体任务完成度。

关键词: 无人机, 任务分配, 深度强化学习, 动态性, 任务完成度

Abstract: Aiming at the problem of low task completion caused by task completion time constraints in the scenario where tasks are randomly assigned, a distributed and dynamic multi-UAV task allocation method based on deep reinforcement learning is proposed. The method uses the interaction between UAVs to quantify the time constraints, task size, task priority and other characteristics of the tasks being performed and new tasks in real time. At the same time, new task priority features are generated dynamically during task execution. Then, the task features after UAV interaction are regarded as the global task shared by UAV, and constantly updated to form a dynamic decision-making basis. Finally, according to the real-time task completion and the time constraints, the behavioral decision-making of the UAV is made based on the deep reinforcement learning, so as to improve the task completion by achieving the dynamic allocation of new tasks and ongoing tasks. This behavioral decision is to realize the dynamic assignment of tasks to improve the task completion. Simulation results show that this method can improve the overall task completion of the system under time constraints.

Key words: UAV, task allocation, deep reinforcement learning, dynamics, task completion

中图分类号:

TP181

唐峯竹, 唐欣, 李春海, 李晓欢. 基于深度强化学习的多无人机任务动态分配[J]. 广西师范大学学报（自然科学版）, 2021, 39(6): 63-71.

TANG Fengzhu, TANG Xin, LI Chunhai, LI Xiaohuan. Dynamic Task Allocation Method for UAVs Based on Deep Reinforcement Learning[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(6): 63-71.

参考文献

[1] 庞强伟, 胡永江, 李文广, 等.多无人机协同侦察任务规划方法研究综述[J]. 电讯技术, 2019, 59(6): 741-748. DOI:10.3969/j.issn.1001-893x.2019.06.020.
[2] MAHMUD I, CHO Y Z. Detection avoidance and priority-aware target tracking for UAV group reconnaissance operations[J]. Journal of Intelligent & Robotic Systems, 2018, 92(2): 381-392. DOI:10.1007/s10846-017-0745-9.
[3] ZHENG Y J, DU Y C, LING H F, et al. Evolutionary collaborative human-UAV search for escaped criminals[J]. IEEE Transactions on Evolutionary Computation, 2020, 24(2): 217-231. DOI:10.1109/TEVC.2019.2925175.
[4] CHEN X, LIU Y T, YIN L Y, et al. Cooperative task assignment and track planning for multi-UAV attack mobile targets[J]. Journal of Intelligent & Robotic Systems, 2020, 100(3/4): 1383-1400. DOI:10.1007/s10846-020-01241-w.
[5] 牛轶峰, 沈林成, 李杰, 等. 无人-有人机协同控制关键问题[J]. 中国科学: 信息科学, 2019, 49(5): 538-554. DOI:10.1360/N112019-00008.
[6] ZHOU S L, YIN G Y, WU Q P. UAV cooperative multiple task assignment based on discrete particle swarm optimization[C]//2015 7th International Conference on Intelligent Human-Machine Systems and Cybernetics. Los Alamitos, CA: IEEE Computer Society, 2015: 81-86. DOI:10.1109/IHMSC.2015.206.
[7] 赵晓林, 张可为, 李宗哲, 等. 多无人机动态侦察资源分配问题研究[J].电光与控制, 2020, 27(6): 11-15, 31. DOI:10.3969/j.issn.1671-637X.2020.06.003.
[8] ZHENG X C, WANG F, LI Z H. A multi-UAV cooperative route planning methodology for 3D fine-resolution building model reconstruction[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 146: 483-494. DOI:10.1016/j.isprsjprs.2018.11.004.
[9] MÁTHÉ K, BUŞNIU L. Vision and control for UAVs: a survey of general methods and of inexpensive platforms for infrastructure inspection[J]. Sensors, 2015, 15(7): 14887-14916. DOI:10.3390/s150714887.
[10] CHEN Y B, YANG D, YU J Q. Multi-UAV task assignment with parameter and time-sensitive uncertainties using modified two-part wolf pack search algorithm[J]. IEEE Transactions on Aerospace and Electronic Systems, 2018, 54(6): 2853-2872. DOI:10.1109/TAES.2018.2831138.
[11] LUAN H Y, XU Y T, LIU D X, et al. Energy efficient task cooperation for multi-UAV networks: a coalition formation game approach[J]. IEEE Access, 2020, 8: 149372-149384. DOI:10.1109/ACCESS.2020.3016009.
[12] GHAMRY K A, KAMEL M A, ZHANG Y M. Multiple UAVs in forest fire fighting mission using particle swarm optimization[C]//2017 International Conference on Unmanned Aircraft Systems (ICUAS). Piscataway, NJ: IEEE, 2017: 1404-1409. DOI:10.1109/ICUAS.2017.7991527.
[13] 田震,王晓芳. 基于多基因遗传算法的异构多无人机协同任务分配[J]. 飞行力学,2019,37(1): 39-44. DOI:10.13645/j.cnki.f.d.20181106.015.
[14] 邓可, 连振江, 周德云, 等. 基于改进量子粒子群算法的多无人机任务分配[J]. 指挥控制与仿真, 2018, 40(5): 32-36. DOI:10.3969/j.issn.1673-3819.2018.05.007.
[15] 许可, 宫华, 秦新立, 等. 基于分布式拍卖算法的多无人机分组任务分配[J]. 信息与控制, 2018, 47(3): 341-346. DOI:10.13976/j.cnki.xk.2018.8013.
[16] 王庆贺, 万刚, 柴峥, 等. 基于改进遗传算法的多机协同多目标分配方法[J]. 计算机应用研究, 2018, 35(9): 2597-2601. DOI:10.3969/j.issn.1001-3695.2018.09.008.
[17] 刘广瑞, 王庆海, 姚冬艳. 基于改进人工蜂群算法的多无人机协同任务规划[J]. 郑州大学学报(工学版), 2018, 39(3): 51-55. DOI:10.13705/j.issn.1671-6833.2017.06.026.
[18] WEI Y, BLAKE M B, MADEY G R. An operation-time simulation framework for UAV swarm configuration and mission planning[J]. Procedia Computer Science, 2013, 18: 1949-1958. DOI:10.1016/j.procs.2013.05.364.
[19] KIM K, CAMPBELL J, DUONG W, et al.DisCoF⁺: asynchronous DisCoF with flexible decoupling for cooperative pathfinding in distributed systems[C]//2015 IEEE International Conference on Automation Science and Engineering (CASE). Piscataway, NJ: IEEE, 2015: 369-376. DOI:10.1109/CoASE.2015.7294106.
[20] ZHEN Z Y, XING D J, GAO C. Cooperative search-attack mission planning for multi-UAV based on intelligent self-organized algorithm[J]. Aerospace Science and Technology, 2018, 76: 402-411. DOI:10.1016/j.ast.2018.01.035.
[21] 李相民, 唐嘉钰, 代进进, 等. 异构多智能体联盟动态任务分配[J]. 西北工业大学学报, 2020, 38(5): 1094-1104. DOI:10.1051/jnwpu/20203851094.
[22] 孙鹏, 陈冠宇, 张杰勇, 等. 基于突发事件的任务计划动态调整模型及算法[J]. 控制与决策, 2020, 35(5): 1052-1062. DOI:10.13195/j.kzyjc.2018.1096.
[23] ATENCIA C R, SER J D, CAMACHO D. Weighted strategies to guide a multi-objective evolutionary algorithm for multi-UAV mission planning[J]. Swarm and Evolutionary Computation, 2019, 44: 480-495. DOI:10.1016/j.swevo.2018.06.005.
[24] 王然然, 魏文领, 杨铭超, 等. 考虑协同航路规划的多无人机任务分配[J]. 航空学报, 2020, 41(S2): 724234. DOI:10.7527/S1000-6893.2020.24234.
[25] YAO W R, QI N M, WAN N, et al. An iterative strategy for task assignment and path planning of distributed multiple unmanned aerial vehicles[J]. Aerospace Science and Technology, 2019, 86: 455-464. DOI:10.1016/j.ast.2019.01.061.
[26] 吴蔚楠, 关英姿, 郭继峰, 等. 基于SEAD任务特性约束的协同任务分配方法[J]. 控制与决策, 2017, 32(9): 1574-1582. DOI:10.13195/j.kzyjc.2016.0858.
[27] LUO Y S, HU Q Q, WANG Y F, et al. Revenue optimization of a UAV-fog collaborative framework for remote data collection services[J]. IEEE Access, 2020, 8: 150599-150610. DOI:10.1109/ACCESS.2020.3016779.
[28] ZHOU Z Y, FENG J H, GU B, et al. When mobile crowd sensing meets UAV: energy-efficient task assignment and route planning[J]. IEEE Transactions on Communications, 2018, 66(11): 5526-5538. DOI:10.1109/TCOMM.2018.2857461.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于深度强化学习的多无人机任务动态分配

Dynamic Task Allocation Method for UAVs Based on Deep Reinforcement Learning

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

Metrics

本文评价

推荐阅读 10

[1]	唐堂, 魏承赟, 罗晓曙, 丘森辉. 基于附加惯性项人群搜索算法的四旋翼无人机姿态控制研究[J]. 广西师范大学学报（自然科学版）, 2018, 36(4): 12-19.
[2]	唐堂,罗晓曙,吕万德,刘欣. 四旋翼无人机滑模自抗扰控制[J]. 广西师范大学学报（自然科学版）, 2018, 36(2): 56-62.
[3]	苏诚, 陈文娜, 周玲, 黄冬梅. 面向海洋空间数据集成的多Agent任务分配机制[J]. 广西师范大学学报（自然科学版）, 2011, 29(2): 205-209.