Human-in-the-loop Control of Multi-agent Aerial Systems Under Intermittent Communication


The objective of this research is decentralized control of heterogeneous multi-agent systems in degraded communication environments. We propose a novel design of an HMI that allows a human to become a supervisor, when necessary, instead of a single unit operator. By deploying dexterous aerial robots as components of multi-agent systems, we allow the supervisor to interact with its surroundings. We envision a scenario in which a group of agents has to be deployed and controlled by an operator in order to perform mission(s) in a coordinated fashion by joining, breaking, maintaining and reconfiguring formation. To accomplish such a mission, each agent has to take-off autonomously, reach formation with others in a decentralized manner, and then fly semi-autonomously by human-in-the-loop control. Upon reaching a designated target, agents, if required, autonomously break their formation allowing the operator to take over the control over a single agent while others remain in coordinated hover. Now, the operator controls a multi-DOF manipulator, attached to the agent, to perform a required task. Once the task is completed, the agent autonomously returns in the formation. Meanwhile, the operator takes over the control of another agent, equipped with different sensors/actuators, and executes another task. In case cooperative actions are required (e.g., heterogeneous urban reconnaissance, surveillance, and target acquisition), a semiautonomous flight is activated in which the operator becomes a supervisor that issues a single prede ned command to reconfigure/break the formation and achieve desired area coverage as well as acquire all available data to become aware of the situation. Several research groups have already demonstrated similar scenarios. However, there are a few fundamental differences between their approaches and our envisioned mission. In those approaches, agent trajectories are planed on-line, centralized coordination is considered, each agent has a designated ground station and/or an operator, missions are aborted when sporadic communication failures occur, etc.