Media release

From: Springer Nature

Technology: A champion-level drone-flying AI (N&V) *IMAGES &

An artificial intelligence (AI) system named Swift, capable of flying drones to win races against human competitors in a head-to-head championship, is presented in a Nature paper. The findings represent a milestone for mobile robotics and machine intelligence, which may inspire the deployment of hybrid learning-based solutions in other physical systems — such as self-driving ground vehicles, aircraft and personal robots.

Deep reinforcement learning systems have previously outperformed humans in various games, including Atari, chess, StarCraft and Gran Turismo. However, these have been mostly limited to simulation and board game environments as opposed to in physical competitions. In first-person view drone racing, professional competitors fly high-speed aircraft through three-dimensional circuits. Each pilot sees the environment from their drone’s perspective via video streamed from an onboard camera. Reaching the level of professional pilots with an autonomous drone is challenging because the robot needs to fly at its physical limits while estimating its speed and location in the circuit exclusively from onboard sensors.

Elia Kaufmann and colleagues designed an autonomous system that can race physical vehicles at the level of human world champions. The system combines deep reinforcement learning in simulation with data collected in the physical world. In a series of races on a real-world track designed by a professional drone-racing pilot, Swift competed against three human champions, including the world champions of two international leagues. The human pilots were given one week of practice on the race track, after which each pilot competed against Swift in multiple head-to-head races. The system won multiple races against each of the human champions (winning a total of 15 out of the 25 races) and also achieved the fastest recorded race time over the course with a lead of half a second over the best time clocked by a human pilot.

In an accompanying News & Views, Guido de Croon says that further developments in a more realistic, varying environment will be necessary to realise the full potential of the technology. To beat human pilots in any racing environment, the system will have to deal with external disturbances like the wind, changing light conditions, gates that are less clearly defined, other racing drones and many other factors — all of which pose sizeable challenges to existing AI techniques.