DeepMind develops AI capable of playing multi-player video game
Image credit: Dreamstime
Google’s DeepMind has developed a neural network capable of engaging with human players in a multi-player video game, Quake III Arena Capture the Flag (CTF), through an adapted approach to machine learning
DeepMind Technologies is a UK-based AI (artificial intelligence) company best known for building neural networks capable of playing video games. In 2016, its AlphaGo program hit headlines after it beat the world champion in a five-game Go match. A more general artificial intelligence, AlphaZero, was shown to be able to play expertly at Go, Chess, and Shogi following mere hours of unsupervised reinforcement learning (a category of machine learning in which behaviour is learnt by performing actions and observing the results of these actions).
Now, the company has presented a paper explaining how it built an AI capable of mastering the tactics, strategy and strong teamwork required to engage with multiplayer video games as effectively as a human player.
This requires multi-agent learning; the ability of different people to come together in a team to perform tasks. While this an ability people – as well as many animals – learn with ease as they mature, it has proven a challenge for neural networks. The DeepMind engineers looked at the example of 3D first-person multiplayer video games, hoping that their AI could learn teamwork from the game.
They focused on Quake III Arena CTF, which requires teams to gain possession of another team’s flag while protecting their own. They used a version of CTF in which the terrain changes with every match, preventing the AI from simply learning the best strategy for a single map.
In this approach to reinforcement learning, the DeepMind engineers trained an entire population of AI agents to play with each other. The neural networks were never explicitly provided with information about the rules of the game by the developers, and instead they learnt by playing the game. The researchers were able to identify particular artificial neurons which were activated for certain events, such as if the agent’s flag is taken.
DeepMind ended up with an optimised AI agent, which they named the ‘For The Win’ (FTW) agent. This agent was capable of playing CTF to a high human standard, cooperating with human and other FTW agents in various environments. The FTW agents were able to mimic the behaviour of human players, such as occupying the opponent’s base.
They ran a tournament with 40 human players which randomly assigned mixed teams of human and FTW agents, which allowed the FTW agents to improve their gameplay further and eventually beat the humans’ performance.
Video games are increasingly being used to train neural networks, due to the worlds of video games being potentially complex but also controllable. For instance, some neural networks responsible for controlling future autonomous vehicles have been trained to navigate hazards using the chaotic world of Grand Theft Auto.