Learning to learn: how Minecraft is helping Microsoft researchers take the next steps in artificial intelligence innovation.
Minecraft, the international gaming phenomenon, is about to level up. Its owner, Microsoft, plans to use it as a testing ground for conducting artificial intelligence (AI) experiments, news which has got computer scientists and amateurs alike hugely excited.
Project Malmo is a platform developed by researcher Katja Hofmann and her colleagues at Microsoft’s research lab in Cambridge, UK, in order to use Minecraft as a way of building more intelligent technology. The company maintains that the Minecraft world is more advanced than any current AI research simulations and cheaper to use than building a robot, and, as such, provides the perfect environment for running experiments in artificial intelligence.
The very reason that Minecraft is so compulsively fascinating to the millions of fans who gladly enter its distinctive virtual worlds every day makes it so perfect for AI research. Unlike other computer games, Minecraft provides gamers with endless possibilities within its immersive landscapes, from simple tasks like searching for treasure, to complex missions that might involve building several structures in collaboration with other players.
As Katja Hofmann explains: “Minecraft is the perfect platform for this kind of research because it’s this very open world. You can do survival mode, you can do ‘build battles’ with your friends, you can do courses, you can implement our games. This is really exciting for artificial intelligence because it allows us to create games that stretch beyond current abilities.”
In recent years, AI researchers have achieved impressive results in teaching computers to do specific, often complex, tasks. They can now understand and translate speech, for example. However, despite such leaps forward, computers still lack what researchers call general intelligence, which is akin to the nuanced and sophisticated way that humans learn and make decisions.
A computer might be able to carry out a given task as well as, or even better than, your average adult (I’m not naming names here…), but it still can’t learn on the job. A human infant brain is taking in a myriad of stimuli at any given moment from all of its senses and is learning constantly from these experiences. A computer won’t cry if it needs recharging, but on the upside it won’t poop its metaphorical pants either!
This learning process, mostly, comes extremely easily to us humans. We are constantly taking in all manner of inputs throughout our daily lives and making decisions based upon them, but it’s this ‘awareness’ that computers find so difficult.
As Hofmann explains, AI researchers can take elements of this awareness and build algorithms to carry out a particular task, like understanding words, or recognising images, but they haven’t succeeded in combining them in the way that the human brain does so efficiently and effortlessly.
The researchers at Microsoft are attempting to train an artificial intelligence agent to learn how to do things like climb to the highest point in Minecraft’s virtual world, using the same types of resources a human has access to when they learn a new task.
The agent begins its journey without any knowledge of its environment or even what its goal might be. It quickly needs to understand its surroundings and work out for itself what is important, i.e. it needs to prioritise. To do this it must learn from trial and error, such as falling into water or lava. It will also need to understand when it has achieved, or is on the way to achieving, its goal. The agent, therefore, is on the steepest of learning curves. Instead of being programmed to accomplish a specific task, the research team are programming it to learn.
Researchers have been able to conduct theoretical research into AI relatively problem-free, but when it comes to practical ways to test their ideas they’ve been pretty limited. Building a robot and teaching it to climb a mountain is costly and impractical, especially if it’s going to be falling into rivers or lava flows.
This is where Hofmann’s Minecraft platform really comes into its own. Your robot can fall off that mountain, be attacked by giant spiders and zombies, have its feet melted off in a lava pit and you’ll never have to repair or replace it. In addition, the platform was particularly attractive to the researchers because it allows players to make complicated decisions that have consequences, as well as being able to add increasingly more difficult elements as gamers’ skill levels increase. It also allows for collaboration which means researchers can experiment with how humans and AI agents could work together.
It’s this collaboration that’s got the gaming world abuzz, with the potential for interested amateurs to become involved in the project come the summer. From the outset, the aim of Project Malmo was to build a system that would be useful for Microsoft’s own research as well as for the wider AI research community.
To this end, in July, Project Malmo will become available via an open-source licence, consisting of a mod for the Java version and code that enables the AI agents to sense and act within the Minecraft virtual world as well as providing feedback on its performance. The two elements can run on Windows, Linux or Mac OS, and users can program their agents in any programming language, although it is not a consumer product.
It is hoped that the project will help accelerate the pace of artificial intelligence innovation by attracting a wide range of academic researchers and amateurs, with all levels of programming skills, background and goals. Let the learning begin…