Minecraft teaches robots better planning

Minecraft computer game to improve robots' planning skills

A new algorithm tested with the help of the Minecraft video game is being developed to improve planning skills of robots.

The algorithm, developed by researchers from Brown University in Providence, Rhode Island, does for robots what intuition does for humans. It enables them to automatically ignore objects and actions that are not critical for completing a task.

The lack of this ability frequently leaves the robots completely overwhelmed by inputs from the environment, which makes efficient planning nearly impossible and leads to the robot performing unnecessary actions.

"It's a really tough problem," said Stefanie Tellex, assistant professor of computer science at Brown. "We want robots that have capabilities to do all kinds of different things, but then the space of possible actions becomes enormous. We don't want to limit the robot's capabilities, so we have to find ways to shrink the search space."

The algorithm augments standard robot planning algorithms using  the so called goal-based action priors - a set of objects and actions in a given space that are most likely to help an agent achieve a given goal. The priors for a given task can be supplied by an expert operator, but they can also be learned by the algorithm itself through trial and error.

The game Minecraft, as it turns out, provided an ideal world to test how well the algorithm learned action priors and implemented them in the planning process.

"Minecraft is a really good a model of a lot of these robot problems," Tellex said. "There's a huge space of possible actions somebody playing this game can do, and it's really cheap and easy to collect a ton of training data. It's much harder to do that in the real world."

Tellex and her colleagues started by constructing small domains, each just a few blocks square, in a model of Minecraft they developed. Then they placed a character into the domain and gave it a simple task such as mining some buried gold or building a bridge to cross a chasm.

The agent, powered by the algorithm, then had to try different options in order to learn the task's goal-based priors and identify the best actions to get the job done.

"It's able to learn that if you're standing next to a trench and you're trying to walk across, you can place blocks in the trench. Otherwise don't place blocks," Tellex said. "If you're trying to mine some gold under some blocks, destroy the blocks. Otherwise don't destroy blocks."

After the algorithm ran through a number of trials of a given task to learn the appropriate priors, the researchers moved to a new domain that it had never seen before to see if it could apply what it learned - and it could.

After the virtual world training, the researchers then added the algorithm to a real robot’s software.

The robot was assigned a task of helping a human to bake brownies. The algorithm, supplied with multiple task-related action priors, helped the robot to make decisions about required actions. For example, by knowing that eggs often need to be beaten with a whisk, the robot was able to promptly deliver a whisk to the cook as soon as a carton of  eggs arrived.

In light of the results, Tellex said she sees goal-based action priors as a viable strategy to help robots cope with the complexities of unstructured environments - something that will be important as robots continue to move out of controlled settings and into the everyday world.

Recent articles

Info Message

Our sites use cookies to support some functionality, and to collect anonymous user data.

Learn more about IET cookies and how to control them