Social feedback algorithm allows robots to fetch items with intelligence

An algorithm that lets robots understand social interactions and alter their behaviour in response has been developed by robotics researchers at Brown University.

The project is designed to improve human-robot collaboration in order to allow them to become helpers in people’s homes and workplaces.

“Fetching objects is an important task that we want collaborative robots to be able to do,” said professor Stefanie Tellex who worked on the project.

“But it’s easy for the robot to make errors, either by misunderstanding what we want, or by being in situations where commands are ambiguous. So what we wanted to do here was come up with a way for the robot to ask a question when it’s not sure.”

The lab had previously developed an algorithm that enables robots to receive speech commands as well as information from human gestures. It demonstrated that when robots combine speech commands with gestures, they were better at correctly interpreting user commands.

However, the system proved flawed and ran into problems when there were lots of very similar objects in close proximity to each other.

For example, simply asking for “a wrench” isn’t specific enough and the robots would not be able to determine which one a person is pointing to if a number of wrenches are clustered close together.

“What we want in these situations is for the robot to be able to signal that it’s confused and ask a question rather than just fetching the wrong object,” Tellex said.

The new algorithm does that. It enables the robot to quantify how certain it is that it knows what a user wants. When its certainty is high, the robot will simply hand over the object as requested.

When it’s not so certain, the robot makes its best guess about what the person wants, then asks for confirmation by hovering its gripper over the object and asking, “This one?”

One of the important features of the system is that the robot doesn’t ask questions with every interaction. It asks intelligently.

“When the robot is certain, we don’t want it to ask a question because it just takes up time,” said research student Eric Rosen. “But when it is ambiguous, we want it to ask questions because mistakes can be more costly in terms of time.”

Even though the system asks only a very simple question, “it’s able to make important inferences based on the answer,” said co-author on the research paper David Whitney.

For example, when a user asks for a wrench and there are two wrenches on a table, if the user tells the robot that its first guess was wrong, the algorithm deduces that the other wrench must be the one that the user wants. It will then hand that one over without asking another question. Those kinds of inferences, known as implicatures, make the algorithm more efficient.

To test their system, the researchers asked untrained participants to come into the lab and interact with Baxter, a popular industrial and research robot.

Participants asked Baxter for objects under different conditions. The team could set the robot to never ask questions, ask a question every time, or to ask questions only when uncertain. The trials showed that asking questions intelligently using the new algorithm was significantly better in terms of accuracy and speed compared to the other two conditions.

The system worked so well, in fact, that participants thought the robot had capabilities it actually didn’t have.

For the purposes of the study, the researchers used a very simple language model - one that only understood the names of objects. However, participants told the researchers they thought the robot could understand prepositional phrases like, “on the left” or “closest to me,” which it could not. They also thought the robot might be tracking their eye-gaze, which it wasn’t. All the system was doing was making smart inferences after asking a very simple question.

In future work, Tellex and her team would like to combine the algorithm with more robust speech recognition systems, which might further increase the system’s accuracy and speed.

Ultimately, Tellex says, she hopes systems like this will help robots become useful collaborators both at home and at work.

Recent articles

Info Message

Our sites use cookies to support some functionality, and to collect anonymous user data.

Learn more about IET cookies and how to control them