Artificial interaction nears Turing Test rate

Machines have come the closest yet to demonstrating interactive skills that fool humans that they are dealing with other humans, in a contest at the University of Reading.

In a contest at the University of Reading, all of the competing artificial conversational entities (ACEs) fooled at least one of their human interrogators, with one achieving a dupe-rate just 5 per cent shy of the 30 per cent pass rate mandated by mathematician Alan Turing. His 1950 Turing Test imitation game postulates that if during text-based conversation, a machine is indistinguishable from a human, then it could be said to be 'thinking' and, therefore, could be attributed with intelligence.

Five ACEs competed in a relays of five minute-long, unrestricted conversation Turing Tests with a team of 40 volunteer human interrogators. Interrogators did not know if they were conversing with an ACE or with a real human elsewhere in the building, when they rated the session. The competition formed part of the 18th Loebner Prize, and was won by the programme ‘Elbot’, whose creator Fred Roberts was awarded the $3000 Loebner Bronze Award by its sponsor Hugh Loebner.

The results show a more complex story than a straight pass or fail by one machine, says the event’s organiser Kevin Warwick, professor of Cybernetics at University of Reading: “Where the machines were identified correctly by the human interrogators as machines, the conversational abilities of each machine was scored at 80 per cent and 90 per cent - demonstrating how close machines have got to communicating with us in a way with which we are comfortable. Software that is adaptive learns to pick-up the nuances of language. They can joke with you or swear at you - and thus fool you.”

However, one aspect that machines still slip-up on, Warwick says, is medium-term memory: “They can remember and refer to facts that have been recently articulated, but forget them in longer conversations. But humans are skilled at interpolating older information.”

Warwick points out that the aptitudes of the human testers play a significant part in determining the ACEs’ success rates: “It is difficult to assemble ‘average’ people to interact in a technical way with a machine; we aimed at as much of a cross-section of testers as possible.”

He believes that ACEs need not necessarily achieve 100 per cent human fidelity in order for users to accept them as human operators for online and telephone service applications. He even envisages ACEs designed to fulfill socio-therapeutic roles that allow aggrieved souls just to ‘sound off’ at them.

More information:

Recent articles

Info Message

Our sites use cookies to support some functionality, and to collect anonymous user data.

Learn more about IET cookies and how to control them