Keeping a learned eye on human behaviour
A cognitive computational system designed to spot patterns of public behaviour in a variety of environments such as surveillance and accident prevent has been developed at Barcelona’s Universitat Autònoma in collaboration with a consortium of European academic research bodies.
HERMES (human expressive graphic representation of motion and their evaluation in sequences) analyses human behaviour based on video sequences captured at three focus levels: individual facial expressions; the individual body postures; and individuals as relatively distant objects. The data is then processed by HERMES’ computer vision and artificial intelligence algorithms, which enable it to interpret and recognise movement patterns.
HERMES introduces two innovations in the field of computer vision, explains Professor Juan José Villanueva, emeritus professor of the Department of Computer Science at Universitat Autònoma de Barcelona: “The first is the natural-language description of movement captured by the cameras, through simple and precise phrases which appear on the computer screen in real time. HERMES uses an avatar to talk and describe this information”.
The system can, in addition, analyse and discover potentially aberrant human behaviour, and issue warning alerts based on what it ‘observes’, Professor Villanueva adds.
The HERMES initiative comprises seven sub-projects developed by research groups across Europe that have been engaged on the project. In the UK, the Active Vision Group in the Department of Engineering Science at University of Oxford has worked on HERMES’ Intelligent Surveillance component: this integrates the fields of activity recognition with active sensing.
The aim of its work is to automatically identify in a surveillance situation the direction in which human figures are facing from a distant surveillance camera to provide input to HERMES higher-level reasoning systems. According to Active Vision Group leader Dr Ian Reid, the direction in which somebody is facing provides an estimate of their ‘gaze direction’, which can help infer familiarity between people observed, or their interest in surroundings.
“The application advantages of HERMES are mainly in the fields of intelligent surveillance and the prevention of accidents or crimes,” says Universitat Autònoma de Barcelona’s José Villanueva. “However, researchers consider that there is much to be gained with the use of this tool in sectors such as marketing or psychology.”