The first ever computer program capable of identifying objects in hand-drawn pictures better than humans can has been built by British researchers.
The program, dubbed Sketch-a-Net, has reportedly outperformed humans in tests by identifying 74.9 per cent of sketches correctly. People involved in the Queen Mary University of London (QMUL) study achieved a 73.1 per cent success rate.
Moreover, the computer program was considerably more successful in understanding details of the sketches. For example, it was able to distinguish the exact type of bird drawn in 42.5 per cent of all cases while humans only successfully recognised 24.8 per cent.
“It’s exciting that our computer program can solve the task even better than humans can,” said Timothy Hospedales, co-author of the study and lecturer at QMUL’s School of Electronic Engineering and Computer Science.
“Sketches are an interesting area to study because they have been used since pre-historic times for communication and now, with the increase in use of touchscreens, they are becoming a much more common communication tool again.”
The team believes the program could pave the way for the development of new forms of human-computer interaction. For example, a computer would be able to find specific images after a user sketches what he or she is looking for on the touchscreen. Such a system could replace traditional keyword search but would also help in various research or forensic applications by, for example, matching drawings of criminals with mugshots and CCTV databases.
“This could really have a huge impact for areas such as police forensics, touchscreen use and image retrieval, and ultimately will help us get to the bottom of visual understanding,” said Hospedales.
Sketch-a-Net is a so-called deep neural network, a type of computer program designed to emulate the processing of the human brain. It can understand the order in which strokes were drawn, which may be critical for recognising the objects in the pictures.
Recognising free-hand sketches is challenging for computers because they are abstract, varied and consist of black and white lines rather than coloured pixels like a photo. Solving sketch recognition will lead to a greater scientific understanding of visual perception.