Lego employs AI to translate instructions into braille and audio

The Danish toy company has launched a global trial with four building sets and hopes to release more next year depending on user feedback.

Using the design script used by Lego creators, known as LXFML data, the Austrian Research Institute for Artificial Intelligence developed software that is able to turn the instructions into braille and English audio.

The idea came about after blind entrepreneur and Lego enthusiast Matthew Shifrin previously relied on a friend to translate instructions into braille, which enabled him to build independently for the first time.

After his friend died, Mr Shifrin was introduced to Lego's Creative Play Lab and decided to push for development of the technology: "This is extremely important for blind children because there aren't a lot of places where we can say 'Look Mum and Dad! I built this on my own... I did this'," he said.

"For blind children, we don't have access to what sighted kids are used to. Lego bricks enable us to learn about our environment, to see the world. It is so important because blind kids get left out of a lot of social stuff, especially in elementary school, but Lego building is one of the things we can do."

Lego said there is still "a lot of progress" to be made with the AI software and automating the process, but it hopes to introduce more languages in the future.

"As I build a set, I develop a better sense of what a building looks like and how it is laid out and constructed," Shifrin added. "For blind people, Lego sets act as miniature 3D substitutes for real-life buildings in lieu of two-dimensional photographs. Lego bricks allow me to see things that are impossible to explore by touch, such as the arches of a Middle Eastern palace or the towers of the London Tower Bridge."

Lego Group creative director Fenella Blaize Charity said: "[Shifrin's] project will help visually impaired children around the world experience the same joy of building and pride of creation that all our fans feel."

Paul Welham, chief executive of CereProc, the speech synthesis company that provided Lego with the Software Development Kit, said: "This fantastic initiative is a major step forward for people living with visual impairment. By adopting Text-to-Speech technology in this way, Lego will not only improve the lives of those using its products, but hopefully also inspire other companies to follow suit with characterful synthetic speech."

Lego has a number of products in place that extend its building sets beyond their basic play function. Earlier this year, for example, the company announced its new Star Wars collection, designed to help boost children's STEM skills. The 'Lego Star Wars Boost Droid Commander' set enables children to build, code and play with three iconic Star Wars droids.

The Danish toymaker has also been obliged to consider its role in global plastic consumption, given that its ongoing commercial success is predominantly based on a petroleum-based product. Accordingly, in 2018, Lego confirmed that sustainable plant-based bricks would become gradually available, rolled out piece by piece. It has started manufacturing “botanical elements” using a sustainable material, starting (appropriately enough) with leaves, bush and tree Lego pieces sold in its packs. These elements account for approximately one to two per cent of the total plastic elements produced by the toy manufacturer.

It was also revealed in 2018 that UK police firms had started to acquire rare Lego pieces in order to help educate firms about cyber crime. London's Metropolitan Police force now owns around a dozen Lego sets, which are used as part of a ‘Decisions and Disruptors’ board game exercise played with business executives. Officers are reportedly not allowed to play with the sets in their spare time.

Despite the global success of its products, Lego has faced a number of challenges in its manufacturing operation, as E&T discovered when it visited the Lego factory in Denmark.