Google's Glass project is synonymous with augmented reality, but the technology is already part of everyday life for many people.
A story told by Georgia Institute of Technology associate professor Thad Starner, now the technical lead on Google's Glass project to build a headset to make augmented reality mainstream, illustrates both the advantages and potential social problems for the technology.
While Starner was at the Massachusetts Institute of Technology (MIT), a student friend went to the top of the Great Dome that overlooks the campus along the north bank of the Charles River. Once at the top, he found he could not get back into the elevator because it had descended to the ground floor. He decided his best bet was to send an email to a fellow student in their lab to ask them to come and send the lift back to the top. He used a keypad similar to that used on the Microwriter proto-PDA launched in the early 1990s – strapped to his arm.
"How are you typing this?" came the puzzled reply.
"I am the Borg," he tapped back.
Starner said at the time that the exotic headgear and armware needed was, socially, 'a great icebreaker'. But for mass-market acceptance, looking like an extra from a later series of 'Star Trek' is probably not going to work even if it could at some point be fleetingly fashionable.
Fashion conscious
Google has made the Glass product steadily more acceptable, stripping a headset down to an unobtrusive pair of spectacles. Those who have tried the Glass report how it makes them feel self-conscious in public but so many times we have seen utility trump other problems – for example, it makes sense to be able to walk along the street in an unfamiliar city without constantly having to keep glancing at the maps app on a phone.
People happily wear oversized glossy headphones because they want to hear the music and believe them to be fashionable. Others wear Bluetooth earpieces in the car. Some once openly flaunted cellular phones with batteries the size of a brick in images that seem ridiculous to us now. It does not seem to be a massive step to make and the $1,500 target price tag could, as with so many other electronic devices, come down quickly.
Projects such as Glass, however, mask the fact that augmented reality is already here and that's it not just about giving everyone a personal head-up display.
Ambarish Mitra, CEO and co-founder of Blippar points out augmented reality has been on our TV screens for some time: "When the Gary Linekers of the world point and draw around players on a football field."
Most museum visitors will have encountered a primitive form of augmented reality in the form of the audio guide. Traditional guides are only barely automated but newer devices are being designed that activate the audio recording for an exhibit as you approach its RFID tag. More sophisticated sensors track your progress.
Cambridge-based Aurasma aims to go several steps further and demonstrated one application for interactive exhibits at a recent TED Talk event. When the screen of a smartphone was pointed at a painting of poet Rabbie Burns it showed the image begin to move and then, to a background of bagpipes, start to recite one of his works. As the phone moved, the image of the moving painting moved with it.
Head of partnerships Matt Mills says the software in the phone tracked the image in the screen and replaced it with the video, rotating and tilting in the same way the painting would if the Aurasma software were not intercepting the image.
"There's no trickery here. There's nothing done to the image. What's great about this is the technology is allowing the phone to start to see and understand in much the same way as the human brain," claims Mills.
Mitra says: "Augmented reality as a technology is very processor intensive, very camera-resolution intensive, it's very memory intensive. But several macro factors are moving in favour of augmented reality. We have better, smarter mobile phones, better cameras and better connectivity."
Aurasma's and Blippar's software runs on the processors inside one of today's phones. "The thing that's incredible about this is how advanced these devices have become," says Mills.
Beyond marketing
Laurent Julliard, director of solutions and software services at French startup Kalray, believes processing has to go further to make augmented reality better than the real thing. The company aims to put its MPPA multiprocessor device into computers that can be taken out by maintenance engineers to help them fix systems in situ.
At Embedded World 2013 in Nürnberg, Kalray demonstrated a system running on a board carrying its processor array inside a tower PC that detected objects picked up by a camera and showed how to take them apart.
"As you move the camera it tracks it. You can't do this on a processor today below 130W of power. We do it with 12W, so it could be built into a mobile system," Julliard claims. "We are working on a project where we have a tablet with the MPPA in it. We have a number of customers, particularly in Japan, who are interested in it."
Aurasma and Blippar are promoting the idea of augmented reality primarily as a device for brand extension. At the Turing Festival Conference last year, Mitra gave a largely unenthusiastic audience the opportunity to be 'photographed' with Justin Bieber – the company took part in a promotion run by the singer's record label.
Last Christmas, Robbie Williams ran a competition for fans to get their album cover idea onto the sleeve of his latest release 'Take The Crown' through augmented reality. During advent, the software would play a different video every day if the phone's camera caught sight of the sleeve. Lorraine Sands' winning entry now appears as one of the images when the app is used.
Look further than the mobile phone and it is apparent that augmented reality is sneaking into our lives by other means. Similar concepts are being developed for car drivers, says David Price, CTO at automotive engineering specialist Pi Innovo: "There is useful information that can be used in collision avoidance and within other systems. Head-up displays could enhance what the driver is aware of in the outside environment."
Assisted driving
At the Embedded World Show in February, a number of companies ran demonstrations of advanced driver assistance systems that detect potentially hazardous situations, such as objects or people in the path of the car. One system built by DDC using an Altera-made programmable-logic device uses a combination of radar and cameras to flag nearby objects that are getting closer in red while those moving away are left alone or, for the purposes of demonstration, coloured in green.
"Road-sign recognition is coming in that will provide speed warnings to the driver. The radar, optical and infrared systems will provide vision enhancements and information for head-up displays, all providing the driver with more information on what's out there," says Price.
Distraction is a concern for the vehicle designers as there is the possibility of overwhelming the driver with visual information. Worse, if the system attempts to class everything in the field of view as a hazard, drivers will tend to ignore much of what they are being shown.
In the early trials for the UK government-funded Foot-Lite system, which was designed to gauge the effectiveness of real-time information for improving both fuel economy and safety, drivers reported increased stress from the quantity of pop-up messages that the display produced.
"You are into the similar problems as with people using mobile phones: their capacity to process information is being used elsewhere," says Price.
Audio and touch provide other senses that can be exploited by these augmented-reality systems. Aircraft designers exploit the latter already, making controls resist certain movements or vibrate strongly as the risk of stalling the plane increases. Drivers are already familiar with the rumble strips used along the side of some motorways to alert them to the fact they have drifted off the main carriageway. Similarly, some lane-departure warning systems vibrate the seat to simulate the effect of the rumble strip.
There are concerns as to whether augmented reality in what can be life-threatening situations will help or hinder. What happens when assistance features fail silently?
"With a blind-spot warning, if an icon isn't flashing, it might mean nothing is there. It might mean that the system has failed. It says in the handbook you are responsible under any conditions. But will people know this?" Rivett asked at the Safety Critical Systems conference last year. "We are thinking through many of these scenarios. It is not clear what the answers are."
Price says: "These systems are trying to reduce blind spots as much as they can. But nothing is ever 100 per cent foolproof."
These systems are gradually moving into more and more production vehicles with the result that augmented reality has become a large and increasingly important market without people being aware of it happening. It will take sometime for wearable goggles to catch up.
Sign up to the E&T News e-mail to get great stories like this delivered to your inbox every day.
