Blind man driving RoMeLa car guided by non-visual stimuli

Non-visual feedback and the blind driver

The innovative use of non-visual feedback techniques is enabling blind people to drive a car, and could have important lessons for other technologies that must interface with humans, reports E&T.

In the summer of 2009 several young blind people in the United States of America enjoyed an experience previously impossible for them they drove a motor vehicle, retaining full control over speed and manoeuvring.

The innovative technologies that made the venture possible included a laser scanner on the front of the vehicle to record details of the surrounding environment, on-board computing power to convert the scanned data into a digital 3D 'map', the integration of the steering wheel with a set of headphones to enable the driver to respond immediately to audible prompts and an 'active' seatbelt that communicated information about speed by means of a vibratory signalling system.

The really important factor was the outcome: a sense of exhilaration and empowerment that the drivers got from the experience something that was evident to all those who were present. As Dr Dennis Hong says: 'It was all over their faces.'

Dr Hong, though, was more than just a passive spectator. He is head of the Robotics and Mechanisms Laboratory (RoMeLa) at Virginia Tech in Blacksburg, Virginia, in the US. The vehicle was developed by a team of a dozen students working under his guidance within RoMeLa in a project entitled the Virginia Tech Blind Driver Challenge (BDC).

Blind driver challenge

According to Dr Hong a number of factors came together to make the project possible. One of them was a base of relevant knowledge gained in previous work to develop autonomous vehicle technology for the US Defense Advanced Research Projects Agency (DARPA), a division of the US Department of Defense.

But a more immediate spur was a specific Blind Driver Challenge to develop technologies to enable blind people to drive issued in the middle of the last decade by the Jernigan Institute, the research and education arm of the US National Federation of the Blind (NFB), in Baltimore, Maryland. The task, as Dr Hong admits, was a tough one so much so that when the RoMeLa team began work on the project in the early part of 2008 it became the first and only representative of the US higher education sector bold enough to do so.

Nevertheless, he says, he had no problem finding enough candidates to form the project team. 'It really caught their imagination,' he explains, adding that the students were all about to embark on their fourth and final year of study as engineering undergraduates at Virginia Tech.

The first step in the whole process, though, was not about technology development, but familiarisation with the day-to-day realities of life for blind people. Accordingly both Dr Hong and the students spent a good deal of time visiting schools for the blind for one-to-one conversations and even formal 'brainstorming' sessions. Quite simply, says Dr Hong: 'If you are going to develop technologies for blind people, you have to work with blind people.'

Not the least of the insights that came out of the consultation exercise was confirmation of just how determined the blind students were to achieve active control of the technologies. Dr Hong says that he and his team quickly realised that, while the previous DARPA work might provide some technical input to the new venture, its basic premise that of replacing driver control with automated capabilities was of no interest to their blind clients. 'They did not want that,' he states bluntly. 'They wanted to drive the car.'

Consequently, he says, the team had to address three distinct areas in which the real world whether the physical surroundings or the driver had to interact with vehicle systems. These were:

  • sensing - gathering information that might normally be collected visually;
  • planning - enabling decision-making by the driver;
  • acting - communicating driver instructions to the vehicle.

Equally important was the fact that all these operations had to be integrated in a way that allowed for real-time decision-making and control by the driver. Autonomy, not automation, was the watchword.

As such the heart of the whole system was a Hokuyo laser range finder on the front of the vehicle linked to a National Instruments CompactRIO programmable controller, which acted as the central processor. The drivers received information from the vehicle's control system through their senses of hearing and touch.

Manoeuvring, for instance, was negotiated by a feedback system involving both the steering wheel and a pair of headphones worn by the driver. The headphones communicated instructions to the driver to turn the wheel right or left by a specified number of audible 'clicks', which the driver then heard through the headphones as they turned the wheel. By counting the clicks they turned the wheel the correct degree.

Meanwhile starting and stopping and the intermediate acceleration and deceleration were carried out according to patterns of vibration generated by electronically controlled actuators fastened to the seatbelt. It sounds complicated but Dr Hong says that, given their aptitude for dealing with non-visual stimuli, it did not prove too difficult for blind drivers to learn how to use the system. In contrast, he adds, his students found the procedure much more tricky when they acted as test-drivers. 'The blind drivers were much better than us,' he notes wryly. Actual physical design of the various hardware elements, by the way, was carried out using the SolidWorks 3D CAD system.

But is this more than a collection of clever technical tricks? Could it have fundamental long-term significance?

Jernigan institute

Someone whose answer to those questions is emphatically positive is Mark Riccobono, executive director of the Jernigan Institute. He explains that when the institute was inaugurated in 2004 it aimed not just to help develop technologies that could support blind people in their day-to-day lives, but also to change the perception of blind people merely as passive users of technologies developed on their behalf by others.

The institute pursues the latter objective by two means. The first is to involve blind people actively in its work both by consulting and employing them: Riccobono himself is blind. The second is to counter assumptions elsewhere in society about the limited capabilities of blind people. The Blind Driver Challenge is aimed squarely at those assumptions.

'The idea that blind people could drive a car is out of synch with a lot of people's perceptions,' Riccobono observes, though he adds that the stated objective of allowing blind people to drive is a serious one in its own right.

Nevertheless, Riccobono is under no illusions about the scale and difficulty of the task. Indeed, he compares it to the US Space programme of the 1960s to land a man on the Moon, not just for its technical complexity but also for its wider implications in terms of both attitudinal changes and possible spin-off technologies.

But the difficulties are not deterring either Virginia Tech or the NFB, and plans are being made for an ambitious extension of the whole project. The aim, says Riccobono, is to fit out a normal on-road vehicle with technologies that will enable it be driven over much more complex test-tracks than that initial flat car park. Moreover there is a tight schedule of just 18 months or so, since the intention is to set up a series of such tracks at strategic points on the 900 miles between Baltimore and Orlando, Florida, where the US National Convention of the Blind will take place in the summer of 2011.

Details are still to be finalised, though Dr Hong says that one idea already being explored at Virginia Tech is that of feeding information to the driver via a varying pattern of compressed air against the fingertips through a series of pinholes in the steering wheel a sort of 'dynamic braille'. Such a system, he adds, would also obviate the audible instructions used last year, which the drivers perceived as an interfering 'backseat driver'. The notion is one that particularly enthuses Mark Riccobono, who points out that it could, for instance, lead to a relatively inexpensive way of allowing blind people to interact with and control a desktop computer.

Whatever the specific technologies, the underlying aims will remain those of empowering blind people and ensuring that the world at large recognises their potential for independent action. As Mark Riccobono puts it: 'Blind people are not broken people to be mended.'

Recent articles

Info Message

Our sites use cookies to support some functionality, and to collect anonymous user data.

Learn more about IET cookies and how to control them