Head-up display uses lidar to alert drivers of upcoming hazards

The technology, which has been developed by a team at the University of Cambridge, the University of Oxford, and University College London (UCL) allows drivers to “see through” objects, alerting them to potential hazards.

Lidar, a technology employed in autonomous vehicles to build models of their surroundings, uses the time taken for pulses of laser light to be reflected from surfaces and returned to the sensor in order to measure distances. In this case, the data are used to create ultra-HD holographic representations of objects on the road. The information is then beamed directly to the driver’s eyes, instead of 2D windscreen projections used in most head-up displays.

While the technology has not yet been tested in a car, early tests - based on data collected from a busy street in central London - showed the holographic images appearing in the driver’s field of view to match their actual position (augmented reality). This could be particularly useful where objects such as road signs are hidden by large trees or trucks, for example, allowing the driver to “see through” visual obstructions intuitively.

“Head-up displays are being incorporated into connected vehicles, and usually project information such as speed or fuel levels directly onto the windscreen in front of the driver, who must keep their eyes on the road,” said lead author of the Optics Express paper Jana Skirnewskaja, a PhD candidate at Cambridge’s Department of Engineering. “However, we wanted to go a step further by representing real objects in as panoramic 3D projections.”

The researchers scanned a busy street on the UCL campus in central London with a lidar array. Millions of pulses were sent out from different positions along the street and the lidar data was then combined with point cloud data, building up a 3D model.

“This way, we can stitch the scans together, building a whole scene, which doesn’t only capture trees, but cars, trucks, people, signs, and everything else you would see on a typical city street,” said co-author Dr Phil Wilkes. “Although the data we captured was from a stationary platform, it’s similar to the sensors that will be in the next generation of autonomous or semi-autonomous vehicles.”

When the 3D model of the street was completed, the researchers then transformed objects on the street into holographic projections. The lidar data, in the form of point clouds, were processed by separation algorithms to identify and extract target objects.

Another algorithm was used to convert the target objects into computer-generated diffraction patterns, which were used to project 3D holographic objects into the driver’s field of view. The optical setup is capable of projecting multiple layers of holograms, using advanced algorithms. The holographic projection can appear at different sizes and is aligned with the position of the represented real object on the street. For example, a hidden street sign would appear as a holographic projection relative to its actual position behind the obstruction, acting as an alert mechanism.

In future, the researchers hope to refine their system by personalising the layout of the head-up displays and have created an algorithm capable of projecting several layers of different objects. These layered holograms can be freely arranged in the driver’s vision space. For example, in the first layer, a traffic sign at a further distance can be projected at a smaller size. In the second layer, a warning sign at a closer distance can be displayed at a larger size.

“This layering technique provides an augmented reality experience and alerts the driver in a natural way,” said Skirnewskaja. “Every individual may have different preferences for their display options. For instance, the driver’s vital health signs could be projected in a desired location of the head-up display.”

“Panoramic holographic projections could be a valuable addition to existing safety measures by showing road objects in real time. Holograms act to alert the driver but are not a distraction.”

The researchers are now working to miniaturise the optical components used in their holographic setup so they can fit into a car. Once the setup is complete, vehicle tests on public roads in Cambridge will be carried out.