Autonomous cars: will we be driven to distraction?

Personal transport has reached a pivotal point in its evolution as self-driving cars shift drivers’ focus away from the road and towards the interior cabin, where an array of technological distractions are being developed to help them work, play, shop or socialise.

Experts predict autonomous cars will become mobile entertainment/infotainment ‘pods’ where consumers watch immersive films and TV, game in virtual reality (VR), or use windscreens as a lens to the outside world using augmented reality (AR) to overlay content and advertisements.

Concepts for self-driving cars feature interiors that more closely resemble luxury private jets than vehicles from the past, with flexible interior layouts, swivel seats and big screens.

As car makers, tech giants and electronics firms pump millions into R&D, policy makers are changing regulation to pave the way for a future where immersive in-car entertainment is the norm.

In April, the UK government revealed proposed changes to the Highway Code that would allow drivers to watch TV at speeds of up to 37mph when fully autonomous cars are rolled out.

The rules would mean drivers are not responsible for crashes in autonomous cars, with the burden instead falling on insurance companies, and come ahead of a full regulatory framework due to be implemented by 2025.

Across the pond in the US, federal vehicle safety regulators cleared the way in March for the production and deployment of driverless vehicles that do not have traditional manual controls, including steering wheels or pedals. This will free up valuable real-estate on the dashboard for technology like big in-car displays, interactive maps and more. It immediately prompted Tesla CEO Elon Musk to promise a “dedicated robotaxi”, with no steering wheel or pedals, ready for volume production as soon as 2024.

The prospect of a driverless world is enticing, but the sector must negotiate several bumps in the road before it becomes a reality. Trial cabins with new seat layouts and displays have raised safety concerns and caused motion sickness. High-speed wireless connectivity needed for streaming and graphically intensive applications remains patchy, and it’s not yet clear how passengers and car makers want content to be accessed and viewed.

Then there’s the deeper question of when driverless cars will be rolled out at scale. Full autonomy is needed to free up drivers to engage in entertainment, but ensuring cars are safe on the road demands some serious onboard number-crunching not currently available.

“There are many years to go before what, arguably, is the greatest change to how we travel since the invention of the motor car is fully realised,” says David Wong, senior technology and innovation manager at the Society of Motor Manufacturers and Traders (SMMT), trade association for the UK motor industry. “But it’s also true to say the use of these technologies may no longer solely be in the realm of science fiction.”

My earliest memory of ‘in-car entertainment’ was in 1988 when, rather than endure the stress of going sailing with my dad, I stubbornly waited behind in our Volvo estate listening to hip-hop cassettes. Fast-forward to today and, although the underlying technologies I use have changed to Spotify and YouTube, I’m still listening to audio, whether it’s music, news or podcasts.

Audio isn’t going away, but the arrival of driverless cars opens the door to an array of new multimedia experiences, ranging from VR and AR to movies on a widescreen, console gaming or even holographics.

A 2019 report on connected and automated vehicles by the SMMT and Frost & Sullivan suggests that if self-driving vehicles are deployed in substantial numbers this decade, in-car entertainment, plus other digital services and aftersales, could generate some £18bn for the UK economy.

A recent survey of 2,500 motorists worldwide, by management consultants Horváth & Partners and Fraunhofer Institute for Industrial Engineering IAO, found that after sleeping and relaxing, entertainment was the activity most motorists would like to indulge in in a driverless car, followed by eating and drinking, working and being productive, then beauty, wellness and fitness.

Increasing adoption of electric vehicles is fuelling the transition to the high-tech in-car multimedia experience. Where – historically – car makers created differentiation between products through external styling, handling and the sound and vibration of the engine, electric motors all sound and feel more or less the same. This shifts focus onto creating a distinct user experience through the human-machine interface, particularly the infotainment system.

Thus, many announcements by car makers at the CES2022 consumer electronics show were infotainment-based, such as Amazon’s plan to integrate its Fire TV streaming service into two Ford models. BMW used the occasion to reveal its new Theatre Screen, a massive 31-inch, 8K display designed to enable backseat passengers to stream their favourite music and movies on the go.

Yet it is visionary designs for concept cars that reveal the inspirational possibilities of life inside future autonomous vehicles.

Cadillac’s Inner Space concept for a curvy self-driving coupe resembles a luxury cabin for two. Rather than separate driver and passenger seats, there’s a plush ‘loveseat’ that appears to be constantly in recline mode, a huge display occupies most of the windscreen, and there is no steering wheel or pedals.

Luxury is also at the heart of Audi’s Grandsphere concept for a Level 4 autonomous car (the car is in control almost all of the time without any human input) with its spacious lounge-style environment, including “individually staged displays and ambient light” and seats and climate control that automatically adjust to passenger preferences as soon as they sit down.

A spokesperson for Audi says: “Digitalisation and, above all, automated driving are undoubtedly making the interior the centre of attention. The interior of the future will be bigger and much more homely than today. The car is supposed to become the new living space – next to the office and your home.”

Car makers are already implementing immersive technologies like VR, AR and video gaming into production-line vehicles, and the more widespread availability of high-speed 5G could open the door for vehicle-to-vehicle games.

Audi’s Holoride system adapts VR content, viewed through a headset like Meta’s Oculus, to the car’s movements in real time. For example, turning the car to the right might cause a spaceship in the virtual world to turn right; hitting the accelerator could make a vehicle in a game speed up.

Pitched as the first car designed for the metaverse, the WayRay Holograktor concept car processes data from onboard sensors, cameras and map info to precisely overlay holographic imagery onto real-life scenery visible through the windows. The ride-hailing vehicle would be subsidised by holographic advertising pitched at passengers.

AR head-up displays that project vital information on the driver’s field of vision are already available in some car models, but driverless cars could take advantage of entire windscreens to highlight anything from points of interest, to travel information, to options for making a pit stop for shopping. This dynamic, location-based content could provide a new platform for advertisers and local businesses to offer deals and tap into passing trade.

If autonomous infotainment pods are the future, car makers and their technology partners are still scratching heads over how to lay out cabin interiors and install multimedia tech.

One area of uncertainty relates to road safety and occupant wellbeing, explains Tobias Schneiderbauer, who leads on connectivity, future interior and data modernisation within the McKinsey Centre for Future Mobility: “The ability to relax more within the vehicle could potentially mean turning the front seats so that four people can sit together and watch a movie. But that raises operational challenges, in terms of how you turn seats and make them safety compliant.” Driverless concept cars include more screens in the vehicle, but their positioning can have a major impact on motion sickness. “I just came off a conference call with a couple of OEMs where tests revealed high levels of motion sickness,” adds Schneiderbauer.

Wireless connectivity is a major challenge for in-car entertainment: 4G coverage is still not guaranteed on every mobile network across UK roads, and 5G needed for certain entertainment/infotainment applications such as gaming and VR is currently only prioritised in urban and commercial areas.

“When there is limited or no connectivity, the system still needs to operate continuously and seamlessly,” says Tomasz Dzikowski, automotive product manager at video user experience specialists 3SS. However, this problem has a solution. He adds: “Entertainment while flying is a kind of autonomous transportation and the same functionality we’re used to on commercial aircraft needs to be brought into cars. It’s just a matter of time.”

Passenger preferences for how content should be accessed and viewed are another area of contention. Should a car feature built-in displays with dedicated software for streaming films and gaming, for instance, or should it be treated like a ‘room’ into which passengers bring their own laptop or tablet with integrated apps to view content?

People spend more time using smartphones and tablets than driving cars, and automotive versions of the world’s most popular mobile operating systems, Apple CarPlay and Android Automotive OS, are already available in most new vehicles sold in Europe and North America.

Marius Mailat, digital services managing director and CTO at autonomous vehicle consultancy and software developer P3, says: “Automakers are on the path of a huge digital revolution ... each is making the pivotal decision on their role in this new entertainment-rich driving environment. Some want complete control and ownership over the in-car experience, others are more interested in providing an enabling platform. They view the path towards next-level in-car entertainment as an open ecosystem, and inviting third parties in as providers/suppliers.”

P3 recently partnered with 3SS to co-develop an in-car infotainment system built on Android Automotive OS.

Manufacturers that do not wish to cede ground to a third-party are more aggressively recruiting software developers and user-interface designers to realise their own visions for multimedia cabins. Some significant computing horsepower is required to run next-generation infotainment systems, as well as advanced driver-assistance systems, so the need to form direct relationships with chipset makers is an advancing area of cooperation.

Nvidia’s Drive Thor centralised computer for autonomous vehicles, launched in September, can process up to 2,000 teraflops and handle functions including automated and assisted driving, parking, and in-vehicle infotainment. Qualcomm has licensed its cloud-enabled platform Snapdragon Digital Chassis to various car brands to power their future in-car communication and infotainment systems.

It’s inspiring to imagine a future where robotic vehicles keep us relaxed and entertained, but this remains a shifting and uncertain landscape where new ideas and partnerships continue to evolve. “It is fair to anticipate an in-vehicle entertainment/infotainment ecosystem involving car manufacturers, content providers and platforms with new extended business models. However, any predictions at this stage of how they might look are at best speculative,” says SMMT’s Wong.

At a more fundamental level, the central premise of autonomous cars – that computers and AI will dramatically reduce accidents caused by human error – also remains elusive.

Making cars that drive more safely than people is difficult because self-driving software systems, though highly sophisticated, lack humans’ ability to rapidly assess and anticipate risk, especially when encountering unexpected incidents or ‘edge cases’.

Martin Kellner, associate partner at management consultancy McKinsey & Company and member of the McKinsey Center for Future Mobility, says: “The big challenge is to cover all possible edge cases. There are so many things humans can directly interpret – imagine a ball rolling onto the street or a child running between cars – but in the case of an autonomous car, you need to collect all these edge cases and train the algorithms to handle them.” Layer in all the possible variants for each edge case and it could add up to millions of outputs.

Companies like General Motors, BMW, Google and Tesla are investing to solve the puzzle, but rollout schedules for fully autonomous cars have continued to slip.

GM originally planned to add the Cruise Origin to its commercial ride-sharing fleet in 2019, but production is now slated for spring 2023. Musk promised to deliver a million Tesla robotaxis “next year for sure” in 2019, but in an interview on YouTube this summer, he admitted developing self-driving cars was “harder than I originally thought, by far”.

Until car makers deliver on the potential of this controversial but potentially life-​changing technology, we’ll all need to keep our eyes on the road and our hands on the wheel.