Automotive manufacturers are quickly taking over the annual CES in Las Vegas, but what does this mean to the future of residential AV? Terry Martin investigates.

The annual Consumer Electronics Show (CES) in Las Vegas is fast becoming a must-attend event for motor vehicle manufacturers and automotive suppliers who are using the world’s biggest technology showcase to present their latest and greatest – not only cars, but in-car tech systems and, let’s face it, some pretty wild ideas.

This not only demonstrates how technology is rapidly changing the experience in the car, but highlights the increasingly strong connections between home, office and personal transport, as well as applications and concepts that might start from the vehicle before quickly expanding into other areas.

The 2019 CES is a case in point, from VR and AI breakthroughs to new noise-cancelling tech that dispenses with headgear and applies to an entire cabin.

So let’s take a deeper dive into some of the highlights.

Audi VR platform

As we’ve witnessed a stunning rise in quality home cinematic systems, the auto world has quickly turned its attention to transforming – and in Audi’s case, “redefining” – in-car entertainment.

The days of seatback screens are well and truly over before they really even took off – among mainstream brands, at least – with Audi taking to CES to show how virtual reality (VR) glasses can be used as part of a fully immersive experience, using technology that enables the virtual content to respond to vehicle movements in real time.

Partnering with Disney, Audi demonstrated the system using a game, Marvel’s Avengers: Rocket’s Rescue Run, which gives back seat passengers a unique VR experience in that every movement of the car is seamlessly reflected in the experience. So if the vehicle takes a sharp right-hand turn, the spaceship in the experience does too, curving around an oncoming hazard. When the car accelerates quickly, the ship in the experience does the same.

That sounds to us like a sure bet for travel sickness, but Audi insists that the synchronisation between the visual experience and the user’s actual perception means the chances of throwing up are “significantly reduced”.

While the technology has debuted on a video game, and will easily translate to movies, educational formats and real-life immersive experiences, the developers say there are “almost no limits to what is possible”. This is particularly relevant as intelligent transport systems, such as V2X (or, ‘vehicle to everything’) communications, expand and enable traffic events encountered along the driving route to be integrated into the VR experience.

The aim is to have the system on the market within the next three years using standard VR glasses. To get there so quickly, the German prestige car-maker, through its Audi Electronics Venture subsidiary, has co-founded a start-up company, Holoride, which will look to commercialise the technology via an open platform made available to other car-makers and content developers.

WayRay to go

Head-up displays (HUDs) are becoming increasingly common on today’s new vehicles in Australia, projecting useful information such as vehicle speed and traffic sign information onto a small portion of the windscreen.

They work well in keeping the driver’s eyes on the road, but the technology is really cranking up, as demonstrated Hyundai Motor Group and Swiss deep-tech start-up WayRay which brought to CES what is billed as the world’s first holographic augmented reality (AR) navigation system.

Equipped in a Genesis G80 luxury sedan, this next-generation visual tech takes the in-car experience to the next level. While conventional HUD units project a reflected image indirectly through an LCD screen mounted on the dash, the holographic AR display projects a stereoscopic image through the windscreen, displaying it on the actual road.

There’s no headset or earpiece involved, and the images and information are constantly adjusted in real time according to vehicle speed and the driver’s viewing angle. The virtual image measures 1,310mm high and 3,152mm wide and is projected at a distance of 15m from the driver’s eyes.

What’s more, the level of detail here is quite stunning, with the technology already sophisticated enough to not only show navigational features such as current speed and destination points, but incorporate advanced driver-assist features and alert the driver of oncoming hazards.

Hyundai is among several global investors in WayRay – Porsche is another – and expects annual growth to the tune of 30% in the holographic AR display sector.

The tech will also soon be able to display pedestrians, objects, buses, bike lanes and footpaths, while the major players expect the incorporation of V2X technology and connectivity features will allow data such as traffic signals, surrounding vehicle information and road and weather information to be built into the system, all provided in real time.

Traditional automotive is just the tip of the iceberg as WayRay works on AR concepts across land, air and water transportation, spanning safety, navigation and infotainment streams. It also recently launched a True AR Software Development Kit that allows third-party developers to integrate virtual objects into the real world, creating new applications that run on its holographic AR displays.

Given that glass is such a big component of our smart cars, buildings and cities, the opportunities for this emerging tech look to be huge – and the possibilities endless.

Kia reads the mood

Hyundai’s sister brand Kia also attracted plenty of attention at CES as the Korean car-maker looked ahead to a time – not all that far away, depending on who you believe – when fully autonomous driving becomes the norm and new forms of technology will be applied to enhance the driving experience.

Sorry, that should be ‘human mobility experience’ given ‘driver’ will by then be an archaic term and the cabin will not be encumbered with a steering wheel and other controls. It will be more like the home or office, where technology dominates, and at CES Kia turned up with what it claims is “the automotive industry’s first technology converging human senses-oriented in-cabin environment control and AI-based emotional intelligence”.

Or, to put it more simply, the tech is called READ, short for Real-time Emotion Adaptive Driver.

Developed in collaboration with the Massachusetts Institute of Technology, the READ system is designed to optimise and personalise the cabin space by analysing a driver’s emotional state in real time via AI-based bio-signal recognition tech.

Kia says the technology monitors a driver’s emotional state using sensors to read his or her facial expressions, heart rate and electrodermal activity. It then tailors the cabin environment according to its assessment in an effort to create “a more joyful mobility experience”.

AI deep-learning technology enables the system to establish a baseline in user behaviour, and then identify patterns and trends to customise the cabin accordingly.

Forming part of the READ system is another claimed world first in the form of virtual touch-type gesture control technology. Dubbed V-Touch, this application employs a 3D camera to monitor users’ eyes and fingertips and allows the occupants to control cabin features such as climate, lighting and infotainment via a head-up display and using simple hand gestures, thus eliminating the need for conventional switchgear or even touch screens.

Capping it off, the READ system also includes music-response vibration seats, where occupants can ‘feel’ their favourite songs as well as listen to them. Sensory-based signal processing technology adapts the seat vibrations according to sound frequencies of the music being played.

The vibrating seats also have settings for massage and, should something go wrong in this utopian accident-free autonomous environment, can provide haptic warnings from the advanced driver-assist systems on-board.

Along similar lines, AI company Nuance also used the CES to introduce a new innovation in its ‘Dragon Drive’ intelligent automotive assistant platform using voice, sight, gesture and emotion interaction that “transforms it into a conversational, humanised mobility assistant that will be core to the digital, button-free car of the future”.

An alternative route

Just as Audi has forged close ties with Disney, American tech giant Intel has joined forces with Warner Bros – using a specially modified BMW X5 SUV – to explore the potential of next-generation entertainment when the vehicles are driving by themselves.

Kia’s future self-driving cars might look to read the occupants’ mood, but here Intel tech and Warner Bros blockbusters combine to make the journey one that could potentially be ‘controlled’ by a fictional character and the trip itself set in a fictional place, breaking the boredom of the daily commute or a long-haul drive.

In this case, the virtual ride – complete with giant screen, projectors, sensory and haptic feedback and immersive audio and lights – takes place in Gotham City, moderated by Batman’s trusted butler Alfred, who comes to life, in a sense, by interacting with the occupants, keeping them comfortable and informed of actual events occurring outside in the real world: traffic jams, road closures, route changes, and so on.

But the chaperone/navigator could be anyone or anything, and the environment anywhere, highlighting the prospect of an entertaining but nonetheless safe future driving experience backed by Intel, which via its Mobileye subsidiary is positioning itself as a leading player in autonomous vehicle development with the hi-tech computing power required to make it all work.

It also brings into play the marketing opportunities for big media giants like WB, which have a captive audience in a highly connected car with which to build strong relations, screen movies, run trailers of future films, identify nearby cinemas, help them purchase movie tickets, and so on. And those are just some of the obvious applications we can imagine today.

Intel says the so-called “passenger economy” brought with autonomous cars will free up more than 250 million hours of commuting time per year in the world’s most congested cities, while the future market for new in-vehicle apps and content is an estimated $US200 billion – hence the heavy investments now being made into R&D and exploring how consumers will interact with emerging forms of entertainment within cars “once they are uncoupled from the steering wheel”.

So much more to see

There was so much more on display at CES from the auto giants, including Mercedes showing off the latest application of its still-fresh MBUX (or Mercedes-Benz User Experience) infotainment system – via the new-generation CLA Coupe – which is underpinned by AI and deep learning, including intelligent voice control with natural speech recognition.

Among others, Nissan was there demonstrating ‘Invisible to Visible’ (I2V) technology which merges the real and virtual worlds using sensors inside and outside the vehicle with data from the cloud, enabling the system to anticipate what’s ahead – even behind a building or around a corner.

Some smaller players also won plenty of attention, notably electric vehicle start-up Byton with its M-Byte SUV concept featuring a mammoth 48” ‘Shared Experience Display’ that runs the full length of the dashboard. Not stopping there, M-Byte also includes a 7” tablet floating on the steering wheel pad that stays upright as the tiller is turned, and another 8” touchpad on the centre console for the front passenger.

Byton calls the cabin a ‘mobile digital lounge’ and, while we’re yet to see the production version, the company says M-Byte “represents the transformation of the traditional car into a next-generation smart device”.

As part of this, Byton is one of several car-makers – others being Toyota/Lexus, Audi, BMW and Ford – to join forces with Amazon and its Alexa virtual assistant and is working with the e-commerce juggernaut to develop advanced in-vehicle voice control functionality.

Indeed, Amazon’s work in this area was a major talking point at the show, with its potential to tap into the huge automotive sector – both OEM and aftermarket – and pave the way for new advances in connectivity between car and smart home.

Equally smart in our view was audio giant Bose’s ‘Road Noise Control’ (RNC) system that, for the first time, adapts its intelligent wireless QuietComfort noise-cancelling headphone tech to a vehicle’s cabin – no small feat, given controlling noise in a car is much more difficult than the relatively small area around your ears.

Vast resources are pumped into reducing noise inside a vehicle, generally focusing on sound-deadening material and tyre spec, but here Bose has used a combination of accelerometers, proprietary signal-processing software, microphones and the audio system to electronically control unwanted sound, calculating an “acoustic cancellation signal” and then delivering this through the car’s speakers to reduce the targeted noise.

Harman was also there at CES with parent company Samsung, presenting an updated version of their digital cockpit which is based on Samsung’s Bixby intelligent platform and, in similar fashion to Amazon’s tech, offers high-level connectivity between vehicle and home, such as enabling the owner to remotely check the fuel tank or start up the climate controls before reaching the car.

This is the sort of functionality we are not just getting our head around, but are beginning to expect, which shows just how far the car-makers have come in recent years – and how we’re ready for the ride.