User interfaces are constantly evolving but what if you could have an interface that is, essentially, invisible? Anna Hayes explores the concept of natural user interfaces.

To understand the concept of a natural user interface (NUI), one must first consider the question of which aspects of
human interaction are truly natural and universal?

Much of the work in this area is focused on voice, spatial recognition and sensors – devices that can be asked to do
something, or already know to do it in your presence.

There are a number of views on what is considered the ‘Holy Grail’ for UI design, some positive and some more conservative.

David Meyer, technical research consultant at CEDIA plays a bit of devil’s advocate on the topic: “I think there’s a bit of hype around NUI which is not warranted at the moment and, just because we can do something does that mean that we should?”

Questions need to be posed, he feels, such as: is it going to be better for an end user? Will it give more versatility, more control than before?

The idea of an NUI has been around for quite a while, with varying degrees of definition.

Josh.ai is one of the more natural voice-controlled home automation systems offering a more conversational interface than a command-driven one.

Josh.ai head of business development Nader Dajani believes that ease-of-use is a major consideration for anyone trying to develop natural interfaces, pointing out that they often see, in the luxury home market, that one person in the family loves technology, hires an integrator to set up a home automation system, and is the only person in the house to use it regularly.

“It’s mainly because all the other household members are either intimidated or don’t want to learn a new technology system. That’s the great thing about voice: everyone knows how to speak. With the right voice control system you can have a setup that everyone will use.”

My mind is drawn to a recent visit home to Ireland and the amusing sight of my six and four-year-old nephews yelling at Alexa to turn on and off the conservatory lights, to the point where she must have been considering joining a union… David’s idea of what is natural is different.

“I think one of the ultimate NUIs is the one that doesn’t seem to be included in NUIs and that’s the good old fashioned touchscreen. It’s not sexy because it’s not new so people aren’t talking about it.”

He refers to a CEDIA Expo keynote where the speaker showed a video of an 18-month old interacting with an iPad, making pages flip and enlarging images. The same toddler was shown trying to interact in the same way with a normal magazine, obviously to no avail.

“I thought that touch is such a natural user interface that even a toddler can use it.”

But the definition of an NUI, he says, is constantly changing and the focus has turned more to UIs that are ‘invisible’, with interaction happening through voice, gesture, eye tracking, etc.

“For anyone who uses voice control they know that there’s a long way to go and it’s not natural.”

I can empathise, having fought with two different options before begrudgingly neutralising my Irish accent for the sake of functionality.

In most cases, voice control is command-based and, even at that, it doesn’t always work.

“There’s a long way to go. I think we’ll get there but the question is: do people want it? Some people actually prefer what I would refer to as a tactile user interface – the light switch. The click is a form of feedback which isn’t always forthcoming from voice control.”

He adds that in the last number of years, there has been mega investment into making voice control good, and there have been some significant advances through the use of artificial intelligence.

In terms of other NUIs, David says that the automotive industry is a step ahead, implementing technologies such as eye-tracking to test fatigue, as well as being a step ahead when it came to lighting control.

“It’s interesting to look at that channel and see what’s happening there.”

For Josh.ai, Nader says that one of their mantras is: ‘no UI is the best UI’ and the home interface should not be limited to an ‘or’ but building towards an ‘and’.

“It will never be voice or remote controls or light switches or sensors. It will be voice and remote controls and light switches and sensors. Everyone interacts with technology in their own way, and so there needs to be multiple options to make the experience universal.”

Nader believes that, in future, a combination of sensors, spatial recognition, voice recognition, occupancy sensors and more will comprise the smart home.

“Ideally, you should be able to walk into a room in your home, and the room knows who you are, where you are, and can make a decision based on those factors as well as other factors, such as time of day, previous commands, etc.”

David agrees, albeit accepting that it is some time away.

“Ultimately that is the end goal, when it becomes truly natural like Ketra lighting which can automatically dim when it gets to a certain light level outside, change colour temperature during the day, and turn on and off lights at night.”

Being able to automate something to the point where you don’t have to think about it anymore is what a truly natural
interface will mean, David says.

For integrators, he says it is important to get to know the technology.

“Install it in your own home. Live with it, let your family live with it, see how people react. Get to know it and, if you don’t already, get lots of feedback from customers.”

He says that there will be tech savvy people who will jump straight onto any new technology but he warns integrators not to sell it on the notion that it will make everyone’s life easier.  This technology is undoubtedly going to improve in time – David says to approach selling and promoting the current offering with caution and common sense.

Nader says that integrators need to be mindful of how they program devices in order to make it easier to use.

“We’ve seen integrators name different rooms as 001, 002, 003 which is not natural for a user to speak. They need names that are human readable and easily remembered and understood.”

Simplicity, he says, will ensure usability and functionality, adding that integrators should also spend some time on a formal training/handoff to the end-user as they may, at first, not know what to say.

In the end, David says, the naturalism does not live in the technology but in the quality of the user’s experience: “To me, NUI is anything which the user can more intuitively learn how to use, it’s not about the technology itself being natural, it’s about the user experience.”