Live from the 
Instead of your next video conference call, imagine having a virtual meeting room where you and your colleagues are all present. Or the next time your child has a class at home, one of their favorite cartoon characters “appears” and they are able to virtually interact and ask question after question. Such compelling experiences used to require expensive hardware and headsets, but now due to advancements in augmented reality, we can create these experiences using technologies we all already have.
Challenging times, like the one we are facing today, are the perfect time for businesses to explore how they are connecting with customers and employees. We can use these new interface technologies to create and maintain personal connections with each other – and overcome the challenges of physical distancing measures. This is part of what we refer to as “augmented collaboration”.
In this article, we explore the significant advances in the quality and availability of natural user interfaces, and examine via key use cases, how businesses can create new and compelling experiences.
The theory: People react to technology as people
In their influential Media Equation theory, the academics Byron Reeves and Clifford Nass, argue that people often respond to and treat technology (and other forms of media) as an actual person. They highlight examples such as when people describe robots in distinctly human terms.
Their research showed that people even provide “polite” answers directly to a computer when the computer asks them about its performance, much in the same way as we would provide a polite answer to a person.
Thus it’s important to understand that when people use any type of media, they are likely to assign it a personality – despite of course knowing that, for example, a computer is just a box made of wires and processors.
What does this theory mean for product design and development? If your product is clear and helpful to its end users via its interface, users will reciprocate and invest in it in the same way that they would invest in their personal relationships.
So to get a customer hooked to its offerings, businesses need to tap into their attentional space, and provide access to information and services seamlessly in the most frictionless manner possible.
The user interface is the link between the individual and the business. Therefore, organizations have focused on making the interface as amiable, natural and “human” as possible.
Technology advancements have meant that user interfaces have gone from simple text-based command-line interfaces to rich graphical user interfaces, from desktop computers to a myriad of form factors. And now we’re seeing a more natural form of communication using voice, gestures and extended reality experiences.
These natural or reality-based interfaces make the experience intuitive and direct so consumers can connect with the product from anywhere and at any time.
Let’s look at three forms of natural user interfaces:
1. Extended reality
Extended reality complements the goal of providing natural experiences by embedding information in the user’s three-dimensional space and immersing him or her in it, making the experience more realistic, engaging and satisfying.
Mixed reality experiences or Extended reality are simply the usage of augmented (AR) and virtual (VR) reality. While VR requires people to use special hardware to immerse themselves in the experience, the entry barrier to creating AR experiences is easier to overcome. Indeed, building augmented experiences for mobile devices has become significantly easier in recent years.
In addition, recent advancements in browser-based technologies have helped create these types of experience. The simplicity and ubiquity of the browser make it an excellent medium to provide immersive experiences. One study reveals that by providing AR experiences on the mobile browser rather than specialized native apps, the addressable market nearly doubles in size.
2. Voiced-based UI (VUI)
Conversational forms of interfaces have gradually evolved from text to using voice to connect with users. Voice-based interaction is effortless and can have a calming, confidence-inducing effect on the user. The rise of voice-based digital assistants like Google Home, Apple’s HomePod, or Amazon Alexa is a testament to their popularity.
3. Gesture-based interactions
Such interactions are no longer science fiction. We have seen avant-garde examples of this over the past few years with technologies such as Xbox Kinect and Google’s Project Soli. These systems are aware of your presence and naturally respond to your bodily gestures. In recent developments, machine learning libraries such as TensorFlow aid in creating gesture-based interactions. They are now ripe for businesses to explore opportunities across web and mobile channels.
A combination of voice, gestures and mixed reality experiences using AI techniques can make the UI smart and provide highly immersive natural experiences. This has the potential to disrupt the very definition of conventional user interfaces.
Use cases of futuristic interactions
There are widespread examples of how organizations are already using natural user interactions to engage with their customers and employees. Let’s look at five possible applications of natural user interfaces, along with some examples based on proof of concepts that we have built.
1. The visual embodiment of digital assistants to create trust and confidence.
Providing a visual “body” to your digital assistant via augmented reality can make the experience of interacting with a digital assistant more compelling and trustworthy. It’s a paradigm shift from talking to a faceless digital assistant to a more realistic character exhibiting social traits through expressions and gestures.
Research has demonstrated that an agent with a visual body using AR and demonstrating natural social behaviors increases the user’s confidence in the service.
Imagine applying this to educational technology (EdTech). A child could interact with his/her favorite cartoon character in AR and ask it endless questions. The congenial personality of the character makes it more fun for the child to keep learning from it.
Businesses can apply this strategy to their virtual customer care executives. A pleasing personality can help customers identify with it and enhance the brand. This technique can be used for various chatbots and virtual assistants.
2. Improved e-commerce experiences.
Online commerce grew rapidly as it gave each one of us the convenience to shop round the clock from the comfort of our homes. However, for retailers, a common problem is dealing with a much higher proportion of returns than their brick-and-mortar counterparts. This is particularly the case for fashion retailers. These technologies can help people shopping for clothing try on the item, and help them make the right choice around the size and the style.
The website can prompt the customer to turn on their camera and help them try it on virtually using augmented reality (privacy issues can be alleviated as the camera feed is processed locally at the client’s end on the browser and no data is transmitted to the server). The future of retail, therefore, is definitely going to be ‘phygital’.
3. Touchless kiosks will solve the problem of touch-based systems.
Kiosks in public places such as shopping malls, museums, movie theaters, and fast-food restaurants provide interactive access and display information via touch. Due to an increased focus on hygiene resulting from the Covid-19 pandemic, many people will be wary of using such touch-based systems going forward. Gestural interfaces can come to the rescue. They provide the obvious benefits of hygiene, performative interaction and gamification.
4. New experiences for the fitness industry.
With most fitness facilities affected by the pandemic, the fitness industry has shifted to providing services online via remote or pre-recorded sessions. There is however significant potential to improve such services.
Imagine the next time you use a fitness app and it has a virtual personal trainer guiding you through the workout. With AI techniques, it is possible for the UI to detect your bodily movements and guide you through the exercise. The interface exhibits all the traits of a personal trainer such as counting repetitions, demonstrating the correct posture and technique, while challenging and encouraging the individual through the exercise procedure. We believe this technology can present major new opportunities for the fitness industry.
5. Virtually teleport your colleagues into your next meeting.
While video calls have made the entire remote communication experience pleasurable, we could significantly enhance the experience by having the person you are interacting with virtually teleported into your natural setting. While systems exist to present holographic experiences for such use cases, they rely on specialized hardware such as Microsoft HoloLens. The need for such specialized hardware limits broader adoption (at least as of today).
However, with AI techniques, we could strip the background of the person we are communicating with and experience a holographic impression of him or her via augmented reality. This takes the remote communication experience to the next level. The simplicity of AR can make this experience available for the masses. The below image demonstrates how we created such a solution at Globant.
Image is from an augmented reality proof of concept developed by Globant
It’s time to revolutionize how we interact remotely
Applying the techniques mentioned in this article, businesses can revolutionize the way they connect with their customers and employees. However, when developing such next-generation interfaces, it will be critical to keep the human element of the end-user in mind. I recommend always keeping the below quote by Jef Raskin in mind as a guiding principle while developing user interfaces:
“An interface is humane if it is responsive to human needs and considerate of human frailties.”
