Your message is highly valuable for us. One of our experts will follow up with you within 1-2 business days to discuss your request or to inquire for additional information if needed.
Reading time: 12 minutes
Years ago, Morton Heilig, a cinematographer and father of VR, might not have suspected that people with small handheld devices would rush the streets in a search of their virtual friend. Сraze for Pokémon Go, beards and rabbit’s ears in MSQRD app and Snapchat lens – augmented reality has been a toy in our hands for many years. The affinity with VR largely exploited in gaming industry has slightly cast a shadow over augmented reality attributed mainly to an entertainment sector.
Augmented Reality Importance for Automotive Industry
However, AR goes beyond gaming and AR masks creation. Besides already existing educational apps (e.g., BBC’s  ) or instructional augmented reality apps (e.g., Street View mode in Google Maps   atop the real world;   for easier navigation in the airport terminal buildings), AR finds a variety of ways to be innovatively applied in the automotive industry:
- Inside a vehicle (e.g.,   built into a car – read more in the Use Cases section below);
- Outside a vehicle as a part of its maintenance (e.g.,   for service technicians);
- And for the vehicle creation. Augmented reality in manufacturing becomes a far-reaching integration that automakers aspire implementing on their factories. For example,   to automate their models setup, configuration, calibration, and quality checkup.
Generally, today all industries see larger potential for adopting augmented reality. For example, virtual reality and augmented reality market is foreshadowed to hit   whilst   reckon on livening up their augmented reality strategies within 12 months.
Going by the number of investments in AR, automotive industry will apparently leapfrog other industries by 2020. See the table below:
So, why do automakers and automotive software/hardware companies anticipate implementing augmented reality as part of their business strategies?
Augmented reality accompanied with computer vision and IoT sustains the core of advanced navigation solutions and  . The latter results in enhanced safety for a driver and contributes to the development of an  . Besides, AR will be soon indispensable from autonomous augmented driving features offering a new level of interaction.
Recently at CES 2019, Nissan has presented its groundbreaking   that creates “a 360-degree virtual space around the car” to provide real-time road data and make manual navigation assistance smarter and safer. Actually, this is a new-age AR HUD that extends beyond the limits of a windshield glass. But what about distractions? A half-autonomous driver support system is expected to assist driver in becoming more focused together with Omni-sensing technology, onboard AI system that can also track whether a driver is focused on the road. Moreover, merging data from sensors inside and outside the vehicle and cloud, the AI-enabled system is capable of monitoring the vehicle’s nearby objects and everything that lays far ahead. In the future of fully autonomous vehicles, the I2V technology relying on   (a virtual environment used by avatars of real people for communication and their personal representation) will be used for the social interaction with virtual avatars of friends or family members appearing inside the vehicle. Watch this video for more details:
Another reason why AR should be considered by businesses is that it will continue stirring up drivers’ interest who are eager to try a new toy in their hands. Most likely, these are the drivers who have unintentionally created the virus effect around augmented reality mobile applications. So, imagine how many potential buyers automakers can receive from the list of mobile app users.
Finally, implementation of AR systems for vehicles has become easier these days. Relative affordability of hardware and development frameworks creates broader opportunities for augmented reality adoption. Besides, technological advancements let augmented reality applications move from labs to the world of commerce bringing it closer to people. Among them are:
- Appearance of 3G, 4G, and 5G networks enabled higher speed data transfer and lower latency;
- Smart cameras and faster speed of rendering. Here’s a good example of a powerful smart camera used in modern vehicles and driven by Computer Vision and Data Fusion.
- Appearance of more powerful processors, sensors and IoT devices;
- Higher computation power;
-  , tools, frameworks at AR software developers’ disposal.
What is Augmented Reality Compared to Virtual and Mixed Reality?
Basically, there are three types of extended reality (XR) depending on the degree of the user’s immersion. They can be used interchangeably or concurrently to cover the whole range of business purposes.
Augmented reality is the process of overlaying digital objects on an existing natural environment to provide the partial immersion into the virtual environment.
Unlike AR, virtual reality substitutes the real world with an artificial surroundings creating total visionary and sensory immersion.
Mixed reality is about naturally adding the virtual objects (holograms) to our physical environment in such a way that you can interact with them by using holographic devices.
So, how does augmented reality work? To understand this, have a look at this visually rich comparison of all XR types:
Augmented Reality Hardware and its Use Cases in Automotive
When you interact with an augmented reality app, your real world and experience as well can be enhanced with the added computer-generated text, graphics, sounds, and even haptic feedback. For many users, AR is about headsets, smartphones, and applications. But do you know what actually stands behind AR technology? Here are some key components it reposes on:
- Sensors and cameras that allow collecting data by scanning our physical space and objects. At this phase, depth tracking and simultaneous localization and mapping (SLAM) technologies are utilized to determine distance to the objects.
- Augmented reality devices require enough processing power to function as mini computers to accurately process gathered data from sensors and cameras. Usually, standard AR devices include such elements as CPU, GPU, flash memory, RAM, Bluetooth/WiFi microchip, and GPS microchip. However, the advanced AR devices like Microsoft HoloLens rely on their custom   and inertial measurement unit (IMU) consisting of accelerometer, magnetometer, and gyroscope.
- Mounted projector, which sends the collected data onto the surface of AR headsets, smart glasses or lenses.
- Reflection mirrors that by reflecting light to a camera and the human’s eye properly align an image.
Depending on the hardware used by automakers, there’s a variety of possible use cases in automotive:
- Augmented Reality HUD (head-up display) allows to project the augmented navigation hints atop the car’s windshield displaying road signs, warnings and real-time route navigation to help a driver stay more focused on the road. To understand how to implement AR into an embedded vehicle navigation system together with computer vision and sensor fusion technologies, check this case study.
Today, AR HUDs are already available in a range of car models (e.g., Chevrolet Camaro, Toyota Camry, Lexus LS 500, Mazda6, etc.) and present speedometer, tachometer, and navigation system’s information just at the line of the human’s sight. And as the automakers are trying to catch up with  , augmented reality HUDs already incorporate advanced driver assistance systems (e.g.,  ) that are expected to advance to  .
Will a HUD still be a trend in 2019? Certainly, it will! One study shows that HUD’s adoption by different brands is only  . The spread of connected and semi-autonomous electric cars will definitely encourage automakers to equip their vehicles with an advanced HUD. The latter is only at the stage of development, however, already whets end users’ appetite with its pioneering technology at such international tech trade shows as   (January 8-11). At the event, Hyundai in cooperation with WayRay have announced the development of one-of-a-kind  , which exceeds   in a few points:
- It projects stereoscopic images such as guidance hints, navigational information, destination points, and alerts directly through the windshield glass onto the road compared to the indirectly displayed reflected images through the LCD screen of existing HUDs.
- Holographic images are accurately adjusted to the viewing angle of a driver in such a way that they are perceived as a part of the road.
- The displayed information already includes such ADAS features as forward collision warning, lane departure warning in the form of written recommendations (e.g., “Slow down”) and in future – rear side warning and highway driving assist.
An interesting fact is that mobile AR is   and in view of its affordability as against AR/VR headsets, apparently it will have the highest revenue ($B).
Unfortunately, the project was soon closed which again proves that AR glass hardware is still not ready for the mass adoption.
Virtual Reality and its Use Cases in Automotive
Although today’s VR hardware surpasses first VR headset appliances fixed to the ceiling not to crash a man’s head, it is still too bulky and expensive for the mass use. Moreover, as VR has a long history of its use for gaming and entertainment needs (e.g., watching 3D movies, 3D virtual tours etc.), businesses see less opportunities for investing in VR – $90 billion of investments in AR compared to $30 billion in VR by 2020  . Moreover, the choice for using VR in automotive is restricted to a few use cases, apart from in-car entertainment (e.g., at CES 2019, Audi demonstrated a demo of its   for the rear passengers). The most popular today is virtual reality apps for   that actually transform a traditional approach to the first ride. At the pre-sale stage, VR can be a sort of a marketing bait to entice potential buyers to try out a new vehicle model or features in the dealer’s centers. This way VR test drives allow generating more audience (e.g., 1,500 people tried Ford’s VR test drives during one auto show), cut costs on vehicle transportation and collect firsthand feedback.
Mixed Reality and its Use Cases in Automotive
This term says for itself combining the best parts of virtual and augmented reality (e.g.,  ,  ,  ). Some sources also call it   when the connection of a user with the real world isn’t blurred but enhanced with virtual objects realistically filling the space around us. Imagine one of the scenes from the famous Jumanji movie, like rhinos naturally crashing through the living room or monkeys riding the police car. By using mixed reality technology, ginning up similar game settings becomes quite attainable today.
For the vehicle design and engineering teams, the potential of AR and MR allows achieving the desired accuracy of the car model virtual representation. Thanks to it,   can be replaced with 3D visualization which designers can interact with in real-time. Most importantly, it vanishes the need to create a physical prototype allowing to create dozens of iterations of a vehicle model or its separate elements, their shape, position, etc. This way AR helps speeding up the development process and delivering the vehicle prototype faster.
A good example is Ford’s new-age design studio that puts into practice Microsoft’s HoloLens Mixed Reality to reshape own approach to the layout process of a new car model:
Mixed Reality and Haptic Feedback
What if you could touch holograms and sense what they are, feel their texture and motion? Haptic feedback is a next level of mixed reality which still needs to be developed, tested and refined until it becomes ready for mass distribution and use. The only form of haptic feedback we can easily access today is our smartphones’ ringtone vibration. Others require special haptic glove which is still in its development stage.
There are two essential kinds of haptic feedback that can help us sense the virtual world:
- Kinesthetic haptics or force feedback is about influencing the human’s body with force, vibration, or motion. Gamers are well accustomed to force feedback which is already available in their gaming wheel, seat, mouse and other parts of a playset. But even with all its advantages, a study of a human-robot interaction reports that force-based haptics is less effective in recreation of realistic VR experience.
- On contrary, cutaneous haptics that influence our skin with a complex of slight vibrations and touches is able to fool the human mind better making it believe that it actually touches something real.
Let’s look at the way such types of feedback are realized in a haptic glove relying on 120 sensors to make a person feel the legs of spider or rain:
Although the bulky glove with wires can hardly fit your home settings, we are still to see great advancements in this area. Besides, the ability of haptic feedback to result in genuine senses may ramp up impact of VR on larger audiences and justify high costs of devices. For automotive industry, haptic feedback may completely change an approach to sales process, replacing a standard dealership center with a comprehensive VR showroom for making a test drive of any vehicle at any time.
The bottom line is that AR, VR and MR have a full potential to amplify a number of other use cases and integrate into the ecosystem of a vehicle, its production, sales process and generally, the entire automotive value chain. The more hardware improves, the deeper AR will root into all aspects of automotive industry. Find out in our second article “Augmented Reality Benefits for the Entire Automotive Value Chain” what opportunities AR brings to OEMs and Tier-1 suppliers and what challenges should be tackled to start large-scale AR adoption.