13+ AI Applications & Use Cases of Augmented Reality in 2024

Augmented Reality (AR) is a digital media that allows the user to integrate virtual context into the physical environment in an interactive multidimensional way. AR software derives information about the surrounding environment from cameras and sensors.

Implementing AI enhances the AR experience by allowing deep neural networks to replace traditional computer vision approaches, and add new features such as object detection, text analysis, and scene labeling.

How does AI transform AR?

Historically, AR software used traditional computer vision techniques called Simultaneous Localization and Mapping (SLAM). SLAM algorithms compare visual features between camera frames in order to map and track the environment.

However, modern AR applications rely on deep learning to provide more advanced functionality.

AI Applications in AR

AR developers can leverage AI algorithms to offer AR features like enhanced interaction with the surrounding physical environment. AI technologies such as machine learning and deep learning are well suited to AR environments because:

• There is an opportunity to collect more data for the AI algorithm to train on, because cameras are always on
• The input to the AI algorithm is detailed because AR environments rely on multiple sensors (e.g. the device’s gyroscopes, sensors, accelerometers and GPS). This provides better reliability compared to systems that rely only on a single sensor.

AI applications in AR include:

1. Object labeling

Object labeling utilizes machine learning classification models. When a camera frame is run through the model, it matches the image with a pre-defined label in the user’s classification library, and the label overlays the physical object in the AR environment. For example, Volkswagen Mobile Augmented Reality Technical Assistance (MARTA) labels vehicle parts, and provides information about existing problems and instructions on how to fix them.

2. Object detection and recognition

Object detection and recognition utilize convolutional neural network (CNN) algorithms to estimate the position and extent of objects within a scene. After the object is detected, the AR software can render digital objects to overlay the physical one and mediate the interaction between the two. For example, IKEA place ARKit application scans the surrounding environment, measures vertical and horizontal planes, estimates depth, and then suggests products that fit the particular space.

For more, feel free to read our image recognition article.

3. Text recognition and translation

Text recognition and translation combine AI Optical Character Recognition (OCR) techniques with text-to-text translation engines such as DeepL. A visual tracker keeps track of the word and allows the translation to overlay the AR environment. Google Translate offers this functionality.

4. Automatic Speech Recognition

Automatic Speech Recognition (ASR) uses neural network audiovisual speech recognition (an algorithm that relies on image processing to extract text). Specific words trigger an image in the library labeled to fit the word description, and the image is projected onto the AR space. An example is Panda sticker app.

For more, please read our collection of top speech recognition use cases.

9 AI/AR applications in industry

AR has been used in many applications, especially entertainment and construction. Other industries that can benefit from AI/AR include:

• Construction: Architecture, design, project planning, site revision, safety and inspection, underground constructions, and training.
• Education: Scene expedition (Museum, factory), model experiments in labs (chemistry, physics, geometry, anatomy)
• Entertainment: real time info from sports arenas, augmented music concerts, interactive ads, movies, and games.
• Medicine: diagnosis, surgical navigation, training surgeons on new procedures, and modeling drug effects.
• Logistics: warehouse planning and operations, transportation optimizations, and inventory management
• Manufacturing: design and prototyping, maintenance, repair and training,
• Military: aircraft navigation, weapon aiming, and telepresence in military operations.
• Real estate: Marketing, interior design, floor planning, construction staff training.
• Fashion: try-before-you-buy, in-store navigation, personalized shopping, AR window shopping, and makeup apps.

AI enabled AR Software Vendors

According to Statista, the global market of augmented reality (AR), virtual reality (VR), and mixed reality (MR) is estimated to reach $31B in 2021. Companies such as Apple and Google are in the market of developing AI enabled AR software to enhance customers’ AR experience.

Here are the top AI enabled AR software vendors:

Apple ARKit

ARKit is Apple’s augmented reality (AR) development platform for iOS iPhones and iPads. ARKit provides object labeling, people occlusion, motion capture, and multiple face tracking. ARKit has been used in:

• Education: to model practical experiments in science, physics, or chemistry labs, such as Labster
• Construction and architecture: to measure spatial dimensions and suggest products or solutions, such as IKEA place.
• Entertainment, such as Pokemon GO.

Google ARCore

ARCore is Google’s AR platform, ARCore integrates digital content into the physical environment via motion capture and object detection and recognition. ARCore has been used in:

• Real estate: to visualize, decorate, and design empty spaces. Such as Sotheby’s Curate app
• Lifestyle and maintenance: to connect users with professionals who provide guidance and instruction, take measurements, and project potential solutions into the space. For example Streem app.
• Entertainment: such as TendAR virtual pet game app.

Others

Other AI/AR software vendors include:

• Amazon Sumerian
• Microsoft Mesh
• Unity
• Vuforia Engine
• ZapWorks

For more on AI and AR

To learn about other uses of AI, feel free to read our 100+ AI Use Cases & Applications: In-Depth Guide

These articles about AI may also interest you:

• Ultimate Guide to the State of AI technology
• Future of AI according to top AI experts
• Advantages of AI according to top practitioners

You can also check our data-driven lists of:

• AR Tools
• AI Platforms
• AI Consultants
• AI Development Services

And if you have a specific business challenge, we can help you find the right vendor to overcome that challenge.