AR is the term that's used to describe the technology that allows users to visualize part of the real world through the camera of a technological device (smartphone, tablet, or AR glasses) with virtual graphical information that's been added by this device. The device adds this virtual information to existing physical information. By doing this, tangible physical elements combine with virtual elements, thus creating augmented reality in real-time. The following image shows how AR works:

Now, we are going to look at the beginnings of AR and learn how AR can be divided according to its functionality.

AR is not a new technology. The beginnings of AR begin with the machine that was invented by Morton Heilig, a philosopher, visionary, and filmmaker, when, in 1957, he began to build a prototype with an appearance similar to the arcade video game machines that were very popular in the 90s. The following image shows a schema of how the prototype worked:

Morton called his invention Sensorama, an experience that projected 3D images, added a surround sound, made the seat vibrate, and created wind that was thrown as air at the viewer. The closest similar experience we can feel today is seeing a movie in a 4D cinema, but these experiences were created more than 60 years ago.

In 1968, Harvard Electrical Engineering professor Ivan Sutherland created a device that would be the key to the future of the AR technology known as the Human-Mounted Display (HMD). Far from the AR glasses that we know of today, this HMD, called the Sword of Damocles, was a huge machine that hung from the ceiling of a laboratory and worked when the user was placed in the right place. In the following image, you can see what this invention looked like:

In 1992, Boeing researcher Tom Caudell invented the term AR, and at the same time, AR technology was boosted from two other works. The first AR system, from L.B. Rosenberg, who works for the United States Air Force, is a device that gives advice to the user on how to perform certain tasks as they are presented, something like a virtual guide. This can be seen in the following image:

The other research in this area was led at Columbia University, where a team of scientists invented an HMD that interacted with a printer. The device, baptized as Karma (AR based on knowledge for maintenance assistance), projected a 3D image to tell the user how to recharge the printer, instead of going to the user manual.

The following diagram is a representation of the continuum of advanced computer interfaces, based on Milgram and Kishino (1994), where we can see the different subdivisions of the MIXED REALITY (MR) that go from the REAL ENVIRONMENT to the VIRTUAL REALITY. AR that's located nearer to the REAL ENVIRONMENT is divided between spatial AR and see-through AR. However, the appearance of mobile devices in the 21st century has allowed a different version of AR, where we can display it using the device screen and camera:

Now that we have introduced the beginnings of AR, let's learn how this technology can be classified depending on the trigger that's used to show virtual elements in the real world.

AR can be created in many ways; the main challenge is how to make the combination of the real and virtual worlds as seamless as possible. Based on what is used to trigger the virtual elements to appear in the real world, AR can be classified as follows:

• GPS coordinates: We use GPS coordinates, compasses, and accelerometers to locate the exact position of the user, including the cardinal point they are looking at. Depending on where the user is pointing to, they will see some virtual objects or others from the same position.
• Black and white markers: We use very simple images, similar to black and white QR codes, to project virtual objects on them. This was one of the first AR examples, although nowadays they are used less often as there are more realistic ways to create the AR experience.
• Image markers: We use the camera of the mobile device to locate predefined images (also called targets or markers) and then project virtual objects over them. This type of AR has substituted black and white markers.
• Real-time markers: The user creates and defines their own images with the mobile camera to project any virtual object in them.
• Facial recognition: Through the camera, we capture the movements of the face to execute certain actions in a request, for example, to give facial expressions to a virtual avatar.
• SLAM: Short for Simultaneous Localization And Mapping, this technology understands the physical world through feature points, thereby making it possible for AR applications to recognize 3D objects and scenes, as well as to instantly track the world, and overlay digital interactive augmentations.
• Beacons: eBeacons, RFID, and NFC are identification systems that use radio frequency or bluetooth, similar to GPS coordinates, to trigger the AR elements.

Now, you have a better grasp of what AR is and where it comes from. We have covered the basics of AR by looking at the first prototypes, and classified different types of AR according to the element that triggers the virtual images so that they appear on the screen. The next step is to see what is required to work with it.

AR projects need some basic elements to fulfill its function:

• Digital support, mainly tablet, mobile phone, or glasses. These will be the devices we will use to recognize what we are seeing and incorporate the information into AR.
• Sensors to perceive reality as a camera, GPS, 3D scanner, and so on. They are usually integrated into mobile devices.
• An algorithm or application to understand the reality that is being observed, interpreted, and displayed with the associated information.
• Digital content to enrich reality. This is all the information that you incorporate into AR.

The result is an interface in which reality is observed with additional information located in the space it needs to be in. The following subsections present different AR libraries and SDKs, as well as how they function.

ARToolKit was the world's first open source software library and allows the creation of AR applications, in which virtual images overlap in the real world. It uses video tracking capabilities to calculate the position of the camera and the orientation relative to the position of the physical markers in real-time. Once the position of the real camera is established, the virtual camera is placed at the same point and a virtual layer of information, such as a 3D object, is superimposed on the real marker. ARToolKit was originally developed by Hirokazu Kato and the initial release was published in 1999 by the HIT Laboratory of the University of Washington. It is currently maintained as an open source project hosted at SourceForge with commercial licenses available at ARToolworks: http://www.artoolkitx.org/.

Now, we will introduce the AR SDKs that we will use in this book. From the first appearance of ARToolKit, AR SDKs have very much evolved. Let's introduce and describe the different SDKs we are going to use in this book to create our own AR experiences:

• Vuforia
• EasyAR
• ARCore
• ARKit

Let's talk a little bit about each of them.

Initially developed by Qualcomm, and currently run by PTC, Vuforia is one of the oldest and most well-known AR SDKs, and it's one of the most stable and best-performing pieces of software in the market. Vuforia offers a wide variety of features, including 2D and 3D tracking, markerless AR, and fiducial markers called Vumarks. It provides multiple examples and extra features, such as virtual buttons, runtime image target creation, and background video texture manipulation.

An AR engine that allows us to create AR solutions in an easy way and offers multiple AR features, including technology that recognizes and tracks the position, rotation, and scale of a previously selected image in the real world, such as a book cover, a photograph, or a business card. It allows us to map a real place and keep track of the objects within it and contains technology to localize and track the position and orientation of real 3D objects instead of flat images and features, which allow us to take videos of the AR scene while we are playing it.

The first version was launched in February 2018 and is Google's platform for creating AR applications for Android and iOS. It makes use of different capabilities, such as motion tracking, to estimate the position and orientation of the mobile device regarding the real world; the environmental location, to find and track horizontal surfaces such as the ground, tablets, or walls; and light estimation, to place 3D elements realistically into the real world. Targeted at Android 7.0 and above, not all devices in the market support this technology, although this number has increased greatly since the first SDK version.

Apple launched the first version of ARKit in 2017, along with Xcode 9 and iOS 11, to bring AR to iOS devices. The framework, which is included in Xcode, lets users place virtual content in the real world. Months after its official release, it added new features such as 2D image detection and face detection. In 2018, with the iOS 12 release, ARKit 2 was launched with new features such as 3D object tracking, adding realistic textures to the scenes, and creating multiuser AR experiences. At the time of writing this book, the latest version is iOS 13, with ARKit 3 promising a huge improvement on the current state as it will add a new way of interacting with virtual elements, such as hiding virtual objects when a person is detected in front of them. It also allows users to interact with 3D objects via gestures and capture not only facial expressions but the body motions of a person.

Now that we are acquainted with the main AR tools we will use throughout this book, let's look at how we will apply this software to different enterprise fields.

AR can be a valuable asset and technology in multiple fields. In this section, we will learn how it is being used in sectors such as industry, marketing, retail, and tourism. In the first case, we will learn how industry 4.0 has put AR in the spotlight, since it's the natural interface for the human-machine interaction in areas such as production, maintenance, training, and more.

This section's main objective is to give you a better insight into how to apply AR in your own projects before you start to code. This way, you will be able to adapt the examples in this book to your own needs. 

Let's start introducing the uses of AR in the industry field.

The use of this technology in the industrial sector is growing. Its application focuses, among other aspects, on being able to prepare for the breakdown of a machine and thus know what part has been damaged and how to proceed to repair it. This use of AR represents a significant cost saving as it favors rapid action before the breakdown in question. You can also use the AR to visualize a plant or product before carrying it out. This means that you can perfect what to build before turning it into something physical. The use of AR in industry favors the application of one of the enabling technologies of industry 4.0 that help us increase the competitiveness and productivity of a company.

Some examples are as follows:

• Smart assistance
• Virtual prototyping
• Logistics and warehouse management
• Digital Twins

Let's look at these examples in more detail.

The use of AR with maintenance process management platforms provides direct support to operators during the operation of industrial machines, thus shortening the time and increasing accuracy in the performance of the task or repair. Some tasks you can have under this case are as follows:

• Follow guided instructions and workflows
• Assign activities in real-time
• Access checklists and list of assigned tasks
• Create a digital report of the interventions
• Automatically detect 3D objects
• View 3D manuals and work orders in real-time
• Have an expert remote collaborator through video calls
• Extract and display real-time data from machines

In this case, the use of AR technology improves business productivity by reducing costs and the possibility of errors.

AR can be applied to visualize and configure 3D industrial plants and machinery in real-time. 3D models can be shown in the industrial environment where the validation is carried out, but also in exhibition spaces such as fairs and events. Through AR, you can visualize the hologram of the industrial machinery in the place where it will be installed to evaluate its dimensions, escape spaces, and safety margins. By doing this, you optimize the production processes. In addition, the packaging, shipping, or transport of heavy industrial machinery from one part of the world to another for presentation at fairs has a very high cost. With AR, you can see your product as if it were real in the center of your stand.

Through 3D viewers or glasses connected to a warehouse management system, a warehouse operator receives information for the storage of merchandise directly on the device, which allows them to improve the times and quality of their work and, ultimately, the production yield.

Thanks to AR technologies, you can create a virtual and dynamic replica of a real system, product, or service that, through the information that's obtained from sensors or automatisms, models its behavior. In this way, it allows intelligent monitoring and analysis, as well as the realization of simulations and experiments in the replica, without risks or costs, with the aim of improving the performance and effectiveness of the real system.

In this book, we are going to learn how to use AR for manufacturing by creating a prototype automation with an AR guide and training.

The industry is not the only field where AR can be of value. The versatility of this technology makes it the perfect communication item between the users and the final product or process and adds a visual and interactive layer to engage them. Whether it's used to make a product more attractive or help users understand a process with added information, AR can be a great asset in many fields. Let's look at some other enterprise areas that we will later cover in this book where AR adds value. 

AR has many uses in this area, such as prototyping, production, and training. In these areas, AR can help speed up and reduce costs in prototyping by visualizing the designs over the real world and allowing designers, workers, and potential customers to walk around and through the model and manipulate its three dimensions.

AR has become a valuable asset in education and training. It allows us to visualize concepts in three dimensions over an image or directly in the room so that we can access the information in a quicker and more dynamic way (just pointing with the camera instead of looking through a book or searching the internet for the information) or to create deeper personal projects (giving life to a painting, adding instructions or extra information over a handcrafted project, creating an animated presentation, and so on).

Using AR, the user is in charge of their experience and can interact with it. Now, brands go beyond the wow effect and create functional experiences to promote and sell their products. Examples of this include augmented catalogs, which show the products in 3D over their flat images; virtual mirrors, where you can buy the glasses or complements you are trying on AR; packages that come to life to explain the elements inside the fabrication process; and so on.

Retail is one of the fields where AR offers a wider range of possibilities, from satisfying and engaging the consumer in order to reduce returned products, to linking products with social media or personalizing the shopping experience. Some examples of this include trying products before buying them so that users can visualize clothes, shoes, glasses, or even makeup over themselves before actually buying the products. By doing this, they can see how a product such as a piece of furniture, art, or even wall paint looks on their homes with AR.

Other examples include seeing extra information about a product, such as comments and reviews, before buying a product, receiving geopositioned information and discounts from the stores on it, and in supermarkets or big stores, orienting customers through sections to the product they want.

AR is the natural interface and connection to the Internet of Things (IoT) and Big Data. It allows workers to visualize and interact with the data coming from the machines and sensors of a factory in an easy and attractive way, either using mobile devices or AR headsets.

AR use in automation can go from the facial recognition of an employee to getting access to a concrete machine, to real-time on-site surveillance of the production process or remote access and control to the system through AR glasses.

The thing that's attractive about AR is the visual content it displays over the real world. That makes this technology perfect for enhancing traveling experiences, from showing skyline information to making animals in a museum come to life or even translating signs and guides in real-time. Some of the most common uses of AR in tourism include serving as a live guide in the streets of a new city, where a user can go around the city while the AR app is showing them where the most interesting points to see are; showing attractions and points of interest (POIs) over a map when a user points at a map with the camera; giving all kinds of extra information on paintings, sculptures, or monuments so that when a user points at a painting it can show a video about the author, the place and time where it was painted, or even make it come to life in a 3D simulation. Apart from all these experiences, when AR is combined with other immersive technologies, such as virtual worlds or 360º videos, the experience goes one step ahead, allowing users to visit several places at the same time (for example, in a museum network, while visiting one of them, to be able to virtually visit the others).

In this section, we have looked at several examples of how we can use AR for very different fields, from visualizing a prototype before building it, to attracting customers or adding value to art pieces. Now, you have a better understanding of the possibilities AR offers in the world of enterprises and how to apply it according to the needs of each sector.

In this chapter, we have taken our first look at AR by learning what it is, where it comes from, and what type of software and elements we need in order to create it. We have learned about the names and main features of the SDKs we are going to use in this book and are now ready to start creating our own AR projects.

In the next chapter, we will introduce Untiy 3D, a 3D tool that can be used as the base for nearly any AR project since most AR SDKs are compatible with it. You will learn about how it works and create a simple example that will be the basis for the AR projects that follow.