This book is for developers who are interested in learning and gaining real hands-on experience by building AR applications. We do not assume you have knowledge of the Unity game engine, C# language, 3D graphics, or previous experience with VR or AR development; although, any prior experience from novice to seasoned expert will be helpful.
Our focus will be on visual augmented reality. We will include some audio, which can be very important to complete an AR experience, but not other senses.
Mobile devices are natural platforms for AR, both mobile phones and tablets, and Android and iOS. We refer to this as handheld AR with video see-through tracking. All the projects in the book can be built and run on mobile devices, including iPhone, iPad, and Android phones and tablets.
Much of today's interest and excitement about AR is the wearable eyewear AR with optical see-through tracking, such as Microsoft HoloLens. Most of the projects in this book can also be built and run on eyewear AR devices. Cases where changes are required in the user interface and interactivity, for example, will be called out.
There is a risk in trying to cover such a wide range of target devices and platforms in one book. We will do our best to separate any device-specific dependencies in our step-by-step tutorial instructions and make it easy for you to follow the instructions relevant to your setup and skip those that do not pertain to you.
We've included topics regarding setting up your Windows or Mac development machine to build AR applications (sorry, Linux not included) and uploading the app onto your device.
For app development, we use the Unity 3D game development platform (https://unity3d.com/), which provides a powerful graphics engine and full-featured editor that you can drive using C# language programming. There are many sources that review and discuss the virtues and benefits of using Unity. Suffice to say, Unity includes native support for photorealistic rendering of computer graphics, humanoid and object animations, physics and collision, user interface and input event systems, and more. With Unity, you can create a project and then build it for any number of supported target platforms, including Windows, Android, iOS, as well as many other popular consoles and mobile devices.
For our AR toolkit, we will teach you how to use the popular and professional Vuforia AR SDK (https://www.vuforia.com/). AR development requires some sophisticated software algorithms and device management, much of which is handled quite elegantly by Vuforia. It was first published in 2008 by Daniel Wagner in a paper titled Robust and unobtrusive marker tracking on mobile phones, then it grew into the award-winning Vuforia by Qualcomm and was later acquired by PTC in 2015. Today, Vuforia supports a wide range of devices, from handheld mobiles to wearable eyewear, such as HoloLens. As we will see throughout this book, the SDK supports many types of tracking targets, including markers, images, objects, and surfaces; therefore, it can be used for many diverse applications. They also provide tools and cloud-based infrastructure for managing your AR assets.
Vuforia requires you have a license key in each of your apps. At the time of writing this, licenses are free for the first 1,000 downloads of your app; although, it displays a watermark in the corner of the display. Paid licenses start at $499 per app.
An alternative to Vuforia is the free and open source ARToolkit SDK (http://artoolkit.org/). ARToolkit was perhaps the first open source AR SDK and certainly the one that lasted the longest; it was first demonstrated in 1999 and released as open source in 2001. ARToolkit is now owned by DAQRI (https://daqri.com/), a leading industrial AR device and platform manufacturer.
As of this writing, the current ARToolkit 5 version is focused on marker and image-based targeting, and it is more limited than Vuforia. (ARToolkit version 6 is in Beta and it promises exciting new features and an internal architecture, but it is not covered in this book.) It does not support as wide a range of devices as Vuforia out of the box. However, since it is open source and has a sizable community, there are plugins available for just about any device (provided you're willing to tinker with it). The Unity package for ARToolkit does not necessarily support all the features of the native SDK.
Apple's ARKit for iOS (https://developer.apple.com/arkit/) is also in Beta and requires iOS 11 (also in Beta as of this writing). ARKit works on any iPhone and iPad using an Apple A9 or A10 processor. Unity provides an asset package that simplifies how to use ARKit and provides example scenes (https://bitbucket.org/Unity-Technologies/unity-arkit-plugin).
We are pleased to provide an introduction to Google ARCore (https://developers.google.com/ar/) in this book, but only an introduction. ARCore is brand new and at the time of writing this, it is in early preview only. The documentation and demo scene they provide is very bare-bones. The setup will likely be different when Unity supports ARCore in the final release. Things such as installing a preview of AR services APK will change. The list of supported Android devices is very short. Please refer to the the GitHub repository for this book for new implementation notes and code using Google ARCore for Android: https://github.com/ARUnityBook/. The principles are very similar to ARKit, but the Unity SDK and components are different.
Microsoft HoloLens is a Windows 10 MR (mixed reality) device. See https://www.microsoft.com/en-us/hololens/developers for more information. Using its companion, MixedRealityTooklit (formerly, HoloToolkit) components (https://github.com/Microsoft/MixedRealityToolkit) facilitates development using this fascinating AR device.
AR technology is moving ahead quickly. We really want you to learn the concepts and principles behind AR and its best practices, apart from specific SDK and devices.
The following table sorts out the various combinations of platforms, devices, and tools that will be covered in this book:
|
Target device |
Dev platform |
Dev engine |
AR SDK |
|
Android mobile |
Windows |
Unity |
Vuforia |
|
Android mobile |
Windows |
Unity |
ARToolkit |
|
Android mobile |
Windows |
Unity |
Google ARCore |
|
iOS mobile |
macOS |
Unity |
Vuforia |
|
iOS mobile |
macOS |
Unity |
Apple ARKit |
|
HoloLens |
Windows |
Unity |
Vuforia |
|
HoloLens |
Windows |
Unity |
MixedRealityToolkit |
