Unity3D has several options readily available via plugins for creating AR applications and games:
- Vuforia AR Starter Kit
- ARCore (Tango)
- ARToolKit
- ARKit
It should be noted that Vuforia Starter Kit has been fully integrated into Unity3D and is quite easy to jump right into creating projects with. ARKit and ARCore, however, are slightly different. Because they are still in the experimental and early developmental phases, Apple and Google have not released full and proper SDKs for Unity Technologies to incorporate into the engine. There is a plugin that exists for both of them that you will have to compile to get it to work with your project for Apple and Android devices, and we will go into how to compile and integrate into Unity3D to get it to work appropriately later in this chapter. For now, it is good to know that there is a little bit more setup involved with getting AR to work with iOS and Android.
With the introduction out of the way, we can finally begin to really talk about AR, what it is exactly, and how to set up Unity3D to take advantage of the SDKs available to create your own AR games and applications. Without further ado, let's define what AR actually is.
AR is augmented reality. Augmented reality is taking some form of reality and changing it in a specific way to enhance the experience. What augmented reality typically refers to is as follows:
- Sound:
Sound perception
- Video:
The text in this image is not important. We are just showcasing the video perception overlay
- Graphics:
The text in this image is not important. We are just showcasing the Graphic perception overlay
- Haptics:
Haptic perception
- GPS data:
GPS perception
This means that we could boost visual graphics of an object and view it in a different perspective than what we are accustomed to, or we could add something that isn't there. The video is a little bit different, as it requires the software to interface with specialized hardware, such as glasses, cell phones, HUDs, and other devices.
We could boost the auditory aspect of the world around us. We could take words we see in one language and have it say those words in another, or we could pick up those faint sounds that we always hear and tune out, then amplify them and really bring them to the forefront.
Haptic perception is a little more difficult, but it could be achieved with sensors that emulate touch. We could make something vibrate softly or harshly to simulate various effects, or we could make the game or application entirely touch- or motion-sensor-based. There are many other things we could use for haptic perception in applications or games. It is a field that is constantly being researched and expanded upon.
For GPS data, we can use the user's location to know where the user is in the app or game world. Another use for GPS data would be to know if something of interest to the user should be shown to them.
Because Unity3D loves to take care of most of the finer details of implementation for us, we don't have to worry too much about incorporating DLLs (dynamic-link libraries) or writing the wrapper classes to work with the majority of the popular AR and VR devices. There are exceptions to this rule, depending on the platform and whether the engine has been updated to specifically work with those devices.
Android and iOS are the most popular devices for applications and games to have AR incorporated into them, however, the various tech giants have been working hard to add more and more devices into the mix, with varying amounts of success. Some of these will obviously not have implementation with Unity3D, although you can write wrapper classes as previously mentioned, however, that is outside the scope of this book.
Let's take a quick look at some AR-capable devices. This should give us a slightly better idea of the different types of devices we can use and deploy to:
- The Meta 2 is a head-mounted display headset that uses a sensory array for hand interactions and positional tracking, it has a visual field view of 90 degrees, and a resolution of 2560 x 1440:
Meta 2
- AR displays can be rendered on devices resembling eyeglasses, such as the Google Glass:
Google Glass
- Another such device is the HoloLens:
HoloLens
- There is something called a head-up display, which is commonly referred to as a HUD. It is an alternate implementation of AR displays:
HUD
- There are many new devices being researched on and created all the time. Augmented reality is still in its infancy.
All throughout this book, we will create applications and games that are inspired by the definition of AR. Since there are four main aspects of AR, we will use those four aspects and create a specific application or game for that chapter. Since we have a finite number of sensors that are available for Android and iOS devices, some sensors will be utilized in multiple projects.