The spectrum of VR devices

As with most technologies, there is a spectrum of products for virtual reality ranging from the simplest and least expensive to the very advanced.

Old fashioned stereoscopes

Cardboard is at the low end of the VR device spectrum. Well, you could even go lower if you consider the ViewMaster that you may have played with as a child, or even the historic stereoscope viewer from 1876 (B.W. Kilborn & Co, Littleton, New Hampshire), as shown in the following image:

In these old fashioned viewers, a pair of photographs display two separate views for the left and right eyes that are slightly offset to create parallax. This fools the brain into thinking that it's seeing a truly three-dimensional view. The device contains separate lenses for each eye that allow you to easily focus on the photo close up.

Similarly, rendering these side-by-side stereo views is the first job of a Google Cardboard application. (Leveraging their legacy, Mattel recently released a Cardboard-compatible ViewMaster brand VR viewer that uses a smartphone, which can be found at http://www.view-master.com/).

Cardboard is mobile VR

Cardboard's obvious advantages over stereoscopic viewers are like the advantages of digital photographs over traditional ones. Digital media can be dynamically stored, loaded, and manipulated right within our smartphones. That's a powerful leap on its own.

On top of that, Cardboard uses the motion sensors in the phone in such a way that when you turn your head left-right or up-down, the image is adjusted accordingly, effectively obliterating the traditional frame edges of the image. Framing the image is a very important part of traditional visual media, such as painting, photography, and cinematography. For centuries, artists and directors have established a visual language using this rectangular frame.

However, not so much in VR. When you move your head in VR your view direction changes, and the scene is updated as if the camera is rotating along with you, providing a fully immersive view. You can rotate it horizontally 360 degrees as you look side to side and 180 degrees up and down. In other words, you can look anywhere you want. There is no frame in VR! (Albeit your peripheral vision might be limited by the optics and display size, which determine the device's field of view or FOV). In this way, the design considerations may be more akin to sculpture, theatre-in-the-round, or even architectural design. We need to think about the whole space that immerses the visitor.

The Google Cardboard device is simply a casing for you to slip your smartphone into. It uses the smartphone's technology, including the following:

• Display
• CPU (the main processor)
• GPU (the graphics processor)
• IMU (the motion sensor)
• Magnetometer and/or touchscreen (the trigger sensor)

We'll talk more about how all this works a little later.

Using a mobile smartphone for VR means great things, such as ease of use, but also annoying constraints, such as limited battery life, slower graphics processing, and lower accuracy/higher latency motion sensors.

The Samsung Gear VR is a mobile VR headset that is smarter than a simple Cardboard viewer. Android-based but not compatible with Cardboard apps (and only works with specific models of Samsung phones), it has a separate built-in higher precision IMU (motion sensor), which increases the accuracy of the head motion tracking and helps reduce the motion-to-pixel latency when updating the display. It's also ergonomically designed for more extended use and it includes a strap.

Desktop VR and beyond

At the higher end of consumer virtual reality devices are the Oculus Rift, HTC Vive, and Sony PlayStation VR, among others. These products go beyond what Cardboard can do because they're not limited by the capabilities of a smartphone. Sometimes referred to as "desktop VR," these devices are head-mounted displays (HMD) tethered to an external PC or console.

On desktop VR, the desktop's powerful CPU and GPU do the actual computation and graphics rendering and send the results to the HMD. Furthermore, the HMD has higher quality motion sensors and other features that help reduce the latency when updating the display at, say, 90 frames per second (FPS). We'll learn throughout this book that reducing latency and maintaining high FPS are important concerns for all VR development and the comfort of your users on all VR devices, including Cardboard.

Desktop VR devices also add positional tracking. The Cardboard device can detect the rotational movement on any of the X, Y, and Z axes, but it unfortunately cannot detect the positional movement (for example, sliding along any of these axes). The Rift, Vive, and PSVR can. The Rift, for example, uses an external camera to track the position using infrared lights on the HMD (outside-in tracking). The Vive, on the other hand, uses sensors on the HMD to track a pair of laser emitters placed strategically in the room (inside-out tracking). The Vive also uses this system to track the position and rotation of a pair of hand controllers. Both strategies achieve similar results. The user has a greater freedom to move around within the tracked space while experiencing moving around within the virtual space. Cardboard cannot do this.

Note that innovations are continually being introduced. Very likely, at some point, positional tracking will be included with the Cardboard arsenal. For example, we know that Google's Project Tango implements visual-inertial odometry, or VIO, using sensors, gyroscopes, and awareness of the physical space to provide motion and positional tracking to mobile apps. Refer to https://developers.google.com/project-tango/overview/concepts. Mobile device companies, such as LG and Samsung, are working hard to figure out how to do mobile positional tracking, but (at the time of this writing) a universal, low-latency solution does not yet exist. Google's Project Tango shows some promise but cannot yet achieve the time-to-pixel latency required for a smooth, comfortable VR experience. Too much latency and you get sick!

At the very high end are industrial and military grade systems that cost thousands or millions of dollars, which are not consumer devices, and I'm sure can do really awesome things. I could tell you more about it, but then I'd have to kill you. Solutions such as these have also been around since the 1980s. VR is not new—consumer VR is new.

The spectrum of VR devices is illustrated in the following diagram:

When we develop for Cardboard, it is important to keep in mind the things it can and cannot do relative to other VR devices. Cardboard can display stereoscopic views. Cardboard can track rotational head movement. It cannot do positional tracking. It has limitations of graphics processing power, memory, and battery life.