




















































This virtual reality thing calls into question, what does it mean to "be somewhere"?
Before cell phones, you would call someone and it would make no sense to say, "Hey, where are you?" You know where they are, you called their house, that's where they are.
So then cell phones come around and you start to hear people say, "Hello. Oh, I'm at Starbucks," because the person on the other end wouldn't necessarily know where you are, because you became un-tethered from your house for voice communications.
So when I saw a VR demo, I had this vision of coming home and my wife has got the kids settled down, she has a couple minutes to herself, and she's on the couch wearing goggles on her face. I come over and tap her on the shoulder, and I'm like, "Hey, where are you?"
It's super weird. The person's sitting right in front of you, but you don't know where they are.
-Jonathan Stark, mobile expert and podcaster
In this article, by Jonathan Linowes, author of the book Unity Virtual Reality Projects, we will define virtual reality and illustrate how it can be applied not only to games but also many other areas of interest and productivity.
Welcome to virtual reality! In this book, we will explore what it takes to create virtual reality experiences on our own. We will take a walk through a series of hands-on projects, step-by-step tutorials, and in-depth discussions using the Unity 5 3D game engine and other free or open source software. Though the virtual reality technology is rapidly advancing, we'll try to capture the basic principles and techniques that you can use to make your VR games and applications feel immersive and comfortable.
This article discusses the following topics:
Today, we are witnesses to the burgeoning consumer virtual reality, an exciting technology that promises to transform in a fundamental way how we interact with information, our friends, and the world at large.
What is virtual reality? In general, VR is the computer-generated simulation of a 3D environment, which seems very real to the person experiencing it, using special electronic equipment. The objective is to achieve a strong sense of being present in the virtual environment.
Today's consumer tech VR involves wearing a head-mounted display (such as goggles) to view stereoscopic 3D scenes. You can look around by moving your head, and walk around by using hand controls or motion sensors. You are engaged in a fully immersive experience. It's as if you're really there in some other virtual world. The following image shows a guy experiencing an Oculus Rift Development Kit 2 (DK2):
Virtual reality is not new. It's been here for decades, albeit hidden away in academic research labs and high-end industrial and military facilities. It was big, clunky, and expensive. Ivan Sutherland invented the first head-mounted display in 1966, which is shown in the following image. It was tethered to the ceiling! In the past, several failed attempts have been made to bring consumer-level virtual reality products to the market.
In 2012, Palmer Luckey, the founder of Oculus VR LLC, gave a demonstration of a makeshift head-mounted VR display to John Carmack, the famed developer of Doom, Wolfenstein 3D, and Quake classic video games. Together, they ran a successful Kickstarter campaign and released a developer kit called Oculus Rift Development Kit 1 (DK1) to an enthusiastic community. This caught the attention of investors as well as Mark Zuckerberg, and in March 2014, Facebook bought the company for $2 billion. With no product, no customers, and an infinite promise, the money and attention that it attracted has helped fuel a new category of consumer products. Others have followed suit, including Google, Sony, Samsung, and Steam. New innovations and devices that enhance the VR experience continue to be introduced.
Most of the basic research has already been done and the technology is now affordable thanks in large part to the mass adoption of devices that work on mobile technology. There is a huge community of developers with an experience in building 3D games and mobile apps. Creative content producers are joining in and the media is talking it up. At last, virtual reality is real!
Say what? Virtual reality is real? Ha! If it's virtual, how can it be... Oh, never mind.
Eventually, we will get past the focus on the emerging hardware devices and recognize that content is king. The current generation of 3D development software (commercial, free, and open source) that has spawned a plethora of indie, or independent, game developers can also be used to build non-game VR applications.
Though VR finds most of its enthusiasts in the gaming community, the potential applications reach well beyond that. Any business that presently uses 3D modeling and computer graphics will be more effective if it uses the VR technology. The sense of immersive presence that is afforded by VR can enhance all common online experiences today, which includes engineering, social networking, shopping, marketing, entertainment, and business development. In the near future, viewing 3D websites with a VR headset may be as common as visiting ordinary flat websites today.
Presently, there are two basic categories of head-mounted displays for virtual reality—desktop VR and mobile VR.
With desktop VR (and console VR), your headset is a peripheral to a more powerful computer that processes the heavy graphics. The computer may be a Windows PC, Mac, Linux, or a game console. Most likely, the headset is connected to the computer with wires. The game runs on the remote machine and the head-mounted display (HMD) is a peripheral display device with a motion sensing input. The term desktop is an unfortunate misnomer since it's just as likely to be stationed in either a living room or a den.
The Oculus Rift (https://www.oculus.com/) is an example of a device where the goggles have an integrated display and sensors. The games run on a separate PC. Other desktop headsets include HTC/Valve Vive and Sony's project Morpheus for PlayStation.
The Oculus Rift is tethered to a desktop computer via video and USB cables, and generally, the more graphics processing unit (GPU) power, the better. However, for the purpose of this book, we won't have any heavy rendering in our projects, and you can get by even with a laptop (provided it has two USB ports and one HDMI port available).
Mobile VR, exemplified by Google Cardboard (http://www.google.com/get/cardboard/), is a simple housing (device) for two lenses and a slot for your mobile phone. The phone's display is used to show the twin stereographic views. It has rotational head tracking, but it has no positional tracking. Cardboard also provides the user with the ability to click or tap its side to make selections in a game. The complexity of the imagery is limited because it uses your phone's processor for rendering the views on the phone display screen. Other mobile VR headsets include Samsung Gear VR and Zeiss VR One, among others.
Google provides the open source specifications, and other manufacturers have developed ready-made models for purchase, with prices for the same as low as $15. If you want to find one, just Google it! There are versions of Cardboard-compatible headsets that are available for all sizes of phones—both Android and iOS.
Although the quality of the VR experience with a Cardboard device is limited (some even say that it is inadequate) and it's probably a "starter" device that will just be quaint in a couple of years, Cardboard is fine for the small projects in this book, and we'll revisit its limitations from time to time.
It's probably worthwhile clarifying what virtual reality is not.
A sister technology to VR is augmented reality (AR), which superimposes computer generated imagery (CGI) over views of the real world. Limited uses of AR can be found on smart phones, tablets, handheld gaming systems such as the Nintendo 3DS, and even in some science museum exhibits, which overlay the CGI on top of live video from a camera.
The latest innovations in AR are the AR headsets, such as Microsoft HoloLens and Magic Leap, which show the computer graphics directly in your field of view; the graphics are not mixed into a video image. If the VR headsets are like closed goggles, the AR headsets are like translucent sunglasses that employ a technology called light fields to combine the real-world light rays with CGI. A challenge for AR is ensuring that the CGI is consistently aligned with and mapped onto the objects in the real-world space and eliminate latency while moving about so that they (the CGI and objects in real-world space) stay aligned.
AR holds as much promise as VR for future applications, but it's different. Though AR intends to engage the user within their current surroundings, virtual reality is fully immersive. In AR, you may open your hand and see a log cabin resting in your palm, but in VR, you're transported directly inside the log cabin and you can walk around inside it.
We can also expect to see hybrid devices that somehow either combine VR and AR, or let you switch between modes.
The consumer-level virtual reality starts with gaming. Video gamers are already accustomed to being engaged in highly interactive hyper-realistic 3D environments. VR just ups the ante.
Gamers are early adopters of high-end graphics technology. Mass production of gaming consoles and PC-based components in the tens of millions and competition between vendors leads to lower prices and higher performance. Game developers follow suit, often pushing the state-of-the-art, squeezing every ounce of performance out of hardware and software. Gamers are a very demanding bunch, and the market has consistently stepped up to keep them satisfied. It's no surprise that many, if not most, of the current wave of the VR hardware and software companies are first targeting the video gaming industry. A majority of the demos and downloads that are available on Oculus Share (https://share.oculus.com/) and Google Play for the Cardboard app (https://play.google.com/store/search?q=cardboard&c=apps) are games. Gamers are the most enthusiastic VR advocates and seriously appreciate its potential.
Game developers know that the core of a game is the game mechanics, or the rules, which are largely independent of the skin, or the thematic topic of the game. Gameplay mechanics can include puzzles, chance, strategy, timing, or muscle memory (twitch). VR games can have the same mechanic elements but might need to be adjusted for the virtual environment. For example, a first-person character walking in a console video game is probably going about 1.5 times faster than their actual pace in real life. If this wasn't the case, the player would feel that the game is too slow and boring. Put the same character in a VR scene and they will feel that it is too fast; it could likely make the player feel nauseous. In VR, you will want your characters to walk a normal, earthly pace. Not all video games will map well to VR; it may not be fun to be in the middle of a war zone when you're actually there.
That said, virtual reality is also being applied in areas other than gaming. Though games will remain important, non-gaming apps will eventually overshadow them. These applications may differ from games in a number of ways, with the most significant having much less emphasis on game mechanics and more emphasis on either the experience itself or application-specific goals. Of course, this doesn't preclude some game mechanics. For example, the application may be specifically designed to train the user at a specific skill. Sometimes, the gamification of a business or personal application makes it more fun and effective in driving the desired behavior through competition.
In general, non-gaming VR applications are less about winning and more about the experience itself.
Here are a few examples of the kinds of non-gaming applications that people are working on:
Wow, that's quite a list! This is just the low-hanging fruit.
The purpose of this book is not to dive too deeply into any of these applications. Rather, I hope that this survey helps stimulate your thinking and provides a perspective towards how virtual reality has the potential to be virtually anything for everyone.
This book takes a practical, project-based approach to teach the specifics of virtual reality development using the Unity 3D game development engine. You'll learn how to use Unity 5 to develop VR applications, which can be experienced with devices such as the Oculus Rift or Google Cardboard.
However, we have a slight problem here—the technology is advancing very rapidly. Of course, this is a good problem to have. Actually, it's an awesome problem to have, unless you're a developer in the middle of a project or an author of a book on this technology! How does one write a book that does not have obsolete content the day it's published?
Throughout the book, I have tried to distill some universal principles that should outlive any near-term advances in virtual reality technology, that includes the following:
Once VR becomes mainstream, many of these lessons will perhaps be obvious rather than obsolete, just like the explanations from the 1980's on how to use a mouse would just be silly today.
If you are interested in virtual reality, want to learn how it works, or want to create VR experiences yourself, this book is for you. We will walk you through a series of hands-on projects, step-by-step tutorials, and in-depth discussions using the Unity 3D game engine.
Whether you're a non-programmer who is unfamiliar with 3D computer graphics, or a person with experience in both but new to virtual reality, you will benefit from this book. It is not a cold start with Unity, but you do not need to be an expert either. Still, if you're new to Unity, you can pick up this book as long as you realize that you'll need to adapt to the pace of the book.
Game developers may already be familiar with the concepts in the book, which are reapplied to the VR projects while learning many other ideas that are specific to VR. Engineers and 3D designers may understand many of the 3D concepts, but they may wish to learn to use the game engine for VR. Application developers may appreciate the potential non-gaming uses of VR and want to learn the tools that can make this happen.
Whoever you are, we're going to turn you into a 3D Software VR Ninja. Well, OK, this may be a stretch goal for this little book, but we'll try to set you on the way.
There is not just one kind of virtual reality experience. In fact, there are many. Consider the following types of virtual reality experiences:
In this book, we will implement a number of projects that demonstrate how to build each of these types of VR experience. For brevity, we'll need to keep it pure and simple, with suggestions for areas for further investigation.
You will learn about the following in this book:
We will write scripts in the C# language and use features of Unity as and when they are needed to get things done.
However, there are technical areas that we will not cover, such as realistic rendering, shaders, materials, and lighting. We will not go into modeling techniques, terrains, or humanoid animations. Effective use of advanced input devices and hand and body tracking is proving to be critical to VR, but we won't have a chance to get into it here either. We also won't discuss game mechanics, dynamics, and strategies. We will talk about rendering performance optimization, but not in depth. All of these are very important topics that may be necessary for you to learn (or for someone in your team), in addition to this book, to build complete, successful, and immersive VR applications.
In this article, we looked at virtual reality and realized that it can mean a lot of things to different people and can have different applications. There's no single definition, and it's a moving target. We are not alone, as everyone's still trying to figure it out. The fact is that virtual reality is a new medium that will take years, if not decades, to reach its potential.
VR is not just for games; it can be a game changer for many different applications. We identified over a dozen. There are different kinds of VR experiences, which we'll explore in the projects in this book.
VR headsets can be divided into those that require a separate processing unit (such as a desktop PC or a console) that runs with a powerful GPU and the ones that use your mobile phone for processing. In this book, we will use an Oculus Rift DK2 as an example of desktop VR and Google Cardboard as the example of mobile VR, although there are many alternative and new devices available.
We're all pioneers living at an exciting time. Because you're reading this book, you're one, too. Whatever happens next is literally up to you. As the personal computing pioneer Alan Kay said, "The best way to predict the future is to invent it."
So, let's get to it!
Further resources on this subject: