Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Augmented Reality with Kinect
Augmented Reality with Kinect

Augmented Reality with Kinect: If you know C/C++ programming, then this book will give you the ability to develop augmented reality applications with Microsoft's Kinect. By the end of the course you will have created a complete game.

eBook
€8.99 €22.99
Paperback
€28.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Table of content icon View table of contents Preview book icon Preview Book

Augmented Reality with Kinect

Chapter 1. Getting Started with Kinect

Before the birth of Microsoft Kinect, few people were familiar with the technology of motion sensing. Similar devices have been invented and developed originally for monitoring aerial and undersea aggressors in wars. Then in the non-military cases, motion sensors are widely used in alarm systems, lighting systems and so on, which could detect if someone or something disrupts the waves throughout a room and trigger predefined events. Although radar sensors and modern infrared motion sensors are used more popularly in our life, we seldom notice their existence, and can hardly make use of these devices in our own applications.

But Kinect changed everything from the time it was launched in North America at the end of 2010. Different from most other user input controllers, Kinect enables users to interact with programs without really touching a mouse or a pad, but only through gestures. In a top-level view, a Kinect sensor is made up of an RGB camera, a depth sensor, an IR emitter, and a microphone array, which consists of several microphones for sound and voice recognition. A standard Kinect (for Windows) equipment is shown as follows:

The Kinect device

The Kinect drivers and software, which are either from Microsoft or from third-party companies, can even track and analyze advanced gestures and skeletons of multiple players. All these features make it possible to design brilliant and exciting applications with handsfree user inputs. And until now, Kinect had already brought a lot of games and software to an entirely new level. It is believed to be the bridge between the physical world we exist in and the virtual reality we create, and a completely new way of interacting with arts and a profitable business opportunity for individuals and companies.

In this book, we will try to make an interesting game with the popular Kinect technology for user inputs, with the major components explained gradually in each chapter. As Kinect captures the camera and depth images as video streams, we can also merge this view of our real-world environment with virtual elements, which is called Augmented Reality (AR). This enables users to feel as if they appear and live in a nonexistent world, or something unbelievable exists in the physical earth.

In this chapter, we will first introduce the installation of Kinect hardware and software on personal computers, and then consider a good enough idea compounded of Kinect and augmented reality elements, which will be explained in more detail and implemented in the following chapters.

Note

Before installing the Kinect device on your PCs, obviously you should buy Kinect equipment first. In this book, we will depend on Kinect for Windows or Kinect for Xbox 360, which can be learned about and bought at:

http://www.microsoft.com/en-us/kinectforwindows/

http://www.xbox.com/en-US/kinect

Please note that you don't need to buy an Xbox 360 at all. Kinect will be connected to PCs so that we can make custom programs for it. An alternative choice is Kinect for Windows, which is located at:

http://www.microsoft.com/en-us/kinectforwindows/purchase/

The uses and developments of both will be of no difference for our cases.

Installation of Kinect


It is strongly suggested that you have a Windows 7 operating system or higher. It can be either 32-bit or 64-bit and with dual-core or faster processors.

Note

Linux developers can also benefit from third-party drivers and SDKs to manipulate Kinect components, which will be introduced in the Appendix, Where to Go from Here, of this book.

Before we start to discuss the software installation, you can download both the Microsoft Kinect SDK and the Developer Toolkit from:

http://www.microsoft.com/en-us/kinectforwindows/develop/developer-downloads.aspx

In this book, we prefer to develop Kinect-based applications using Kinect SDK Version 1.5 (or higher versions) and the C++ language. Later versions should be backward compatible so that the source code provided in this book doesn't need to be changed.

Setting up your Kinect software on PCs

After we have downloaded the SDK and the Developer Toolkit, it's time for us to install them on the PC and ensure that they can work with the Kinect hardware. Let's perform the following steps:

  1. Run the setup executable with administrator permissions. Select I agree to the license terms and conditions after reading the License Agreement.

    The Kinect SDK setup dialog

  2. Follow the steps until the SDK installation has finished. And then, install the toolkit following similar instructions.

  3. The hardware installation is easy: plug the ends of the cable into the USB port and a power point, and plug the USB into your PC. Wait for the drivers to be found automatically.

  4. Now, start the Developer Toolkit Browser, choose Samples: C++ from the tabs, and find and run the sample with the name Skeletal Viewer.

  5. You should be able to see a new window demonstrating the depth/skeleton/color images of the current physical scene, which is similar to the following image:

    The depth (left), skeleton (middle), and color (right) images read from Kinect

Why did I do that?

We chose to set up the SDK software at first so that it will install the motor and camera drivers, the APIs, and the documentations, as well as the toolkit including resources and samples onto the PC. If the operation steps are inversed, that is, the hardware is connected before installing the SDK, your Windows OS may not be able to recognize the device. Just start the SDK setup at this time and the device should be identified again during the installation process.

But before actually using Kinect, you still have to ensure there is nothing between the device and you (the player). And it's best to keep the play space at least 1.8 m wide and about 1.8 m to 3.6 m long from the sensor. If you have more than one Kinect device, don't keep them face-to-face as there may be infrared interference between them.

Note

If you have multiple Kinects to install on the same PC, please note that one USB root hub can have one and only one Kinect connected. The problem happens because Kinect takes over 50 percent of the USB bandwidth, and it needs an individual USB controller to run. So plugging more than one device on the same USB hub means only one of them will work.

The depth image at the left in the preceding image shows a human (in fact, the author) standing in front of the camera. Some parts may be totally black if they are too near (often less than 80 cm), or too far (often more than 4 m).

Note

If you are using Kinect for Windows, you can turn on Near Mode to show objects that are near the camera; however, Kinect for Xbox 360 doesn't have such features.

You can read more about the software and hardware setup at:

http://www.microsoft.com/en-us/kinectforwindows/purchase/sensor_setup.aspx

The idea of the AR-based Fruit Ninja game


Now it's time for us to define the goal we are going to achieve in this book. As a quick but practical guide for Kinect and augmented reality, we should be able to make use of the depth detection, video streaming, and motion tracking functionalities in our project. 3D graphics APIs are also important here because virtual elements should also be included and interacted with irregular user inputs (not common mouse or keyboard inputs).

A fine example is the Fruit Ninja game, which is already a very popular game all over the world. Especially on mobile devices like smartphones and pads, you can see people destroy different kinds of fruits by touching and swiping their fingers on the screen.

With the help of Kinect, our arms can act as blades to cut off flying fruits, and our images can also be shown along with the virtual environment so that we can determine the posture of our bodies and position of our arms through the screen display.

Unfortunately, this idea is not fresh enough for now. Already, there are commercial products with similar purposes available in the market; for example:

http://marketplace.xbox.com/en-US/Product/Fruit-Ninja-Kinect/66acd000-77fe-1000-9115-d80258410b79

But please note that we are not going to design a completely different product here, or even bring it to the market after finishing this book. We will only learn how to develop Kinect-based applications, work in our own way from the very beginning, and benefit from the experience in our professional work or as an amateur. So it is okay to reinvent the wheel this time, and have fun in the process and the results.

Summary


Kinect, which is a portmanteau of the words "kinetic" and "connect", is a motion sensor developed and released by Microsoft. It provides a natural user interface (NUI) for tracking and manipulating handsfree user inputs such as gestures and skeleton motions. It can be considered as one of the most successful consumer electronics device in recent years, and we will be using this novel device to build the Fruit Ninja game in this book.

We will focus on developing Kinect and AR-based applications on Windows 7 or higher using the Microsoft Kinect SDK 1.5 (or higher) and the C++ programming language. Mainly, we have introduced how to install Kinect for Windows SDK in this chapter. Linux and Mac OS X users can first read the Appendix, Where to Go from Here, which provides an alternative method to call Kinect functionalities on other systems. Developers of .NET, processing, or other language tools may also find useful resources in the last chapter of this book.

In the next chapter, we will learn how to start and shut down the Kinect device in our applications, and prepare a basic framework for further development.

Left arrow icon Right arrow icon

Key benefits

  • Understand all major Kinect API features including image streaming, skeleton tracking and face tracking
  • Understand the Kinect APIs with the help of small examples
  • Develop a comparatively complete Fruit Ninja game using Kinect and augmented Reality techniques

Description

Microsoft Kinect changes the notion of user interface design. It differs from most other user input controllers as it enables users to interact with the program without touching the mouse or a trackpad. It utilizes motion sensing technology and all it needs is a real-time cameras, tracked skeletons, and gestures. Augmented Reality with Kinect will help you get into the world of Microsoft Kinect programming with the C/C++ language. The book will cover the installation, image streaming, skeleton and face tracking, multi-touch cursors and gesture emulation. Finally, you will end up with a complete Kinect-based game. Augmented Reality with Kinect will help you get into the world of Kinect programming, with a few interesting recipes and a relatively complete example. The book will introduce the following topics: the installation and initialization of Kinect applications; capturing color and depth images; obtaining skeleton and face tracking data; emulating multi-touch cursors and gestures; and developing a complete game using Kinect features. The book is divided in such a way so as to ensure that each topic is given the right amount of focus. Beginners will start from the first chapter and build up to developing their own applications.

Who is this book for?

This book is meant for readers who are familiar with C/C++ programming and want to write simple programs with Kinect. The standard template library can also be used as it is simple enough to understand.

What you will learn

  • Install a Kinect device and SDK packages on your computer
  • Initialize Kinect with C++ APIs in your own application
  • Obtain image streaming data from color and depth cameras
  • Acquire skeleton data and face tracking results for use
  • Use hand positions to emulate multi-touch cursors and gestures
  • Develop a Fruit Ninja game with different Kinect functionalities
  • Study with simple and interesting examples using a uniform OpenGL framework
  • Find more open source and commercial resource on the Web

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Jul 11, 2013
Length: 122 pages
Edition : 1st
Language : English
ISBN-13 : 9781849694391
Vendor :
Microsoft
Category :
Languages :
Tools :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Jul 11, 2013
Length: 122 pages
Edition : 1st
Language : English
ISBN-13 : 9781849694391
Vendor :
Microsoft
Category :
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 91.97
Augmented Reality with Kinect
€28.99
Kinect for Windows SDK Programming Guide
€37.99
Kinect in Motion - Audio and Visual Tracking by Example
€24.99
Total 91.97 Stars icon
Banner background image

Table of Contents

6 Chapters
Getting Started with Kinect Chevron down icon Chevron up icon
Creating Your First Program Chevron down icon Chevron up icon
Rendering the Player Chevron down icon Chevron up icon
Skeletal Motion and Face Tracking Chevron down icon Chevron up icon
Designing a Touchable User Interface Chevron down icon Chevron up icon
Implementing the Scene and Gameplay Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.3
(8 Ratings)
5 star 37.5%
4 star 50%
3 star 12.5%
2 star 0%
1 star 0%
Filter icon Filter
Top Reviews

Filter reviews by




J. Swanson Aug 08, 2013
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Having a few books dealing with both Kinect and a couple about the relatively new buzz phrase 'augmented reality' under my belt, I was very anxious to read Rui Wang's book. Although I have not yet finished the book I feel confident saying it is a well thought out book on the subject covered.Let me say up front that this is not a complete beginners book - if you are not familiar with the C++ language and how to use the Visual Studio IDE, compiling code and some knowledge of OpenGL, then go and get yourself familiar with them and then tackle the contents of this book, I assure you it's worth it.I consider myself a hobbyist when it comes to my own involvement in the Kinect device. My interest stems from my love of both 3D graphics and computer programming. As for my C++ programming abilities, I consider myself at an intermediate level. I already had the Microsoft Kinect for Windows SDK and Visual Studio Express installed so I just had to add FreeGLUT and FreeImage libraries to get going. The instructions for setting up these libraries, so they are accessible, was well done.It's great that the code is available for download - I think this made trying things out a lot easier. I liked the fact that there is an actual project we work on the - 'Fruit Ninja game' - to see the techniques learned in action. With every thing set up we get right in to coding thereby building our knowledge of the concepts step by step. This is an instruction technique I find works best for me. Throughout the text I see an 'Understanding the code' section which I found helpful. Sometimes the code we are instructed to use purposely does not give the desired results. We are then walked through the method to correct the problem. This is another strong instruction tool I find useful when I'm faced with learning new concepts. At the end of the chapters there is a summary given, and throughout the book we have some links to additional useful information.All in all I like the book. I have not yet finished it (there's a lot of info packed in here) but I feel this is a worthwhile book if you are interested in combining augmented reality and the Kinect device. As of this writing I am half way through the Skeletal Motion and Face Tracking chapter. I will try to come back here and update the review when I finish the book. But for now I have no trouble recommending this book.
Amazon Verified review Amazon
James Ashley Aug 19, 2013
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Rui Wang's Augmented Reality with Kinect from Pakt Publishing is my new favorite book about the Kinect sensor. It's a solid 5 out of 5 for me and if you want to learn how to use the Kinect 4W SDK 1.5 and above with C++, then this is the book for you. That said, however, it is also an incredibly frustrating software programming book.The first issue I have with it is that it isn't really about Augmented Reality, as such. The way AR fits in is simply that the central project created in the course of the book is a Fruit Ninja-style game using Kinect and with a player overlay. AR seems very much incidental to the book.What it actually is is an intro book to C++ and the Kinect for Windows SDK. This is actually a much needed resource in the Kinect community and one I have been on the lookout for for a long time. I'm not sure why the publisher decided to add this "AR" twist to the concept for the book. It really wasn't necessary.Second, the book's tool chain is Visual Studio 2012, C++, Kinect for Windows SDK 1.5 and OpenGL. One of these is not like the others! In the second chapter, we are then told that the book covers OpenGL rather than DirectX because "...it is only used under Windows currently, and can hardly support languages except C/C++ and C#." Hmmm.With those reservations out of the way, this is a really fine book about programming for the Kinect sensor. C++ is the right way to do vision processing and this is a great introduction to the topic. Along the way, it even includes a nice overview of face tracking.
Amazon Verified review Amazon
Dick Mandemaker Sep 22, 2013
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book really is a guide to help you understand how a Kinect works and how to create your own applications in a structured way, using the SDK and libraries. I was surprised how clear this is written by the author!Step by step you are guided to build the example Augmented Reality Ninja-Fruit game. It surprised me how quickly I understood what the author was doing, and the 'understand how the code works' part at the end of each section is a great thing!I have learned how to build a Visio Studio project, in a structured way how to write code, and at the same time getting to know the way a Kinect works. It is clearly explained how data from Kinect (be it color, depth, skeleton, face) is easily captured and used in your application. Also how to control with touch and swipe is easy to learn and apply.So in summary it is the easiest way to start building Kinect applications yourself.
Amazon Verified review Amazon
Junie B Jones Aug 11, 2013
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
Reminiscent of the Wrox Press books, "Augmented Reality with Kinect" takes the reader, step-by-step, through the process of creating a simple C++ "Fruit Ninja" application using Kinect with the OpenGL framework. The writing style was easy to follow. I'm new to C++ but was able to following the code. Book made good use of screenshots, though screenshots of the code within the IDE might be a good addition. I would like to see another publication building on this example: grabbing the fruits and tossing them in a basket, etc. I would definitely read more from this author.
Amazon Verified review Amazon
Tinux Aug 09, 2013
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
This book is a practical guide to Kinect NUI (Natural User Interface). The author in about 100 pages will show how to use depth detection, video streaming and motion tracking in order to develop a simple Fruit Ninja-style game on Windows 7 (or higher) using the Microsoft Kinect SDK 1.5 (or higher) and the C/C++ programming language.I think that it is a nice and interesting example to guide the reader in the development of Kinect-based applications.As prerequisite, the reader should have basic knowledge of C++ programming and some experience of programming real-time graphics APIs (OpenGL) may be useful but not is not required. If you don't have a good C++ coding skill don't worry, the code is simple and well written ;)Chapter One, Getting Started with Kinectshows how to setting up the Kinect software on Windows and introduce the idea of the Augmented Reality based Fruit Ninja like game.Chapter Two, Creating Your First Programshows how to prepare the development environment and how to initialize the Kinect device with C++.Noteworthy, this chapter demonstrates how to create an OpenGL-based framework utilizing the FreeGLUT library.Chapter Three, Rendering the Playerdemonstrates how to obtain color and depth images from Kinect and how to display them in own OpenGL-based framework. Furthermore, shows a traditional way for background subtraction and how to generate a color image form a depth image.Chapter Four, Skeletal Motion and Face Trackingshows how to render the skeleton data calculated by the Kinect SDK and introduces the Face Detection APIs. In particular how to detect a face from the camera and how to draw the parametric face model.Chapter Five, designing a Touchable User Interfacedemonstrates how to simulate multi-touch inputs. How to draw cursors using two hands and how to utilize it to emulate the Windows mouse.Chapter Six, Implementing the Scene and Game Playshows how to make the Fruit Ninja like game. In particular how to integrate the shown code in the prior chapters and how to implement a simple game logic.Appendix, Where to Go from hereIt's a reference for libfreenect, OpenNI and other resources.I personally would have suggested to the reader open and commercial libraries/products compatibles with the framework utilized in this book. (Microsoft Kinect SDK)In conclusion...If you are a C++ developer and want to start to develop Kinect-based applications/games this book it's a good start. ;)If you are interested in Augmented Reality I suggest to integrate it with others books focused on AR.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.