Pick the stick – 3D selection using ray picking

3D interaction on desktop computers made use of a limited set of devices, including the keyboard, mouse, or joystick (for games). On a smartphone (or tablet), interaction is mainly driven by touch or sensor input. From an interaction input (the sensor data, such as x and y coordinates on the screen, or the event type, such as click or dwell), you can develop different interaction techniques, such as ray picking, steering navigation, and so on. For mobile AR, a large set of interaction techniques can be used for 2D or 3D interactions. In this section, we will look at using touch input combined with a technique named ray picking.

The concept of ray picking is to use a virtual ray going from your device to your environment (which is the target) and detect what it hits along the way. When you get a hit on some object (for example, the ray intersects with one of your virtual characters), you can consider this object picked (selected) and start to...