Recognizing hand landmarks in real time
One of the additions to Vision in iOS 14 is hand detection. This new feature to detect hands in images and video allows developers to find with great detail the positions of the wrist and the individual fingers in a video frame or photo.
In this section, we are going to explain the basics behind hand detection, and we will demonstrate how it works with a sample project. Let's start with the hand landmarks that we will be able to recognize.
Understanding hand landmarks
There are 21 landmarks that we will be able to detect in a hand:
- 4 in the thumb
- 4 in each finger (16 in total)
- 1 in the wrist
As you can see, Vision differentiates between finger and thumb. In both the finger and thumb, there are 4 points of interest. The following figure shows how these landmarks are distributed:
Figure 15.04 – Finger and thumb landmarks
Note how there is also a landmark in the middle of the...
