Hand gestures are analyzed by the recognize function; this is where the real magic takes place. This function handles the entire process flow, from the raw grayscale image to a recognized hand gesture. It returns the number of fingers and the illustration frame. It implements the following procedure:
- It extracts the user's hand region by analyzing the depth map (img_gray), and returns a hand region mask (segment), like this:
def recognize(img_gray: np.ndarray) -> Tuple[int,np.ndarray]:
# segment arm region
segment = segment_arm(img_gray)
- It performs contour analysis on the hand region mask (segment). Then, it returns the largest contour found in the image (contour) and any convexity defects (defects), as follows:
# find the hull of the segmented area, and based on that find the
# convexity defects
contour, defects = find_hull_defects(segment)
- Based on the contour found and the convexity defects, it detects the number of extended fingers (num_fingers) in the image. Then, it creates an illustration image (img_draw) using (segment) image as a template, and annotates it with contour and defect points, like this:
img_draw = cv2.cvtColor(segment, cv2.COLOR_GRAY2RGB)
num_fingers, img_draw = detect_num_fingers(contour,
defects, img_draw)
- Finally, the estimated number of extended fingers (num_fingers), as well as the annotated output image (img_draw), are returned, as follows:
return num_fingers, img_draw
In the next section, let's learn how to accomplish hand region segmentation, which we used at the beginning of the procedure.
