Tracking colorful objects using MeanShift and CamShift

We have seen that background subtraction can be an effective technique for detecting moving objects; however, we know that it has some inherent limitations. Notably, it assumes that the current background can be predicted based on past frames. This assumption is fragile. For example, if the camera moves, the entire background model could suddenly become outdated. Thus, in a robust tracking system, it is important to build some kind of model of foreground objects rather than just the background.

We saw various ways of detecting objects in Chapter 5, Detecting and Recognizing Faces, Chapter 6, Retrieving Images and Searching Using Image Descriptors, and Chapter 7, Building Custom Object Detectors. For object detection, we favored algorithms that could deal with a lot of variation within a class of objects, so that our car detector was not too particular about what shape or color of car it would detect. For object tracking, our needs...