Optical flow is the pattern of apparent motion between two consecutive frames of video. We select feature points in the first frame and try to determine where those features have gone in the second frame. This search is subject to a few caveats:
- We make no attempt to distinguish between camera motion and subject motion.
- We assume that a feature's color or brightness remains similar between frames.
- We assume that neighboring pixels have similar motions.
OpenCV's calcOpticalFlowPyrLK function implements the Lucas-Kanade method of computing optical flow. Lucas-Kanade relies on a 3 x 3 neighborhood (that is, 9 pixels) around each feature. Taking each feature's neighborhood from the first frame, we try to find the best matching neighborhood in the second frame, based on least squares error. OpenCV's implementation of Lucas-Kanade uses...
