Optical flow

Optical flow is an algorithm that detects the pattern of the motion of objects, or edges, between consecutive frames in a video. This motion may be caused by the motion of the object or the motion of the camera. Optical flow is a vector that depicts the motion of a point from the first frame to the second.

The optical flow algorithm works under two basic assumptions:

• The pixel intensities are almost constant between consecutive frames

• The neighboring pixels have the same motion as the anchor pixel

We can represent the intensity of a pixel in any frame by f(x,y,t). Here, the parameter t represents the frame in a video. Let's assume that, in the next dt time, the pixel moves by (dx,dy). Since we have assumed that the intensity doesn't change in consecutive frames, we can say:

f(x,y,t) = f(x + dx,y + dy,t + dt)

Now we take the Taylor series expansion of the RHS in the preceding equation:

Cancelling the common term, we get:

Where .

Dividing both sides of the equation by dt we get:

This...