Optical Flow

Optical flow is how each point in the image appears to move from one frame to the next. Its pattern says a lot about how the camera is moving: flowing outward from a point when moving forward, sliding sideways when moving sideways, and sweeping evenly when turning. Drones and insects use it to dodge obstacles and judge speed.

Move forward Move sideways Turn (yaw) Add a near obstacle

The flow equation

For a camera moving with velocity $(v_x, v_y, v_z)$ and turning at $\omega_y$, a point at image position $(x, y)$, in normalized units, and depth $Z$ moves at

$$\dot x = \frac{-v_x + x\,v_z}{Z} - (1 + x^2)\,\omega_y \qquad \dot y = \frac{-v_y + y\,v_z}{Z} - x y\,\omega_y$$

The part from moving depends on depth. The part from turning does not.

What to notice

Moving forward, everything streams out from one point, the focus of expansion, which shows where the camera is heading. Near things flow faster, so the near obstacle stands out with long arrows: that difference in flow is how insects and some drones sense obstacles with one eye. Turning makes everything slide by nearly the same amount, whatever its depth, so turning gives no depth information. Algorithms like Lucas-Kanade estimate this flow from real images by tracking small patches.