Epipolar Geometry

When two cameras see the same scene, a point in one image does not tell you where its match is in the other, but it does narrow it down to a single line. That line comes from the geometry of the two cameras. It turns a 2D search for matches into a 1D search, which is what makes stereo matching fast.

Second camera offset Second camera turn Chosen point

Or click a point in the left image.

The epipolar constraint

For matching points $\mathbf{x}$ and $\mathbf{x}'$ in the two images, written with an extra 1, the fundamental matrix $F$ satisfies

$$\mathbf{x}'^\top F\,\mathbf{x} = 0, \qquad \boldsymbol\ell' = F\,\mathbf{x}$$

$\boldsymbol\ell'$ is the epipolar line in the second image. With known intrinsics $K$, $F = K^{-\top} [\mathbf{t}]_\times R\, K^{-1}$, where $R$ and $\mathbf{t}$ are the second camera's rotation and translation.

What to notice

The chosen point's true match always sits on its line, whatever its depth. All the epipolar lines in an image pass through one point, the epipole, which is where the other camera's center would appear. Slide the second camera straight sideways with no turn and the lines become horizontal, which is why stereo cameras are rectified: each match can then be found on the same row.