The Pinhole Camera Model

Almost all robot vision starts from one simple model: light from a 3D point passes through a single tiny hole and lands on the image plane. Dividing by depth is what makes far things look small.

Focal length (pixels) Cube distance Z Cube offset sideways X Cube turn

Projection

A point $(X, Y, Z)$ in the camera frame, with $Z$ pointing forward, lands at pixel $(u, v)$:

$$u = f_x\,\frac{X}{Z} + c_x \qquad v = f_y\,\frac{Y}{Z} + c_y \qquad \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} \sim \underbrace{\begin{bmatrix} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix}}_{K} \begin{bmatrix} X \\ Y \\ Z \end{bmatrix}$$

$K$ is the intrinsic matrix. $f_x$ and $f_y$ are the focal length in pixels, and $(c_x, c_y)$ is the image center.

What to notice

Double the distance and the cube looks half as big. Raise the focal length and it grows, like zooming in, but the field of view shrinks. In the side view, the real image behind the pinhole is upside down, so cameras flip it, which is the same as putting the image plane in front of the hole.

Dividing by $Z$ also loses information: a small near cube and a big far cube can make the same image. Recovering depth takes a second view, a known object size, or a depth sensor.