Structure from Motion

Structure from motion recovers both the 3D shape of a scene and the path of the camera from nothing but images. It matches features between views, works out how the camera moved, triangulates the points, and refines everything. One thing it cannot recover from images alone is the true size of the scene.

Camera views used Assumed distance between first two views Bearing noise

The pipeline

Match features across images. From two views, estimate the relative camera motion, for example with the essential matrix. Triangulate each matched point by intersecting its rays. Add more views by locating each new camera against the points, then run bundle adjustment over everything.

Scale ambiguity

Images measure only directions. If the whole scene and the whole camera path were twice as large, every image would look exactly the same:

$$\pi(s\,T,\ s\,\mathbf{p}) = \pi(T,\ \mathbf{p}) \quad \text{for any scale } s > 0$$

What to notice

Change the assumed distance between the first two views and the whole rebuilt scene grows or shrinks around the first camera, but its shape stays the same. A robot fixes the scale with something extra: a stereo camera with a known baseline, an IMU, wheel odometry, or an object of known size. With only two nearby views, the rays to far points are almost parallel, so a tiny bearing error moves the triangulated point a long way in depth, and the worst point can be meters off. More views, spread further apart, cross each point's rays at wider angles, so the shape gets much cleaner.