Monocular Depth Estimation

Neural networks can now guess depth from a single image, using cues like size, texture, and perspective. The result looks impressive, but it is usually only relative: right about what is nearer and farther, wrong about the actual meters. Stereo measures meters directly, but gets noisy far away.

Align the single-image depth with 3 known points Stereo baseline

Relative versus metric depth

Many single-image models predict depth only up to an unknown scale $s$ and shift $b$, often in inverse depth. If a few points have known depth, for example from LiDAR or a known object size, the two numbers can be fit by least squares:

$$Z_{\text{metric}} \approx s\,\hat Z + b \qquad \min_{s,\,b}\ \sum_{k} \big(Z_k - (s\,\hat Z_k + b)\big)^2$$

Stereo depth error grows with the square of distance, $\delta Z \approx Z^2 \delta d / (f B)$.

What to notice

Before alignment, the single-image curve has the right shape, with the same bumps and order, but it sits at the wrong scale. Three known points are enough to snap it close to the truth. Stereo is accurate up close but spreads out far away, especially with a small baseline. Robots often combine the two: stereo or LiDAR for metric scale, and a learned model to fill in gaps and textureless areas where stereo fails.