Visual-Inertial Odometry

A single camera can track how it moves, but not how far, since a small close scene and a large distant one look the same. An IMU measures real acceleration, with true scale, but drifts within seconds. Fused together, the camera stops the drift and the IMU supplies the scale.

True scene scale (unknown to the camera) Accelerometer bias Camera tracking noise

What each sensor knows

A monocular camera measures position only up to an unknown scale $s$. The accelerometer measures acceleration with a bias $b$. An EKF with state $(p, v, b, \lambda)$, where $\lambda = 1/s$, fuses them:

$$\dot p = v \qquad \dot v = a_{\text{IMU}} - b \qquad z_{\text{camera}} = \lambda\,p + \text{noise}$$

The camera reading is nonlinear in the state, since it multiplies $\lambda$ and $p$, so the filter uses the EKF's linearization.

What to notice

The camera-only position, gray, has the right shape but the wrong size. The IMU-only position, amber, has the right size at first but drifts away quickly, because the bias is integrated twice. The fused estimate, red, follows the truth, and the scale estimate settles on the true value after the robot has moved back and forth a few times, since motion is what makes scale observable. VIO runs on phones for AR, on drones, and on headsets.