Object Tracking

A detector finds objects in each frame on its own, with no idea which object is which from one frame to the next. A tracker links them over time: it predicts where each track should be now, matches new detections to those predictions, and starts or ends tracks as objects appear and disappear.

Use motion prediction Missed detections Detection noise Matching gate

Predict, then match

Each track keeps a position and velocity, often in a Kalman filter. Each frame it predicts the next position, then assigns detections to tracks by solving a matching problem on the distances, usually with the Hungarian algorithm, ignoring pairs farther than a gate:

$$\hat{\mathbf{x}}_{t} = \mathbf{x}_{t-1} + \mathbf{v}_{t-1}\,\Delta t \qquad \min \sum_{(i,j)} \lVert \hat{\mathbf{x}}_i - \mathbf{z}_j \rVert \ \text{ over one-to-one pairs}$$

This page matches greedily, closest pair first, which is a common simplification. SORT and its successors use exactly this structure.

What to notice

When two objects cross, a tracker without motion prediction often swaps their identities, since each detection is closest to the wrong track's old position. With prediction, the tracks carry their momentum through the crossing. Missed detections are survived by coasting on the prediction for a few frames. Too small a gate drops tracks during fast moves, and too large a gate lets tracks steal each other's detections.