Object Detection: Boxes, IoU, and NMS

An object detector outputs many candidate boxes, each with a class and a confidence score. Most overlap: several boxes for the same object. Two steps clean them up: drop low-confidence boxes, then use non-maximum suppression to keep only the best box among each group that overlaps too much.

Confidence threshold NMS overlap limit (IoU) Apply non-maximum suppression

Intersection over union

IoU measures how much two boxes overlap, from 0 for no overlap to 1 for identical boxes:

$$\operatorname{IoU}(A, B) = \frac{\lvert A \cap B\rvert}{\lvert A \cup B\rvert}$$

Non-maximum suppression

Sort boxes by confidence. Take the best one, remove every remaining box whose IoU with it is above the limit, and repeat. A detection counts as correct if its IoU with a true object box is at least 0.5.

What to notice

Without NMS, each object has a cluster of boxes, so precision is poor. With NMS, each cluster shrinks to one. Set the overlap limit too low and two objects standing close together, like the two mugs here, merge into one detection. A high confidence threshold removes false alarms but starts missing real objects. That trade-off is what precision and recall measure.