Plane and Object Segmentation in Point Clouds

A robot looking at a tabletop sees one big flat surface and a few objects on it. A classic pipeline finds the table with RANSAC plane fitting, removes those points, and then groups what is left into objects by putting nearby points in the same cluster. Each cluster becomes something to recognize or grasp.

Plane inlier distance Cluster distance

Two steps

RANSAC picks a few random points, fits a plane (a line in this side view) through them, and counts how many points lie within a distance $\epsilon$. After many tries it keeps the best plane. Euclidean clustering then joins any two remaining points closer than $d$, and each connected group is an object:

$$\text{inlier: } \lvert \mathbf{n}^\top \mathbf{p} + c\rvert < \epsilon \qquad \text{same cluster: } \lVert \mathbf{p}_i - \mathbf{p}_j\rVert < d$$

What to notice

Too small an inlier distance leaves table points behind as noise, which then form fake little clusters. Too large and the bottoms of the objects get eaten as part of the table. A small cluster distance splits one object into pieces, and a large one merges two objects that sit close together. This pipeline, often run with PCL or Open3D, is still a fast, dependable baseline for tabletop grasping.