The Pick and Place Pipeline

Picking up an object and putting it somewhere else takes a chain of steps, each depending on the last: find the object, choose a grasp, plan an approach, close the gripper, lift, carry, and place. Real systems also check after each step, because a failure early on ruins everything after it.

The object gets bumped during the approach Check the result after each stage Speed

The stages

Perceive: detect the object and estimate its pose. Grasp: choose where and how to hold it. Approach: plan a collision-free path to a pre-grasp pose just above it. Grasp: move in and close. Lift and carry: plan to above the target. Place: lower and open. In MoveIt or a similar framework, each stage is a separate call that can fail:

$$P(\text{task succeeds}) = \prod_{k} P(\text{stage } k \text{ succeeds})$$

What to notice

Turn on the bump with checking off: the arm closes on empty air, carries nothing, and reports success anyway. With checking on, the gripper notices it closed too far, goes back to perceiving, and recovers. Because success multiplies across stages, even 95 percent per stage gives only about 70 percent for seven stages, so checking and retrying is what makes pick and place reliable.