Visual Servoing

Visual servoing closes the control loop through the camera. Instead of planning a motion and executing it blind, the robot keeps comparing what it sees with what it wants to see, and moves to shrink the difference. Image-based servoing works directly on feature positions in the image. Pose-based servoing first estimates the 3D pose, then controls that.

Method Gain λ Starting rotation error

Image-based servoing

Stack the feature positions into $\mathbf{s}$ and the desired ones into $\mathbf{s}^*$. The interaction matrix $L$ says how features move when the camera moves with velocity $\mathbf{v}_c$. The control law drives the error to zero exponentially:

$$\dot{\mathbf{s}} = L\,\mathbf{v}_c \qquad \mathbf{v}_c = -\lambda\,L^{+}\,(\mathbf{s} - \mathbf{s}^*)$$

This page estimates $L$ numerically for a camera that can move in x, y, height, and rotate about its axis.

What to notice

With image-based servoing, the features move along nearly straight lines in the image, so they stay in view, but the camera itself may take a curving path, and with large rotation errors it backs away before closing in. Pose-based servoing gives a straight camera path, but the features can curve out of the image. Visual servoing is used for docking, insertion, and catching moving targets, where accumulated errors from blind motion would be too large.