Vision-Language-Action (VLA) Models

A vision-language-action model starts from a vision-language model and trains it to output robot actions as well as text. Given a camera image and an instruction like "put the apple in the bowl", it produces the next motion. One early approach turns each action dimension into a discrete token, like a word. Newer models add a continuous action head for smoother, faster control.

Action output Bins per action dimension (tokens) Control rate needed

Actions as tokens

Models such as RT-2 and OpenVLA split each action dimension's range into $B$ bins, often 256, and treat each bin as a token in the vocabulary. A 7-number action, position and rotation changes plus the gripper, becomes 7 tokens generated one after another:

$$\text{token}_j = \Big\lfloor B\,\frac{a_j - a_{\min}}{a_{\max} - a_{\min}} \Big\rfloor, \qquad \text{error} \le \frac{a_{\max} - a_{\min}}{2B}$$

Continuous heads, such as π0's flow matching action expert, output whole chunks of real-valued actions at once.

What to notice

With few bins, the discretized trajectory is a staircase, too coarse for fine work. More bins shrink the error, but each action still takes several tokens, generated one at a time, which limits the control rate for a large model. The continuous head is smooth and outputs a whole chunk in one pass, which is why newer VLAs can control at 50 Hz. The throughput numbers on this page are illustrative.