Fine-Tuning Robot Foundation Models

Robot foundation models are pretrained on huge, varied datasets, then adapted to a specific robot or task with a little new data. Adapting means changing weights, and weights shared with old skills can be overwritten, called catastrophic forgetting. Methods differ in how many weights they change, and mixing some original data back in is a simple, effective guard.

Method Mix in 30 % original data (replay) New task examples Training steps

Three ways to adapt

Start from pretrained weights $W_0$. Full fine-tuning updates every weight. LoRA freezes $W_0$ and learns a low-rank change $\Delta W = BA$, with thin matrices $B$ and $A$, so far fewer parameters train. Last-layer-only keeps the network's features fixed and retrains just its output:

$$W = W_0 + BA, \qquad B \in \mathbb{R}^{d \times r},\ A \in \mathbb{R}^{r \times d},\ r \ll d$$

The model here is a small network with one nonlinear hidden layer, $y = \mathbf{b}^\top \tanh(W\mathbf{x})$, trained exactly, so you can watch real behavior.

What to notice

In this small example, all three methods fit the new task, and on their own all three forget a similar amount of the original skill, because each one changes weights that the original skill also uses. Even last-layer-only training, which changes just 10 numbers, forgets, since that one layer is shared by everything. The methods differ more in cost: LoRA trains a small fraction of the parameters, which makes it cheaper and lets many task adapters sit on top of one base model. Turning on replay, mixing original examples into each batch, cuts the forgetting for every method, and most of all for LoRA here. On large models the gaps between methods are bigger and depend on the task, so recipes for adapting models such as OpenVLA and π0 commonly combine LoRA with co-training on original data.