World Models

A world model learns to predict what will happen next given what the robot sees and does. With one, a robot can imagine the results of actions before taking them, or even train its policy entirely in imagination. But every prediction has small errors, and when the model predicts from its own predictions, those errors pile up.

Model accuracy How far ahead to imagine (steps) Correct with a real observation every 20 steps

Predicting from predictions

A world model $f_\theta$ predicts the next state from the current state and action. Imagining $H$ steps ahead means feeding its own predictions back in:

$$\hat s_{t+1} = f_\theta(\hat s_t, a_t), \quad \hat s_{t+2} = f_\theta(\hat s_{t+1}, a_{t+1}), \ \dots$$

A small one-step error $\epsilon$ can grow quickly, especially through events like bounces where a tiny difference in timing changes everything that follows.

What to notice

A good model's imagined paths stay close to reality for a while, then fan out, and the error curve climbs with the horizon. A poorer model fans out almost immediately. Correcting the imagination with a real observation every so often keeps the error bounded, which is what model-predictive control does by replanning. Modern world models, such as Dreamer, or video models like Genie and Cosmos, learn this from pixels, and the same compounding problem limits how far ahead they can be trusted.