Supervised Learning, Overfitting, and Validation

Supervised learning fits a model to examples with known answers. The danger is overfitting: a model flexible enough to pass through every training point also fits the noise, and then predicts badly on new data. Holding out some examples as a validation set reveals this, because the model never trains on them.

Model complexity (polynomial degree) Training examples Noise in the data

Train, validate, test

Fit the model by minimizing error on the training set, then measure error on separate validation data:

$$\text{train error} = \frac{1}{n}\sum_{i \in \text{train}} (y_i - \hat y_i)^2 \qquad \text{validation error} = \frac{1}{m}\sum_{j \in \text{val}} (y_j - \hat y_j)^2$$

Choose the model complexity where validation error is lowest. A final test set, used only once, gives an honest estimate of real performance.

What to notice

Training error always falls as complexity grows. Validation error falls at first, then rises once the curve starts chasing noise between the points, the classic U shape. More training data pushes the overfitting point to higher complexity. The same trap applies to robot policies: a policy that is perfect on its demonstrations can fail badly in a new kitchen, which is why robot learning results should always be judged on held-out scenes.