Avoiding Overfitting

“Classic” issue in learning.

Standard approaches:

• Restricted hypotheses

  • Limits the overfitting capability of the learner
  • Example: restrict # of parents or # of parameters

• Minimum description length

  • Description length measures complexity
  • Choose model that compactly describes the training data

• Bayesian methods

  • Average over all possible parameter values
  • Use prior knowledge

Previous slide

Next slide

Back to first slide

View graphic version