Likelihood Score for Structure (cont.)
Intuitive explanation of likelihood score:
- The larger the dependency of each variable on its parents, the higher the score
- Likelihood as a compromise among dependencies, based on their strength
Adding arcs always helps
- I(X;Y) ? I(X;Y,Z)
- Maximal score attained by “complete” networks
- Such networks can overfit the data --- the parameters they learn capture the noise in the data