Likelihood Score for Structure (cont.)

Rearranging terms:

where

• H(X) is the entropy of X

• I(X;Y) is the mutual information between X and Y

  • I(X;Y) measures how much “information” each variables provides about the other
  • I(X;Y) ? 0
  • I(X;Y) = 0 iff X and Y are independent
  • I(X;Y) = H(X) iff X is totally predictable given Y

Previous slide

Next slide

Back to first slide

View graphic version