Expectation Maximization (EM)

• A general purpose method for learning from incomplete data

Intuition:

• If we had access to counts, then we can estimate parameters

• However, missing values do not allow to perform counts

• “Complete” counts using current parameter assignment

1.30.41.71.6

X

Z

N (X,Y )

X

Y

#

HTHHT

Y

??HTT

T??TH

HTHT

HHTT

P(Y=H|X=T,?) = 0.4

Expected Counts

P(Y=H|X=H,Z=T,?) = 0.3

Data

Current

model

Previous slide

Next slide

Back to first slide

View graphic version