Color Segmentation: CS223b
Mitul Saha & Rohit Singh
MATLAB code organization:
trainOnData.m: Trains the one-gaussian-per-category model on the data.
The data is specified in terms of images, skin-color masks and
background-color masks . The data calculates model-parameters for 3
different color-spaces (RGB, HSV, HS) and stores them all in trainData.mat.
color_segment.m: This function takes an image-file and an optional
argument indicating the color-space to use. The default color-space is
RGB. It produces output in the desired format.
getgaussian.m: Calculate the probability of a k-dimensional point bein
generated by a k-variate gaussian with a specified mean and variance.
The images used for training skin color and background color were the
same and were:
P1010032_s.jpg, P1010040_s.jpg, P1010044_s.jpg, P1010053_s.jpg,
P1010055_s.jpg, P1010059_s.jpg, P1010063_s.jpg, P1010067_s.jpg,
P1010071_s.jpg, P1010072_s.jpg, P1010076_s.jpg, P1010079_s.jpg,
P1010082_s.jpg, P1010088_s.jpg, P1010090_s.jpg, P1010092_s.jpg,
The Mean and Covariance matrices are stored in trainData.mat in the
skinMean(1,:,:): mean for RGB
skinMean(2,:,:): mean for HSV
skinMean(2,:,1:2) mean for HS
and so on...similarly, for skinCov, bakgndMean and bakgndCov.
The basic observation we made was than RGB had a problem with intensity
variation. Also, it was rather conservative in its estimation of skin
pixels. HSV/HS, on the other hand, seemed to be *too* liberal in categorizing
pixels as skin color.
We tested on a lot of images. Here are the more interesting ones:
HSV seems to work better than RGB- nonuniform lighting on face:
RGB output(not so good):
In the following RGB (P1010054_s.jpg) seems to be the best: uniform lighting:
RGB output(seems better):
HSV output(not so good):
The following one (P1010037_s.jpg) was interesting because the student was
wearing a T-shirt with some colors that matched skin colors. RGB was too
conservative here- focusing only on the real skin but not recognizing all
parts of it equally well. HSV/HS did recognize the whole face, but they
also mis-recognized significant portions of the T-shirt. Its debatable
which one you consider better:
HSV output(too liberal):
RGB output(too conservative):
EXTRA CREDIT PART:
em.m: Runs EM on a mixture-of-K-gaussians model where K is user
specified (or 5). Runs for 20 iterations (10 iterations were not
enough, at times) or until the point when relative difference in the norm
of successive log-likelihood values goes below 0.001.
>> em ( 'image-file-name')
4 gaussians (or any number of gaussians):
>> em ( 'image-file-name', 4)
With 3 gaussians on P1010054_s.jpg, we get very good results.
EM output(3 gaussians):
Using more gaussians is not always good: on P1010039_s.jpg:
We first tried 5 gaussians, however the skin part seemed to get
split into two gaussians, a low intensity and a high intensity (too much
light) part. Then, we repeated EM with 4 gaussians. The results
were very good. We got one coherent skin-color region. It even detected
the watch!. see:
EM output(5 gaussians: not so good):
EM output(4 gaussians: just right!..detecting the watch!!!):