HomeWork 3

Color Segmentation: CS223b

Mitul Saha & Rohit Singh

Part 1

MATLAB code organization:

trainOnData.m: Trains the one-gaussian-per-category model on the data. The data is specified in terms of images, skin-color masks and background-color masks . The data calculates model-parameters for 3 different color-spaces (RGB, HSV, HS) and stores them all in trainData.mat.

color_segment.m: This function takes an image-file and an optional argument indicating the color-space to use. The default color-space is RGB. It produces output in the desired format.

getgaussian.m: Calculate the probability of a k-dimensional point bein generated by a k-variate gaussian with a specified mean and variance.

Training Data:

The images used for training skin color and background color were the same and were:

P1010032_s.jpg, P1010040_s.jpg, P1010044_s.jpg, P1010053_s.jpg,
P1010055_s.jpg, P1010059_s.jpg, P1010063_s.jpg, P1010067_s.jpg,
P1010071_s.jpg, P1010072_s.jpg, P1010076_s.jpg, P1010079_s.jpg,
P1010082_s.jpg, P1010088_s.jpg, P1010090_s.jpg, P1010092_s.jpg,
P1010096_s.jpg, P101000_s.jpg

The Mean and Covariance matrices are stored in trainData.mat in the current directory.

skinMean: 3x1x3:
skinMean(1,:,:): mean for RGB
skinMean(2,:,:): mean for HSV
skinMean(2,:,1:2) mean for HS
and so on...similarly, for skinCov, bakgndMean and bakgndCov.

Test Data:

The basic observation we made was than RGB had a problem with intensity variation. Also, it was rather conservative in its estimation of skin pixels. HSV/HS, on the other hand, seemed to be *too* liberal in categorizing pixels as skin color.

We tested on a lot of images. Here are the more interesting ones:
P1010082_s.jpg:

HSV seems to work better than RGB- nonuniform lighting on face:
RGB output(not so good):

HSV output(good):

In the following RGB (P1010054_s.jpg) seems to be the best: uniform lighting:

RGB output(seems better):

HSV output(not so good):

The following one (P1010037_s.jpg) was interesting because the student was wearing a T-shirt with some colors that matched skin colors. RGB was too conservative here- focusing only on the real skin but not recognizing all parts of it equally well. HSV/HS did recognize the whole face, but they also mis-recognized significant portions of the T-shirt. Its debatable which one you consider better:

HSV output(too liberal):

RGB output(too conservative):

EXTRA CREDIT PART:

MATLAB file:

em.m: Runs EM on a mixture-of-K-gaussians model where K is user specified (or 5). Runs for 20 iterations (10 iterations were not enough, at times) or until the point when relative difference in the norm of successive log-likelihood values goes below 0.001.

To RUN:

5 gaussians(default):
>> em ( 'image-file-name')
or
4 gaussians (or any number of gaussians):
>> em ( 'image-file-name', 4)

RESULTS

With 3 gaussians on P1010054_s.jpg, we get very good results.

EM output(3 gaussians):

Using more gaussians is not always good: on P1010039_s.jpg:

We first tried 5 gaussians, however the skin part seemed to get split into two gaussians, a low intensity and a high intensity (too much light) part. Then, we repeated EM with 4 gaussians. The results were very good. We got one coherent skin-color region. It even detected the watch!. see:

EM output(5 gaussians: not so good):

EM output(4 gaussians: just right!..detecting the watch!!!):