Things and Stuff (TAS) Model Learning Package
Last Updated August 15, 2008
by Geremy Heitz
Stanford University

Hello,

Thanks for downloading the TAS package.  This package includes two things:

1) All the code you need to run the algorithm from our TAS paper (ECCV
2008, see citation below).  2) A small example dataset that this
algorithm can be run on.

This README provides a walkthrough for getting this package to work.
Hopefully, substituting your own data in will be relatively easy once
you have it working.  Please send any questions/concerns/comments to
Geremy Heitz at gaheitz@stanford.edu.

*******************************************
         WALKTHROUGH
*******************************************

**** STEP 0 - Data Setup

This method assumes that you have a set of images, that these images
have already been segmented into superpixel regions, and that you have
run an object detector on these images to pull out the candidate
objects of interest.  If you need some code to give superpixels, I
point you to the Ren&Malik superpixel code available at
http://www.cs.sfu.ca/~mori/research/superpixels.  For a good
off-the-shelf detection algorith, I recommend the HOG Detector of
Dalal&Triggs, available at http://pascal.inrialpes.fr/soft/olt. Let's
assume that you are running your experiments from a base directory
called BASE_DIR from here on.  For each image that you wish to use,
and each object class you have, you should have:

*BASE_DIR/Data/Images/<image_name>.jpg - The jpg images that you wish
to use *BASE_DIR/Data/Regions/<image_name>_S.mat - The regions in a
segmentation mask with filenames given by <image_name>_S.mat
*BASE_DIR/Data/Regions/<image_name>_FTR.mat - The features for the
regions.  This is an M x D matrix where each row corresponds to a
region and each column to a feature.
*BASE_DIR/Data/Candidates/<class>/<image_name>.cand - A file with the
candidate detections.  The format of each line in this file is: BBleft
BBtop BBwidth BBheight detector_score

**** STEP 1 - Params & Code

Open matlab from the "example" directory.  First, add the codebase to
your path using:

         init;

Load in your parameters file, which gives constants, and tells which
mode to learn in:

         run('Params/search.m');

You can look into this parameters file, and the fields should have
comments.  Feel free to tweak these to see what effect they have.
         
**** STEP 2 - Load Data

Load your data into the data structure used for TAS processing:

          train_data = load_tas_data('cars', 'Data', tas_params, tas_params.train_indices)
          
Once you have done this, you can visualize the detections to make sure
that everythign looks right:

          view_detections(train_data); 
          
**** STEP 2 - Learn TAS

Now you will learn the model.  On my computer, this takes about 1 hour.

         model = learn_tas_em_indicators_structure(train_data, tas_params);

**** STEP 3 - Test it

Once you have learned a model, you can "apply" it.  We can start by
making sure that it improves the results on the training set (about 2
minutes):

         train_tas  = infer_tas_indicators(train_data, model, tas_params, infer_params);

We can look at the results in train_tas with the following commands:

         prior_pr = det_rpc(train_tas,train_tas.scores,tas_params.truth_threshold,'r');
         posterior_pr = det_rpc(train_tas,train_tas.pl,tas_params.truth_threshold,'b');

The blue curve, which represents the posterior (TAS result) should
have a better (higher) PR curve than the red curve (detector only).
With this verified, you can now run on the test set:

         test_data = load_tas_data('cars', 'Data', tas_params, tas_params.test_images);
         test_tas  = infer_tas_indicators(test_data, model, tas_params, infer_params);
         figure;
         prior_pr = det_rpc(test_tas,test_tas.scores,tas_params.truth_threshold,'r');
         posterior_pr = det_rpc(test_tas,test_tas.pl,tas_params.truth_threshold,'b');

You can also look at the actual detections:

         view_detections(test_tas, 3, model, 0.2);

The second parameter is what to score the detections based on.  

*********************************************************
Congratulations, you have just run TAS.  Here are those commands again in order:
         init;
         run('Params/search.m');
         train_data = load_tas_data('cars', 'Data', tas_params, tas_params.train_images);
         model = learn_tas_em_indicators_structure(train_data, tas_params);
         train_tas  = infer_tas_indicators(train_data, model, tas_params, infer_params);
         prior_pr = det_rpc(train_tas,train_tas.scores,tas_params.truth_threshold,'r');
         posterior_pr = det_rpc(train_tas,train_tas.pl,tas_params.truth_threshold,'b');
         test_data = load_tas_data('cars', 'Data', tas_params, tas_params.test_images);
         test_tas  = infer_tas_indicators(test_data, model, tas_params, infer_params);
         figure;
         prior_pr = det_rpc(test_tas,test_tas.scores,tas_params.truth_threshold,'r');
         posterior_pr = det_rpc(test_tas,test_tas.pl,tas_params.truth_threshold,'b');
*********************************************************

Again, please let me know if you need any help.  Here is the paper
that you should cite if you use this method as a baseline or to
reference it in general:

** Geremy Heitz and Daphne Koller. "Learning Spatial Context: Using
Stuff to Find Things." ECCV, 2008.

Well done on making it this far.  Here is my contact info:

Geremy Heitz,
gaheitz@stanford.edu