Alireza Fathi's Egocentric Vision Toolbox
Table of Contents:
Download the Source Code (Beta 09/Aug/2013) here.
README
In order to compile this code and use all its capabilities, you need to have the following libraries installed and hardware available:
- CUDA
- OpenCV compiled with GPU option ON
- Matlab
- VL Feat
Spectral Clustering Functions:
- matrixToD
- getLaplacianMatrix
- getNormalizedLaplacianMatrix
- normalizedSpectralClustering
- unnormalizedSpectralClustering
Image Processing Operations:
Matrix Operations:
- repmat
- powMatrix
- expMatrix
- subMatrix
- indMatrixRows
- indMatrixCols
- matrixMaxValue
- matrixMinValue
- matrixMaxValueDiagonal
- matrixMinValueDiagonal
- matrixMinValueGreaterThanZero
Miscellaneous Functions:
- randsample
- intersectHist
- timeToFrame
Write Functions:
- writeMatrixToFile
- writeMaskAsImage
Read Functions:
- readMatrixFromFile
- readMaskFromImage
Functions:
- writeMatrixToMatFile
- readFloatMatrix
- readBSDSegmentation
- readUCM
Functions:
- eigs
- matrixToD
- getLaplacianMatrix
- getNormalizedLaplacianMatrix
- normalizedSpectralClustering
- unnormalizedSpectralClustering
- printMatrix
Functions:
- getDenseFlow_eccv04 (borx on gpu)
- affineEstimate
- affineMotionRansac
- getMotionChannels_spectralClustering
- getMotionChannels_clustering
- xiaofeng_getInliersOpticalFlowAffine
- getBackgroundMaskPb
Grouping Functions:
- felzenszwalb_segment
- meanShiftSegmentation
- berkeley_segment
- graphCut
Post-processing regions:
- getContours
- getContoursBetweenTwoRegions
- getSegmentCenters
- getWeightedAdjacencyMatrix
Functions:
- getColorHistogram
- getHog
- getColorLab
- getHierarchicalFeature
- featureAroundPoint
Functions:
- harmonicSemiSupervised: It is used for interactive and semi-supervised object segmentation. Refer to Fathi et al., BMVC 2011 for further detail.
Reading and Processing Annotations:
- readActionAnnotations_SMI
Dataset class:
- getScale
- getFps
- getImType
- getSequence
- getDatasetName
- getDataDir
- getTrainSeqs
- getTestSeqs
- getImagePath
- getContourPath
- getGTObjectMaskPath
- getGazePath
- getObjectNames
- getActions
- getGazeLocations
- frequentActionsInTraining
- frequentActionsInTesting
- frequentActions
- addSequence
- setTrainSeqs
- setTestSeqs
- pushObjectName
DatasetSeq Class
- getSeqLength
- getStartFrame
- getEndFrame
- getSeqName
Prior Class get Functions:
- Prior(image, frameNumber): constructor that uses Felzsenswalb segmentation
- Prior(image, ucm, k, frameNumber): constructor that uses Berkeley segmentation
- getImage: returns the image
- getFrameNumber: returns the frame number
- getSegmentation: returns the image segmentation
- getNumSegments/getM: returns the number of segmented regions
- getCenters: returns the (y,x) center of segmented regions
- getAdjacencyWeights: returns a sparse adjacency matrix that contains the similarity of regions to each other
- getAdjacencyLengths: returns a sparse adjacency matrix that contains the length of the contour between neighbor regions
- getW: returns a sparse pairwise matrix that can be given to graphcut for image/video segmentation
- getPrior_size: returns a vector containing the sizes of segmented regions
- getPrior_location: given a learned location pattern, assigns a score to each segmented region
- getPrior_motionChannel: assigns a score to each segmented region based on its likelihood of belonging to foreground
- getColorMap: assigns a color bin to each pixel of the image
- getHogMap: assigns a HOG bin to each pixel of the image
- getFeatureAroundPoint_CLab: returns the histogram of color bins appearing in a circle around a given point
- getFeatureAroundPoint_HOG: returns the histogram of HOG bins appearing in a circle around a given point
- maskToLabel: given an image mask, assigns a label to each segmented region
- labelsToMask: given a set of labels assigned to segmented regions, returns an image mask
Pairwise Class Functions:
- Pairwise(frame1, frame2): constructor
- getImage1: returns the image of the first frame
- getImage2: returns the image of the second frame
- getSegmentation1: returns the segmentation of the first frame
- getSegmentation2: returns the segmentation of the second frame
- setFlow: computes the dense Brox optical flow between two frames
- getVX: returns the horizontal optical flow motion vectors
- getVY: returns the vertical optical flow motion vectors
- getFlow: return the flow vectors
- getTransitionFlow_unnormalized: returns a sparse matrix that contains the number of corresponding pixels between every pair of regions in the first and second frames
- getTransitionFlow_normalized: normalized version of getTransitionFlow_unnormalized, sum of the values assigned to each region in the second frame is 1
- transferFrom1to2: given a prior on segmented regions of the first frame, transfers it to the second frame and returns a new prior on segmented regions of the second frame
- computeTemporal: returns a sparse weighted matrix that is ready to be given to graphcut for video segmentation
Sequence Class Functions:
- Sequence(dataDir, seqName, imType, startFrame, endFrame, seg_type, resizeFactor): constructor. A sequence contains a chain of frames, starting at startFrame and ending at EndFrame.
- getStartFrame: returns the startFrame
- getEndFrame: returns the endFrame
- push_back: adds a frame (Prior) to the end of the sequence
- push_front: adds a frame (Prior) to the beginning of the sequence
- getPrior(frame): returns the Prior object corresponding to frame
- getPairwise(frame1, frame2): returns the Pairwise object for frame1 and frame2
- pop_back: removes the last frame
- pop_front: removes the first frame
- getLength: returns the length of the sequence
- transferPriorFromFrameToFrame: transfers a prior through chain of optical flow motion from one frame to another
- computeW: returns a sparse matrix that is ready to be given to graphcut for video segmentation
Foreground Segmentation Functions:
- batch_fgSegmentation(datasetName, seqNumber, startFrame, endFrame, train_version): segments foreground out of background in images of the seqNumber sequence. Refer to Ren and Gu, CVPR 2010 and Fathi et al., CVPR 2011 for further detail.
Object Classification Functions:
- trainingDataHierarchicalObjectClassifier: Loads the training data from object mask annotations. For each segmented region it computes hierarchical color and hog features. Refer to Fathi et al., ECCV 2012 for further detail.
- trainHierarchicalObjectClassifier: Trains a linear SVM classifier on hierharchical features computed at each segment.
- hierarchicalObjectClassification: Runs the learned object classifier
Action Classification Functions:
- trainActionClassifier_circle: trains an action classifier by learning from the appearance of a circle around the gaze point in training videos.
- actionClassification_circle: classifies actions based on the appearance of a circle around the gaze point. Refer to Fathi et al., ECCV 2012 for further detail.
Functions:
- cvShowManyImages
- showSegmentation
- showConfusionMatrix
- showSegmentationConfusionMatrix
- showContours
- showContoursOnImage
- showFzsbSegmentation
- showNonzeroElementsMatrix
- showSegmentationRandom
- showParticularRegion
- showAdjacencyMatrix
- showSpatialOnContours
- showPrior
- showFloatMat
- showLabels