CS223b: Computer Vision

HW#2 : Stereo

Rohit Singh [rohitsi@stanford.edu]

Mitul Saha [mitul@stanford.edu]

Part 1: Correlation-based stereo

Here is the image with maxDisp = 15, windowSize = 5, SSD and no filtering:
Left Image
Right Image
Disparity Map
Uniform Intensity regions
The biggest problem seems to happen when a large region has (roughly) the same intensity for all pixels in it. In this case, intensity based error minimization might pick very small differences and assign somewhat arbitrary disparities to the region This seems to be the case with the blank wall in the upper-right corner , the glare in the bookshelfs and the shadow between the bust and the table. Obviously, in such a case, a belief-propagation like approach might help.

Maximum Disparity Threshold (maxDisp)
If maxDisp is set too low for the image, the algorithm can?t differentiate well between the objects that are closer than a threshold minimum distance (dependent on the maxDisp). For example with maxDisp = 8, this is the result we get:

Observe that the lamp and the bust are too close for the current disparity limit. More often than not, the maximum disparity (i.e. 6) gives the lowest error for such cases so that most of these are colored white. However, at times, the error given by a lower disparity (say 0) would be lower than the error from high disparity (say 6) so that low disparity is chosen for that region. This is especially true of regions where there isn't too much intensity variation and its easy for the algorithm to get confused.

SSD is more sensitive than SAD (because it squares the difference). However, it also seems to be more susceptible to noise. Look at the following:
Look at the small black dot in bust in the SSD image. It is not present in the SAD image. Similarly the black spot on the tripod stand is smaller for SAD. Due to squaring, large error terms win over (many) small errors and hence even a single noisy pixel can screw things.

Windowsize seems to have a blurring effect on the disparity map. Increasing window size helps handle noise better, however increasing window size also blurs the disparity map, thus reducing accuracy. The following is a disparity map for windowSize = 20:

Laplacian of Gaussian
This operator basically extracts edge information. Running the algo on such an image gave decent results for the edges but terrible results for the (uniform intensity) portion between the edges [Pls refer to the earlier discussion about problems with regions of uniform intensity].
Left Image after Laplacian of Gaussian
The disparity map of filtered image

Performance on the Lamp Image
The performance on the lamp image is very bad: Here is the result with maxDisp = 15, windowSize = 5, SSD and no filtering:
Left Image
Right Image
Disparity Map
Clearly, the reason for the bad performance is the existence of large regions on uniform intensity. The disparity calculations for the edges seem to be correct. However, it is in the regions between the edges where the algorithm gets confused, since even small differences in intensity can change the disparity.

Extra Credit Part
In correlation_match, support for 'dir' the 4th argument has been added. Similarly, the function correlation_match2 has also been implemented. However, the results of left-right consistency check are not encouraging- there are many pixels where the disparity values are not consistent

Discussion of Other Approaches
The belief propagation algo seems to have a lag, i.e. it doesn't change its estimate of the disparity until it has seen some pixels from the region with the new disparity. Obviously, as such the boundaries of objects are not crisp/accurate.Also it didn't handle the upper-right corner (uniform intensity region) well

We felt that the Graph Cuts + Occlusion algo (ranked 2, number 4) was better. The edges were much crisp and the whole disparity map seemed to be free of noise. Moreover, regions of uniform intensity were very well handled. Obviously, the criteria to determine which algorithm is better are not only subjective (to the human being evaluating) but the criteria will differ for different sorts of cases. Dynamic programming seems to be introducing a lot of horizontal streaks in the disparity map. The realtime algorithm is ranked 12 in the given set.

Matlab Codes