Part 1: Correlation-based stereo

Left Image	Right Image

Disparity Map

Uniform Intensity regions

The biggest problem seems to happen when a large region has (roughly) the same intensity for all pixels in it. In this case, intensity based error minimization might pick very small differences and assign somewhat arbitrary disparities to the region This seems to be the case with the blank wall in the upper-right corner , the glare in the bookshelfs and the shadow between the bust and the table. Obviously, in such a case, a belief-propagation like approach might help.

Maximum Disparity Threshold (maxDisp)

If maxDisp is set too low for the image, the algorithm can?t differentiate well between the objects that are closer than a threshold minimum distance (dependent on the maxDisp). For example with maxDisp = 8, this is the result we get:

Observe that the lamp and the bust are too close for the current disparity limit. More often than not, the maximum disparity (i.e. 6) gives the lowest error for such cases so that most of these are colored white. However, at times, the error given by a lower disparity (say 0) would be lower than the error from high disparity (say 6) so that low disparity is chosen for that region. This is especially true of regions where there isn't too much intensity variation and its easy for the algorithm to get confused.

SAD vs SSD

SSD is more sensitive than SAD (because it squares the difference). However, it also seems to be more susceptible to noise. Look at the following:

SSD	SAD

Look at the small black dot in bust in the SSD image. It is not present in the SAD image. Similarly the black spot on the tripod stand is smaller for SAD. Due to squaring, large error terms win over (many) small errors and hence even a single noisy pixel can screw things.

WindowSize

Windowsize seems to have a blurring effect on the disparity map. Increasing window size helps handle noise better, however increasing window size also blurs the disparity map, thus reducing accuracy. The following is a disparity map for windowSize = 20:

Laplacian of Gaussian

This operator basically extracts edge information. Running the algo on such an image gave decent results for the edges but terrible results for the (uniform intensity) portion between the edges [Pls refer to the earlier discussion about problems with regions of uniform intensity].

Left Image after Laplacian of Gaussian	The disparity map of filtered image

The performance on the lamp image is very bad: Here is the result with maxDisp = 15, windowSize = 5, SSD and no filtering:

Left Image	Right Image

Disparity Map

Clearly, the reason for the bad performance is the existence of large regions on uniform intensity. The disparity calculations for the edges seem to be correct. However, it is in the regions between the edges where the algorithm gets confused, since even small differences in intensity can change the disparity.