Description
For Assignment 3, you will implement a few functions in a stereo pipeline example
provided by mathworks.com
Provided files:
In your Matlab Command Window, type
>> edit DepthEstimationFromStereoVideoExample
What You Have to Do:
Task 1 (30 points): Calculate disparity using the SSD algorithm
Implement the SSD matching algorithm for a range of window sizes, and create a
disparity image D(x, y) such that Left(x, y) = Right(x + D(x, y), y) when matching
from left to right. Remember to reduce the search only over epipolar lines. You
should also restrict the search using the disparity range (see line 56 in the original
script).
Write a function to replace the disparity built-in function on line 54. Your function
needs to accept as an input value the size of the window search. Call your function
three times for the following window sizes: 1, 3, 5.
Create a 2×2 subplot and display the disparity map results from these three function
calls, plus the one obtained using Matlab’s built-in disparity function.
Note: when using a window size bigger than 1, it is common practice to use a
Gaussian weighting of the window, so that the center has greater influence than the
periphery. Convolve your window matrices with a Gaussian filter prior to
computing the sum of squared differences.
Task 2 (30 points):
Calculate disparity using the NCC algorithm
Implement the normalized cross correlation (NCC) matching algorithm for a range
of window sizes, and create a disparity image D(x, y) such that Left(x, y) = Right(x +
D(x, y), y) when matching from left to right. Similarly, remember to reduce the
search only over epipolar lines and to restrict the search using the disparity range
(see line 56 in the original script).
Information on how to compute NCC for image-processing applications here:
https://en.wikipedia.org/wiki/Cross-correlation#Normalized_cross-correlation
The inner product of vectors approach mentioned on Wikipedia, also here, slide 66:
http://www.gris.tu-darmstadt.de/teaching/courses/ws0910/cv2_ws0910/slides/l3-stereo-v1_0.pdf
Write a function to replace the disparity built-in function on line 54. Your function
needs to accept as an input value the size of the window search. Call your function
three times for the following window sizes: 3, 5 and 7.
Create a 2 x 2 subplot and display the disparity map results from these three
function calls, plus the one obtained using Matlab’s built-in disparity function.
Task 3 (30 points):
Generate outliers map – Refine the disparity by performing a
left-right consistency check.
The disparity map dLR(x) is acquired considering as reference image the left image
of the stereo pair. If the right image is considered as reference, then the disparity
map dRL(x) is acquired. The disparity maps dLR(x) and dRL(x) can be useful in
detecting problematic areas, especially outliers in occluded regions and depth
discontinuities. One strategy for detecting outliers is the Left–Right consistency
check introduced by 1.
In this strategy, the outliers are disparity values that are not
consistent between the two maps and therefore, they do not satisfy the relation:
| dLR(x) −dRL(x−dLR(x)) | ≤ TLR, where TLR is a user-defined threshold.
Write a function that accepts as inputs two disparity maps (LR and RL) and a value
for TLR and returns a binary image where the outliers have the value 1 and the
inliers have the value 0.
Call this function twice, passing as inputs the LR and RL disparity maps from SSD
and NCC matching with window size of 3. Use a TLR value of 1. With this value, pixels
with difference equal to 1 in the Left–Right consistency check are not considered
outliers. Plot the two resulting binary images side by side.
Task 4 (10 points): Compute depth from disparity
Now that we have a disparity image, computing depth at every pixel in the left image
should be very straightforward. Use equation 11.1 from the Szeliski textbook.
Write a function to replace the reconstructScene built-in function on line 64. Your
function should have the same input arguments as the built-in version and it should
return a matrix of depth values for each pixel location from the left image.
Submitting the assignment:
Make sure each script or function file is well commented and it includes a block
comment with your name, course number, assignment number and instructor name.
Zip all the .m and image files together and submit the resulting .zip file through
Moodle as Assignment 2 by Monday, October 17th, by 11:55pm.
1 M. Humenberger, T. Engelke, W. Kubinger, A census-based stereo vision algorithm
using modified semi-global matching and plane-fitting to improve matching quality,
CVPR ECV Workshop, 2010.
