Description
This is a programming assignment. You are asked to experiment with Selective Search on the given images
with groundtruth bounding boxes. We suggest you use OpenCV with version 4.1.0 or later, since some
functions may not be implemented in earlier versions.
1 Image Data
The image data is in CSCI 677 HW2 Code.zip folder in the DEN class page. Three color images are in
“JPEGImages” folder; their corresponding ground truth bounding boxes are in the “Annotations” folder.
An example of the bounding box coordinates (saved in .xml file) is in the following:
where the object label is “aeroplane”, and the bounding box is described in the format of (xmin, ymin, xmax,
ymax) = (68, 76, 426, 209), Namely, the left-top and right-bottom points of the box. You can visualize it by
using cv2.rectangle function. We provide the function parse annotation to load the groundtruth annotations.
This function returns a list of bounding boxes for the groundtruth.
2 Code
We provide a framework of code in the compressed file along with the images. Please finish the TODO parts
of this code. You need to implement the following functions.
2.1 Use OpenCV to read an image.
2.2 Complete the selective search function. You may follow the instructions in the comment of the code.
2.3 Finish the function of calculating IoU between two bounding boxes.
2.4 For each image, calculate the “recall” of groundtruth (GT) bounding boxes by computing the fraction
of GT boxes that are overlapped with at least one proposal box with Intersection over Union (IoU)
≥ 0.5.
2.5 Finish the function of plotting the bounding box in an image.
2.6 (optional) Finish the function of visualizing or saving the image with bounding boxes.
For selective search in OpenCV, you can find some related documentation in: [Selective Search Documents].
You may restrict experiments to use only the RGB space. To segment with strategy using texture similarity,
for example, do the following:
• Create SS strategy object by calling createSelectiveSearchSegmentationStrategyTexture().
• Create SS object by calling createSelectiveSearchSegmentation() and use addStrategy() in SS object
to add strategies. You may find some references in this document.
To feed an image into SS object, you can do the following:
1 ss = cv2 . ximgproc . segmentation . createSelectiveSearchSegmentation ()
2 ss . setBaseImage ( img )
To get the output bounding boxes, you may call
1 bboxes = ss . process ()
3 Report
For the report, please finish the following tasks.
3.1 Apply the selective search method to the provided data. Please experiment with two strategies: color
only and all four similarities (color, texture, size, and fill).
3.2 For each strategy, show the bounding boxes for proposals compared with groundtruth bounding boxes.
3.3 Display the proposal boxes whose IoU ≥ 0.5 with the groundtruth bounding boxes, along with the
corresponding GT, on each image. If there are multiple proposals with IoU ≥ 0.5 with the same
groundtruth bounding box, plot the one with the biggest IoU.
4 What to Submit?
You should submit a compressed file, e.g., .zip, including the following:
4.1 Your code with comments (if any).
4.2 An analysis of your test results. Base your conclusions on your test results rather than on the observations made by the instructor or the authors of the books and papers.
