Sherman Ng

NetID: ssn35

CS 6670 Project 1

Custom Feature Descriptor:

The custom feature descriptor I designed considers a 5 by 5 window around a feature point detected by the Harris operator. Instead of only storing the intensity values of the 25 pixels in the window into the data field of the feature data structure, the feature descriptor function stores the partial derivatives of intensity in the x and y directions of each pixel in the 5 by 5 window along with the pixel's intensity value into the data field of the feature data structure. Since there are 25 pixels in the 5 by 5 window, there are 75 total elements stored in the data field of each feature data structure corresponding to a feature point. The partial derivatives of intensity of each pixel in the 5 by 5 window are used to describe the region around a feature point because the simple descriptor that only stores pixel intensity values would not be very robust to changes in illumination. By considering differences between neighboring pixels, this custom feature descriptor should be more robust to variations in illumination between 2 pictures.

Major Design Choices:

To implement the Harris detector, the only major design choice I had to make was the choice in determining a threshold to use in the Harris value local maximum function. From running the entire Harris detector multiple times, I found that setting the threshold to be 0.001worked fairly well in suppressing the non-maximum Harris values. There weren't many design choices to make in the Harris value computation function, as I just summed the values in the Harris matrices over the 5 by 5 window and calculated the Harris values after the summation.

In designing the MOPS feature descriptor, I calculated the angle of a feature point in the compute Harris features function so that the compute MOPS descriptor function, once it is called, will already have a calculated angle value stored in the feature data structure input argument to use in calculating feature data to store.

For the boundary cases, I made sure that the computations of the Harris values or MOPS descriptors, which uses windows, would not access memory outside of the range of the input image. To do this, I just neglected pixels located near the edges and calculated Harris values and MOPS descriptors for pixels not near edges. For pixels on the edges, I just set the Harris values to 0 and did not calculate MOPS descriptors for those pixels.

Harris value images:

Yosemite:

Graf:

Benchmark Images AUC:

Average AUC values for the different descriptor and matching code combinations:

ROC Curves:

ROC plot for Yosemite images

ROC plot for Graf images

Strengths and Weaknesses:

From the results of running the entire feature detection and feature matching algorithm developed in this project, it seems like the MOPS descriptor and the custom feature descriptor I designed using pixel intensity values and partial derivatives performs better on average than the simple 5 by 5 window descriptor. The areas under the ROC curves of the MOPS and the custom feature descriptors are larger on average than the area under the ROC curve of the simple descriptor. The custom feature descriptor is more invariant to variations in illumination than the simple descriptor, while the MOPS descriptor is more invariant to orientation than the simple descriptor, which may explain why the overall performances of the MOPS and custom descriptors are better. However, a weakness common to all three descriptors is that their performances are fairly sensitive to the number of feature points detected by the Harris detector. As seen from the ROC curves of the Yosemite and Graf images, the performances of all three descriptors are better on the Yosemite images, which have fewer feature points, compared with the Graf images with more feature points. To ensure that the descriptors achieve maximum performance, the threshold to suppress non-maximum Harris values must be set to a suitable value.

Running Harris detector on additional images:

fridge.jpg:

stove.jpg: