CS 4670 Introduction to Computer Vision (Fall 2010)

Project 2b: Object Matching and Panorama Stitching

Released: Monday, Oct. 18, 2010
Due: Tuesday, Nov. 2, 2010, by 8:59pm
Last Modified: Wednesday, Oct. 20, 2010


Overview

In this two-part project, you will write code to do (a) object matching and (b) panorama stitching on the Nokia N900.

Description

Part I. Object Matching

In this part, you will reuse the code you implemented in Project 2a. You will take a picture and extract its features. You will need to estimate a homography between the database image and the search image using RANSAC, and return the best matched image. You will also write a new feature matcher that uses new data structures to do faster nearest-neighbor matching. Using our provided image database, you will get the following result.



Here are some screenshots on the cell phone.

Part II. Panorama Stitching

In this part you will shoot two photos and make them into a panorama by applying the homography. You will need to implement RANSAC to evaluate feature matches. You will also reuse the homography code to align two images, then use alpha blending and feathering to stitch the images together. Here's a screenshot doing panorama stitching on the cell phone.

Click here to download the skeleton code.

To Do

The only file that you will need to edit is Features.cpp. Where appropriate, you may copy in code from Project 2A if you choose. You are encouraged to use the SURF features (feature type 3 in this code), although you can also try your MOPS features from Project 2A. Here's a list of functions you need to implement.

Running and testing your code

Part I. Object Matching

After writing your code, you should follow the document to compile it. We provide some testing images, which can be downloaded here. We also provide test images for cell phones. There are instructions for building your own image database inside the tarball. You can use this as your image database and test your code in Vision Lab. This database feature is accessible on the desktop version through a new menu item in the GUI used in Part A of the project.

Part II. Panorama Stitching

You can use the following command to test your code on the desktop.
./Features panorama img1 img2 destimage [featuretype] [matchtype]

You should test your panorama stitcher on these two Yosemite images. Please include your panorama create from this pair in your writeup (see below).

If you would like a larger set to test on, try out this larger set of Yosemite images.


Phone Checkout

First you need to sign up for phone checkout. You are given 5 days for this project. There is a sign up sheet in front of Upson 4144. In general, there are two slots available in a week, Friday to Wednesday and Wednesday to Monday. By Friday, October 22, you should signup for one slot. There's an emergency slot from Monday, 11/1 - Tuesday, 11/2. If you need additional time, you can signup for that slot (not open yet), priority will be given to the team who signs up for Friday to Wednesday slot.


Submission

You should turn in their code, as well as an artifact webpage. The webpage should contain the results of the object matching on the N900, panorama results on our test images, and at least two panoramas that you create yourself, on the N900. Compress the webpage into a zip file called webpage.zip and compress your source codes into code.zip, and upload them to CMS. In addition, you will need to demo the result to the TAs.


Bells and Whistles

Here is a list of suggestions for extending the program for extra credit. You are encouraged to come up with your own extensions as well!

Monster Bells

Disclaimer: please consult the course staff before spending any serious time on these. They are quite difficult, and credit can vary depending on the quality of your method and implementation.

Web-scale object matching

Write a script to scrape all images of book covers from Amazon, and exterd your object matching to recognize any book in the world.

Real-time panorama creation

Write an app to "paint" a scene with the N900 as you move it around, constructing and displaying a panorama in real-time on the fly (see the "360 Panorama" app for the iPhone).