CS4670/5670: Computer Vision, Fall 2013

Project 4 (Single View Modeling) User Interface

Overview

The interface allows you to load an image and add points, lines, and polygons. After you compute the 3D position of those points, you can save the model and reload it for further editing. When you are done, you can dump the model in VRML 2.0 format and view it in a VRML viewer.

Working with the Point Stack

To compute the homography or the 3D positions of points, you will primarily interact with the program through the "point stack". You can push a point onto the stack by pointing at it with the mouse and hitting spacebar. To pop a point off the stack, point the mouse somewhere else and hit spacebar. When a point is put on the stack, a number that indicates its position on the stack (0 = bottom) will appear next to it, as shown in the image below:

Once a point's 3D position is computed, a small box will appear around it, which looks like this: . It can then be used to compute 3D positions of other points. A more detailed description on how to do this for different menus is given
below.

Steps

To create a 3D model using the program, you will follow the steps below:

  1. Add 3 or more lines (Edit->X Line/Y Line/Z Line) parallel to X, Y, Z axis, respectively, and compute vanishing points (Tools->ComputeVPs).

  2. Specify 2 or more reference points on the reference plane, and 1 reference point off of the plane for the reference height. See below (Edit->Point) for a more detailed description on how to do this.

  3. Push the on-plane reference points on the point stack and compute the homography for the reference plane (Tools->Compute Homography).

  4. Add polygons (Edit->Polygon) and compute the 3D position of each corner point by pushing appropriate points onto the point stack and using the appropriate method (Tools->sameZPlane/sameXY).

  5. When you are done, save the model in VRML format and the texture map in .tga images (File->Save VRML). Note that the texture images must be converted into .gif format to view it in VRML viewer.

Menu

The file IO related functions are under "File" submenu as usual. 

Open Image: Load an input image of .tga format.

Open Model: Load a saved model from a text file. Loads the reference coordinate frame (vanishing points, homography) and points, lines and polygons that you last worked with.

Save Model: Save the reference coordinate frame (i.e., the vanishing points, homography for the reference plane) and the model (i.e., points, lines, polygons) into a text file (with the file name extension .txt) for later use.

Save VRML: Save the 3D model and texture images into VRML 2.0 format so that it can be viewed in VRML viewer.

Save unwrapped model: Saves an unwrapped version of the scene that can be printed out and folded into a physical 3D model. This works for both normal and inverted scenes.

The "Edit" submenu, you have the following choices:

Point: Add or delete points. To add a point, left click. To delete a point, move to the point till it is high lighted as white, then press "Backspace". The point can be deleted if it is not used by any other lines or polygons. For a reference point, the 3D coordinate can be entered by moving the mouse over it and pressing the enter key. A dialogue box will pop up where you can enter the 3D coordinates for the point directly. To specify the reference height, check the box on the dialogue window that says "use for reference height".

X Line, Y Line, Z Line, Other Line: Add or delete lines. To add a line, the first left click defines the start point, and the second left click defines the end point. If you want to reuse one of the existing points as start/end point, just press "Ctrl" when you left click. To delete a line, move the mouse onto it till it becomes white and press "Backspace". In "X Line" edit mode, the lines you add are supposed to be parallel to the X axis in 3D. Similar meaning for the "Y Line" and "Z Line" mode. Lines added in "Other Line" mode may have any orientation. You can reuse an existing point for a new line endpoint by holding down Ctrl while left-clicking on that point.

Polygon: Add or delete polygons. Each polygon consists of a list of points. To add a polygon, you sequentially left click on desired positions and then press "Enter". A closed a polygon will be drawn. ( You don't have to click on the first point to make the polygon closed, the system automatically does it for you. ) To delete a polygon, move the mouse to the center of the polygon, shown as a white square, and press "Backspace". Every time you create a new polygon, you will give a name for it, e.g, "ceiling", "floor", which will be used as the texture file names when you save the model in VRML. The texture file name for a particular polygon will be the polygon name with ".gif" extension. As in the case of lines, existing points may be reused as new polygon corners by holding down Ctrl while clicking on it.

XZ Rectangle: Create a rectangle parallel to the XZ plane. To specify a rectangle, you must click on a known reference point in the scene (this point is the anchor for the rectangle), then drag to specify the size and shape of the rectangle. Clicking will specify the opposite corner of the rectangle, and create a new polygon (you'll be asked to give the rectangle a name). Note that each of the vertices of this rectangle has an index starting from 0, as shown in this image:

You will need to implement the solveForOppositeCorners routine for this feature to work.

Sweep rectangle: Sweep out (extrude) an existing XZ rectangle to create a box. To sweep out the rectangle, click on an anchor point and drag. The mouse position specifies how large the box should be (the line opposite the horizontal line through the anchor point will pass through the mouse position). Click to create a box (creating five new polygons). Again, the vertices of the box are indexed starting from 0, as follows:

You will need to implement the solveForOppositeFace and find3DPositionsBox routines for this feature to work.

The "Draw" submenu, you can toggle the following options:

Points: Draw points or not.

Lines: Draw lines or not.

Polygons: Draw polygons or not.

Draw 3D: Draw it in 2D or 3D mode. 

When "Draw 3D" is not checked, all the image and points, lines, and polygons are drawn in image plane. You can edit them and

zoom in/out: Ctrl+/-;

move image: Drag with right button;

When "Draw 3D" is checked, all the points, lines, and polygons are drawn in 3D (based on your computation of X,Y,Z for each point). The image is texture mapped onto the polygon (based on your estimation of homograph H, invH). You can not edit in this mode, but you can

scale up/down: Ctrl+/-;

move model parallel to the viewing plane: Drag with left button;

move model further/closer: Drag with left button upwards/downwards, with Alt down;

rotate around X: Drag with left button vertically, with Ctrl down;

rotate around Y: Drag with left button horizontally, with Ctrl down;

rotate clockwise/counterclockwise: Drag with left button to the right/left, with Shift down;

The "Tools" submenu contains the main features of the program for computing 3D positions and camera parameters. Most of your work will be about implementing these features.

Compute VPs: Compute vanishing points using the lines you added along each axis by Edit->X Line/Y Line/Z Line command.

Compute Homography: Compute the homography from the reference plane to the image plane. 2 to 4 reference points should be on the point stack. If fewer than 4 reference points are on the stack, x and y vanishing points are used as remaining reference points.

Same Z Plane: Compute the 3D position of a new point that is on the same plane as another point whose 3D position is already known. The new point and the known point should be on the point stack, with the new point on the top.

Same XY: Compute the 3D position of a new point that is directly above another point whose 3D position is already known. The new point and the known point should be on the point stack, with the new point on the top.

Compute Camera Params: Compute the position and projection matrix of the camera in the scene.

Invert scene: Invert the scene in order to produce a reverse perspective model.

Guidelines: When this option is enabled, whenever the cursor is near enough to the line connecting either two points on the stack or one point on the stack and a vanishing point, the line will appear in the window and you can use it to guide your work. If you want to snap to the line, hold down Ctrl while placing your point. This can be useful, for instance, when you want to specify a distant point that may have the same X, Y coordinate as an existing point, or that lie on the same Z plane as an existing point, or both. These points may be good to use as the corners of a polygon later on. .



Last modified November 8, 2013