CS4621 PPA1 GPURay

Out: Thursday September 24, 2015

Due: Thursday October 15, 2015 at 11:59pm

Work in groups of 2


In this programming assignment, you will implement a simplified GPU version of the CS4620 raytracer (ray1). The goal here is not to reproduce all the features we have seen in the other assignment but rather to expose you to some of the basics of modern graphics hardware programming.

The cs4621.GPUray and cs4621.GPUray.shaders packages under the framework contain all the relevant source code which we provide as a starting point. There are several things that need to be done before we can render images using the GPU. We begin with setting up the relevant libraries that are needed to communicate with the hardware, and go on to check that the graphics card in your machine can support the needed functionality using a simple diagnostic program. Then we call the appropriate existing methods for ray1 to parse the same xml scene files used in the cs4620 ray1 assignment. This data will need to be packaged and formatted in a specific way before it is communicated to the graphics processor. Note that not all the scenes will be supported so we need to make sure we only pass the data our GPU implementation can handle. The OpenGL API also provides a family of methods that allow you to bind and communicate the data to the GPU. We saw in class that the data layout has to be consistent on both sides (Java-client and gpu-host) for this to work. The GPU may have the data but we still need to program it to do something useful with it, such as showing it one screen. This is done by writing GLSL code for each shader in the GLProgram. We will also write some code to show off how fast our GPU raytracer is by enabling some simple user interaction through LWJGL to rotate the camera and light source positions around the scene. Finally, we quantitavely compare the performance of our GPU implementation against that of the ray1 using a series of benchmark scenes.

Task List:

  1. Setting up the CS4620/4621 framework and OpenGL diagnostic
  2. Parsing ray1 xml scenes
  3. Writing GLSL shader code
  4. Creating the GLProgram and binding buffers to the GPU
  5. Adding user interaction
  6. Performance Benchmark

We have marked all the functions or parts of the functions you need to complete for the assignment with TODO#PPA1 in the source code. To see all these TODO#PPA1 annotations in Eclipse, select the Search menu, then File Search and type TODO#PPA1 (They can also be viewed through the Task List). You only need to modify the sections marked with TODO unless of course you want to add some additional cool shader or user interface functionality for extra credit. All other parts have been implemented for you.

Task 1: Framework LWJGL configuration and Diagnostic

Getting Started

We have prepared a short diagnostic program that tests out the CS4620/4621 OpenGL framework on your machine, and we have also released a separate CMS assignment, due Friday Sep 25th at 11:59pm, for you to turn in the results it generates (Diag.png and GLDiag.txt found in the root folder of the framework). Please take a minute to pull the new framework from Github (CornellCS4620/Framework), run it, and submit the output on CMS. See the README comment section in cs4620.util.DiagnosticApp for directions on setting up Eclipse to run OpenGL applications in Java.

Update: a revised diagnostic program cs4620.util.LightweightDiagnosticApp can also be run to query for additional information in case you are running into any issues or do not seem to be meeting the hardware specifications. This generates another log file in your project root (LWGLDiag.txt). There aren't any points associated with this part, but there is the benefit of decreased probability of compatibility problems if we have your system info.

A new commit has been pushed to the CS4620 class Github page in the master branch. We recommend switching to your master branch, pulling this commit, and creating a new branch (e.g. PPA1 solution) and committing your work there. The new commit contains all the framework changes and additions necessary for this project. We use LWJGL (a java wrapper library for OpenGL) in this project, so you may need to update some settings of your build to have this external library work properly. We have included the correct jar files, but you may need to configure your native library location. LWJGL is available in the Framework, but it requires some configuration specific to your machine to use it properly. We can access the settings under:

Project -> Properties -> Java Build Path -> Select Libraries -> Select the LWJGL Drop Down Menu -> Modify Native Library Location.

To do this in Eclipse, you can follow the more detailed step-by-step instructions below to modify the LWJGL settings so that it matches your OS (windows | linux | macosx)

  1) Right-click on your project and select properties
  2) In the window that pops up, select the tab on the left
     labelled "Java Build Path"
  3) Select the tab called "Libraries" and expand "lwjgl.jar"
  4) Select the option "Native library location:" and press the "Edit" 
  5) Enter in the correct path to your native library location. It should
     be one of the following (pick your OS):
  6) You should now be fully configured to use LWJGL on your machine

OpenGL Diagnostic

We can finally execute the diagnostic program and see if the hardware can support the expected level of functionality for the current and future assignments. Run cs4620.util.DiagnosticApp. An image of a Cornell box (see image below) will be visible on screen and the program should terminate successfully with a message to the console saying that the image has been written to disk.

You will see something similar to the output below in the Diag.txt file under the root folder of the framework:

TIME:                         Tue Sep 15 23:19:02 EDT 2015

=== Java System Properties ===
java.version:                 1.7.0_79
java.runtime.name:            OpenJDK Runtime Environment
java.vm.name:                 OpenJDK 64-Bit Server VM
java.vm.version:              24.79-b02
os.name:                      Linux
os.version:                   3.2.0-74-generic

====== OpenGL Properties =====
GL_VENDOR:                    NVIDIA Corporation
GL_VERSION:                   4.3.0 NVIDIA 346.72
GL_RENDERER:                  GeForce GTX TITAN Black/PCIe/SSE2

Please notify the course staff, as soon as possible, if you cannot see the image or if the GL_VERSION is lower than 3.3 (Use the piazza poll to post your results). There is a certain likelihood that systems without dedicated graphics hardware may not meet the mark. In such a case, you can use the machines in the Gates Hall CSUG and MEng labs to run your assignments for this class. The lab machines have been tested and support the required specs without any trouble.

Task 2: Parsing the ray1 xml scenes

Our next step is to load the same xml scenes we were using in the first CS4620 raytracing assignment. We achieve this by calling the CS4620 ray1 scene resolver and xml parser (see cs4620.ray1.Parser and cs4620.ray1.RayTracer.ScenePath). We want to extract all the triangle (Wavefront Obj) meshes, corresponding material and light sources as well as other parameters from the scene structure, and format it in a way that can be sent off to the GPU. That will also involve removing meshes/data that will not be supported on the GPU by our implementation (for instance sphere and cylinder primitivies or Phong materials). Note that you can eventually extend the functionality to handle these kinds of features for extra credit. The scene camera information should also be parsed and initialized accordingly. Implement the missing parts in setupScene() as instructed by the inline comments and proceed to call this method using the build() method under RayTracerScreen. You will also need to make sure that during construction time the meshes do not exceed a certain hardcoded conservative maximum size that the GPU can handle.

We show a typical ray1 xml file below. Notice that there are fields we wish to parse (camera, Mesh, light) and ones that we will ignore (Box, Phong, image):

<?xml version="1.0" encoding="UTF-8" ?>
<!--  Stanford Bunny mesh -->
  <camera type="PerspectiveCamera">
    <viewPoint>4 6 8</viewPoint>
    <viewDir>-4 -6 -8</viewDir>
    <viewUp>0 1 0</viewUp>
    450 450
  <shader name="bunny" type="Lambertian">
    <diffuseColor>.05 1 0.2</diffuseColor>
  <shader name="cube" type="Phong">
    <diffuseColor>0.8 0.4 0.1</diffuseColor>
    <specularColor>0.9 0.7 0.3</specularColor>

  <surface type="Mesh">
    <shader ref="bunny" />
  <surface type="Box">
  	<minpt>-3 -2 -3</minpt>
  	<maxpt>3 -0.9 3</maxpt>
  	<shader ref="cube" />
  	<position>3 10 5</position>
  	<intensity>9 9 9</intensity>

Task 3: GLSL Shader Code

The current task (and the following one) will walk you through the steps to set up the necessary OpenGL plumbing that connects the Java side with the GPU shader program. The cs4621.GPUray.shaders package contains the shader source files raytracer.vert and raytracer.frag. We have provided the vertex shader code but it is your job to complete the missing parts in the fragment shader. That involves declaring the required global parameters which include uniforms for the camera, lights, vertex and triangle information. Will we need any varying variables and/or attributes (Why or why not?). The modern GLSL syntax uses the keywords in and out for these parameters.

The function stubs setup_camera(), intersectCube(), intersectTriangle(), compute_shadow(), shade_lambertian() and main() have inline comments that describe what each method should be doing which boils down to porting over the appropriate ray1 code. This is an exercise in understanding how data flows around the shader program and using GLSL syntax and data types. Feel free to extend the shader implementation to handle sphere intersections and Phong shading as well, if you wish.

Debugging GLSL

Remember that uniforms that are declared but not used are “optimized” out and OpenGL will throw an error if you try to set a nonexistent uniform (a very common pitfall). Don't forget to also specify the compiler #pragma for the GLSL version of the shader code and make sure it corresponds to the OpenGL context you request on the Java side.

It is good practice to explicitly specify which parameters are inputs to a shader stage and which are outputs from a stage using the in and out keywords. When editing the shader files the eclipse IDE has a quirk where you have to refresh and rebuild the project otherwise the changes might not be reflected regardless of whether you saved the source files or not.

If you run into issues, there are a couple of different ways to debug your code:

The egl.GLError.get() method can be used to query the GL state which retrieves a diagnostic error code that you can look up online for more information on the issue (egl.GL has enums for some of the common errors glGetError()).

The glGet* family of methods can also be used but that can become tedious fast. Unfortunately, we cannot print to the console directly from the shader code, though once you have things on screen, we can output whatever values we wish for visual debugging by color-coding the information and setting it to vFragColor.

visualizing different parameters on screen (normals, t-values, shadow mask)

Task 4: Binding Parameters and creating the GLProgram

Once we have our GLSL code, we need to ask the framework to create a GLProgram with the requested shader source files using the quickCreateResource() method. The program.use() and program.unuse() methods should be called, at the beginning and the end, whenever we want to do something with our shader program. We also need to make sure the data is put in the appropriate containers which in our case will be GLBuffers and Uniforms. We can use IntBuffers and FloatBuffers to store the triangle indices, positions and colors. Use glUniform3() and glUniform1i() with getUniformArray() or getUniform() to retrieve the ids and bind the data to variables that will have the equivalent names on the GLSL shader code side (see Task 3). Don't forget to call the .rewind() for your buffers every time you end up using them. Finally, we want to avoid sending the data over to the graphics accelerator multiple times, so you should structure the code in a way that avoids overhead from repetitive transfers unless it concerns data that is expected to change from one frame to the next (Hint: look at the different methods that RayTracerScreen inherits from GameScreen).

Task 5: Controls and animation (mode toggle, camera and lights)

In this part we add user interaction so that we can better show off our GPU raytracer. We will use keyboard controls to allow the user to orbit the scene with the camera as well as move the first light parsed from the xml scene (Hint: look at egl.math.Matrix for some useful methods and feel free to augment the camera controls with more advanced ones such as the ones found in the CS4620 scene assignment). Complete the missing code in the update() method as instructed by the inline comments. Another useful feature to have while debugging the shader code is the ability to toggle between different shading modes on screen. Create an LWJGL KeyBoardEventDispatcher that listens to your toggle key, bind this key to a uniform and pass it to the fragment shader where you can use it in control logic to decide between different modes such as visualizing the surface normals, the ray t-values, a binary shadow mask (see the example images above).

Task 6: GPU/ray1 performance comparison

The graphics hardware seems fast(er) than the ray1 implementation. Let's try to quantify that in a performance comparison. The task at hand is to run both the ray1 and GPUray implementations with a series of scenes where we collect some timing information. ray1 already outputs the time it took for rendering a particular image at the end of the execution. Use the gameTime parameter to compute a framerate and the time taken per frame during each draw() call. What kind of speedup did you end up with (is the comparison fair)? Tabulate some of these results and put them in a README file which you should submit along with your implementation. If you have additional time and are feeling particularly creative you can also use a modeling environment, such as blender, to produce a scene with some interesting geometry, export it as an obj file and use it in GPUray where you can experiment with the cast shadows.

What to Submit

Submit a zip file containing your solution organized the same way as the code on Github. All the code you have written should be well commented and easy to read. The header comments for all modified files should indicate the appropriate authorship. Include a readme in your zip that contains:

Upload on CMS