CS5630 Physically Based Realistic Rendering

This class covers the mathematics and physics of light transport as it pertains to the visual appearance of the world, and the algorithms and practice of using these models to generate beautiful realistic images. This includes radiometry, colorimetry, and radiative transfer; formulations of light transport in terms of recursive integrals or integral equations in path space; Monte Carlo methods for computing approximate solutions; and the craft of building fast and usable rendering software. These methods are the ones currently universally used in the film industry for animation and visual effects, and they are becoming available in the lastest hardware for games and VR as well.

In the first half of the semester, you will be given a bare-bones renderer that initially cannot render anything meaningful. Through a series of assignments, you will turn this starter code into a full path tracer capable of rendering scenes with glossy, rough, transparent, and reflective materials under a variety of illumination conditions:

To help you understand the underlying mathematical background, there will be several theory-oriented notebooks that should be completed alongside the implementation work.

The second part of the class is all about the final project. You will extend the basic renderer to support more advanced rendering techniques of your choosing. The class will culminate in a rendering competition where you will present renderings produced with your code to a panel of judges.

For inspiration and to see what a final implementation of your renderer can achieve, we recommend looking at the results of rendering competitions held at other universities,: CS-440@EPFL, CS87/287@Dartmouth, or CG@ETH.

Disclaimer: This is a brand new class and the syllabus and assignments may change as we refine the course over time.

Schedule

Here is a tentative schedule and the relevant reading material. We will update the schedule if necessary. If the assignment due date differs between this page and CMSX, the CMSX date overrides this one. All the assignments are due at 11:59pm.

Date		Topic	References	Assignments
20	Jan	Intro
22	Jan	Ray tracing acceleration	PBRT 7.3	Assignment 0 is due on Jan 23
27	Jan	Probability and Monte Carlo integration • notes | notebook | notebook	PBRT Ch.2
29	Jan	Probability and Monte Carlo integration
3	Feb	Sampling techniques • notebook
5	Feb	Radiometry • notes	PBRT 4.1	Assignment 1 is due on Feb 6
10	Feb	Light reflection	PBRT 4.3
12	Feb	Illumination integrals	PBRT 4.2, 4.4	Notebook 2 is due on Feb 13
17	Feb	—February break—
19	Feb	Assignment 2 help and discussion
24	Feb	Multiple importance sampling		Assignment 2 is due
26	Feb	Microfacet reflection model
3	Mar	Transparency and translucency		Notebook 3 is due
5	Mar	The rendering equation
10	Mar	Path tracing		Assignment 3 is due
12	Mar	Next event estimation and MIS in path tracing
17	Mar	Textures, antialiasing, and image-based lighting		Notebook 4 is due
19	Mar	Denoising
24	Mar	Real time ray tracing		Assignment 4 is due
26	Mar	Many lights and resampled importance sampling		Project proposal is due on March 27
31	Mar	—Spring break—
2	Apr	—Spring break—
7	Apr	Radiative Transport
9	Apr	Participating media
14	Apr	Translucent materials
16	Apr	Scattering from fibers
21	Apr	Survey of reflectance models
23	Apr	Layered reflectance models
28	Apr	Measuring surface appearance
30	Apr	TBD
5	May	TBD
TBD		Rendering competition

Coding assignments and notebooks

The coding assignments are built in Wenzel Jakob's educational renderer, Nori, though our assignments will be somewhat different in the specifics. The details are on the CS5630 Nori page. Assignments 2, 3 and 4 are acompanied by notebooks that should be completed before the correspoinding coding assignment. (There are no Notebooks 0 and 1) The notebooks are there to help you understand the mathematical background behind the algorithms you will be implementing.

The planned assigments and notebooks are as follows:

• Assignment 0 (5 pts) to get the framework set up and compiled.
• Assignment 1 (100 pts) to implement an acceleration datastructure for ray tracing.
• Notebook 2 (50 pts) in which you study Monte Carlo method and various variance reduction techniques using simple 2-3 dimensional examples.
• Assignment 2 (100 pts) in which you implement the basic tools of Monte Carlo integration for rendering and your very first and very simple physically based rendering algorithm.
• Notebook 3 (50 pts) where you get to play with multiple importance sapmling (MIS) technique.
• Assignment 3 (100 pts) to implement MIS in the renderer and to figure out how to render diffuse, dielectric and rough surfaces.
• Notebook 4 (50 pts) will explain what to do with recursive integrals.
• Assignment 4 (100 pts) to implement a complete path tracer.

Projects

During the second half of the semester, you will extend your renderer to support new rendering effects of your choosing, culminating in a rendering competition to select the technically and aesthetically best images. To help you decide on the details of your project, we will provide a list of features that students commonly implement in this class. Each feature will be assigned a value based on its complexity, and a strong final project should reach an appropriate level of overall complexity. You are more than welcome to implement features that are not on the list; the value of such custom features will be determined on a case-by-case basis after discussion with the course staff.

To help you plan your project, you will be required to submit a project proposal by March 27, and we will provide feedback on it. Projects will be graded based on written reports and in-person demos. The grading of the project will not take the aesthetics of the images into account, only the correctness of the implementation and technical complexity.

Course policies

Questions, help, discussion: The instructor and TAs are available to answer questions, advise on projects, or just to discuss interesting topics related to the class at office hours and by appointment as needed. For electronic communication we are using Ed Discussion (access via Canvas).

Late submissions: We allow for the total of 4 late days. You can use these late days for any assignments except for the final project submission. Each late day allows you to submit an assignment up to 24 hours after the deadline without penalty. Once you have used all 4 late days, grade for any further late submission will be reduced by 50%. Assignments in this class are very dependent on each other, so this policy is here to encourage you to submit on time.

Collaboration: You are welcome (encouraged, even) to discuss projects and assignments among yourselves and to help one another track down problems. But when it comes to the design and implementation, your code needs to be your own work. In particular, it's certainly never OK to copy parts of one person's writeup, code, or results into another's, even if the general solution was worked out together.

AI use and open source renderers: Various LLM chatbots and co-pilots can be useful tools for coding and learning, can help you figure out how to get around a new codebase faster or debug tricky issues. Thus, we do not prohibit the use of AI in this class. We, however, require you to disclose any use of AI tools in your assignment and project reports and mention which tools you used for what purpose.

There are multiple open source renderers that you may find online. You are welcome to study them to understand how certain algorithms are implemented, but you may not copy code from them into your assignments or projects. If you do refer to any open source renderer in your work, please cite it properly in your report.

Ultimately, it is very easy to decide when to use and when not to use AI or open source code. Here is a two step strategy:
1. Remember that you are taking the class to learn and improve your understanding of the subject.
2. Aks the question: Does using this tool or code help me learn and understand the subject better? If the answer is yes, go ahead and use it. If the answer is no, don't use it.

Regardless of which external resources or tools you used you are responsible for understanding everything you submit. Assignments and projects will be graded based on the understanding of the content.

Academic integrity: We assume the work you hand in is your own, and the results you hand in are generated by your program. You're welcome to read whatever you want to learn what you need to do the work, but we do expect you to build your own implementations of the methods we are studying. If you're ever in doubt, just include a citation in your code or report indicating where some idea came from, whether it be a classmate, a web site, another piece of software, or anything—this always maintains your honesty, whether the source was used in a good way or not. The principle is that an assignment is an academic document, like a journal article. When you turn it in, you are claiming that everything in it is your original idea (or is original to you and your partner, if you're handing in as a pair) unless you cite a source for it.

School can be stressful, and your coursework and other factors can put you under a lot of pressure, but don't let that lure you into dishonesty. If you feel you can't complete the work on your own, come talk to the professor, or your academic advisor, and we can help you figure out what to do.

For more information see Cornell's Code of Academic Integrity.

Books