The goal of the final project is for you to implement and evaluate a computational imaging system both in hardware and software. An exceptional final project should be something that could become a conference or journal publication after some further refinements. The final project is open-ended and can be based on your ideas and interests. Some suggestions for types of final projects:
- Re-implementing and thoroughly evaluating a published computational imaging paper in a new scenario not covered in the paper itself.
- Applying ideas and algorithms from this class in your own research area on some kind of imaging system.
- Coming up with and evaluating a new algorithm for imaging forward models or inverse problems.
- Modifying an existing computational imaging setup in an interesting way to produce some new effect.
Key deadlines
Key logistics
Teams: Project should be completed in teams of 2-3 students. If you already have a clear project idea, or want to do something related to your research area, talk to your classmates and convince them to join your project team. We might allow individual projects in special circumstances, but come talk to us first.
Imaging hardware: Final projects can (and are encouraged) to make use of imaging hardware (cameras, microscopes, lights, depth sensors, light field cameras, special lenses, and so on). If you have access to equipment from your research lab, great! You are encouraged to leverage this. If you would like to do a project that requires specialized equipment that you do not have access to, come talk to me before writing/submitting your project proposal. You might be able to loan out equipment for the project.
Deliverables and Grading
The final project will be worth 50% of your grade for the class. Make sure to start early. During weeks when there is no homework, you should be spending on average 6 hours/week on the final project. This amount of time is necessary to make sufficient progress on the final project.
Group formation (5%)
We will help facilitate project groups in class with a group forming activity. Please read this whole document and brainstorm some ideas for your final project before attending this group forming activity. Working on a team rather than working alone can help you think of more creative ideas, will allow you to tackle a more ambitious project, and potentially get access to more resources. You are expected to come prepared to the group formation lecture with ideas to discuss with your peers, and to actively engage in our group formation activities.
Project proposal (15%)
Each team will submit a proposal of their plan for the final project. Each project should utilize some hardware, either from your own research lab or from the camera kits provided by the class. We have some other imaging hardware that can be loaned out, but please talk to us in advance! We will give you feedback on this proposal to make sure it's within the scope of the course goals and project timeline.
The written project proposal should be a PDF between 1-2 pages, to be submitted on Canvas. It should have at least the following sections and content:
- Title. Provide the title of your project.
- Summary. Summarize your project in no more than 2-3 sentences. Describe what you plan to do and what will be learned.
- Background. Describe in 1-2 paragraphs why this project is hard, useful, and/or interesting.
- Resources. Describe the resources (cameras and other imaging hardware, starter code, dataset, any special computing resources, etc.) you will use. If you are building off of an existing codebase, or an existing hardware setup, please explicitly say so. If there are any books or papers that you are using as references, please provide the citations. Make sure to explain what data (images, videos, etc.) you will use to evaluate your results. If you are doing a hardware project, explicitly mention so and list what equipment you will need. Please also explain whether you already have access to these resources, or whether you would like teaching staff to provide them to you.
- Goals and deliverables. Describe the deliverables or goals of your project. Make sure to separate your goals into what you plan to achieve (that is, the minimum set of goals you believe must be reached for the project to be successful), as well as what you hope to achieve (additional goals you would like to see happen if the project goes really well).
- Schedule. Provide a tentative schedule for your project. List what you plan to get done each week from now until the project due date.
- Citations. Please cite at least 2-3 relevant papers, and make sure to cite these in the background section.
Background presentations (10%)
Each team will prepare a presentation on background information relevant to understanding your project. Each person will have 2 minutes to present, so a 2-person team will have 4 minutes total and a 3-person team will have 6-minutes total. These presentations will occur during lecture and will be evaluated based on clarity of presentation. Your goal should be to prepare a mini-lecture that explains the background and relevant state of the art necessary to understand your project and put it in context with work in the field. It should also summarize the forward models and inverse problems used in your chosen area. This is difficult to do, but is good preparation for conference talks. We will provide feedback on your background presentations so that you can improve and make a great final presentation. As part of your grade, you will be evaluating and providing feedback to your peers on their presentations.
Check-ins (10%)
Each team will present a status update on their project to the teaching staff. These will be scheduled during the week of Nov. 11th during office hours and other times outside of class. During this update, you will talk about your project goals and the current status of your project. Please include photos of any initial hardware prototypes or intermediate results. This is a great time to discuss any roadblocks or challenges you are encountering in order to leverage the expertise of the teaching staff for suggestions.
Project presentation (30%)
Each team will prepare a final presentation to showcase your project and results. These final presentations will occur during the last two days of class. Each person will have 4 minutes total to present, so a 2-person team will have 8 minutes and a 3-person team will have 12 minutes to present. Your presentations will be judged by a panel of experts to determine the best presentation award for the year. In addition to the in-class presentation, you will upload a polished and condensed video recording (no longer than 5 minutes) of your final presentation to Canvas. These video recordings will be uploaded to YouTube in a public channel to highlight and showcase your work. Your video should be similar to a conference spotlight video, and should succinctly and clearly communicate your project results, contributions, and how it fits into the literature.
Project report (30%)
Your final report should be a PDF of length 4 pages (1-person groups), 6 pages (2-person groups) or 8 pages (3-person groups), plus any additional pages for references. Your report should be written as a SIGGRAPH paper (you can use the author kit for the formatting). This paper should be in the format of an academic conference or journal paper and include at least a title, abstract, intro + background/related-work, methods, results, conclusions/future work sections, and a section that summarizes each team member's contributions. See these tips for how to write a paper.
Project Ideas
Coming up with project ideas: Imagining something exciting and new to do as a project is hard. Please take time to brainstorm and think about potential project ideas, and come discuss potential topics with the teaching staff to help narrow down your project. Here are some places where you can find inspiration:
- Most lectures include teasers of advanced subjects that relate to the lecture's overall theme. These subjects are not discussed in detail, but the references at the end of the lecture provide pointers to related literature. You can follow up on those pointers.
- If the overall theme of some lecture strongly appealed to you, you can do a literature search to find more recent papers in that area, and peruse those for ideas. Google Scholar is also your friend, especially the option to show citations of a paper, which you can use to search through recent research on topics and papers we discuss in class lectures.
- You can look at final projects from previous offerings of related courses: CMU 15-463 Computational Photography (F20, F21, F22, F23) and Stanford EE367/CS448I: Computational Imaging (W20,W21, W22).
- You can binge-watch videos on the ICCP YouTube channel, to find talks and related papers and research topics that strongly appeal to you.
Our suggestions for projects. If you can't think of your own project, below are a few project ideas that we think could be exciting to explore:
- Dual pixel cameras: estimating depth with dual pixel cameras, split aperture 2 in 1 cameras, depth from dual pixels, synthetic defocus on stereo and monocular mobile phones (your Canon R10 cameras have dual pixels!)
- Fun with projectors: optical gradient descent.
- Speckle imaging: seeing through scattering, motion tracking, tampering detection, neural field for seeing through scattering.
- Fun with polarization: depth sensing, dehazing.
- Fun with lightfields: pinhole lightfield camera, build your own plenoptic camera, motion estimation, shape estimation, reconstructing transparent objects, schlieren photography.
- Fun with motion blur: flutter shutter.
- Fun with apertures and defocus: coded aperture, confocal stereo, extended depth of field, focal flow, depth from focus on your phone, depth from defocus in the wild.
- Fun with cheap lenses: imaging with cheap lenses, depth from cheap lenses.
- Lensless cameras: 3D diffuser cam, hyperspectral imaging, optimizing masks, using information theory, single-shot video.
- Fun with hyperspectral cameras: DIY hyperspectral imaging, DIY using printer paper.
- Fun with imaging around corners: corner camera, computational periscopy, accidental pinholes.
- Others: learning sensor noise, black hole imaging, neural models for imaging.