Digital cameras have become one of the most ubiquitous forms of computer in existence, but we still don't have a good handle on how to program them, what data we should program them to capture, how that data should be combined to make the photograph, and if a photograph is even the desired output. I'll talk about two of the tools we've developed to explore these questions.
The first is the "Frankencamera", which is an open API and architecture for programmable cameras that makes it easy to deterministically control the various components of a camera at high frame rates. The goal of the Frankencamera is to enable research in photography, and indeed in graduate-level classes we've found that students equipped with this API are much more able to prototype their novel camera applications than with conventional camera APIs.
The second tool is the discrete Gauss transform, which represents a class of algorithms that are widely used in photographic image processing for denoising, sharpening, and controlling tone and contrast. Incarnations of the discrete Gauss transform include the bilateral filter, the joint bilateral filter, and non-local means. We'll talk about making discrete Gauss transforms fast by representing the problem as resampling in a higher-dimensional space.
Finally, time permitting, I'll talk about what we need to do to get these kinds of photographic image processing algorithms to run efficiently on the agglomeration of processors found inside today's cameras.