My research is dedicated to making multimedia data types first class entities in our computing environment. Currently, they are not well integrated, and video, image, and audio data are much more difficult for programmers to use than text and numbers. Programming languages, operating systems, compilers, and networks do not "understand" multimedia data. To address these shortcomings, we are building tools to support multimedia applications. These tools treat audio, video, and images as first class entities, and the applications validate and motivate the tools stretch the paradigms used in the tools. Our current projects are a toolkit (CMT) for building distributed multimedia applications that are portable, scaleable, and fault tolerant and a programming language, Rivl (pronounced "rival").
CMT addresses storage and transportation problems associated with multimedia data. Basic video support can be added to an application with just a few lines of code. This strategy is similar to that of graphical user interface toolkits. CMT is being used to develop a distributed continuous media file server and playback application, which will be used to develop advanced multimedia applications and serve as a basis for studying communication, storage, and processing algorithms for multimedia data.
Rivl, an extension to Tcl/Tk, treats images and video as first class data types. Rivl differs from other image processing systems because it includes a compiler that understands the semantics of image and video operations and because Rivl programs are independent of the resolution of the images or the video. These two features allow the Rivl compiler to perform many optimizations automatically, including combining operators and choosing the most appropriate implementation for resolution, image format, and operation. The optimizations lead to faster execution and improved image quality. For example, the Rivl compiler detects the common case when the input and output images are compressed in the same format and resolution and simply copies the compressed data, tweaking it as necessary. The resulting execution is faster and more accurate than conventional implementations. Rivl provides both a framework for integrating image processing software and an efficient language for manipulating images and video that is more easily automated than windowing user interfaces like Photoshop, but more easily used than C libraries.
I am also exploring algorithms for processing compressed audio and video data. I recently developed a method for computing transition effects on Motion-JPEG compressed video data. The algorithms process compressed video data without decompressing it, which eliminates the time-consuming processing associated with decompression and compression and reduces the volume of data, allowing the data to be processed in software at video data rates. We recently used these algorithms to solve the MPEG-to-Motion-JPEG transcoding problem: we can efficiently convert data from MPEG format to Motion-JPEG format entirely in the compressed domain.
If you have questions or comments please contact: firstname.lastname@example.org.