Comparing Images Using Joint Histograms

Greg Pass
Ramin Zabih


Color histograms are widely used for content-based image retrieval due to their efficiency and robustness. However, a color histogram only records an image's overall color composition, so images with very different appearances can have similar color histograms. This problem is especially critical in large image databases, where many images have the same color histogram. In this paper we propose an alternative to color histograms called a joint histogram, designed for use in large databases. A joint histogram incorporates additional information from the image without sacrificing the robustness of color histograms. This is accomplished through careful selection of a set of local features. Each entry in a joint histogram contains the number of pixels in the image that are described by a particular combination of feature values. We describe a number of different joint histograms, and evaluate their performance for image retrieval on a database with over 210,000 images. On our benchmarks, joint histograms outperform color histograms by an order of magnitude.

This paper entitled Comparing Images Using Joint Histograms has appeared in the ACM Journal of Multimedia Systems. The paper is available in PDF format. (Free PDF readers can be obtained for many platforms from Adobe.) The images which appear in this paper can be viewed in color.

Older Papers

A paper entitled Histogram Refinement for Content-Based Image Retrieval appeared in the 1996 Workshop on the Applications of Computer Vision. This paper is available in Postscript and PDF format.

A slightly older paper entitled Comparing Images Using Color Coherence Vectors appeared in the 1996 ACM Conference on Multimedia, and is available in PDF format here.