Manifold Learning
Manifold Learning (often also referred to as non-linear dimensionality reduction) pursuits the goal to embed data that originally lies in a high dimensional space in a lower dimensional space, while preserving characteristic properties. This is possible because for any high dimensional data to be interesting, it must be intrinsically low dimensional. For example, images of faces might be represented as points in a high dimensional space (let’s say your camera has 5MP -- so your images, considering each pixel consists of three values [r,g,b], lie in a 15M dimensional space), but not every 5MP image is a face. Faces lie on a sub-manifold in this high dimensional space. A sub-manifold is locally Euclidean, i.e. if you take two very similar points, for example two images of identical twins, you can interpolate between them and still obtain an image on the manifold, but globally not Euclidean -- if you take two images that are very different --- for example Arnold Schwarzenegger and Hillary Clinton -- you cannot interpolate between them. I develop algorithms that map these high dimensional data points into a low dimensional space, while preserving local neighborhoods. This can be interpreted as a non-linear generalization of PCA.
Relevant publications:
[PDF][CODE][BIBTEX] Wenlin Chen, Kilian Q. Weinberger, and Yixin Chen. Maximum Variance Correction. Proceedings of 30th International Conference on Machine Learning (ICML), Atlanta, GA (in press).
[PDF][BIBTEX] J. M. Lewis, P. M. Hull, K. Q. Weinberger, and L. K. Saul (2008). Mapping Uncharted Waters: Exploratory Analysis, Visualization, and Clustering of Oceanographic Data. In Proceedings of the Seventh International Conference on Machine Learning and Applications (ICMLA-08). San Diego, CA.
[BIBTEX] K. Q. Weinberger PhD Thesis, Metric Learning with Convex Optimization. University of Pennsylvania. PhD committee: Lawrence K. Saul (chair), Fernando C. N. Pereira, Daniel D. Lee, Gert Lanckriet, Ben Taskar
[PDF][CODE][BIBTEX] K. Q. Weinberger, F. Sha, Q. Zhu and L. K. Saul (2007). In B. Schoelkopf, J. Platt, and T. Hofmann (eds.). Graph Laplacian Regularization for Large-Scale Semidefinite Programming. Advances in Neural Information Processing Systems (NIPS) 19. MIT Press: Cambridge, MA.
[PDF] [CODE][BIBTEX] K. Q. Weinberger and L. K. Saul (2006). An Introduction to Nonlinear Dimensionality Reduction by Maximum Variance Unfolding. National Conference on Artificial Intelligence (AAAI), Nectar paper, Boston MA
[PDF] [CODE][BIBTEX] K. Q. Weinberger and L. K. Saul (2006) International Journal of Computer Vision. Unsupervised Learning of Image Manifolds by Semidefinite Programming. Please download from www.springerlink.com. In Special Issue: Computer Vision and Pattern Recognition-CVPR 2004 Guest Editor(s): Aaron Bobick, Rama Chellappa, Larry Davis, Pages 77-90, Volume 70, Number 1, Springer Netherlands
[PDF][BIBTEX] L. K. Saul, K. Q. Weinberger, J. H. Ham, F. Sha, and D. D. Lee (2006). Spectral methods for dimensionality reduction. In O. Chapelle, B. Schoelkopf, and A. Zien (eds.), Semisupervised Learning, pages 293-308. MIT Press: Cambridge, MA.
[PDF] [CODE][BIBTEX] K. Q. Weinberger, B. D. Packer, and L. K. Saul (2005). Nonlinear dimensionality reduction by semidefinite programming and kernel matrix factorization. In Z. Ghahramani and R. Cowell (eds.), Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics (AISTATS), pages 381-388. Barbados, West Indies. Outstanding student paper award.
[PDF][BIBTEX] J. Blitzer, K. Q. Weinberger, L. K. Saul, and F. C. N. Pereira (2005). Hierarchical distributed representations for statistical language models. In L. K. Saul, Y. Weiss, and L. Bottou (eds.), Advances in Neural Information Processing Systems 17 (NIPS), pages 185-192. MIT Press: Cambridge, MA.
[PDF][CODE][BIBTEX] K. Q. Weinberger, F. Sha, and L. K. Saul (2004), ICML: Learning a kernel matrix for nonlinear dimensionality reduction, In Proceedings of the Twenty First International Confernence on Machine Learning (ICML-04), Banff, Canada. Outstanding student paper award.
[PDF][CODE][BIBTEX] K. Q. Weinberger and L. K. Saul (2004). Unsupervised learning of image manifolds by semidefinite programming. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR-04), vol. 2, pages 988-995. Washington D.C. Outstanding student paper award.