AI Seminar - Focusing on the Representation

Focusing on the Representation

Abstract:  What is the point of machine learning?  Arguably it is to learn how to extract useful representations of data.  Despite the representation being the implicit goal, most objectives in machine learning are naturally generative in nature.  With modern variational approaches, we can instead directly optimize constrained information-theoretic objectives that put the representation first. Doing so has demonstrated improvements in generalization, adversarial robustness, calibration, and out-of-sample detection for supervised-learning. It has also led to better understanding of some of the deficiencies present in modern unsupervised-learning in the form of variational autoencoders (VAEs).  In this talk, I'll try to convince you that representations matter, highlighting some of their successes and discussing their potential future.