Smooth Wasserstein Distance: Metric Structure and Statistical Efficiency

Abstract: The Wasserstein distance has seen a surge of interest and applications in machine learning. This stems from many advantageous properties it possesses, such as metric structure (it metrizes weak convergence), robustness to support mismatch, compatibility to gradient-based optimization, and rich geometric properties. In practice, however, we rarely have access to the true distributions and only get samples from them, which highlights a major drawback of the Wasserstein distance: it suffer from the curse of dimensionality when estimated from samples, limiting its applicability in high dimensions. 

This talk proposes a novel smooth 1-Wasserstein distance (W1), that achieves the best of both worlds -- preserving the Wasserstein metric structure while alleviating the empirical curse of dimensionality. Specifically, we will show that the empirical approximation error under smooth W1: (i) decays at a dimension-free rate of n^{-1/2} in expectation, and (ii) satisfies a high-dimensional central limit theorem. This contrasts the classic (unsmooth) W1 case, where the expected convergence rate (for d-dimensional distributions) is n^{-1/d} and a limit distribution result is known only when d=1. These statistical properties pose smooth W1 as favorable alternative to its classic counterpart for high-dimensional analysis and applications. As such, we will explore the utility of the proposed framework for generative modeling.

Bio: Ziv Goldfeld is an Assistant Professor in the School of Electrical and Computer Engineering at Cornell University, joined in July 2019. During the 2017-2019 academic years, he was a postdoctoral research fellow in the Laboratory for Information and Decision Systems (LIDS) of the Electrical Engineering and Computer Science Department at MIT. Before that, Ziv graduated with a B.Sc., M.Sc. and Ph.D. (all summa cum laude) in Electrical and Computer Engineering from Ben Gurion University, Israel, in 2012, 2012 and 2017, respectively.

Ziv's research interests include optimal transport theory, statistical machine learning, information theory, high-dimensional and nonparametric statistic, applied probability and interacting particle systems. He seeks to understand and design engineering systems by formulating and solving mathematical models. A main focus of his is a principled approach towards machine learning rooted in theoretical principles.

Honors include NSF CISE Research Initiation Initiative (CRII) award, Rothschild postdoctoral fellowship, the Ben Gurion postdoctoral fellowship, the Feder award, a best student paper award in the IEEE 28-th Convention of Electrical and Electronics Engineers in Israel, the Basor fellowship for outstanding students in the direct Ph.D. program, the Lev-Zion fellowship and the Minerva Short-Term Research Grant (MRG).