Quantifying Availability and Discovery in Recommender Systems via Reachability

Abstract: Personalized preference models mediate access to a large portion of online content through their use in recommendation systems. It is therefore important to look beyond measures of accuracy towards notions of access. In this talk, I propose an evaluation based on reachability, which quantifies the maximum probability of recommending a target piece of content over a set of allowable strategic modifications of a user's features. While much recent work in robust machine learning casts strategic manipulation as undesirable, we view the ability of users to influence the model in a positive light.

The proposed framework allows us to compute an upper bound on the likelihood of a recommendation with minimal assumptions about user behavior. Stochastic reachability can be used to detect biases in the availability of content and diagnose limitations in the opportunities for discovery granted to users. We will see that this metric can be computed efficiently and that it is not inherently at odds with accuracy. We demonstrate evaluations of recommendation algorithms trained on large datasets and end with a discussion of open problems.

Joint work with Mihaela Curmei, Ben Recht, and Sarah Rich.

Biography: Sarah is an incoming assistant professor in the Computer Science Department at Cornell. She recently completed her PhD in EECS from UC Berkeley, where she was advised by Ben Recht, and is currently a postdoc with Jamie Morgenstern at the University of Washington. Sarah is interested in the interplay between optimization, machine learning, and dynamics in real-world systems. Her research focuses on understanding the fundamentals of data-driven control and decision-making, broadly categorized into two thrusts: guaranteeing safety in feedback control and ensuring values in social-digital systems. This work is grounded in and inspired by collaborative projects in applications ranging from robotics to recommendation systems.