Learning from Implicit Feedback Through Online Experimentation

Project Goals

The goal of the project is to harness the information contained in users' interactions with information systems (e.g. query reformulations, clicks, dwell time) to train those systems to better serve their users' information needs. The key challenge lies in properly interpreting this implicit feedback and collecting it in a way that provides valid training data. Moving beyond existing passive data collection methods, the project draws on multi-armed bandit algorithms, experiment design, and machine learning to actively collect implicit feedback data. Developing these interactive experimentation methods goes hand-in-hand with developing machine learning algorithms that can use the resulting training data, and empirical evaluations that validate the models of user behavior assumed by the algorithms.

This research will improve retrieval quality for important applications like intranet search and desktop search. Additionally, the project will provide an operational full-text search engine for the Physics E-Print ArXiv and potentially other digital libraries, thus forming a test-bed for the research while also providing a valuable service and dissemination tool to the academic community beyond computer science. Including an REU Supplement, the project provides interesting and motivating research opportunities to undergrads and international exchange students, and the PI's will include relevant material into the undergraduate and graduate curriculum. Finally, the project will provide easy-to-use software that enables research and teaching, via this project website.

People

• Thorsten Joachims (PI)
• Robert Kleinberg (Co-PI)
• Pannaga Shivaswamy (Postdoc)
• Yisong Yue (Ph.D. Student)
• Yue Gao (Ph.D. Student)
• Karthik Raman (Ph.D. Student)
• Joshua Moore (Ph.D. Student)
• Jacob Banks (Ugrad Student)
• Christina Brandt (Ugrad/Master Student)
• Andrew Hoelscher (Ugrad Student)
• David Kupiec (Ugrad Student)
• Hooyeon Haden Lee (Ugrad Student / Research Assistant)

Related Publications

Related Software

• Dynamic Ranking - software for Dynamic Ranked Retrieval.
• OSMOT Search Engine - open-source search engine for learning ranked retrieval functions and evaluating the learned rankings.

Related Courses

• T. Joachims, Advanced Topics in Machine Learning: "Learning with Humans in the Loop", Spring 2010.

Related Presentations

• T. Joachims, The Value of User Feedback. Invited Keynote Talk, European Conference on Information Retrieval, 2011.
• P. Shivaswamy, Multi-armed Bandit Problems with History, The Learning Workshop, 2011.
• T. Joachims, Dynamic Ranked Retrieval, Conference on Web Search and Data Mining (WSDM), 2011.
• T. Joachims, Learning with Humans in the Loop. Invited Talk, Machine Learning Distinguished Lecture Series, Carnegie Mellon University, 2010.
• T. Joachims, Optimizing Search Engines as a Dueling Bandits Problem. Invited Talk, ICML Workshop on Reinforcement Learning and Search in Very Large Spaces, Haifa, Israel, 2010.
• Yisong Yue, An Interactive Learning Approach to Optimizing Information Retrieval Systems.
  • Carnegie Mellon University, Pittsburgh, PA, 2010.
  • Yahoo! Research, Santa Clara, CA, 2010.
  • Google Zurich, Zurich, Switzerland, 2010.
  • Microsoft Research Asia, Beijing, China, 2010.
• Yisong Yue, New Learning Frameworks for Information Retrieval.
  • Microsoft Research, Redmond, WA, 2010. (video)
  • Google, Mountain View, CA, 2010.
  • Johns Hopkins University, Baltimore, MD, 2010.
  • Yahoo! Research, Sunnyvale, CA, 2010.
  • Carnegie Mellon University, Pittsburgh, PA, 2010.
  • Cornell University, Ithaca, NY, 2010.
  • IBM TJ Watson, Hawthorne, NY, 2009.
• T. Joachims, Evaluating and Optimizing Search Engines through Interactive Experiments. Invited Talk, Microsoft Research, Cambridge, UK, 2009.
• T. Joachims, Measuring Result Utility through Interactive Experiments. Invited Talk, Seminar on Interactive Information Retrieval, Dagstuhl, Germany, 2009.
• Yisong Yue, Interactively Optimizing Information Retrieval Systems as a Dueling Bandits Problem.
  • New York Academy of Sciences Machine Learning Symposium, New York, NY, 2009.
  • Cornell University, Ithaca, NY, 2009.
• T. Joachims, How does Clickthrough Data Reflect Retrieval Quality? Invited Talk, University of Glasgow, Glasgow, UK, 2009.
• Yisong Yue, Towards Interactive Approaches to Learning to Rank. Invited Talk. SIGIR 2009 Workshop on Learning to Rank, Boston, MA, 2009.
• T. Joachims, Search Engines that Learn. Invited Talk, Fraunhofer IAIS, Sankt Augustin, Germany.
• R. Kleinberg, Some Vignettes From Learning Theory. Invited Talk, Microsoft Faculty Summit, Redmond, WA, 2009.

Acknowledgement and Disclaimer

This material is based upon work supported by the National Science Foundation under Award IIS-0905467. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation (NSF).

Last change: 7/13/2011