- About
- Events
- Calendar
- Graduation Information
- Cornell Learning Machines Seminar
- Student Colloquium
- BOOM
- Fall 2024 Colloquium
- Conway-Walker Lecture Series
- Salton 2024 Lecture Series
- Seminars / Lectures
- Big Red Hacks
- Cornell University - High School Programming Contests 2024
- Game Design Initiative
- CSMore: The Rising Sophomore Summer Program in Computer Science
- Explore CS Research
- ACSU Research Night
- Cornell Junior Theorists' Workshop 2024
- People
- Courses
- Research
- Undergraduate
- M Eng
- MS
- PhD
- Admissions
- Current Students
- Computer Science Graduate Office Hours
- Advising Guide for Research Students
- Business Card Policy
- Cornell Tech
- Curricular Practical Training
- A & B Exam Scheduling Guidelines
- Fellowship Opportunities
- Field of Computer Science Ph.D. Student Handbook
- Graduate TA Handbook
- Field A Exam Summary Form
- Graduate School Forms
- Instructor / TA Application
- Ph.D. Requirements
- Ph.D. Student Financial Support
- Special Committee Selection
- Travel Funding Opportunities
- Travel Reimbursement Guide
- The Outside Minor Requirement
- Diversity and Inclusion
- Graduation Information
- CS Graduate Minor
- Outreach Opportunities
- Parental Accommodation Policy
- Special Masters
- Student Spotlights
- Contact PhD Office
Cross Lingual Understanding (XLU) through Pretraining: Scaling NLP Across Languages
Abstract: Billions of people around the world use Facebook in over one hundred languages. This linguistic diversity is wonderful, but it presents challenges for Natural Language Processing (NLP) systems. It is simply impossible to annotate data and train a new system for each language. Instead, we rely on Cross-Lingual Understanding (XLU) to learn NLP systems in one language and apply them in languages that are not a part of the original training data.
We have made significant progress in XLU in the last two years. In particular, petraining methods have been an effective way of improving XLU performance. I will give a brief overview of recent pretraining methods such as BERT, XLNet and RoBERTa. I will then cover common XLU benchmark including the Cross-Lingual Natural Language Inference (XNLI) benchmark that we introduced in 2018. I will continue by talking about common methods for XLU and recent progress we have made through pretraining. I will finish by discussing exciting ongoing work in my group.
Bio: Ves is a Research Scientist Manager at Facebook AI focusing on Natural Language Processing (NLP). Before Facebook AI, Ves was a Research Scientist on the Search team at Facebook focusing on NLP uses for Search. Before Facebook, Ves spent three wonderful years as a PostDoc at the Center for Language and Speech Processing at Johns Hopkins University. There he worked with Jason Eisner on Machine Learning for Structured Prediction and was supported by a Computing Innovation Fellowship from the CRA. Ves graduated with a PhD from Cornell University, where he worked with his advisor, Claire Cardie, on opinion analysis. His thesis title is “Opinion Summarization: Automatically Creating Useful Representations Of The Opinions Expressed In Text.” During his PhD, Ves was supported by an NSF Graduate Research Fellowship.