CS 5740 SP20

Time: MoWe 11:00am-12:15pm
Room: Bloomberg Center 131
Listing: CS 5740

Instructor: Yoav Artzi
Teaching assistants: Rishi Bommasani and Max Grusky
Graders: Kuan-Wen Wang and Yong Huang

CMS | Forum

Schedule

Assignments and Exam

  Release Date Due Date
Assignment 1 TBD TBD
Assignment 2 TBD TBD
Assignment 3 TBD TBD
Assignment 4 TBD TBD
Assignment 5 TBD TBD
Final exam May 8 (8am) May 12 (11:59pm)

Lectures

Bold readings are the highest priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.

Date Topic Board Recommended Readings
Jan 22 Introduction   NLP (circa 2001)
Jan 27      
Jan 29      
Feb 3      
Feb 5      
Feb 10      
Feb 12      
Feb 17      
Feb 19      
Feb 24 No class – February break    
Feb 26      
Mar 2      
Mar 4      
Mar 9      
Mar 11      
Mar 16      
Mar 18      
Mar 23      
Mar 25      
Mar 30 No class – spring break    
Apr 1 No class – spring break    
Apr 6      
Apr 8      
Apr 13      
Apr 15      
Apr 20      
Apr 22      
Apr 27      
Apr 29      
May 4      

Upcoming Topics

As we reach a topic, it will be moved to the schedule table above.

Topic Recommended Readings
Text classification M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation
Neural networks Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Computation graphs NN Tips, Intro to Computation Graphs
Lexical semantics and embeddings w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010
Language modeling J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN
Sequence modeling J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger
Recurrent neural networks G14, BPTT, RNN Tutorial, Effectiveness, Luong2015
Dependency parsing J&M 12.7, Nivre2003, Chen2014
Convolutional neural networks G13, Kim2014, Jacovi2018
Transformers Annotated Transformer, Illustrated Transformer
Contextualized representations BERT, The Illustrated BERT, ELMo, and co., Chen2019
IBM translation models J&M 25.5, M&S 13.1-13.2, Collins: IBM Models, IBM Models, Collins: EM (Sec 5-6), HMM alignments, IBM Model 2 EM Notebook, BLEU Score, Neural MT Tutorial
Constituency parsing J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside
Phrase-based machine translation J&M 25.4, 25.8, M&S 13.3, Collins: PBT, Statistical PBT, Pharaoh decoder

If time allows, we will discuss compositional semantics, summarization, and question answering.