CS 5740 SP20

Time: MoWe 11:00am-12:15pm
Room: Bloomberg Center 131
Listing: CS 5740

Instructor: Yoav Artzi
Teaching assistants: Rishi Bommasani and Max Grusky
Graders: Kuan-Wen Wang and Yong Huang

CMS | Forum

Schedule

Assignments and Exam

  Release Date Due Date
Assignment 1 Feb 5 Feb 26 (11:59pm)
Assignment 2 TBD TBD
Assignment 3 TBD TBD
Assignment 4 TBD TBD
Assignment 5 TBD TBD
Final exam May 6 (8am) May 10 (11:59pm)

Lectures

Bold readings are the highest priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.

Video recordings are available with NetID login.

Date Topic Board Recommended Readings
Jan 22 Introduction PDF NLP (circa 2001)
Jan 27 " PDF  
  Text classification   M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation
Jan 29 " PDF  
Feb 3 " PDF  
Feb 5 " PDF  
Feb 10 "    
  Neural networks PDF Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Feb 12 " PDF  
  Computation graphs   NN Tips, Intro to Computation Graphs
Feb 17 " PDF  
  Meta NLP    
Feb 19 Lexical semantics and embeddings   w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010
Feb 24 No class – February break    
Feb 26      
Mar 2      
Mar 4      
Mar 9      
Mar 11      
Mar 16      
Mar 18      
Mar 23      
Mar 25      
Mar 30 No class – spring break    
Apr 1 No class – spring break    
Apr 6      
Apr 8 Guest lecture by Joel Tetreault (Dataminr)    
Apr 13      
Apr 15      
Apr 20      
Apr 22      
Apr 27      
Apr 29      
May 4      

Upcoming Topics

As we reach a topic, it will be moved to the schedule table above.

Topic Recommended Readings
Language modeling J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN
Sequence modeling J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger
Recurrent neural networks G14, BPTT, RNN Tutorial, Effectiveness, Luong2015
Dependency parsing J&M 12.7, Nivre2003, Chen2014
Convolutional neural networks G13, Kim2014, Jacovi2018
Transformers Annotated Transformer, Illustrated Transformer
Contextualized representations BERT, The Illustrated BERT, ELMo, and co., Chen2019
IBM translation models J&M 25.5, M&S 13.1-13.2, Collins: IBM Models, IBM Models, Collins: EM (Sec 5-6), HMM alignments, IBM Model 2 EM Notebook, BLEU Score, Neural MT Tutorial
Constituency parsing J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside
Phrase-based machine translation J&M 25.4, 25.8, M&S 13.3, Collins: PBT, Statistical PBT, Pharaoh decoder

If time allows, we will discuss compositional semantics, summarization, and question answering.