CS 5740 SP20

Time: MoWe 11:00am-12:15pm
Room: Bloomberg Center 131
Listing: CS 5740

Instructor: Yoav Artzi
Teaching assistants: Rishi Bommasani and Max Grusky
Graders: Kuan-Wen Wang and Yong Huang

CMS | Forum

Schedule

Topic and assignment dates are tentative.

Assignments and Exam

  Release Date Due Date (all 11:59pm)
Assignment 1 Feb 5 Feb 26
Assignment 2 Feb 27 March 18
Assignment 3 March 16 April 9
Project April 9 April 19
Assignment 4 April 20 May 8
Assignment 5 April 20 May 12

Lectures

Starting March 12, all classes will be conducted online. Instructions for connecting to the class and watching recordings are here.

Bold readings are the highest priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.

Video recordings from the first part of the semester are available with NetID login.

Date Topic Board Recommended Readings
Jan 22 Introduction PDF NLP (circa 2001)
Jan 27 " PDF  
  Text classification   M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation
Jan 29 " PDF  
Feb 3 " PDF  
Feb 5 " PDF  
Feb 10 "    
  Neural networks PDF Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Feb 12 " PDF  
  Computation graphs   NN Tips, Intro to Computation Graphs
Feb 17 " PDF  
  Meta NLP    
Feb 19 Lexical semantics and embeddings   w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010
Feb 24 No class – February break    
Feb 26 "    
Mar 2 Language modeling PDF J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN
Mar 4 " PDF  
Mar 9 " PDF  
  Sequence modeling   J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger
Mar 11 " PDF  
Mar 16 " PDF  
Mar 18 " PDF  
  Recurrent neural networks   G14, BPTT, RNN Tutorial, Effectiveness, Luong2015
Mar 23 " PDF  
Mar 25 " PDF  
Mar 30 No class – enjoy spring break!    
Apr 1 No class – enjoy spring break!    
Apr 6 Recurrent neural networks PDF  
  Transformers   Annotated Transformer, Illustrated Transformer
Apr 8 " PDF  
  Contextualized representations   BERT, The Illustrated BERT, ELMo, and co., Chen2019
Apr 13 " PDF  
  Convolutional neural networks   G13, Kim2014, Jacovi2018
Apr 15 Dependency parsing PDF J&M 12.7, Nivre2003, Chen2014
Apr 20 " PDF  
Apr 22 " PDF  
Apr 27 Constituency parsing   J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside
Apr 29 " PDF  
May 4 Guest lecture by Joel Tetreault (Dataminr)