| Week 1 - January 22 |
Lecture |
Introduction [slides] |
No reading material |
| January 22 |
Assignment |
HW0 released |
[pdf] [data] [solutions] |
| Week 2 - January 27 |
Lecture |
Language Modeling / N-grams [slides] |
JM chapter3 (focus on 3.1, 3.4, 3.6.1) |
| January 29 |
Lecture |
n-gram LMs revisited; POS tagging; Sequence tagging: HMMs intro [slides] |
JM ch17-17.2; 17.4.4 |
| January 31 |
Assignment |
HW0 due (11.59 p.m.) |
|
| Week 3 - February 3 |
Lecture |
Viterbi; NER [slides] |
JM ch17.3, 17.4.5; JM ch3.6.2 |
| February 3 |
Assignment |
HW1 released |
[hw1-release ed pointer] |
| February 5 |
Lecture |
Viterbi Walkthrough [slides], MEMMs [slides] |
|
| Week 4 - February 10 |
Lecture |
Text Classification [slides] |
JM ch5.1 - 5.62 |
| February 12 |
Lecture |
Word Embeddings [slides] |
JM ch6.3 - 6.5, 6.8-6.11 |
| February 12 |
Assignment |
HW1 Milestone due (11.59 p.m.) |
|
| Week 5 - February 17 |
Lecture |
[NO CLASS - February Break] |
|
| February 19 |
Lecture |
FFNNs[slides] |
JM ch7 - 7.3 |
| February 21 |
Assignment |
HW1 due (11.59 p.m.) |
|
| Week 6 - February 24 |
Lecture |
Error backpropagation; cross-entropy loss; NNs for language modelling[slides] |
JM ch7.4 - 7.5.1; 7.5.5 - 7.8;
For more background, start with JM ch5.4 - 5.6.4 |
| February 24 |
Assignment |
HW2 released |
|
| February 26 |
Tutorial |
Pytorch Tutorial |
|
| Week 7 - March 3 |
Lecture |
RNNs[slides] |
JM ch8 - 8.4 |
| March 5 |
Tutorial |
Midterm Review |
|
| March 6 |
Midterm |
Midterm |
|
| Week 8 - March 10 |
Lecture |
Backprop calculus; computation graphs[slides] |
JM ch7.5.2 - 7.5.5 |
| March 12 |
Lecture |
Encoder-Decoder, Attention[slides] |
JM ch8.6 - 8.8 |
| March 12 |
Assignment |
HW2 Milestone due (11.59 p.m.) |
|
| Week 9 - March 17 |
Lecture |
Transformers [slides] |
JM ch 9.1, 9.3, 9.4 |
| March 19 |
Lecture |
Transformers [slides] |
JM Ch9 |
| March 21 |
Assignment |
HW2 due (11.59 p.m.) |
|
| Week 10 - March 24 |
Lecture |
Transformers (Encoder-Decoder and Encoder-only) [slides] |
|
| March 26 |
Lecture |
Encoder-only models (BERT) [slides] |
JM Ch11 |
| March 26 |
Assignment |
HW3 out |
|
| Week 11 - March 31 |
Lecture |
[NO CLASS - Spring Break] |
|
| April 2 |
Lecture |
[NO CLASS - Spring Break] |
|
| Week 12 - April 7 |
Lecture |
Evaluation Part 1 [slides] |
Perplexity, extrinsic/intrinsic eval covered in JM Ch3.2-3.3; P/R/F in JM Ch4.7 |
| April 9 |
Lecture |
Evaluation of text generation [slides] |
Sampling covered in JM Ch10.2; BLEU and BertScore in
JM Ch13.6.2 and Ch13.6.3 |
| Week 13 - April 14 |
Lecture |
Evaluation of text generation (factuality) [slides] |
|
| April 16 |
Lecture |
Question Answering in NLP [slides] |
JM Ch14 up to but not including 14.1.1 |
| Week 14 - April 21 |
Lecture |
QA and Retrieval Augmented LLMs (RAG)[slides] |
JM Ch14.2 (not including Colbert), Ch14.3-14.4 |
| April 21 |
Assignment |
HW3 due (11.59 p.m.) |
|
| April 21 |
Assignment |
HW4 out |
|
| April 23 |
Lecture |
Transformers Review |
|
| Week 15 - April 28 |
Lecture |
Modern LLMs: In-context Learning, Prompting, Chain of Thought [slides] |
JM Ch12-12.4 |
|
| April 30 |
Lecture |
LLM++: Efficiency |
|
| Week 16 - May 5 |
Tutorial |
Review Session (Final) |
|
| May 6 |
Assignment |
HW4 due (11.59 p.m.) |
|
| May 16 2-4:30pm, STL 185-Aud |
Endsem |
Final Exam |
|