# Lectures

- Lecture 1, Thursday Jan 22: course introduction, beginning of dimensionality reduction.
- Lecture 2, Thursday Jan 27: Principal Component Analysis (PCA), additional lecture notes lecture notes, matlab code for the smilie example for you guys to play with code .
- Lecture 3, Thursday Jan 29: Geometric intuitions regarding
Principal Component Analysis (PCA).
- lecture notes
- handout, with corrections (missing transpose indicators in the data matrix's rows, covariance matrix mistakenly talked about the rows instead of the columns)
- PCA example in Python from class, slightly extended version lec03PCAExample2.py
- Java version of the PCA example done in class, by TA Jack Hessel
- C++ version of a PCA example, by TA Gaurav Aggarwal

- Lecture 4, Tuesday Feb 3: Introduction to canonical correlation analysis (CCA).
- lecture notes
- handouts: announcements, outline and cribsheet, Penn State example
- covariance vs. correlation for the pens-and-pencils example: R program
- Penn State Stats 505 CCA example: detailed writeup, R code, data . Very accessible.
- UCLA IDRE CCA example: detailed writeup, R code, data . Rather accessible.
- Warning regarding CCA implementation in scikit-learn (python)
- Canonical correlation: A tutorial, by Magnus Borga, dated Jan 2001. Not too gentle, but useful.
- Analysis of factors and canonical correlations, Mans Thulin, dated 2011. Not too gentle, but gives a different perspective and an example.

- Lecture 5, Thursday Feb 05: More on Canonical Correlation Analysis.
- Lecture 6, Tuesday Feb 10: Random projections.
- handout: updated cheatsheet on projections

- Lecture 7, Thursday Feb 19: review/A1 overview.
- lecture notes
- handout: updated cheatsheet on projections (links to same file as mentioned in lecture 6)
- handout: selected review questions

- Lecture 8, Tuesday Feb 24: Compressed Sensing and Sparse Recovery.
- Lecture 9, Thursday Feb 26: Introduction (transition) to clustering
- updated lecture handout
- lecture notes
- References:
- Section 10.7 of Richard O. Duda, Peter E. Hart, David G. Stork, 2001,
*Pattern Classification*(2nd ed) - Sections 9.3-9.4 of David Hand, Heikki Mannila, and Padhraic Smyth, 2001,
*Principles of Data Mining*(link gives access to Cornellians; you may need to be coming from a Cornell IP address). Official book link at MIT Press here - Sections 14.3.5-14.3.6 of Trevor Hastie, Robert Tibshirani and Jerome Friedman, 2009
*The Elements of Statistical Learning*(2nd ed)

- Section 10.7 of Richard O. Duda, Peter E. Hart, David G. Stork, 2001,

- Lecture 10, Tuesday Mar 3: k-means clustering
- lecture notes
- updated lecture handout
- k-means visualization by Naftali Harris
- References:
- Section 8.2 ("A k-means clustering algorithm") of John Hopcroft and Ravindran Kannan, draft of November 2014,
*Foundations of Data Science* - Section 4.7 ("Clustering") of Jon Kleinberg and Éva Tardos, 2005,
*Algorithm Design*. Slides by Kevin Wayne here.

- Section 8.2 ("A k-means clustering algorithm") of John Hopcroft and Ravindran Kannan, draft of November 2014,

- Lecture 11, Thursday Mar 05: Single-link clustering's optimality; spectral clustering
- lecture notes on single-link clustering; lecture notes on spectral clustering

- Lecture 12, Tuesday Mar 10: Spectral Clustering continued
- Lecture 13, Thursday Mar 12: Kleinberg's impossibility theorem for clustering
- lecture notes
- lecture handout
- iClicker results analysis
- Reference: Jon Kleinberg. An impossibility theorem for clustering. NIPS 2003.

- Lecture 14, Tuesday Mar 17: More on the impossibility theorem; intro to ((Gaussian) mixture) models
- lecture notes
- lecture handout, with minor updates
- Reference: Sections 10.1-10.3 (ignore the material on identifiability in 10.2) of Richard O. Duda, Peter E. Hart, David G. Stork, 2001,
*Pattern Classification*(2nd ed)

- Lecture 15, Thursday Mar 19: MLE vs. MAP principles, probabilistic modeling, towards EM.
- lecture slides, with minor updates
- Lecture 16, Tuesday Mar 24: EM Algorithm
- lecture slides
- for now, until the typeset notes come out: handwritten notes scan
- Lecture 17, Thursday Mar 26: Mixtures of multinomials and EM
- Lecture 18, Tuesday Apr 7: Development of Latent Dirichlet Allocation (LDA)
- lecture notes
- lecture handout
- summary/comparison of three generative stories
- References:
- Percy Liang and Dan Klein (2007): Structured Bayesian Nonparametric Models with Variational Inference
- David Blei's group's topic modeling software (C, C++. R, Python)

- Lecture 19, Thursday April 9th: Graphical models
- announcement slides
- handwritten lecture transcript (starts late because Prof Lee was uploading the announcement slides at the beginning)

- Lecture 20, Tuesday April 14th: Graphical models
- handwritten lecture transcript (starts late because Prof Lee was handling some course administrative stuff at the beginning)

- Lecture 21, Thursday April 16th: Graphical models
- announcements
- handwritten lecture transcript (typo fix Fri May 1 9:25am)

- Lecture 22, Tuesday April 21: More intuition building for reasoning with graphical models and HMMs
- lecture notes
- handout
- Reference: Sections 20.2 ("Belief propagation for trees") and 20.3 ("The variable elimination algorithm") of Kevin Murphy, 2012. Machine Learning: A Probabilistic Perspective. MIT Press. (link gives access to Cornellians; you may need to be coming from a Cornell IP address).

- Lecture 23, Thursday April 23: Learning parameters for a Bayes net: the case of EM on HMMs
- lecture notes
- lecture handout
- Reference: Chapter 9.3 of Frederick Jelinek (1997),
*Statistical Methods for Speech Recognition.*Google books link

- Lecture 24,
, G01 Gates Hall. No notes available, by lecturer request.__Sunday__April 26th, 5-6pm, Guest lecture by Lars Backstrom - Lecture 25, Thursday April 30th: Graphical models and Wrapping up
- Lecture 26, Tuesday May 5: Valedictory: Lessons learned