Instructors: Kilian Q. Weinberger and Karthik Sridharan

cs4780logo

Contact: Ed Discussions (most questions), cs4780staff@gmail.com (sensitive or discreet inquiries only)

Course staff office hours: Calendar link

Office hours:
Kilian Weinberger : Mondays 9:00 - 10:00 am (Booking Link) in 410 Gates Hall
Karthik Sridharan : Friday 2 - 3 pm (Booking Link) in 424 Gates Hall

Lectures: Tuesday and Thursday from 1:25 pm to 2:40 pm in Uris Hall G01.

Course overview: The course provides an introduction to machine learning, focusing on supervised learning and its theoretical foundations. Topics include regularized linear models, boosting, kernels, deep networks, generative models, online learning, and ethical questions arising in ML applications.

Prerequisites: probability theory (e.g. BTRY 3080, ECON 3130, MATH 4710, ENGRD 2700), linear algebra (e.g. MATH 2940), calculus (e.g. MATH 1920), and programming proficiency (e.g. CS 2110).

Course logistics: For enrolled students the companion Canvas page serves as a hub for access to Ed Discussions (the course forum), Vocareum (for course projects), Gradescope (for HWs), and paper comprehension quizzes. If you are enrolled in the course you should automatically have access to the site. Please let us know if you are unable to access it.

Homework, projects, and exams


Your grade in this course is comprised of three components: homework, exams, and projects. Please also read through the given references in concert with the lectures.

Grading

Final grades are based on homework assignments, programming projects, and the exams. For the 5780 level version of the course, the research comprehension quizzes will also factor in.

For CS 4780 your final grade consists of: For CS 5780 your final grade consists of:

Undergraduates enrolled in 4780 may choose to do the paper comprehension assignments; if completed you will receive the higher of your two grades between the above schemes.

Schedule


A tentative schedule follows, and includes the topics we will be covering, relevant reference material, and assignment information. It is quite possible the specific topics covered on a given day will change slightly. This is particularly true for the lectures in the latter part of the course, and this schedule will be updated as necessary. Please note that the due dates here are mostly correct, but may change. Check Canvas for any changes to assignment due dates.

Homework 7 due 4/28
-->
Date Topic References Notes/assignments
1/23/24 Introduction PML: 1.1; ESL: Ch. 1; and PPA: Ch. 1 slides
1/25/24 ML Basics PML: 1.2, and ESL: 2.1 and 2.2. html pdf
handwritten
1/30/24 K Nearest Neighbors and the curse of dimensionality PML: 16.1 html pdf
handwritten
5780: Cover and Hart 1967
2/1/24 The Perceptron Wikipedia article html pdf
handwritten
2/6/24 Clustering: K-means ESL: 14.3.6 and 14.3.7, and PML: 21.3 html handwritten
k-means demo DBSCAN demo GMM demo DBSCAN Colab Demo
2/8/24 Principal Component Analysis PML: 20.1, ESL: 14.5.1 and 14.5.2 Project 0 due; Project 2 out; Quiz 2 out
html handwritten
PCA Colab demo
2/13/24 MLE and MAP Nice Youtube video for MLE and MAP.
Ben Taskar's lecture notes.
Tom Mitchell's book chapter on MLE and MAP
ESL: 8.2.2-8.3
html pdf
handwritten
Homework 1 due
2/15/24 Naive Bayes ESL: 6.6.3, and Tom Mitchell's book chapter Quiz 1 (optional) due; Project 1 due on 2/17
html pdf
handwritten
2/20/24 Logistic Regression PML: 8.1, 8.2, and 8.3 logistic html logistic pdf;
2/22/24 Gradient Descent, AdaGrad, Newton's method PML: 8.1, 8.2, 8.3, and 8.4 (specifically, see PML 8.4 for SGD) Project 2 due; Eiganfaces Paper Quiz 2 due
GD html GD pdf
handwritten
Jupyter Notebook demo
2/27/23 February break no class
2/29/24 Linear regression PML 11.1, 11.2,11.3 and ESL 3.2 Homework 2 due on 3/1
html pdf
3/5/24 Support Vector Machine Project 3 due
html pdf
3/7/24 Empirical Risk Minimization PML 4.3, 5.4 Quiz 3 due
html pdf
3/12/24 Midterm Review Homework 3 due
3/14/24 Midterm Optional Review A.K.A. Midterm Jeopardy Prelim Location: Statler Hall 185, 196
Prelim Time: 7:30pm
3/19/24 Model Selection html pdf
3/21/24 Bias and Variance Tradeoff html pdf
handwritten
3/26/24 Kernels, part 1 PML: 17.1 Project 4 due
html pdf handwritten
3/28/24 Kernel SVM kernel html kernel pdf kernel slides Kernel Ridge Regression Demo
4/2/24 Spring Break Woohooo!!
4/4/24 Spring Break Woohooo!!
4/9/24 Kernel SVM + CART PML: 17.3 kernel html kernel pdf kernel slides Kernel Ridge Regression Demo
4/11/24 Classification and regression trees Project 5 due, Homework 5 deadline with slip days 4/12, Quiz 4 due
Decision Tree html pdf Classification Tree Demo
Regression Tree Demo
4/16/24 Ensemble Methods: Bagging & random forest html pdf handwritten
4/18/24 Ensemble Methods: Boosting Homework 6 due 4/19
html pdf handwritten
4/23/24 Neural Networks Project 6 due 4/22
handwritten pdf
4/25/24 Neural Networks Cont. Homework 7 due 4/28
handwritten pdf Link to a cool demo!
4/30/24 Convolutional Neural Networks PML: 14.1, 14.2, 14.3,15.4, 15.5 slides
5/2/24 Neural networks: Transformers Transformer Algorithm
Transformers explained
Project 7 due
pdf slides
5/7/24 AI in Human Society Project 8 due, Kaggle due, Quiz 5 due
pdf
5/11/24 Final Exam Location: Barton Hall
Time: 5/11 7PM

References


While this course does not explicitly follow a specific textbook, there are several that are very useful references to supplement the course.

Books

We will not be explicitly following any single textbook in this course. Nevertheless, the book by Murphy roughly cover the material for the course and is recommended. Most suggested readings are assigned out of these two texts. Three additional texts are provided that complement these texts and are useful for further study (or to gain another perspective).

Additional references

Background references

Software

Course policies


Inclusiveness

You should expect and demand to be treated by your classmates and the course staff with respect. You belong here, and we are here to help you learn and enjoy this course. If any incident occurs that challenges this commitment to a supportive and inclusive environment, please let the instructors know so that the issue can be addressed. We are personally committed to this, and subscribe to the Computer Science Department’s Values of Inclusion. [Statement reproduced with permission from Dan Grossman.]

Mental health resources

Cornell University provides a comprehensive set of mental health resources and the student group Body Positive Cornell has put together a flyer outlined the resources available.

Participation

You are encouraged to actively participate in class. This can take the form of asking questions in class, responding to questions to the class, and actively asking/answering questions on the online discussion board.

Collaboration policy

Students are free to share code and ideas within their stated project/homework group for a given assignment, but should not discuss details about an assignment with individuals outside their group. The midterm and final exam are individual assignments and must be completed by yourself.

Academic integrity

The Cornell Code of Academic Integrity applies to this course.

Accommodations

In compliance with the Cornell University policy and equal access laws, we are available to discuss appropriate academic accommodations that may be required for student with disabilities. Requests for academic accommodations are to be made during the first three weeks of the semester, except for unusual circumstances, so arrangements can be made. Students are encouraged to register with Student Disability Services to verify their eligibility for appropriate accommodations.