Instructor: Wen Sun


Office hours: Thursdays 2 - 3 pm in Gates Hall 416b

Lectures: Tuesday and Thursday from 8:40 am to 9:55 am in Bailey Hall 101.

Course staff office hours:Canvas Calendar (location: Rhodes 503)

Course overview: The course provides an introduction to machine learning, focusing on supervised learning and its theoretical foundations. Topics include regularized linear models, boosting, kernels, deep networks, generative models, online learning, and ethical questions arising in ML applications.

Prerequisites: probability theory (e.g. BTRY 3080, ECON 3130, MATH 4710, ENGRD 2700), linear algebra (e.g. MATH 2940), calculus (e.g. MATH 1920), and programming proficiency (e.g. CS 2110).

Course logistics: For enrolled students the companion Canvas page serves as a hub for access to Ed Discussions (the course forum), Vocareum (for course projects), Gradescope (for HWs), and paper comprehension quizzes. If you are enrolled in the course you should automatically have access to the site. Please let us know if you are unable to access it.

Homework, projects, and exams

Your grade in this course is comprised of three components: homework, exams, and projects. Please also read through the given references in concert with the lectures.


Final grades are based on homework assignments, programming projects, and the exams. For the 5780 level version of the course the research comprehension quizzes will also factor in.

For CS 4780 your final grade consists of: For CS 5780 your final grade consists of:

Undergraduates enrolled in 4780 may choose to do the paper comprehension assignments; if completed you will receive the higher of your two grades between the above schemes.


A tentative schedule follows, and includes the topics we will be covering, relevant reference material, and assignment information. It is quite possible the specific topics covered on a given day will change slightly. This is particularly true for the lectures in the latter part of the course, and this schedule will be updated as necessary.

Date Topic References Notes/assignments
8/22 Introduction Slides PML: 1.1; ESL: Ch. 1; and PPA: Ch. 1
8/24 ML Basics Slides, Annotated Slides, Lecture note PML: 1.2, and ESL: 2.1 and PPA: Ch. 1
8/29 K Nearest Neighbors and the curse of dimensionality Slides, Annotated Slides, Lectue note, PML: 16.1
8/31 Clustering: K-means Slides, Annotated Slides, Lecture note, ESL: 14.3.6 and 14.3.7, and PML: 21.3
9/5 Principal Component Analysis Slides, Annotated Slides,Lecture note, PML: 20.1, ESL: 14.5.1 and 14.5.2
9/7 The Perceptron Slides, Annotated Slides, Lecture note, Wikipedia articlea
9/12 MLE and MAP Slides, Annotated Slides, Lecture note, ESL: 8.2.2-8.3 Homework 1 due; Project 1 due
9/14 Naive Bays Slides, Annotated Slides, Lecture note
9/19 Logistic Regression and Gradient descent Slides, Annotated Slides,Lecture note on logistic regression, Lecture note on Optimization, ESL: 4.4, and PML: 10.1, 0.2, and 10.3
9/21 Adaptive Gradient methods Slides, Annotated Slides, Lecture note on Optimization, PML: 8.1, 8.2, and 8.3 Homework 2 due
9/26 Stochastic Gradient Descent Slides,Annotated Slides, PML: 8.1, 8.2, 8.3, and 8.4 (specifically, see PML 8.4 for SGD) Project 2 due
9/28 Linear regression Slides, Annotated Slides, Lecture note, PML 11.1, 11.2,11.3; ESL 3.2
10/3 Support Vector Machine Slides, Annotated Slides, Lecture note Project 3 due
10/5 SVM (continued) and Empirical Risk Minimization Slides, Annotated Slides, PML 4.3, 5.4 Homework 3 due
10/10 Fall break, no class
10/12 Midterm Jeopardy Slides Homework 4 due 10/13
10/17 Exam Review Evening prelim
10/19 Bias and Variance Tradeoff Slides,Annotated Slides, lecture note
10/24 Bias and Variance Tradeoff and Model Selection Slides, Annotated slides, Lecture note, Note Project 4 due
10/26 Kernels, part 1 Slides, Annotated Slides, Lecture node, PML: 17.1
10/31 Kernels, part 2 Slides, Annotated Slides, Lecture Node, PML: 17.3
11/2 Classification and regression trees Slides,Annotated Slides, Lecture node Homework 5 due; Project 5 due
11/7 Ensemble Methods: Bagging & random forest Slides, Annotated slides, Lecture node
11/9 Ensemble Methods: Boosting Slides, Annotated Slides, Lecture note
11/14 Neural Network Slides, Annotated Slides, Lecture note Project 6 due
11/16 Neural Network: backpropagation Slides,Annotated Slides, Note Homework 6 due
11/21 Neural networks: convolutional and residual layers Slides, Annotated Slides
11/23 Thanksgiving, no class
11/28 Neural networks: transformers Slides, Annotated Slides Project 7 due
11/30 Neural networks: LLMs PML: 15.4 and 15.5 Homework 7 due; Project 8 due at Dec 4
12/08 Final Exam location and time tbd


While this course does not explicitly follow a specific textbook, there are several that are very useful references to supplement the course.


Additional references

Background references


Course policies


You should expect and demand to be treated by your classmates and the course staff with respect. You belong here, and we are here to help you learn and enjoy this course. If any incident occurs that challenges this commitment to a supportive and inclusive environment, please let the instructors know so that the issue can be addressed. We are personally committed to this, and subscribe to the Computer Science Department’s Values of Inclusion. [Statement reproduced with permission from Dan Grossman.]

Mental health resources

Cornell University provides a comprehensive set of mental health resources and the student group Body Positive Cornell has put together a flyer outlined the resources available.


You are encouraged to actively participate in class. This can take the form of asking questions in class, responding to questions to the class, and actively asking/answering questions on the online discussion board.

Collaboration policy

Students are free to share code and ideas within their stated project/homework group for a given assignment, but should not discuss details about an assignment with individuals outside their group. The midterm and final exam are individual assignments and must be completed by yourself.

Students are free to use AI tools (e.g., ChatGPT, GPT4) for a given assignment. Students must include a detailed description of how they use the AI tools in their submission. Such details include (but not limited to) what AI tools are used, what prompts are used, how the AI generated content is used in the assignment, etc. Note that while GPTs are great products of ML,these models can still generate incorrect answers (i.e., these models can hallucinate). Always use AI tools with caution.

Academic integrity

The Cornell Code of Academic Integrity applies to this course.


In compliance with the Cornell University policy and equal access laws, we are available to discuss appropriate academic accommodations that may be required for student with disabilities. Requests for academic accommodations are to be made during the first three weeks of the semester, except for unusual circumstances, so arrangements can be made. Students are encouraged to register with Student Disability Services to verify their eligibility for appropriate accommodations.