**Instructors:** Anil Damle and Kilian Weinberger

**Contact:** damle@cornell.edu and kilian@cornell.edu

**Office hours:**
Anil (typically Monday 3:15 pm - 4:15 pm and Wednesday 10:30 am - 11:30 am) and Kilian

**Lectures:** Tuesdays and Thursdays from 11:25 am till 12:40 pm in Statler Hall 185 (Statler Auditorium). Lecture recordings are available on Canvas

**Course overview:** The course provides an introduction to machine learning, focusing on supervised learning and its theoretical foundations. Topics include regularized linear models, boosting, kernels, deep networks, generative models, online learning, and ethical questions arising in ML applications.

**Prerequisites:** probability theory (e.g. BTRY 3080, ECON 3130, MATH 4710, ENGRD 2700), linear algebra (e.g. MATH 2940), calculus (e.g. MATH 1920), and programming proficiency (e.g. CS 2110).

**Office Hours:** Calendar link

For enrolled students the companion Canvas page serves as a hub for access to Ed Discussions (the course forum), Vocareum (for course projects), Gradescope (for HWs), and quizzes (for the placement exam and paper comprehension quizzes). If you are enrolled in the course you should automatically have access to the site. Please let us know if you are unable to access it.

**November 9 — Homework 5 due****November 9 — Project 5 due**

Your grade in this course is comprised of three components: homework, exams, and projects. Please also read through the given references in concert with the lectures.

**Homework:**There will be a number of homework assignments throughout the course, typically made available roughly one to two weeks before the due date. The homework primarily focuses on theoretical aspects of the material and is intended to provide preparation for the exams. Homework may be completed in groups of up to two. The assignments themselves will be made available via Gradescope (through Canvas).

**Projects**To provide hands on learning with the methods we will discuss in class there are a number of programming projects throughout the course. The projects may be completed solo or in a group of two. They are accessed through assignments in Canvas that will launch Vocareum.

**Paper comprehension**Students enrolled in this course at the graduate level (i.e., enrolled in 5780) are required to read assigned research papers and complete the associated online quiz. Papers will be assigned roughly once every one to two weeks.

**Exams:**There will be two exams for this class, an evening prelim and a final exam.

- Midterm:
**7:30 pm on Thursday, October 21 in Statler 185 and 196** - Final:
**2:00 pm on Sunday, December 12**

- Midterm:

Final grades are based on homework assignments, programming projects, and the exams. For the 5780 level version of the course the research comprehension quizzes will also factor in.

For CS 4780 your final grade consists of the maximum of either- Exams: 60%
- Projects: 40%

- Exams: 50%
- Homework: 10%
- Projects: 40%

- Exams: 55%
- Projects: 35%
- Paper comprehension: 10%

- Exams: 45%
- Homework: 10%
- Projects: 35%
- Paper comprehension: 10%

Regardless of which grading scheme you are targeting (or ends up being the maximizer), homework must be completed. Homework (staring with homework 2) will be graded for correctness and these scores will be used to compute your overall homework grade. Provided you make a good faith effort (as specified in class) on the homework they do not factor into your final grade under the first scheme above. However, failure to provide a good faith effort for any homework assignments will result in a 5% penalty per missing assignment.

Undergraduates enrolled in 4780 may choose to do the paper comprehension assignments; if completed you will receive the higher of your two grades between the above schemes.

A tentative schedule follows, and includes the topics we will be covering and relevant reference material (More details are given in the references section).

Date | Topic | References | Notes/assignments |
---|---|---|---|

8/26 | Introduction | MLAPP: 1.1, 1.2, and 1.3; ESL: Ch. 1; and PPA: Ch. 1 | Placement exam due 9/3 |

8/31 | ML Basics | Lecture Notes and ESL: 2.1 and 2.2. Personal class notes (handwritten) | |

9/2 | k nearest neighbors and the curse of dimensionality | Lecture Notes and MLAPP: 1.1, 1.2, 1.4.2, 1.4.3, and 1.4.9 | 5780: Cover and Hart 1967 |

9/7 | K-means clustering | Lecture Notes, ESL: 14.3.6 and 14.3.7, and MLaPP: 11.4.1 and 11.4.2 | |

9/9 | Principal component analysis | Lecture Notes, ESL: 14.5.1 and 14.5.2 | |

9/14 | The Perceptron | Lecture Notes, Perceptron Wiki | Project 0 due |

9/16 | MLE and MAP | Lecture Notes, Nice Youtube video for MLE and MAP. Ben Taskar's lecture notes. ESL: 8.2.2-8.3 | Homework 1 due |

9/17 | Project 1 due | ||

9/21 | Naive Bayes | Lecture Notes; ESL: 6.6.3; Tom Mitchell’s book chapter | |

9/23 | Naive Bayes | Lecture Notes; ESL: 6.6.3; Tom Mitchell’s book chapter | 5780: Turk and Pentland 1991 |

9/28 | Logistic regression | Lecture Notes; MLaPP: 8.2, 8.3, and 8.4; ESL 4.4 | HW 2 due |

9/30 | Gradient descent and Newton's method | Lecture Notes; Additional Lecture Notes; MLaPP: 8.2, 8.3, and 8.4 | Project 2 due |

10/5 | Linear Regression | Lecture Notes; MLaPP 8, 8.1, 8.2, 8.3.1, 8.3.2, 8.3.4, 8.6; Tom Mitchell’s book chapter on Naive Bays and Linear Regression; ISL: 3.1; and ESL 3.2 | |

10/7 | Support vector machines (SVM) | Lecture Notes | Project 3 and HW 3 due |

10/12 | No class, Fall Break | ||

10/14 | SVM + Emperical risk minimization (ERM) | Lecture Notes and MLaPP 6.5-6.5.3.2 | |

10/19 | Midterm Review | HW 4 due | |

10/21 | Anil and Kilian OH in class | Midterm at 7:30 pm | |

10/26 | Bias-variance tradeoff | Lecture Notes | |

10/28 | ERM + Model Selection | Lecture Notes | Project 4 due |

11/2 | Kernels, part 1 | Lecture Notes and MLaPP Ch. 14 | |

11/4 | Kernels, part 2 | Lecture Notes and MLaPP Ch. 14 | |

11/9 | Classification and regression trees | Lecture Notes | Homework 5 and Project 5 due (now Nov 10) |

11/11 | Classification and regression trees | Lecture Notes | |

11/16 | Bagging | Lecture Notes and Introduction to Statistical Learning 8.2 | |

11/18 | Boosting | Lecture Notes and Introduction to Statistical Learning 8.2 | Project 6 due |

11/23 | Neural networks | Homework 6 due | |

11/25 | No Class, Thanksgiving | ||

11/30 | Neural networks | Lecture Notes | |

12/2 | Neural networks | Lecture Notes | Project 7 and Homework 7 due |

12/7 | Project 8 due |

While this course does not explicitly follow a specific textbook, there are several that are very useful references to supplement the course.

**Machine Learning A Probabilistic Perspective by Murphy**

We will provide section numbers to this text alongside many of the lectures (abbreviated as MLaPP in the schedule). This text is available digitally through the Cornell University Library. [Cornell library]**The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman**

This text provides a comprehensive introduction to statistical learning and provides in-depth discussion of many of the topics in this course (abbreviated as ESL in the schedule). The book is available directly from the authors. [Book website]

**An Introduction to Statistical Learning by James, Witten, Hastie, and Tibshirani**

This book provides a good overview of some methods in statistical learning, some of which we will discuss. The book is available online through the books website and via the Cornell Library. [Book website]**Patterns, Predictions, and Actions by Hardt and Recht**

A very nice new book that covers many of the topics we do in this class (abbreviated as PPA in the schedule). The book is available directly from the authors. [Book website]**Fairness and Machine Learning by Barocas, Hardt, and Narayanan**

While a work in progress, this text provides insight into fairness as a central tenet of machine learning. In particular, it highlights ethical challenges that arise in the practice of machine learning. The current version of this book is available directly from the authors. [Book website]

**Linear Algebra by Kahn Academy**

Relive the basics of linear Algebra. Everybody loves Kahn Academy. [Linear Algebra (Kahn Academy)]**Linear algebra course by Strang**

Portions of this course will utilize your knowledge of linear algebra. If you feel you need additional preparation, or would like to revisit the topic, you may find Gilbert Strangs linear algebra course quite useful. [MIT Open Courseware]**Matrix Methods in Data Analysis, Signal Processing, and Machine Learning by Strang**

A subsequent course to the above by Strang covers some of the same topics we will (particularly for the linear algebra part of the course) and you may find the videos a useful additional resource. [MIT Open Courseware]

**Python****NumPy****PyTorch**

We understand that the ongoing global health pandemic impacts all of you in varied and profound ways. Therefore, flexibility is important as we continue to navigate the current state of affairs. While many aspects of this course are built with flexibility in mind, if situations arise that may require additional accommodations please reach out to the instructors to discuss potential arrangements.

Cornell University provides a comprehensive set of mental health resources and the student group Body Positive Cornell has put together a flyer outlined the resources available.

You are encouraged to actively participate in class. This can take the form of asking questions in class, responding to questions to the class, and actively asking/answering questions on the online discussion board.

Students are free to share code and ideas within their stated project/homework group for a given assignment, but should not discuss details about an assignment with individuals outside their group. The midterm and final exam are individual assignments and must be completed by yourself.

The Cornell Code of Academic Integrity applies to this course.

In compliance with the Cornell University policy and equal access laws, we are available to discuss appropriate academic accommodations that may be required for student with disabilities. Requests for academic accommodations are to be made during the first three weeks of the semester, except for unusual circumstances, so arrangements can be made. Students are encouraged to register with Student Disability Services to verify their eligibility for appropriate accommodations.