Course Schedule

Overview

Instructors Kilian Q. Weinberger and Wei-Chiu Ma

Lectures Tuesday and Thursday, 2:55–4:10 pm

Office Hours Kilian Weinberger: Mondays 9:45–10:30 am, booking link, 475 CIS Building Wei-Chiu Ma: Wednesdays 2:30–3:30 pm, booking link, 416A Gates Hall

Course Staff Office Hours Calendar link

Course Overview

This class is an introductory course to deep learning. It covers the fundamental principles behind training and inference of deep networks, deep reinforcement learning, the specific architecture design choices applicable for different data modalities, discriminative and generative settings, and the ethical and societal implications of such models.

Prerequisites

Fundamentals of Machine Learning (CS 4780, ECE 4200, STCS I4740), Python fluency (CS 1110), and programming proficiency (e.g. CS 2110).

Course Logistics

For enrolled students the companion Canvas page serves as a hub for access to Ed Discussions (the course forum) and Gradescope (for HWs). If you are enrolled in the course you should automatically have access to the site. Please let us know if you are unable to access it.

Course Notes

Student-compiled lecture notes are available for many topics covered in the course.

Homework, Projects, and Exams

Your grade in this course is comprised of four components: homework, mid-term exam, project, and participation.

Homework
There will be a number of homework assignments throughout the course, typically made available roughly one to two weeks before the due date. The homeworks will have both theoretical questions and programming questions.
Final Project
To provide hands-on learning with the methods we discuss in class and to get familiar with common ML frameworks, there will be a project. For the project, students will implement the method proposed in an existing research paper and will try to reproduce the results in the paper. Final Project Guidelines
Exam
There will be one mid-term exam for the class based on the material covered in the lectures.
Participation
Students are expected to play an active role in providing constructive feedback. The participation grade will be based on feedback provided for lectures and assignments. There will also be daily quizzes at the start of class; these will be graded for participation and correctness.

Grading

Final grades are based on homework assignments, project, exam, and participation.

CS 4782

Homework 10%

Final Project 35%

Mid-term Exam 40%

Participation 15%

CS 5782

Homework 10%

Final Project 30%

Mid-term Exam 35%

Participation 15%

Paper Quizzes 10%

References

While this course does not explicitly follow a specific textbook, there are useful references on many of the topics covered. Pointers to references will be provided alongside each lecture in the schedule above.

Primary Textbook

Dive Into Deep Learning — Zhang, A., Lipton, Z.C., Li, M. & Smola, A.J.
Includes many of the topics covered in this course with instructive PyTorch implementations. Section numbers are provided alongside lectures (abbreviated DiDL). [Book website]

Background References

Probabilistic Machine Learning: An Introduction — Murphy
A comprehensive introduction to the foundations of machine learning. Available digitally through the Cornell University Library and as a draft from the author. [Book website]
The Elements of Statistical Learning — Hastie, Tibshirani, and Friedman
A comprehensive introduction to statistical learning, available directly from the authors. [Book website]
DiDL Appendix: Mathematics for Deep Learning
An overview of the mathematical topics most relevant to understanding deep learning. [Book appendix]
Linear Algebra — Khan Academy
[Khan Academy]
Linear Algebra — Strang (MIT OCW)
[MIT OpenCourseWare]

Software

Python [Docs]
NumPy [Docs]
PyTorch [Docs]

Course Policies

Inclusiveness

You should expect and demand to be treated by your classmates and the course staff with respect. You belong here, and we are here to help you learn and enjoy this course. If any incident occurs that challenges this commitment to a supportive and inclusive environment, please let the instructors know so that the issue can be addressed. We subscribe to the CS Department's Values of Inclusion.

Mental Health Resources

Cornell University provides a comprehensive set of mental health resources. Please make use of them whenever needed.

Participation

You are encouraged to actively participate in class — asking questions, responding to questions, and engaging on the online discussion board.

Collaboration Policy

Students are free to share code and ideas within their stated project/homework group for a given assignment, but should not discuss details about an assignment with individuals outside their group. The midterm exam is an individual assignment and must be completed by yourself.

Academic Integrity

The Cornell Code of Academic Integrity applies to this course.

Accommodations

In compliance with Cornell University policy and equal access laws, we are available to discuss appropriate academic accommodations that may be required for students with disabilities. Requests are to be made during the first three weeks of the semester. Students are encouraged to register with Student Disability Services to verify their eligibility for appropriate accommodations.