Syllabus for CS4787/5777

Principles of Large-Scale Machine Learning — Fall 2026

[Canvas] [Discussion]

Description: CS4787 explores the principles behind scalable machine learning systems. The course will cover the algorithmic and the implementation principles that power the current generation of machine learning on big data. We will cover training and inference for both traditional ML algorithms such as linear and logistic regression, as well as deep models such as transformers. Topics will include: estimating statistics of data quickly with subsampling, stochastic gradient descent and other scalable optimization methods, mini-batch training, accelerated methods, adaptive learning rates, methods for scalable deep learning, hyperparameter optimization, parallel and distributed training, quantization and model compression, and efficient inference.

Prerequisites: CS3780 or equivalent, CS 2110 or equivalent

Format: Lectures during the scheduled lecture period will cover the course content. Problem sets will be used to encourage familiarity with the content and develop competence with the more mathematical aspects of the course. Programming assignments will help build intuition and familiarity with how machine learning algorithms run. There will be one midterm exam and one final exam, each of which will test both theoretical knowledge and programmming implementation of concepts.

Material: The course is based on books, papers, and other texts in machine learning, scalable optimization, and systems. Texts will be provided ahead of time on the website on a per-lecture basis. You aren't expected to necessarily read the texts, but they will provide useful background for the material we are discussing.

Grading: Students taking CS4787 will be evaluated on the following basis.

CS5777 has an additional paper-reading component, and students taking CS5777 will be evaluated as follows.

New this year, your knowledge of the content from the problem sets and the programming assignments will also be checked via short oral “mastery checks” with the TAs. Logistical details of how these will be conducted and scheduled are still being finalized and will be announced later in the semester.

Inclusiveness: You should expect and demand to be treated by your classmates and the course staff with respect. You belong here, and we are here to help you learn—and enjoy—this course. If any incident occurs that challenges this commitment to a supportive and inclusive environment, please let the instructor know so that we can address the issue. We are personally committed to this, and subscribe to the Computer Science Department's Values of Inclusion.

AI Use Policy: It is an academic integrity violation to represent the output of a generative AI tool as your own work. You may not submit any work produced by generative AI as part of the solution of problem sets, programming assignments, or paper reading assignments. It is an academic integrity violation to use any sort of generative AI to assist you during an exam.

Beyond this, you may use AI as you like to assist your own learning in the course, including by asking the AI about course content, asking it to document functions, asking it for the name of a function that does something, asking it to explain an error message, etc.

Course calendar may be subject to change.

Course Calendar Plan