Course Overview

---

Instructor: Kilian Weinberger

Contact: kqw4@cornell.edu

Faculty Office Hours: Bowers 475, Mondays 9:45–10:45am

Course Staff:

• TA: Yann Hicke (ylh8@cornell.edu)
• Office Hours: Fridays 6–8pm

Credits: 4.0 credits (3.0 for lecture, 1.0 for independent research project)

Prerequisites: CS 4780 or CS 5780 or equivalents, or permission of the instructor

Time and Location: Tuesdays and Thursdays, 14:55–16:10 (27 lectures total)

Course overview: This course covers advanced topics in machine learning, focusing on recent developments in large language models, multimodal models, and their applications. Students will engage with cutting-edge research through presentations, discussions, and a final project.

Course Structure

---

Weekly Schedule

• Tuesdays: Overview of the week’s topic. One team gives a 70-minute presentation.
• Thursdays:
  • The same team from Tuesday presents a paper.
  • 16 other students are assigned roles (4 each: Archaeologists, Academic Researchers, Industry Experts, Ethicists).
  • Role-assigned students submit 2–3 slides before class and present during small-group discussions.

Class Session Structure (Thursdays)

1. Quiz on the paper and previous class (10 minutes)
2. 30-minute presentation by the team
3. 30-minute group discussion with role presentations
4. Optional post-class quiz (5 minutes)

Social Reading Groups

Every main presenter will lead a Perusall group for pre-class paper annotation. Groups consist of one Presenter, one Archaeologist, one Academic Researcher, one Industry Expert, one Ethicist, and six other students without specific roles (11 people total).

Course Topics

---

This course covers the following main topics in advanced machine learning:

1. LLM Foundations: Tokenization, positional embeddings, attention, causal language modeling, in-context learning
2. LLM Extensions: Quantization, adapters, mixture of experts, context window, parallelism, distillation, attention-free architectures
3. To be announced

Schedule

---

The course follows this tentative schedule. Topics and dates may change.

Date	Style	Topic
Tuesday, August 26, 2025	Lecture	Introduction
Thursday, August 28, 2025	Lecture	Introduction
Tuesday, September 2, 2025	Overview	LLM Foundations
Thursday, September 4, 2025	Paper	LLM Foundations
Tuesday, September 9, 2025	Overview	LLM Extensions
Thursday, September 11, 2025	Paper	LLM Extensions

Roles

---

Students will rotate through various roles throughout the semester:

• Reviewer (all students): Provide a critical review following NeurIPS reviewer guidelines.
• Presenter (4 students): Create the main presentation, covering motivation, problem definition, method, and experimental findings.
• Archaeologist (4 students): Contextualize the paper within previous and subsequent work.
• Researcher (4 students): Propose a follow-up project building on the current paper.
• Industry Expert (4 students): Propose a new application or product and discuss its potential impacts.
• Ethicist (4 students): Assess the paper’s potential impact on economy, society, and/or environment.
• Hacker (optional): Implement a simplified version of the paper and share a Jupyter Notebook.

Each student assigned to a role should prepare 2–3 slides and present for ~5–6 minutes, except for the Hacker, who provides a Jupyter Notebook instead of slides.

Weekly Assignment: By 10 PM the day before each Thursday class, all students must submit a review on CMT here.

Role Sign-ups

Sign up for roles using this Google Sheets link. Please sign up as soon as possible.

Presenter Responsibilities

Presenters are also responsible for creating quizzes used to assess the class’s understanding of the material.

Acknowledgment

The role-playing aspect of this course has evolved from an initial design by Alec and Eitan Grinspun at Columbia University.

Grading

---

• 10% Paper reviews: Weekly research paper review submitted by all students.
• 10% In-class quizzes (weekly): Online quiz covering recent material and the assigned paper.
• 40% Independent research project: Groups of four propose and complete a research project.
• 40% In-class participation and presentations:
  • Performance in assigned roles (Archaeologist, Academic Researcher, Industry Expert, Ethicist)
  • Quality of role-based presentations and prepared slides
  • Main team presentations (for those assigned as Presenters)
  • Active participation in group discussions
  • Optional Hacker role contributions

Note: The role-playing aspect of the course is a significant part of participation and presentations. Students are expected to fully engage with roles, prepare thoroughly, and contribute meaningfully to discussions.

Final Project

---

The final project engages students in research related to course topics.

Project Guidelines:

• Extend papers from topics covered in class
• Present outcomes as a research paper in standard conference format
• Work in groups of four
• Include a “contributions” section detailing each member’s work
• Host the project on GitHub
• Provide a written report and a descriptive Jupyter Notebook
• Present the final project during the last two class sessions
• Prepare a PowerPoint or LaTeX final presentation

Course Policies

---

Attendance and Participation

Regular attendance and active participation are essential.

Academic Integrity

All students must adhere to the Cornell University Code of Academic Integrity.

Late Work Policy

Given the collaborative nature of assignments, late work is generally not accepted without prior approval.

Accommodations for Students with Disabilities

If you are registered with Student Disability Services, please meet with the instructor as soon as possible to discuss accommodations. All such appointments are confidential.

Diversity and Inclusion

We are committed to an inclusive learning environment that values diversity and fosters respect for all individuals.