Skip to main content

See Cornell Chronicle's article on the Robot Learning class. (Video)

Time and Place

Tue, Thu: 2:55pm to 4:10pm
Place: Philips 203


Ashutosh Saxena, asaxena @, 4159 Upson Hall.
Office Hours: Tuesday, 5:15pm-6:15pm. 4159 Upson. (or by appointment)

TAsSpecialityoffice hoursemail
Mark VerheggenMachine LearningWednesday, 2-4pm. 328b (tutoring room)
Amit SharmaMachine Learning, AIThursday, 5:15-7:15pm. 4121 Upson
Abhishek AnandProject TAby appointment
Hema KoppulaProject TAby appointment
Ian LenzProject TAby appointment

Class discussions and questions at Piazza.


We study the problem of how a robot can learn to perceive its world well enough to act in it, to make reliable plans, and to learn from its own experience. The focus will be on algorithms and machine learning techniques for autonomous operation of robots. The course has a term project (teams with 1-2 students) involving physical robots.

Topics include:

  • Robot survival kit (kinematics, statistics, ROS robot programming)
  • Filtering and state estimation (Kalman filters, particle filters)
  • Learning (markov decision process, HMM, supervised learning, spiking neurons)
  • Perception (basic computer vision, point cloud library, 3D sensors and algorithms)
  • Reinforcement learning and control

Machine learning algorithms such as HMM, reinforcement learning, particle filters and algorithms for 3D perception will be covered in depth.


Students are expected to have the following background:

  • Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program. (E.g., CS 1114 or CS 2110 or CS 3110 or equivalent.) Knowledge of C/C++ is a pre-requisite.
  • A course in probability/statistics (e.g. CS 2800, ECE 2200, ECE 3100, or ENGRD 2700 or equivalent).
  • Familiarity with the basic linear algebra. (E.g., MATH 2210 is sufficient but not necessary.) Strong mathematical skills are required in this course.
  • A course in Artificial Intelligence or Robotics areas is desirable, but not essential.
  • Motivation and patience to hack for long hours. Robots are cool only if they don't know that you are running out of time. This is also a pre-requisite.

If you're not clear if you satisfy the pre-reqs, please contact the instructor. Masters students should contact the instructor about pre-requisites.


This is a 4-credit course (letter/S-U). There will be a few written homeworks, one midterm, and one major open-ended term project. The homeworks will contain written questions and questions that require some programming. Some of the homework questions will be on your project's robotic platform. For the term project, see project details. It will involve writing a one-page proposal, 4-page midterm report and 8-page final report, along with the final demonstration session. There will be no final exam.

CS 4758

  • 35%: Homeworks (total 5)
  • 20%: Midterm
  • 45%: Major term project. (1+4+8+32)
All homeworks must be done to pass the course.

CS 6758

  • 24%: Homeworks (only need to do any 4 of 5)
  • 20%: Midterm
  • 10%: Paper readings and 1-1 discussions.
  • 46%: Major research project. (1+4+9+32)

Upto 5% bonus credit may be awarded for class participation. Instructor reserves the right to define 5% class participation points later in the course.

Grading will be done separately for 4758 and 6758 versions of the course.

Late Homework Policy

Each student is granted four unpenalized late days for the semester, and can use max one per homework. Homeworks can be handed in no more than four days late, and will receive a 25% penalty for each day late (excluding one day if an unpenalized late day is used). All homeworks must be done to pass the course.

Course Materials

There is no required text for this course. Lecture notes will be posted periodically on the course website. The following books are recommended as optional reading.

Autonomous Mobile Robots. Siegwart and Nourbakhsh. The MIT Press, 2004. (specially recommended for students with no experience in robotics)

Probabilistic Robotics. Thrun, Burgard and Fox. The MIT Press, 2006. (specially recommended for CS 6758 version)

Machine Learning, Tom Mitchell, McGraw Hill, 1997. (Basic machine learning.)

Pattern Recognition and Machine Learning, Bishop. (Advanced Machine Learning for CS 6758 / 6780.)

Course handouts and other materials can be downloaded from here.


Academic Integrity
This course follows the Cornell University Code of Academic Integrity. Each student in this course is expected to abide by the Cornell University Code of Academic Integrity. Any work submitted by a student in this course for academic credit will be the student's own work. Violations of the rules (e.g., cheating, copying, non-approved collaborations) will not be tolerated.
Course website
The course website will be the primary place for posting homeworks and announcements: