Syllabus đź“–
Table of contents
Communication
For communication, we’ll be using Ed. Ed is where you will see all announcements and get help from staff and other students. You will be added to Ed automatically.
Logistics
Lecture: Tuesday - Thursday, 10:10 PM - 11:25 PM at Upson 216
Robot Access: Gates Hall G46
There is no one textbook that covers the content of this course the way we intend on covering it, though we will link supplementary readings under Resources.
About
Robot manipulation is the ability for a robot to interact physically with objects in the world and manipulate them towards completing a task. It is one of the greatest technical challenges in robotics, due primarily to the interplay of uncertainty about the world and clutter within it. As robots become integrated into complex human environments, robot manipulation is increasingly necessary to assist humans in these unstructured environments. Robotic manipulation will enable applications like personal assistant robots in the home and factory worker robots in advanced manufacturing. This course covers the fundamental theory, concepts, and systems of robot manipulation, including both software and hardware.
Topics we will cover this semester include perception, state estimation, robot arm kinematics and dynamics, task and motion planning, machine learning, controls, human-robot interaction towards various robot manipulation tasks. The course features a semester-long group project in which students propose, formalize, and execute a working robotic manipulation system towards a real-world task. The scope of possible components is quite broad and extends beyond traditional robotics issues into other aspects of CS.
This course is offered to prepare a student for Ph.D. research in robot manipulation.
Note that this course counts as the AI area with either the systems or applied research style for the CS Ph.D. breadth requirement.Course Structure
The course has a mixture of lectures, a project component, and a paper-reading component.
The lecture component of the class will consist of lectures on overview of the field of robot manipulation, kinematics and dynamics, task and motion planning, controls, perception, grasping, bi-manual and deformable robot manipulation, and robot learning all in the context of robot manipulation. It will also consist of a tutorial on how to use the Widow-X 250 robot arms.
The paper-reading component of the class is about reading, discussing, and presenting research/survey papers. Papers will be assigned in a group-wise manner. Each class would usually involve two paper presentations. Each paper presentation should not take more than 25 minutes and will be followed by 10 minutes of discussion. The discussions will take place in a debate-like format where the presenter group will take the "Pro" side, and students from other groups will take the "Con" side. For a few classes, there could be only one long paper assigned. In that case, the presentation should not take more than 50 minutes. The participation grade is primarily based on this discussion, so all students are expected to come prepared to discuss each paper. It may be helpful to prepare a list of questions to ask the group. The presenter group would be required to share a joint contribution statement highlighting who contributed to what part of the presentation along with mutually-agreed upon scores (on a scale of 1 to 5) for all the group members.
For the project component of the course, the projects will be completed in small groups. The instructor may provide some project ideas, but students are encouraged to provide their own topics. All projects require validation on a real robot. The instructor will provide access to multiple WidowX 250 robot arm platforms for the group projects. Students with access to other real robot arms are encouraged to use them. The project objective is to produce a conference-paper-style final report that could be submitted to a robotics conference such as ICRA or IROS. The project schedule goes as follows:
- Students individually express preferences for project topics and form a project group.
- Project groups write a project proposal describing
- who is in the group
- what problem they aim to solve
- what is the “twist” that makes their approach special
- a plan of action for the semester with dates, leading to a final paper (report)
- For the mid-term presentation, groups present in class their progress to date and what they plan to do next; they submit an extended abstract version of the report
- In the end, groups give an oral presentation and a demo of their completed project
- Groups submit the completed written report
- Groups submit contribution statements highlighting which member contributed to what part of the project. Group members grade their partners objectively on a scale of 1 to 5.
The sections of a typical conference paper are:
- Abstract (summarize the whole paper in one paragraph)
- Introduction (motivate the research and clearly state the contributions)
- Related Work (survey of literature)
- Method (this varies the most – explain how you did what you did)
- Experiments (if you are building something, you should validate it with experiments. Even if it is primarily a theoretical contribution, you need to show that the theory works in the real world by implementing it on a real robot)
- Discussion
- (can also include other sections as needed)
For the extended abstract, you should include Introduction, Related Work, Method, and Experiments (if any).
The other component of the class is about reading and discussing papers. There are papers assigned for some class days. Papers will be assigned in a group-wise manner. Each class would involve one group presenting two papers, but all students must read each paper. Each paper presentation should not take more than 25 minutes and will be followed by 10 minutes of discussion. The discussions will take place in a debate-like format where the presenter group will take the "Pro" side, and students from other groups will take the "Con" side. The presentation should not take more than 50 minutes for classes with only one paper assignment. The participation grade is primarily based on this discussion, so all students are expected to come prepared to discuss each paper. It may be helpful to prepare a list of questions to ask the group. The presenter group would be required to share a joint contribution statement highlighting who contributed to what part of the presentation along with mutually-agreed upon scores (on a scale of 1 to 5) for all the group members.
Learning Outcomes
- Apply research papers on robot manipulation topics and be able to approach problems in robot manipulation.
- Contribute to the robotics research literature, especially research pertaining to robot manipulation.
- Summarize fundamental principles forming the basis of various robotics disciplines: perception, kinematics, dynamics, learning, controls, grasping, planning, and human-robot interaction - all towards robot manipulation applications.
- Apply principles of robotic systems design, and be able to analyze trade-offs in such designs.
- Integrate various sub-fields of robotics towards developing a real robotic system that can perform intelligent tasks in the real world.
Prerequisites
Graduate standing or permission of the instructor. Students should have taken a previous robotics course such as MAE 4180 or CS 4750 / CS 5750 / MAE 4760 / ECE 4770. Proficiency in C++ or Python is required. Familiarity with ROS is required.
Technology
We will be using several websites this semester. Here’s what they’re all used for:
- Course Website: where all content will be posted.
- Note, in case of a conflict between the syllabus and the course website, students should follow the information on the website. It is the responsibility of the students to check the website frequently.
- Ed: discussion forum where all announcements will be sent, and where all student-TA and student-student communication will occur.
- The site will be monitored on business days by the course staff. Students can expect an answer within one business day.
- Students are expected to communicate in a professional manner.
- Students may NOT write code snippets on the discussion board.
- Canvas: where all the slides/notes/papers/homeworks are published.
- Gradescope: where all presentations and project-related submissions will be made.
Policies
Grading
Here’s a detailed breakdown of how we will compute your grade.
Paper presentation: Research Papers
| Component | Weight (Group-wise) |
|---|---|
| Motivate the problem (show videos and describe the problem well) | 5% |
| Present related work (Describe in detail the related work section. Try to bucket the various papers into categories) | 5% |
| Background tutorial (Explain underlying fundamental concepts) | 5% |
| Explain the method (share the insights, details and inner workings) | 5% |
| Present experimental setup + results (additionally share details from the appendix if it exists) | 5% |
| Slides | 5% |
| Defense | 10% |
| Total | 40% |
Course Project
| Component | Weight |
|---|---|
| Mid-term Project Presentation + Extended Abstract | 5% (Group) |
| Final Report (Paper in ICRA format) - Think of us as reviewers of your paper. We will not judge the paper based on novelty but every other criteria applies - introduction (talk about the significance of the problem) - related work (how thorough this section is) - methods (how detailed they are - reproducible?) - experiments (how extensive they are), - results and discussion (how detailed and insightful they are) - writing, story etc. |
15% (Group) |
| Final Project Presentation (Real-world demos and videos with real robots are a must!) | 10% (Group) |
| Individual Contributions (One statement per group with individual scores) | 10% (Individual) |
| Bonus points: Novelty (you can actually submit a conference paper) | 10% (Group) |
| Total | 40% |
Class Participation
| Component | Weight (Individual basis) |
|---|---|
| Paper Presentation participation: Contributions (One group statement per paper presented with individual scores) | 5% |
| Class participation: Offense | 10% |
| Survey participation: Course Eval | 5% |
| Total | 20% |
To summarize, you will be graded on the following components
| Component | Weight |
|---|---|
| Paper Presentation | 40% |
| Project | 40% |
| Participation | 20% |
| Total | 100% |
| Bonus | 10% |
| Bonus + Total | 110% |
Late Policy
Any late submission for a deliverable will deduct 20% from that deliverable per late day.
Regrade Policy
Addition errors in the total score are always applicable for regrades. Please only make regrade requests when the case is strong and a significant number of points are at stake. Regrade requests should be submitted online via a private post on Ed within one week of when a deliverable is returned to the student. You must provide a justification for the regrade request.
Inclusion Statement
Students in this course come from a variety of backgrounds, abilities, and identities. In order to ensure an environment conducive to learning, all members of the course must treat one another and the course staff with respect. If you feel your needs are not being adequately accommodated by the other students or instruction staff, please contact Prof. Bhattacharjee.
Academic Integrity
Students are expected to follow Cornell’s Code of Academic Integrity which can be found here. The purpose of this code is to provide for an honest and fair academic environment. As such, it should be clear to students what is expected of them in the course (see the collaboration policy) and in case of doubt, students should ask Prof. Bhattacharjee. Copying work (code and/or text) and allowing others to copy work are considered violations of Cornell’s code. Course staff will use software tools (such as MOSS) to detect code plagiarism.
For fairness to all students and to discourage inappropriate behavior, violations of the code related to any homework or assignment, will result in an automatic zero. In addition, at the discretion of Prof. Bhattacharjee, violators will be prosecuted.
Life Happens Policy
In case of a legitimate situation or medical emergency that arises during the semester that is going to hinder your ability to complete the work on time, contact Prof. Bhattacharjee as soon as possible. Extensions (beyond the already assigned slip days) will be granted only in exceptional circumstances, such as documented illness, not for situations such as job interviews or large workloads in other courses. Note, the students are free to use the slip days mentioned above (See Late policy) for any reasons they want.
Student Disability Accommodations
Your access in this course is important to me. Please request your accommodation letter early in the semester, or as soon as you become registered with Student Disability Services (SDS), so that we have adequate time to arrange your approved academic accommodations.
- Once SDS approves your accommodation letter, it will be emailed to both you and me. Please follow up with [Prof. Bhattacharjee, TA, etc.] to discuss the necessary logistics of your accommodations.
- If you are approved for exam accommodations, please consult with [Prof. Bhattacharjee, TA, course staff, etc.] at least two weeks before the scheduled exam date to confirm the testing arrangements.
- If you experience any access barriers in this course, such as with printed content, graphics, online materials, or any communication barriers, reach out to Prof. Bhattacharjee or SDS right away.
- If you need immediate accommodation, please speak with Prof. Bhattacharjee after class or send an email message to him and SDS at sds_cu@cornell.edu.
Mental health and well-being
Your health and wellbeing are important to me. There are services and resources at Cornell designed specifically to bolster undergraduate, graduate, and professional student mental health and well-being. Remember, your mental health and emotional well-being are just as important as your physical health. If you or a friend are struggling emotionally or feeling stressed, fatigued, or burned out, there is a continuum of campus resources available to you: https://mentalhealth.cornell.edu/get-support/support-students. Help is also available any time day or night through Cornell’s 24/7 phone consultation (607-255-5155). You can also reach out to Prof. Bhattacharjee, your college student services office, your resident advisor, or Cornel Health for support. Also, kindly refer to the resource guide compiled by the members of Body Positive Cornell, EARS, Reflect, and Cornell Minds Matter.
Additional resources
Other related resources can be found here: