Table of contents
- Course Structure
- Learning Outcomes
For communication, we’ll be using Ed. Ed is where you will see all announcements and get help from staff and other students. You will be added to Ed automatically.
Lecture: Tuesday - Thursday, 1:00 PM - 2:15 PM at Olin 245
Robot Access: Rhodes 106
Note that the first four classes will be held online on Zoom. Recordings will be made available to enrolled students.
Office Hours: See Ed
There is no one textbook that covers the content of this course the way we intend on covering it, though we will link supplementary readings under Resources.
Robot manipulation is the ability for a robot to interact physically with objects in the world and manipulate them towards completing a task. It is one of the greatest technical challenges in robotics, due primarily to the interplay of uncertainty about the world and clutter within it. As robots become integrated into complex human environments, robot manipulation is increasingly necessary to assist humans in these unstructured environments. Robotic manipulation will enable applications like personal assistant robots in the home and factory worker robots in advanced manufacturing. This course covers the fundamental theory, concepts, and systems of robot manipulation, including both software and hardware.
Topics we will cover this semester include perception, state estimation, robot arm kinematics and dynamics, task and motion planning, machine learning, controls, human-robot interaction towards various robot manipulation tasks. The course features a semester-long group project in which students propose, formalize, and execute a working robotic manipulation system towards a real-world task. The scope of possible components is quite broad and extends beyond traditional robotics issues into other aspects of CS.
This course is offered to prepare a student for Ph.D. research in robot manipulation.
Note that this course counts as the AI area with either the systems or applied research style for the CS Ph.D. breadth requirement.
The course has a mixture of class discussions, a project component, and a paper-reading component.
The projects will be completed in small groups. The instructor may provide some project ideas, but students are encouraged to provide their own topics. All projects require validation on a real robot. The instructor will provide access to multiple WidowX 250 robot arm platforms for the group projects. Students with access to other real robot arms are encouraged to use them. The project objective is to produce a conference-paper-style final report that could be submitted to a robotics conference such as ICRA. The project schedule goes as follows:
- Students individually express preferences for project topics and form a project group.
- Project groups write a project proposal describing
- who is in the group
- what problem they aim to solve
- what is the “twist” that makes their approach special
- a plan of action for the semester with dates, leading to a final paper (report)
- For the mid-term presentation, groups present in class their progress to date and what they plan to do next; they submit an extended abstract version of the report
- On the last two days of class, groups give an oral report and a demo of their completed project
- Groups submit the completed written report
- Groups submit contribution statements highlighting which member contributed to what part of the project. Group members grade their partners objectively on a scale of 1 to 5.
The sections of a typical conference paper are:
- Abstract (summarize the whole paper in one paragraph)
- Introduction (motivate the research and clearly state the contributions)
- Related Work (survey of literature)
- Method (this varies the most – explain how you did what you did)
- Experiments (if you are building something, you should validate it with experiments. Even if it is primarily a theoretical contribution, you need to show that the theory works in the real world by implementing it on a real robot)
- (can also include other sections as needed)
For the extended abstract, you should include Introduction, Related Work, Method, and Experiments (if any).
The second component of the class is about reading and discussing papers. There are papers assigned for the majority of class days. Papers have been assigned in a group-wise manner. Refer to the Schedule page for the list of papers and group assignment. Each class would involve one group presenting two papers, but all students must read each paper. Each paper presentation should not take more than 25 minutes and will be followed by 10 minutes of discussion. The discussions will take place in a debate-like format where the presenter group will take the “Pro” side (defense), and students from other groups will take the “Con” side (offense). The presentation should not take more than 50 minutes for classes with only one paper assignment. The participation grade is primarily based on this discussion, so all students are expected to come prepared to discuss each paper. It may be helpful to prepare a list of questions to ask the group. The presenter group would be required to share a joint contribution statement highlighting who contributed to what part of the presentation along with mutually-agreed upon scores (on a scale of 1 to 5) for all the group members.
- Apply research papers on robot manipulation topics and be able to approach problems in robot manipulation.
- Contribute to the robotics research literature, especially research pertaining to robot manipulation.
- Summarize fundamental principles forming the basis of various robotics disciplines: perception, kinematics, dynamics, learning, controls, grasping, planning, and human-robot interaction - all towards robot manipulation applications.
- Apply principles of robotic systems design, and be able to analyze trade-offs in such designs.
- Integrate various sub-fields of robotics towards developing a real robotic system that can perform intelligent tasks in the real world.
Graduate standing or permission of the instructor. Students should have taken a previous robotics course such as MAE 4180 or CS 4750 / CS 5750 / MAE 4760 / ECE 4770. A background in mathematics is required, especially linear algebra (e.g. MATH 4310) and probability (e.g. MATH 4720). Proficiency in C++ or Python is required. Familiarity with ROS is required.
We will be using several websites this semester. Here’s what they’re all used for:
- Course Website: where all content will be posted.
- Note, in case of a conflict between the syllabus and the course website, students should follow the information on the website. It is the responsibility of the students to check the website frequently.
- Ed: discussion forum where all announcements will be sent, and where all student-TA and student-student communication will occur.
- The site will be monitored on business days by the course staff. Students can expect an answer within one business day.
- Students are expected to communicate in a professional manner.
- Students may NOT write code snippets on the discussion board.
- Canvas: where all the slides/notes/papers/homeworks are published.
- Gradescope: where all presentations and project-related submissions will be made.
Here’s a detailed breakdown of how we will compute your grade.
Paper presentation: Research Papers
|Motivate the problem (show videos and describe the problem well)||5%|
|Present related work (Describe in detail the related work section. Try to bucket the various papers into categories)||5%|
|Background tutorial (Explain underlying fundamental concepts)||5%|
|Explain the method (share the insights, details and inner workings)||5%|
|Present experimental setup + results (additionally share details from the appendix if it exists)||5%|
Paper presentation: Survey/Review Papers
|Motivate the field (why is it important? what’s the significance? - show videos of different applications)||5%|
|Present the history of related work (how the field evolved from the past to the state-of-the-art in terms of approach)||5%|
|Categorize related work into different buckets and explain their approach||5%|
|Discuss insights of the different approaches and perform a critical assessment||5%|
|Identify existing trends and open challenges in the field||5%|
|Mid-term Project Presentation + Extended Abstract||5% (Group)|
|Final Report (Paper in ICRA format)
- Think of us as reviewers of your paper. We will not judge the paper based on novelty but every other criteria applies
- introduction (talk about the significance of the problem)
- related work (how thorough this section is)
- methods (how detailed they are - reproducible?)
- experiments (how extensive they are),
- results and discussion (how detailed and insightful they are)
- writing, story etc.
|Final Project Presentation (Real-world demos and videos with real robots are a must!)||10% (Group)|
|Individual Contributions (One statement per group with individual scores)||10% (Individual)|
|Bonus points: Novelty (you can actually submit a conference paper)||10% (Group)|
|Component||Weight (Individual basis)|
|Paper Presentation participation: Contributions (One group statement per paper presented with individual scores)||5%|
|Class participation: Offense||10%|
|Survey participation: Course Eval||5%|
To summarize, you will be graded on the following components
|Bonus + Total||110%|
Any late submission for a deliverable will deduct 20% from that deliverable per late day.
Addition errors in the total score are always applicable for regrades. Please only make regrade requests when the case is strong and a significant number of points are at stake. Regrade requests should be submitted online via a private post on Ed within one week of when a deliverable is returned to the student. You must provide a justification for the regrade request.
Students in this course come from a variety of backgrounds, abilities, and identities. In order to ensure an environment conducive to learning, all members of the course must treat one another and the course staff with respect. If you feel your needs are not being adequately accommodated by the other students or instruction staff, please contact Prof. Bhattacharjee.
Students are expected to follow Cornell’s Code of Academic Integrity which can be found here. The purpose of this code is to provide for an honest and fair academic environment. As such, it should be clear to students what is expected of them in the course (see the collaboration policy) and in case of doubt, students should ask Prof. Bhattacharjee. Copying work (code and/or text) and allowing others to copy work are considered violations of Cornell’s code. Course staff will use software tools (such as MOSS) to detect code plagiarism.
For fairness to all students and to discourage inappropriate behavior, violations of the code related to any homework or assignment, will result in an automatic zero. In addition, at the discretion of Prof. Bhattacharjee, violators will be prosecuted.
Life Happens Policy
In case of a legitimate situation or medical emergency that arises during the semester that is going to hinder your ability to complete the work on time, contact Prof. Bhattacharjee as soon as possible. Extensions (beyond the already assigned slip days) will be granted only in exceptional circumstances, such as documented illness, not for situations such as job interviews or large workloads in other courses. Note, the students are free to use the slip days mentioned above (See Late policy) for any reasons they want.
Expectations and Accomodations
We expect you to complete your own assignments honestly. If you have any questions, we are here to help! Come to office hours with any question you have, and we are more than happy to help you! However, do not expect us to write code or complete written assignments for you. If you cannot make to any office hours during a certain week, don’t hesitate to reach out to the TAs and schedule a meeting with them.
According to Cornell’s Mask Mandate, all students are required to wear a mask in the class and during office hours. Students unable to wear a mask or face covering due to a medical condition or other protected reason should contact Student Disability Services.