Tuesday, February 19, 2008
4:15 pm
B17 Upson Hall

Computer Science
Spring 2008

Pieter Abbeel
Stanford University

Apprenticeship Learning for Robotic Control with Application to Quadruped Locomotion and Autonomous Helicopter Flight

Many problems in robotics have unknown, stochastic, high-dimensional, and highly non-linear dynamics, and offer significant challenges to classical control methods. Some of the key difficulties in these problems are that (i) It is often hard to write down, in closed form, a formal specification of the control task (for example, what is the objective function for "flying well"?), (ii) It is difficult to build a good dynamics model because of data collection and data modeling challenges (similar to the "exploration problem" in reinforcement learning), and (iii) It is expensive to find closed-loop controllers for high dimensional, highly stochastic domains. In this talk, I will present learning algorithms with formal performance guarantees which show that these problems can be efficiently addressed in the apprenticeship learning setting---the setting when expert demonstrations of the task are available. I will also present how my apprenticeship learning techniques have enabled us to solve real-world control problems that could not be solved before: They have enabled a quadruped robot to traverse challenging terrain and to have a helicopter perform by far the most challenging aerobatic maneuvers performed by any autonomous helicopter to date, including maneuvers such as chaos and tic-tocs, which only exceptional human pilots can fly.

Pieter Abbeel is a Ph.D. candidate in the Computer Science Department at Stanford University. His research focuses on robotics, machine learning and control. In his thesis research, he has developed apprenticeship learning algorithms---algorithms which take advantage of expert demonstrations of a task at hand to efficiently build autonomous controllers with formal performance guarantees. These algorithms have enabled a robot dog to traverse highly challenging terrains. They have also, for the first time, enabled an autonomous helicopter to perform a wide range of expert human level helicopter aerobatics, including maneuvers such as loops, hurricanes, knife-edge, flips, rolls, inverted hover, tic-tocs and chaos.