Site Search  

Resources » Publication Details

Publication Details


Reference TypeConference Proceedings
Author(s)Peters, J.;Schaal, S.
Year2007
TitleReinforcement learning for operational space control
Journal/Conference/Book TitleInternational Conference on Robotics and Automation (ICRA2007)
Keywordsoperational space control, reinforcement learning, weighted regression, EM-Algorithm
AbstractWhile operational space control is of essential importance for robotics and well-understood from an analytical point of view, it can be prohibitively hard to achieve accurate control in face of modeling errors, which are inevitable in complex robots, e.g., humanoid robots. In such cases, learning control methods can offer an interesting alternative to analytical control algorithms. However, the resulting supervised learning problem is ill-defined as it requires to learn an inverse mapping of a usually redundant system, which is well known to suffer from the property of non-convexity of the solution space, i.e., the learning system could generate motor commands that try to steer the robot into physically impossible configurations. The important insight that many operational space control algorithms can be reformulated as optimal control problems, however, allows addressing this inverse learning problem in the framework of reinforcement learning. However, few of the known optimization or reinforcement learning algorithms can be used in online learning control for robots, as they are either prohibitively slow, do not scale to interesting domains of complex robots, or require trying out policies generated by random search, which are infeasible for a physical system. Using a generalization of the EM-based reinforcement learning framework suggested by Dayan & Hinton, we reduce the problem of learning with immediate rewards to a reward-weighted regression problem with an adaptive, integrated reward transformation for faster convergence. The resulting algorithm is efficient, learns smoothly without dangerous jumps in solution space, and works well in applications of complex high degree-of-freedom robots.
Place PublishedRome, Italy, April 10-14
Pages2111-2116
Short TitleReinforcement learning for operational space control
URL(s) http://www-clmc.usc.edu/publications/P/peters-ICRA2007.pdf

Designed by: Nerses Ohanyan & Jan Peters
Page last modified on June 20, 2013, at 07:00 PM