Site Search  

Resources » Publication Details

Publication Details


Reference TypeConference Proceedings
Author(s)Theodorou, E.;Buchli, J.;Schaal, S.
Year2010
TitleReinforcement learning of motor skills in high dimensions: A path integral approach
Journal/Conference/Book TitleRobotics and Automation (ICRA), 2010 IEEE International Conference on
Keywordscontrol engineering computing intelligent robots learning (artificial intelligence) learning systems optimal control stochastic systems complex motor system continuous state-action spaces estimation theory gradient-based policy learning high-dim
AbstractReinforcement learning (RL) is one of the most general approaches to learning control. Its applicability to complex motor systems, however, has been largely impossible so far due to the computational difficulties that reinforcement learning encounters in high dimensional continuous state-action spaces. In this paper, we derive a novel approach to RL for parameterized control policies based on the framework of stochastic optimal control with path integrals. While solidly grounded in optimal control theory and estimation theory, the update equations for learning are surprisingly simple and have no danger of numerical instabilities as neither matrix inversions nor gradient learning rates are required. Empirical evaluations demonstrate significant performance improvements over gradient-based policy learning and scalability to high-dimensional control problems. Finally, a learning experiment on a robot dog illustrates the functionality of our algorithm in a real-world scenario. We believe that our new algorithm, Policy Improvement with Path Integrals (PI2), offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL in robotics.
Pages2397-2403
Date3-7 May 2010
Alternate Journal/TitleRobotics and Automation (ICRA), 2010 IEEE International Conference on
Short TitleReinforcement learning of motor skills in high dimensions: A path integral approach
ISBN/ISSN1050-4729
URL(s) http://www-clmc.usc.edu/publications/T/theodorou-ICRA2010.pdf

Designed by: Nerses Ohanyan & Jan Peters
Page last modified on June 20, 2013, at 07:00 PM