

Publication Details
Reference Type  Conference Paper 
Author(s)  Buchli, J.;Theodorou, E.;Stulp, F.;Schaal, S. 
Year  2010 
Title  Variable impedance control  a reinforcement learning approach 
Journal/Conference/Book Title  Robotics Science and Systems (2010) 
Keywords  reinforcement learning, optimal control, pi2 
Abstract  One of the hallmarks of the performance, versatility,
and robustness of biological motor control is the ability to adapt
the impedance of the overall biomechanical system to different
task requirements and stochastic disturbances. A transfer of this
principle to robotics is desirable, for instance to enable robots
to work robustly and safely in everyday human environments. It
is, however, not trivial to derive variable impedance controllers
for practical high DOF robotic tasks. In this contribution, we accomplish
such gain scheduling with a reinforcement learning approach
algorithm, PI2 (Policy Improvement with Path Integrals).
PI2 is a modelfree, sampling based learning method derived from
first principles of optimal control. The PI2 algorithm requires no
tuning of algorithmic parameters besides the exploration noise.
The designer can thus fully focus on cost function design to
specify the task. From the viewpoint of robotics, a particular
useful property of PI2 is that it can scale to problems of many
DOFs, so that RL on real robotic systems becomes feasible. We
sketch the PI2 algorithm and its theoretical properties, and how
it is applied to gain scheduling. We evaluate our approach by
presenting results on two different simulated robotic systems, a
3DOF Phantom Premium Robot and a 6DOF Kuka Lightweight
Robot. We investigate tasks where the optimal strategy requires
both tuning of the impedance of the endeffector, and tuning
of a reference trajectory. The results show that we can use
path integral based RL not only for planning but also to derive
variable gain feedback controllers in realistic scenarios. Thus,
the power of variable impedance control is made available to a
wide variety of robotic systems and practical applications. 
Place Published  Zaragoza, Spain, June 2730 
Short Title  Variable impedance control  a reinforcement learning approach 
URL(s)  http://wwwclmc.usc.edu/publications/B/buchliRSS2010.pdf


