

Publication Details
Reference Type  Journal Article 
Author(s)  Buchli, J.;Stulp, F.;Theodorou, E.;Schaal, S. 
Year  2011 
Title  Learning variable impedance control 
Journal/Conference/Book Title  International Journal of Robotics Research 
Keywords  variable impedance control, reinforcment learning, PI2, movement primitives 
Abstract  One of the hallmarks of the performance, versatility, and robustness
of biological motor control is the ability to adapt the impedance of
the overall biomechanical system to different task requirements and
stochastic disturbances. A transfer of this principle to robotics is
desirable, for instance to enable robots to work robustly and safely
in everyday human environments. It is, however, not trivial to derive
variable impedance controllers for practical high degreeoffreedom
(DOF) robotic tasks.
In this contribution, we accomplish such variable impedance control
with the reinforcement learning (RL) algorithm PISq ({f P}olicy
{f I}mprovement with {f P}ath {f I}ntegrals). PISq is a
modelfree, sampling based learning method derived from first
principles of stochastic optimal control. The PISq algorithm requires no tuning
of algorithmic parameters besides the exploration noise. The designer
can thus fully focus on cost function design to specify the task. From
the viewpoint of robotics, a particular useful property of PISq is
that it can scale to problems of many DOFs, so that reinforcement learning on real robotic
systems becomes feasible.
We sketch the PISq algorithm and its theoretical properties, and how
it is applied to gain scheduling for variable impedance control.
We evaluate our approach by presenting results on several simulated and real robots.
We consider tasks involving accurate tracking through viapoints, and manipulation tasks requiring physical contact with the environment.
In these tasks, the optimal strategy requires both tuning of a reference trajectory emph{and} the impedance of the endeffector.
The results show that we can use path integral based reinforcement learning not only for
planning but also to derive variable gain feedback controllers in
realistic scenarios. Thus, the power of variable impedance control
is made available to a wide variety of robotic systems and practical
applications.

Short Title  Learning variable impededance control 
URL(s)  http://wwwclmc.usc.edu/publications/B/buchliIJRR2011.pdf


