Reference Type  Conference Proceedings 
Author(s)  Theodorou, E. A.;Buchli, J.;Schaal, S. 
Year  2009 
Title  Path integralbased stochastic optimal control for rigid body dynamics 
Journal/Conference/Book Title  Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09. IEEE Symposium on 
Keywords  matrix algebra
nonlinear control systems
optimal control
partial differential equations
stochastic systems
transforms
HamiltonJacobiBellman equation
logarithmic transformation
nonlinear stochastic systems
partial differential equation
path integral stoc 
Abstract  Recent advances on path integral stochastic optimal control [1],[2] provide new insights in the optimal control of nonlinear stochastic systems which are linear in the controls, with state independent and time invariant control transition matrix. Under these assumptions, the HamiltonJacobiBellman (HJB) equation is formulated and linearized with the use of the logarithmic transformation of the optimal value function. The resulting HJB is a linear second order partial differential equation which is solved by an approximation based on the FeynmanKac formula [3]. In this work we review the theory of path integral control and derive the linearized HJB equation for systems with state dependent control transition matrix. In addition we derive the path integral formulation for the general class of systems with state dimensionality that is higher than the dimensionality of the controls. Furthermore, by means of a modified inverse dynamics controller, we apply path integral stochastic optimal control over the new control space. Simulations illustrate the theoretical results. Future developments and extensions are discussed. 
Pages  219225 
Alternate Journal/Title  Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09. IEEE Symposium on 
Short Title  Path integralbased stochastic optimal control for rigid body dynamics 
URL(s)  http://wwwclmc.usc.edu/publications/t/theodorouADPRL2009.pdf
