Site Search  

Resources » Publication Details

Publication Details


Reference TypeConference Proceedings
Author(s)Theodorou, E. A.;Buchli, J.;Schaal, S.
Year2009
TitlePath integral-based stochastic optimal control for rigid body dynamics
Journal/Conference/Book TitleAdaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09. IEEE Symposium on
Keywordsmatrix algebra nonlinear control systems optimal control partial differential equations stochastic systems transforms Hamilton-Jacobi-Bellman equation logarithmic transformation nonlinear stochastic systems partial differential equation path integral stoc
AbstractRecent advances on path integral stochastic optimal control [1],[2] provide new insights in the optimal control of nonlinear stochastic systems which are linear in the controls, with state independent and time invariant control transition matrix. Under these assumptions, the Hamilton-Jacobi-Bellman (HJB) equation is formulated and linearized with the use of the logarithmic transformation of the optimal value function. The resulting HJB is a linear second order partial differential equation which is solved by an approximation based on the Feynman-Kac formula [3]. In this work we review the theory of path integral control and derive the linearized HJB equation for systems with state dependent control transition matrix. In addition we derive the path integral formulation for the general class of systems with state dimensionality that is higher than the dimensionality of the controls. Furthermore, by means of a modified inverse dynamics controller, we apply path integral stochastic optimal control over the new control space. Simulations illustrate the theoretical results. Future developments and extensions are discussed.
Pages219-225
Alternate Journal/TitleAdaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09. IEEE Symposium on
Short TitlePath integral-based stochastic optimal control for rigid body dynamics
URL(s) http://www-clmc.usc.edu/publications/t/theodorou-ADPRL2009.pdf

Designed by: Nerses Ohanyan & Jan Peters
Page last modified on June 20, 2013, at 07:00 PM