Movement coordination requires some form of planning: every degree-of-freedom (DOF) needs to be supplied with appropriate motor commands at every moment in time. The commands must be chosen such that they accomplish the desired task, but also such that they do not violate the abilities of the movement system. Due to the numerous DOFs in complex movement systems and the almost infinite number of possibilities to use the DOFs over time, there exist actually an infinite number of possible movement plans for any given task.This redundancy is advantageous as it allows a movement system to avoid situations where, for instance, the range of motion of DOFs is saturated, or where obstacles need to be circumvented to reach a goal. But, from a learning point of view, it also makes it quite complicated to find good movement plans since the state spaces spanned by all possible plans it extremely large. What is needed to make learning tractable in such high dimensional systems is some form of additional constraints, constraints that reduce the state spaces in a reasonable way without eliminating good solutions.
The classical way to constrain solution spaces is to impose optimization criteria on the movement planning, for instance, by requiring that the system accomplishes the task in minimum time or with minimal energy expenditure. However, it is not trivial to find the correct cost function that result in an adequate behavior. Thus, our research on trajecotry planning has been focussing on an alternative method of constraining movement planning by requiring that movements are built from movement primitives. We conceive of movement primitives as simple dynamical systems that can generate either discrete or rhythmic movements about every DOF. Only speed and amplitude parameters are initally needed to get a movement started. Learning is required to fine-tune certain additional parameters to improve the movement. This approach allows us to learn movements by just adjusting a relatively small set of parameters. We are currently exploring how these dynamical systems can be used to generate full body movement, how their parameters can be learned with novel reinforcement learning methods, and how such movement primitives can be sequenced and superimposed to accomplish more complex movement tasks. We also consider how our developed models compare to biological behavior to find out which movement primitives biological systems employ, and how such movement primitives are represented in the brain.
Inspiration from biology also motivates a related trajectory planning project that we conduct. A common feature in the brain is to employ topographic maps as basic represenation of sensory signals. Such maps can be built with various neural network approaches, for instance Kohonen's Self-Organizing Maps or the Topology Representing Network (TRN) by Martinetz. From a statistical point of view, topographic maps can be thought of as neural networks that perform probability density estimation with additional knowledge about neighborhood relations. Density estimators are very powerful tools to perform mappings between different coordinate systems, to perform sensory integration, and to serve as basic representation for other learning systems. But in addition to these properties, topographic maps can also peform spatial computations that can generate trajectory plans. For instance, by using diffusion-based path planning algorithms, we demonstrated the feasability of such an approach by learning obstacle avoidance with a pneumatic robot arm. Learning motor control with topographic maps is also highly interesting from a biological point of view, as, in contrast to visual information processing, the usefulness of topographic maps in motor control is far from understood so far.
Contact persons: Jun Nakanishi, Jan Peters, Stefan Schaal
(:clmckeywordsearch Movement Primitives :)