

Publication Details
Reference Type  Conference Proceedings 
Author(s)  D'Souza, A.;Vijayakumar, S.;Schaal, S. 
Year  2003 
Title  Bayesian backfitting 
Journal/Conference/Book Title  Proceedings of the 10th Joint Symposium on Neural Computation (JSNC 2003) 
Keywords  statistical learning, Bayesian variational methods, linear regression, graphical models 
Abstract  We present an algorithm aimed at addressing both computational and analytical intractability of Bayesian regression models which operate in very highdimensional, usually underconstrained spaces. Several domains of research frequently provide such datasets, including chemometrics [2], and human movement analysis [1]. The literature in nonparametric statistics provides interesting solutions such as Backfitting [3] and Partial Least Squares [4], which are extremely robust and efficient, yet lack a probabilistic interpretation that could place them in the context of current research in statistical learning algorithms that emphasize the estimation of confidence, posterior distributions, and model complexity. In order to achieve numerical robustness and low computational cost, we first derive a novel Bayesian interpretation of Backfitting (BB) as a computationally efficient regression algorithm. BBÕs learning complexity scales linearly with the input dimensionality by decoupling inference among individual input dimensions. We embed BB in an efficient, locally variational model selection mechanism that automatically grows the number of backfitting experts in a mixtureofexperts regression model. We demonstrate the effectiveness of the algorithm in performing principled regularization of model complexity when fitting nonlinear manifolds while avoiding the numerical hazards associated with highly underconstrained problems. We also note that this algorithm appears applicable in various areas of neural computation, e.g., in abstract models of computational neuroscience, or implementations of statistical learning on artificial systems. 
Place Published  Irvine, CA, May 2003 
Short Title  Bayesian backfitting 
URL(s)  http://wwwclmc.usc.edu/publications/D/dsouzaJSNC2003.pdf


