Abstract

We introduce and develop moment propagation for approximate Bayesian inference. This method can be viewed as a variance correction for mean field variational Bayes which tends to underestimate posterior variances. We show for some simple models (with two components) that moment propagation can be applied to recover the posterior distribution exactly. We then discuss how this idea can be extended to more complicated models with more than two components and a post-hoc correction that can be applied to any Gaussian based approximate Bayesian inference method. We demonstrate these ideas on a number of models where the moment propagation approximation of the marginal posterior distributions is nearly exact.