A Bayesian Practitioner’s Guide to Expectation Propagation

In the field of approximate Bayesian inference, expectation propagation (EP) is an often overlooked counterpart to its older Laplace approximation and variational Bayes cousins, perhaps owing to a lack of theory (especially convergence guarantees) and a higher implementation overhead, where derivations need to be hand-crafted to the specific distributions being approximated. However, when EP is carefully implemented, it is often more accurate than these alternative approaches. With this in mind, the purpose of this review paper is to describe and to consolidate the current state of research on EP at a high level focusing on examples and applications. Our aim is for this broad-based, practical EP guide to encourage its novel application to both existing and new Bayesian problems, where it has the opportunity to dramatically extend the cutting edge in performance.