Propagation Algorithms for Variational 
Bayesian Learning 
Zoubin Ghahramani and Matthew J. Beal 
Gatsby Computational Neuroscience Unit 
University College London 
17 Queen Square, London WCIN 3AR, England 
{zoubin ,m. beal}gat sby. ucl. ac. uk 
Abstract 
Variational approximations are becoming a widespread tool for 
Bayesian learning of graphical models. We provide some theoret- 
ical results for the variational updates in a very general family of 
conjugate-exponential graphical models. We show how the belief 
propagation and the junction tree algorithms can be used in the 
inference step of variational Bayesian learning. Applying these re- 
sults to the Bayesian analysis of linear-Gaussian state-space models 
we obtain a learning procedure that exploits the Kalman smooth- 
ing propagation, while integrating over all model parameters. We 
demonstrate how this can be used to infer the hidden state dimen- 
sionality of the state-space model in a variety of synthetic problems 
and one real high-dimensional data set. 
1 Introduction 
Bayesian approaches to machine learning have several desirable properties. Bayesian 
integration does not suffer overfitting (since nothing is fit to the data). Prior knowl- 
edge can be incorporated naturally and all uncertainty is manipulated in a consis- 
tent manner. Moreover it is possible to learn model structures and readily compare 
between model classes. Unfortunately, for most models of interest a full Bayesian 
analysis is computationally intractable. 
Until recently, approximate approaches to the intractable Bayesian learning prob- 
lem had relied either on Markov chain Monte Carlo (MCMC) sampling, the Laplace 
approximation (Gaussian integration), or asymptotic penalties like BIC. The recent 
introduction of variational methods for Bayesian learning has resulted in the series 
of papers showing that these methods can be used to rapidly learn the model struc- 
ture and approximate the evidence in a wide variety of models. In this paper we 
will not motivate advantages of the variational Bayesian approach as this is done in 
previous papers [1, 5]. Rather we focus on deriving variational Bayesian (VB) learn- 
ing in a very general form, relating it to EM, motivating parameter-hidden variable 
factorisations, and the use of conjugate priors (section 3). We then present several 
theoretical results relating VB learning to the belief propagation and junction tree 
algorithms for inference in belief networks and Markov networks (section 4). Fi- 
nally, we show how these results can be applied to learning the dimensionality of 
the hidden state space of linear dynamical systems (section 5). 
2 Variational Bayesian Learning 
The basic idea of variational Bayesian learning is to simultaneously approximate the 
intractable joint distribution over both hidden states and parameters with a simpler 
distribution, usually by assuming the hidden states and parameters are independent; 
the log evidence is lower bounded by applying Jensen's inequality twice: 
lnP(y[jM) _> dOQo(O) dxQx(x)ln Qx(X) +in Qo(0) (1) 
= :(Qo(O),Qx(x),y) 
where y, x, 0 and JM, are observed data, hidden variables, parameters and model 
class, respectively; P(0lJM ) is a parameter prior under model class JM. The lower 
bound r is iteratively maximised as a functional of the two free distributions, Qx(x) 
and Qo(O). From (1) we can see that this maximisation is equivalent to minimising 
the KL divergence between Qx(x)Qo(O) and the joint posterior over hidden states 
and parameters P(x, 0ly, 
This approach was first proposed for one-hidden layer neural networks [6] under the 
restriction that Qo (0) is Gaussian. It has since been extended to models with hidden 
variables and the restrictions on Qo(O) and Qx(x) have been removed in certain 
models to allow arbitrary distributions [11, 8, 3, 1, 5]. Free-form optimisation with 
respect to the distributions Qo(O) and Qx(x) is done using calculus of variations, 
often resulting in algorithms that appear closely related to the corresponding EM 
algorithm. We formalise this relationship and others in the following sections. 
3 Conjugate-Exponential Models 
We consider variational Bayesian learning in models that satisfy two conditions: 
Condition (1). The complete data likelihood is in the exponential family: 
P(x, yl0) = f(x,y)g(0)exp {qb(0)n-u(x,y)} 
where qS(O) is the vector of natural parameters, and u and f and g are the functions 
that define the exponential family. 
The list of latent-variable models of practical interest with complete-data likeli- 
hoods in the exponential family is very long. We mention a few: Gaussian mixtures, 
factor analysis, hidden Markov models and extensions, switching state-space mod- 
els, Boltzmann machines, and discrete-variable belief networks.  Of course, there 
are also many as yet undreamed-of models combining Gaussian, Gamma, Poisson, 
Dirichlet, Wishart, Multinomial, and other distributions. 
Condition (2). The parameter prior is conjugate to the complete data likelihood: 
P(01V, v) = h(v, v) g(O)"exp {qb(0)n-,} 
where rl and , are hyperparameters of the prior. 
Condition (2) in fact usually implies condition (1). Apart from some irregular cases, 
it has been shown that the exponential families are the only classes of distributions 
with a fixed number of sufficient statistics, hence allowing them to have natural 
conjugate priors. From the definition of conjugacy it is easy to see that the hyper- 
parameters of a conjugate prior can be interpreted as the number (r/) and values 
(,) of pseudo-observations under the corresponding likelihood. We call models that 
satisfy conditions (1)and (2)conjugate-exponential. 
1Models whose complete-data likelihood is not in the exponential family (such as ICA 
with the logistic nonlinearity, or sigmoid belief networks) can often be approximated by 
models in the exponential family with additional hidden variables. 
In Bayesian inference we want to determine the posterior over parameters and 
hidden variables P(x, Oly , t/, t). In general this posterior is neither conjugate nor in 
the exponential family. We therefore approximate the true posterior by the following 
factorised distribution: P(x, Oly , t], t)  Q(x, O) - Qx(X)Qo(O), and minimise 
f Q(x,O) 
KL(QIIP) = dxdOQ(x,O)lnp(x, Oly, rl, ) 
which is equivalent to maximising r(Qx(x), Qo(O), y). We provide several general 
results with no proof (the proofs follow from the definitions and Gibbs inequality). 
Theorem I Given an lid data set y = (y,..-Yn), if the model satisfies conditions 
(1) and (2), then at the maxima of (Q,y) (minima of KL(QIIP)): 
(a) Qo(O) is conjugate and of the form: 
Qo(O) = h(, ,)g(O)  exp { qb(O)T , } 
where 0 = q+,  = r, + y].in__ (yi), and(yi) = (u(xi, Yi))Q, using (.)Q 
to denote expectation under Q. 
(b) Qx(X) = 1-[in__ Qx(Xi) and Qx(Xi) is of the same form as the known pa- 
rameter posterior: 
Qx(Xi) x f(xi,yi)exp{(O)-Vu(xi,yi)} = P(xilyi,(O)) 
where (0) = 
Since Qo(O) and Qx(Xi) are coupled, (a) and (b) do not provide an analytic so- 
lution to the minimisation problem. We therefore solve the optimisation problem 
numerically by iterating between the fixed point equations given by (a) and (b), and 
we obtain the following variational Bayesian generalisation of the EM algorithm: 
VE Step: Compute the expected sufficient statistics t(y) = -].i(Yi) 
under the hidden variable distributions Qx (xi). 
VM Step: Compute the expected natural parameters (0) under the 
parameter distribution given by 0 and ,. 
This reduces to the EM algorithm if we restrict the parameter density to a point 
estimate (i.e. Dirac delta function), Qo(O) = (0 - 0'), in which case the M step 
involves re-estimating 0'. 
Note that unless we make the assumption that the parameters and hidden variables 
factorise, we will not generally obtain the further hidden variable factorisation over 
n in (b). In that case, the distributions of xi and xj will be coupled for all cases i, j 
in the data set, greatly increasing the overall computational complexity of inference. 
4 Belief Networks and Markov Networks 
The above result can be used to derive variational Bayesian learning algorithms for 
exponential family distributions that fall into two important special classes? 
Corollary 1: Conjugate-Exponential Belief Networks. Let JM be a 
conjugate-exponential model with hidden and visible variables z = (x, y) that sat- 
isfy a belief network factorisation. That is, each variable zj has parents Zpj and 
P(z[0) - Hj P(zj[zpj,O). Then the approximating joint distribution for JM satis- 
fies the same belief network factorisation: 
Q.(z) = H Q(zJlZp ,0) 
J 
2 tutorial on belief networks and Markov networks can be found in [9]. 
where the conditional distributions have exaly the same form as those in the 
original model but with natural parameters b()) - b()). Furthermore, with the 
modified parameters , the expectations under the approximating posterior Qx(X) e 
Q,. (z) required for the VE Step can be obtained by applying the belief propagation 
algorithm if the network is singly connected and the junction tree algorithm if the 
network is multiply-connected. 
This result is somewhat surprising as it shows that it is possible to infer the hidden 
states tractably while integrating over an ensemble of model parameters. This result 
generalises the derivation of variational learning for HMMs in [8], which uses the 
forward-backward algorithm as a subroutine. 
Theorem 2: Markov Networks. Let it4 be a model with hidden and visible vari- 
ables z - (x, y) that satisfy a Markov network factorisation. That is, the joint den- 
sity can be written as a product of clique-potentials j, P(z10 ) - g(O) 1-Ij j(Cj, 0), 
where each clique Cj is a subset of the variables in z. Then the approximating joint 
distribution for it4 satisfies the same Markov network factorisation: 
J 
where j(Cj) = exp {<ln j(Cj, O)>Q } are new clique potentials obtained by averag- 
ing over Qo(O), and  is a normalisation constant. Furthermore, the expectations 
under the approximating posterior Qx(x) required for the VE Step can be obtained 
by applying the junction tree algorithm. 
Corollary 2: Conjugate-Exponential Markov Networks. Let it4 be a 
conjugate-exponential Markov network over the variables in z. Then the approx- 
imating joint distribution for it4 is given by Qz(z) =  1-Ij j(Cj,O), where the 
clique potentials have exactly the same form as those in the original model but with 
natural parameters b(O) = (0). 
For conjugate-exponential models in which belief propagation and the junction tree 
algorithm over hidden variables is intractable further applications of Jensen's in- 
equality can yield tractable factorisations in the usual way [7]. 
In the following section we derive a variational Bayesian treatment of linear- 
Gaussian state-space models. This serves two purposes. First, it will illustrate 
an application of Theorem 1. Second, linear-Gaussian state-space models are the 
cornerstone of stochastic filtering, prediction and control. A variational Bayesian 
treatment of these models provides a novel way to learn their structure, i.e. to 
identify the optimal dimensionality of their state-space. 
5 State-space models 
In state-space models (SSMs), a sequence of D-dimensional real-valued observation 
vectors {y,... ,YT}, denoted Y:T, is modeled by assuming that at each time step 
t, Yt was generated from a K-dimensional real-valued hidden state variable xt, and 
that the sequence of x's define a first-order Markov process. The joint probability 
of a sequence of states and observations is therefore given by (Figure 1): 
T 
P(X:T,Y:T) : P(x)P(yIx ) H P(xtlxt-)P(YtlXt)- 
t----2 
We focus on the case where both the transition and output functions are linear and 
time-invariant and the distribution of the state and observation noise variables is 
Gaussian. This model is the linear-Gaussian state-space model: 
x = Ax__ + w, y = Cx + v 
Figure 1: Belief network representation of a state-space model. 
where A and C are the state transition and emission matrices and wt and vt are 
state and output noise. It is straightforward to generalise this to a linear system 
driven by some observed inputs, ut. A Bayesian analysis of state-space models using 
MCMC methods can be found in [4]. 
The complete data likelihood for state-space models is Gaussian, which falls within 
the class of exponential family distributions. In order to derive a variational 
Bayesian algorithm by applying the results in the previous section we now turn 
to defining conjugate priors over the parameters. 
Priors. Without loss of generality we can assume that wt has covariance equal to 
the unit matrix. The remaining parameters of a linear-Gaussian state-space model 
are the matrices `4 and C and the covariance matrix of the output noise, vt, which 
we will call R and assume to be diagonal, R = diag(p)-, where Pi are the precisions 
(inverse variances) associated with each output. 
Each row vector of the .4 matrix, denoted a7 is given a zero mean Gaussian prior 
with inverse covariance matrix equal to diag(c). Each row vector of C, c, is 
given a zero-mean Gaussian prior with precision matrix equal to diag(pi/3). The 
dependence of the precision of  on the noise output precision Pi is motivated by 
conjugacy. Intuitively, this prior links the scale of the signal and noise. 
The prior over the output noise covariance matrix, R, is defined through the pre- 
cision vector, p, which for conjugacy is assumed to be Gamma distributed 3 with 
hyperparameters a and b: P(p [a,b) D b a a- exp{-bpi}. Here, c, /3 are 
= H= r(a) P 
hyperparameters that we can optimise to do automatic relevance determination 
(ARD) of hidden states, thus inferring the structure of the SSM. 
Variational Bayesian learning for SSMs 
Since A, C, p and X:T are all unknown, given a sequence of observations Yl:T, an 
exact Bayesian treatment of SSMs would require computing marginals of the poste- 
rior P(A, C, p, x:T[y:T). This posterior contains interaction terms up to fifth order 
(for example, between elements of C, x and p), and is not analytically manageable. 
However, since the model is conjugate-exponential we can apply Theorem I to de- 
rive a variational EM algorithm for state-space models analogous to the maximum- 
likelihood EM algorithm [10]. Moreover, since SSMs are singly connected belief 
networks Corollary I tells us that we can make use of belief propagation, which in 
the case of SSMs is known as the Kalman smoother. 
Writing out the expression for log P(`4, C, p, x:T, y:), one sees that it contains 
interaction terms between p and C, but none between `4 and either p or C. This 
observation implies a further factorisation, Q(A, C,p) - Q(A)Q(C, p), which falls 
out of the initial factorisation and the conditional independencies of the model. 
Starting from some arbitrary distribution over the hidden variables, the VM step 
obtained by applying Theorem I computes the expected natural parameters of 
Qo(O), where 0 = (A, C, p). 
SMote generally, if we let R be a full covariance matrix for conjugacy we would give 
its inverse V = R -1 a Wishart distribution: P(V[,S) oc IV[ (-D-1)/2 exp (- 21-tr VS-1), 
where tr is the matrix trace operator. 
We proceed to solve for Q(A). We know from Theorem 1 that Q(A) is multivariate 
Gaussian, like the prior, so we only need to compute its mean and covariance. 
has mean $:- (diag(et) q- W) - and each row of A has covariance (diag(et) q- W) -, 
T T--1 
where $- Y]t=2 <xt_x), W = Et= <xtx), and (.)denotes averaging w.r.t. 
the Q(x:) distribution. 
Q(C, p) is also of the same form as the prior. Q(p) is a product of Gamma densities 
i T 2 
. T ii = b + Si, Si Et: Yti - Ui(diag() + 
0(pd = gd where a = a + E, = 
T W t 
W')-U y Ui : Et=Sti(x[} and : W + {xrx} Given p, each row of 
C is Gaussian with covariance Cov(ci) = (diag() + W')-/pi and mean ei = 
pi Ui Cov(ci). Note that S, W and Ui are the expected complete data sufficient 
statistics  mentioned in Theorem l(a). Using the parameter distributions the 
hyperparameters can also be optimised. a 
We now turn to the VE step: computing O(x:r). Since the model is a conjugate- 
exponential singly-connected belief network, we can use belief propagation (Corol- 
lary 1). For SSMs this corresponds to the Kalman smoothing algorithm, where 
every appearance of the natural parameters of the model is replaced with the fol- 
lowing corresponding expectations under the O distribution: (pie/}, (p/c/c/}, (A}, 
(A A). Details can be found in 
Like for PCA [3], independent components analysis [1], and mixtures of factor 
analysers [5], the variational Bayesian algorithm for state-space models can be used 
to learn the structure of the model as well as average over parameters. Specifically, 
using Y it is possible to compare models with different state-space sizes and optimise 
the dimensionality of the state-space, as we demonstrate in the following section. 
6 Results 
Experiment 1: The goal of this experiment was to see if the variational method 
could infer the structure of a variety of state space models by optimising over et and 
/. We generated a 200-step time series of 10-dimensional data from three models: 5 
(a) a factor analyser (i.e. an SSM with A = 0) with 3 factors (static state variables); 
(b) an SSM with 3 dynamical interacting state variables, i.e. A  0; (c) an SSM 
with 3 interacting dynamical and I static state variables. The variational Bayesian 
method correctly inferred the structure of each model in 2-3 minutes of CPU time 
on a 500 MHz Pentium III (Fig. 2 (a)-(c)). 
Experiment 2: We explored the effect of data set size on complexity of the recov- 
ered structure. 10-dim time series were generated from a 6 state-variable SSM. On 
reducing the length of the time series from 400 to 10 steps the recovered structure 
became progressively less complex (Fig. 2(d)-(j)), to a 1-variable static model (j). 
This result agrees with the Bayesian perspective that the complexity of the model 
should reflect the data support. 
Experiment 3 (Steel plant): 38 sensors (temperatures, pressures, etc) were 
sampled at 2 Hz from a continuous casting process for 150 seconds. These sensors 
covaried and were temporally correlated, suggesting a state-space model could cap- 
ture some of its structure. The variational algorithm inferred that 16 state variables 
were required, of which 14 emitted outputs. While we do not know whether this is 
reasonable structure we plan to explore this as well as other real data sets. 
c and/ = D The 
4The ARD hyperpaxameters become ct -- <a-a>, (CTdiag(p)C}ll ' 
i D 
hyperpaxameters a and b solve the fixed point equations b(a) = In b +  ]i=l (ln pi), and 
i D 
d-D i=l (pi), where (w) = o In F(w) is the digaroma function. 
Parameters were chosen as follows: R = I, and elements of C sampled from 
Unif(-5, 5), and A chosen with eigen-values in [0.5, 0.9]. 
Xt  Xt Yt 
a: 
e f g 
Xt 1 Xt Yt 
Xt 1 Xt Yt 
Xt 1 Xt Yt 
Xt 1 Xt Yt Xt 1 Xt Yt 
Figure 2: The elements of the A and C matrices after learning are displayed graphically. 
i 1 
A link is drawn from node k in xt-1 to node I in xt iff  > e, and either  > e or 
1 
1 ) e, for a small threshold e. Similarly links are drawn from node k of xt to yt if  ) e. 
c l 
Therefore the graph shows the links that take part in the dynamics and the output. 
7 Conclusions 
We have derived a general variational Bayesian learning algorithm for models in the 
conjugate-exponential family. There are a large number of interesting models that 
fall in this family, and the results in this paper should allow an almost automated 
protocol for implementing a variational Bayesian treatment of these models. 
We have given one example of such an implementation, state-space models, and 
shown that the VB algorithm can be used to rapidly infer the hidden state dimen- 
sionality. Using the theory laid out in this paper it is straightforward to generalise 
the algorithm to mixtures of SSMs, switching SSMs, etc. 
For conjugate-exponential models, integrating both belief propagation and the junc- 
tion tree algorithm into the variational Bayesian framework simply amounts to com- 
puting expectations of the natural parameters. Moreover, the variational Bayesian 
algorithm contains EM as a special case. We believe this paper provides the founda- 
tions for a general algorithm for variational Bayesian learning in graphical models. 
References 
[1] H. Attias. A variational Bayesian framework for graphical models. In Advances in 
Neural Information Processing Systems 12. MIT Press, Cambridge, MA, 2000. 
[2] M.J. Beal and Z. Ghahramani. The variational Kalman smoother. Technical report, 
Gatsby Computational Neuroscience Unit, University College London, 2000. 
[3] C.M. Bishop. Variational PCA. In Proc. Ninth ICANN, 1999. 
[4] S. Friiwirth-Schnatter. Bayesian model discrimination and Bayes factors for linear 
Gaussian state space models. J. Royal. Stat. Soc. B, 57:237-246, 1995. 
[5] Z. Ghahramani and M.J. Beal. Variational inference for Bayesian mixtures of factor 
analysers. In Adv. Neur. Inf. Proc. Sys. 12. MIT Press, Cambridge, MA, 2000. 
[6] G.E. Hinton and D. van Camp. Keeping neural networks simple by minimizing the de- 
scription length of the weights. In Sixth A CM Conference on Computational Learning 
Theory, Santa Cruz, 1993. 
[7] M.I. Jordan, Z. Ghahramani, T.S. Jaakkola, and L.K Saul. An introduction to vari- 
ational methods in graphical models. Machine Learning, 37:183-233, 1999. 
[8] D.J.C. MacKay. Ensemble learning for hidden Markov models. Technical report, 
Cavendish Laboratory, University of Cambridge, 1997. 
[9] J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Infer- 
ence. Morgan Kaufmann, San Mateo, CA, 1988. 
[10] R. H. Shumway and D. S. Stoffer. An approach to time series smoothing and fore- 
casting using the EM algorithm. J. Time Series Analysis, 3(4):253-264, 1982. 
[11] S. Waterhouse, D.J.C. Mackay, and T. Robinson. Bayesian methods for mixtures of 
experts. In Adv. Neur. Inf. Proc. Sys. 7. MIT Press, 1995. 
