Daphne Koller
Message Passing
BeliefPropagationAlgorithm
Probabilistic
Graphical
Models
Inference
Daphne Koller
B
D
C
A
2: B,C
1: A,B
3: C,D
4: A,D
C
A
B
D
Cluster Graph
Daphne Koller
Passing Messages
2: B,C
1: A,B
3: C,D
4: A,D
C
A
B
D
2,1 = 1
1(A,B)
4(A,D)
2(B,C)
3(C,D)
1,4 =
B 2,1(B) (A,B)
Daphne Koller
Passing Messages
2: B,C
1: A,B
3: C,D
4: A,D
C
A
B
D
1,2 =
2,1 =
2,3 =
3,2 =
3,4 =
4,3 =
1,4 =
4,1 =
B 2,1(B) (A,B)
D 3,4(D) (A,D)
C 2,3(C) (C,D)
A 1,4(A) (A,D)
B 1,2(B) (B,C)
D 4,3(D) (C,D)
A 4,1(A) (A,B)
C 3,2(C) (B,C)
Daphne Koller
Cluster Graphs
•Undirected graph such that:
–nodes are clusters Ci {X1,…,Xn}
–edge between Ci and Cj associated withsepset Si,j Ci Cj
•Given set of factors , we assign each kto a cluster C(k) s.t. Scope[k] C(k)
•Define
Daphne Koller
Example Cluster Graph
1: A, B, C
3: B,D,F
2: B, C, D
5: D, E
4: B, E
B
B
B
C
D
D
E
1 = 1
2 = 2 3 6
4 = 5
5 = 4
3 = 7
Daphne Koller
Different Cluster Graph
1: A, B, C
3: B,D,F
2: B, C, D
5: D, E
4: B, E
B
B
B,C
D
D
E
Daphne Koller
Message Passing
1: A, B, C
3: B,D,F
2: B, C, D
5: D, E
4: B, E
B
B
B
C
D
D
E
Daphne Koller
Belief Propagation Algorithm
•Assign each factor k to a cluster C(k)
•Construct initial potentials
•Initialize all messages to be 1
•Repeat
–Select edge (i,j) and pass message
•Compute
Daphne Koller
Belief Propagation Run
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0
5
10
15
20
Iteration #
P(a1)
True posterior
Daphne Koller
Different BP Run
Daphne Koller
Summary
•Graph of clusters connected by sepsets
•Adjacent clusters pass information to eachother about variables in sepset
–Message from i to j summarizes everything iknows, except information obtained from j
•Algorithm may not converge
•The resulting beliefs are pseudo-marginals
•Nevertheless, very useful in practice
Daphne Koller
END END END