18.413: Error-Correcting Codes Lab March 16, 2004
Lecture 12
Lecturer: Daniel A. Spielman
12.1 Representing Probabilities, Equality nodes
It turns out that probabilities are not always the best way to represent the quantities we are
considering. To see why, I’ll begin with an examination of the computation at equality nodes. Say
that an equality node has incomming messages pint
i, for i= 1, . . . , k, and we want to compute
pext
1=Qk
i=2 pint
i
Qk
i=2 pint
i+Qk
i=2(1 −pint
i).
It turns out that this computation is much easier if we use likelihood ratios:
lrint
i=pint
i
1−pint
i
.
The reason is that we have
lrext
1=
k
Y
i=2
lrint
i.
To see this, note that
pext
1
1−pext
1
=Qk
i=2 pint
i
Qk
i=2(1 −pint
i).
This observation also makes it easier to compute pext
ifor all iat once: first multiply all of the pint
is
together, and then just divide out the appropriate terms as needed. However, I should warn you
that this solution is better in theory than in practice: in practice it is very easy to get zero divided
by zero this way.
Pushing things a little further, note that it is much easier to add than multiply. So, we could take
the logs of all these terms, and then just add them. This gives the log-likelihood-ratios
llri= log(lri).
Log-likelihood-ratios are nicer than likelihood ratios is that they treat probability values near 0 and
near 1 symmetrically: going to the other side just changes the sign in the llr. On the other hand,
the representation of numbers in floating point creates assymetry in likelihood ratios.
12-1
