Download Error-Correcting Codes Lab 06, Lecture Slide - Engineering and more Slides Applied Mathematics in PDF only on Docsity!
18.413: Error-Correcting Codes Lab February 24, 2004
Lecture 6
Lecturer: Daniel A. Spielman
6.1 Introduction
Begin by describing LDPC codes, and how they are described by many local constraints. Point out that random graphs locally look like trees (from the birthday paradox), and so we will learn to do belief propagation on trees. But first, we must learn to do BP on the simplest of trees: with just 2 and three nodes.
6.2 Two variables
We begin with a further examination of our fundamental formula in the case of just two variables. Let X 1 be a variable taking values in the alphabet A 1 and let X 2 be a variable taking values in the alphabet A 2. Then let C โ A 1 ร A 2 be a code.
Assume that we choose (X 1 , X 2 ) โ C uniformly at random, transmit over a channel, and receive (Y 1 , Y 2 ).
We will show
Lemma 6.2.1.
Ppost^ [X 1 = a 1 |Y 1 Y 2 = b 1 b 2 ] = cb 1 ,b 2 Pprior^ [X 1 = a 1 ] Pext^ [X 1 = a 1 |Y 1 = b 1 ] Pext^ [X 1 = a 1 |Y 2 = b 2 ] ,
where
Pprior^ [X 1 = a 1 ] =
|{a 2 : (a 1 , a 2 ) โ C}| |C|
As we already know that
Ppost^ [X 1 = a 1 |Y 1 Y 2 = b 1 b 2 ] = cb 1 ,b 2 Pprior^ [X 1 = a 1 ] Pext^ [X 1 = a 1 |Y 1 Y 2 = b 1 b 2 ] ,
so it suffices to prove
Lemma 6.2.2.
Pext^ [X 1 = a 1 |Y 1 Y 2 = b 1 b 2 ] = cb 1 ,b 2 Pext^ [X 1 = a 1 |Y 1 = b 1 ] Pext^ [X 1 = a 1 |Y 2 = b 2 ].
Lecture 6: February 24, 2004 6-
Proof. We begin by examining the right-hand-sides. We have
Pext^ [X 1 = a 1 |Y 1 = b 1 ] = cb 1 P [Y 1 = b 1 |X 1 = a 1 ] (6.1)
and
Pext^ [X 1 = a 1 |Y 2 = b 2 ] = cb 2 P [Y 2 = b 2 |X 1 = a 1 ] = cb 2
a 2 :(a 1 ,a 2 )โC
P [Y 2 = b 2 |X 1 X 2 = a 1 a 2 ] P [X 2 = a 2 |X 1 = a 1 ]
= cb 2
a 2 :(a 1 ,a 2 )โC
P [Y 2 = b 2 |X 2 = a 2 ] P [X 2 = a 2 |X 1 = a 1 ]. (6.2)
Now, we examine the left-hand-side:
Pext^ [X 1 = a 1 |Y 1 Y 2 = b 1 b 2 ] = cb 1 ,b 2 P [Y 1 Y 2 = b 1 b 2 |X 1 = a 1 ]
= cb 1 ,b 2
a 2 :(a 1 ,a 2 )โC
P [Y 1 Y 2 = b 1 b 2 |X 1 X 2 = a 1 a 2 ] P [X 2 = a 2 |X 1 = a 1 ]
= cb 1 ,b 2
a 2 :(a 1 ,a 2 )โC
P [Y 1 = b 1 |X 1 = a 1 ] P [Y 2 = b 2 |X 2 = a 2 ] P [X 2 = a 2 |X 1 = a 1 ]
= cb 1 ,b 2 P [Y 1 = b 1 |X 1 = a 1 ]
a 2 :(a 1 ,a 2 )โC
P [Y 2 = b 2 |X 2 = a 2 ] P [X 2 = a 2 |X 1 = a 1 ].
To conclude, we observe that this last term is the product of (6.1) and (6.2).
6.3 Simplifying computations
6.4 Three Variables
We now consider the situation in which X 1 , X 2 and X 3 lie in A 1 , A 2 and A 3 , and (X 1 , X 2 ) โ C 12 โ A 1 ร A 2 and and (X 2 , X 3 ) โ C 23 โ A 2 ร A 3. In particular, we will assume that (X 1 , X 2 , X 3 ) are chosen uniformly subject to this condition.
The variables (X 1 , X 2 , X 3 ) then satisfy what the book calls the โMarkovโ property. That is, for all a 1 , a 2 , a 3 ,
P [X 1 X 3 = a 1 a 3 |X 2 = a 2 ] = P [X 1 = a 1 |X 2 = a 2 ] P [X 3 = a 3 |X 2 = a 2 ].
In this case, we can say that all the information that X 3 contains about X 1 is transmitted through X 2. This fact can be used to simplify the belief computation.
Lemma 6.4.1.
Pext^ [X 1 = a 1 |Y 2 Y 3 = b 2 b 3 ] =
a 2 :(a 1 ,a 2 )โC
P [X 2 = a 2 |X 1 = a 1 ] Pext^ [X 2 = a 2 |Y 2 = b 2 ] Pext^ [X 2 = a 2 |Y 3 = b 3 ].
