Download Error-Correcting Codes Lab: Proof of Markov Property and Probability Computation and more Slides Applied Mathematics in PDF only on Docsity!
18.413: Error-Correcting Codes Lab February 26, 2004
Lecture 7
Lecturer: Daniel A. Spielman
To begin, let me point out that there was a typo in the lecture notes from last lecture. Lemma 6.4.
should have said:
Lemma 7.0.1.
P
ext
[X 1
= a 1
|Y
2
Y
3
= b 2
b 3
] =
a 2 :(a 1 ,a 2 )∈C 12
P [X
2
= a 2
|X
1
= a 1
] P
ext
[X 2
= a 2
|Y
2
= b 2
] P
ext
[X 2
= a 2
|Y
3
= b 3
].
7.1 Markov Property
Last lecture, we considered three variables X 1
, X
2
and X 3
chosen uniformly from those that satisfy
(X
1
, X
2
) ∈ C
12
⊆ A
1
× A
2
and and (X 2
, X
3
) ∈ C
23
⊆ A
2
× A
3
We claimed that the variables (X 1
, X
2
, X
3
) then satisfy what the book calls the “Markov” property.
That is, for all a 1 , a 2 , a 3 ,
P [X
1
X
3
= a 1
a 3
|X
2
= a 2
] = P [X
1
= a 1
|X
2
= a 2
] P [X
3
= a 3
|X
2
= a 2
].
I’ll now sketch a proof. It basically follows by by recalling the definition of the probability of an
even conditioned on X 2 = a 2. We first note that the set of choices for (X 1 , X 2 , X 3 ) given that
X
2
= a 2
is
Sa 2
def
= {(X 1 , a 2 , X 3 ) : (X 1 , a 2 ) ∈ C 12 and (a 2 , X 3 ) ∈ C 23 }.
Conditioning on X 2 = a 2 , we obtain a sample chosen uniformly from Sa 2
. Thus, for (a 1 , a 2 , a 3 ) ∈
S
a 2
P [(X
1
, X
2
, X
3
) = (a 1
, a 2
, a 3
)|X
2
= a 2
] =
|Sa 2
Note that
|S
a 2
| = |{a 1
: (a 1
, a 2
) ∈ C
12
}| |{a 3
: (a 2
, a 3
) ∈ C
23
The claim now follows from observing that
P [(X 1 , X 2 ) = (a 1 , a 2 )|X 2 = a 2 ] =
|{a 3
: (a 2
, a 3
) ∈ C
23
|S
a 2
|{a 1
: (a 1
, a 2
) ∈ C
12
and
P [(X
2
, X
3
) = (a 2
, a 3
)|X
2
= a 2
] =
|{a 1 : (a 1 , a 2 ) ∈ C 12 }|
|Sa 2
|{a 3 : (a 2 , a 3 ) ∈ C 23 }|
Lecture 7: February 26, 2004 7-
7.2 Simplifying Probability Computation
First, lets establish that the fundamental quantities we are interested in have the form
P [X
i
= a i
|E] ,
where E is some event, usually a union of the observed variables. We will typically want these
values for all a i
, so we really want a vector
(P [Xi = a 1 |E] , P [Xi = a 2 |E] ,... , P [Xi = an|E] , ) ,
where a 1 ,... , an are the symbols in the alphabet Ai. We will denote such a vector by
P [X
i
|E].
Using this notation, and letting denote componentwise product ((a, b) (c, d) = (ac, bd)), we
have
P
post
[Xi|E] =
P
prior
[Xi]
P
ext
[Xi|E]
Before returning to our probability computations for (X 1
, X
2
, X
3
), I’ll also introduce the simpler
notation P
ext
[Xi = ai|Yi] for P
ext
[Xi = ai|Yi = bi]. We will use this notation whenever bi is fixed
throughout our computation, which it generally is as it is what was received.
We then have, from Lemma 6.2.1,
P
post
[X
1
|Y
1
Y
2
Y
3
] =
P
prior
[X
1
]
P
ext
[X
1
|Y
1
]
P
ext
[X
1
|Y
2
Y
3
] ,
and, from Lemma 7.0.1,
P
ext
[X 1
= a 1
|Y
2
Y
3
] =
a 2 :(a 1 ,a 2 )∈C 12
P [X
2
= a 2
|X
1
= a 1
] P
ext
[X 2
= a 2
|Y
2
] P
ext
[X 2
= a 2
|Y
3
].
Using these formulas, we go through the following steps to compute
P
post
[X 1 |Y 1 Y 2 Y 3 ].
- Compute
P
ext
[X 2 |Y 3 ]. This computation only depends upon Y 3 , and comes from the formula:
P
ext [X 2
= a 2
|Y
3
] ∼
a 3 :(a 2 ,a 3 )∈C 23
P [X
3
= a 3
|X
2
= a 2
] P [Y
3
|X
3
= a 3
].
- Compute
P
ext
[X
2
= |Y
2
], and compute
P
ext
[X 2 = |Y 2 ] P
ext
[X 2 = a 2 |Y 3 ].
- For each a 1
, compute
P [X 2 |X 1 = a 1 ]
P
ext
[X 2 |Y 2 ]
P
ext
[X 2 |Y 3 ] ,
and then sum the resulting vector.
- Take the output of the previous step, and product it with
P
prior
[X 1 ]
P
ext
[X 1 |Y 1 ].
If you look at the flow of this computation, it can be understood as a vector being passed from X 3
to X 2 between steps 1 and 2, and a vector begin passed from X 2 to X 1 between steps 3 and 4.
