Download Error-Correcting Codes Lab 08, Lecture Slide - Engineering and more Slides Applied Mathematics in PDF only on Docsity!
18.413: Error-Correcting Codes Lab March 2, 2004
Lecture 8
Lecturer: Daniel A. Spielman
8.1 Vector Spaces
A set C ∈ { 0 , 1 }n^ is a vector space if for all x ∈ C and y ∈ C, x + y ∈ C, where we take addition to be component-wise modulo 2. We note that over 0, 1, we do not need to state the property that cx ∈ C for all c ∈ { 0 , 1 }, as it is obvious. Note that the all-0 vector is always in C, as it equals x + x.
Given vector x 1 ,... , xk, we define
span (x 1 ,... , xk) = {a 1 x 1 + · · · + akxk : a 1 ,... , ak ∈ { 0 , 1 }}.
We say that x 1 ,... , xk span C if C = span (x 1 ,... , xk).
The following definition is fundamental.
Definition 8.1.1. The vectors x 1 ,... , xk are a basis for C if they span C and no proper subset of these vectors spans C.
Lemma 8.1.2. The vectors x 1 ,... , xk are a basis for C if and only if they span C and for each i,
xi 6 ∈ span (x 1 ,... , xi− 1 , xi+1,... , xk).
Proof. If some subset of x 1 ,... , xk spans C, then there exists i such that x 1 ,... , xi− 1 , xi+1,... , xk spans C. As xi ∈ C for all i, we then have
xi ∈ span (x 1 ,... , xi− 1 , xi+1,... , xk).
On the other hand, if xi ∈ span (x 1 ,... , xi− 1 , xi+1,... , xk) ,
then we will show that x 1 ,... , xi− 1 , xi+1,... , xk span C. To see this, let xi =
j 6 =i bj^ xj^. We will now show that every vector in span (x 1 ,... , xk) can be expressed without using xi. Let
x =
j
aj xj.
If ai = 0, then x ∈ span (x 1 ,... , xi− 1 , xi+1,... , xk). If ai = 1, then
x = xi +
j 6 =i
aj xj =
j 6 =i
bj xj +
j 6 =i
aj xj =
j 6 =i
(aj + bj )xj ,
and so x ∈ span (x 1 ,... , xi− 1 , xi+1,... , xk).
Lemma 8.1.3. Every vector space C ∈ { 0 , 1 }n^ has a basis.
Proof. We first note that C spans C. Now, if we have a set S of vectors that spans C, but which is not a basis, then we can find a proper subset of S that spans C. If we replace S by this propert subset, and repeat, we will eventually find a basis. The process cannot go on forever because initially S is finite and at each step is gets smaller.
Lemma 8.1.4. Let {x 1 ,... , xk} be a basis for C. Then, for (a 1 ,... , ak) ∈ { 0 , 1 }k^ and (b 1 ,... , bk) ∈ { 0 , 1 }k, if there exists a j for which aj 6 = bj , then ∑
i
aixi 6 =
i
bixi.
Proof. We may assume without loss of generality that aj = 0 and bj = 1. Assume by way of contradiction that (^) ∑
i
aixi =
i
bixi.
Then, (^) ∑
i 6 =j
(ai + bi)xi = xj ,
so xj ∈ span (x 1 ,... , xj− 1 , xj+1,... , xk) ,
contradicting the assumption that x 1 ,... , xk is a basis.
Lemma 8.1.5. If x 1 ,... , xk is a basis of C, then |C| = 2k.
Proof. There are 2k^ vectors of the form (^) ∑ aixi,
and by the previous lemma they are all distinct.
Corollary 8.1.6. Each basis of a vector space has the same number of elements.
If C has a basis of k vectors, then we say that C has dimension k.
8.2 Dual
Definition 8.2.1. If C is a vector space in { 0 , 1 }n, then the dual of C is
D =
y ∈ { 0 , 1 }n^ : ∀x ∈ C, yT^ x = 0
This is where we will see a difference between vector spaces over the reals and { 0 , 1 }: we can have vectors in both C and dual (C). For example, consider
C = { 0000 , 0011 , 1100 , 1111 }.
In this case, we have dual (C) = C.
We can now prove the lemma that we set out to prove.
proof of Lemma 8.2.4. Let x 1 ,... , xk be a basis of C. By Lemma 8.2.5, there exist xk+1,... , xn for which x 1 ,... , xn is a basis of { 0 , 1 }n. Let y 1 ,... , yn be the inverse basis shown to exist in Lemma 8.2.8. We claim that yk+1,... , yn is a basis for dual (C). From (8.1), it is clear that each of these vectors is in dual (C). To show that they span dual (C), let z ∈ dual (C). Express
z =
i
biyi.
If bi is 1 for some i ≤ k, then we will have
xTi z = xTi yi = 1,
contradicting the assumption that z ∈ dual (C). Thus, each vector in dual (C) is spanned by yk+1,... , yn.
8.3 Codes and Matrices
Let C be a linear code over { 0 , 1 }. Then, C can be expressed either as the output of a generator matrix: C =
wG : w ∈ { 0 , 1 }k
where G is a k-by-n matrix whose rows form a basis of C, or as those words satisfying a check matrix C =
x : Hx = ~ 0
where H is a n − k-by-n matrix whose rows form a basis of the dual space of C.
It turns out that particular matrices are more useful that others. For example, consider the case in which G has the form G = [IkP ] ,
where Ik is the k-by-k identity matrix. In this case, the first k bits of wG are w. Thus, the message that we are encoding appears in the codeword. An encoding matrix that has this property is called systematic, and this property is particularly useful if we are trying to estimate the wis from corrupted versions of x. In general, any encoding matrix whose columns can be permuted into this special form is called systematic. One can prove:
Lemma 8.3.1. If G is a matrix whose rows are a basis, then there is a systematic matrix G′^ such that {wG} =
wG′
Sketch. One can obtain G′^ from Gaussian elimination. You begin by finding some column that has a 1 in the first row. You then add this row to every other row that has a 1 in that column. After you do this, that will be the only row with a 1 in that column. You then move on to do the same for the next row, etc.
Lemma 8.3.2. If the span of the rows of G is C, and G has the form
G = [IkP ] ,
then the span of the rows of H =
[
P T^ In−k
]
is dual (C).
