18.413: Error-Correcting Codes Lab February 3, 2004
Lecture 1
Lecturer: Daniel A. Spielman
1.1 Introduction
The purpose of this course is to introduce you to some of the most exciting recent developments in
the field of error-correcting codes. The reason that I can teach this material at an undergraduate
level is that recent revolutions in the field have resulted in new coding techniques that can be taught
without reference to much that happened between the late 60’s and the early 90’s. Before this
revolution, error-correcting codes became highly algebraic, and a working knowledge of Algebraic
Geometry became a prerequisite to understanding the the research that was current at the time.
However, much of that research was divorced from practice, and most practical devices were built
using theory that was not taught in these classes.
The revolution consisted of the introduction of iterative coding techniques powered by the Belief
Propagation heuristic. The field of mathematics most closely connected with these codes is prob-
ability, and the most popular analyses combine computer experiment with theory. Moreover, the
theory that has been developed has close connections to practice, and iterative coding techniques
are already being used by companies today. The reasons that I can teach this material to you are
that
•one only needs to know a little probability to understand these codes, and
•much understanding can be gained from experiment.
Wow, that discussion was a little abstract, and maybe meaningless to those of you who don’t even
know what coding theory is. It’s time for me to tell you.
Error-correcting codes are used to facilitate communication in the presence of interference. Inter-
ference can take many forms. For example, static on a phone line, people screaming in the hall
outside this classroom, or bits randomly flipping from 0 to 1 or vice versa. For now, let’s consider
the simplest model–that of flipping bits. We will assume that a sender is trying to send information
to a receiver through a channel that has a probability pof flipping each bit. We will assume that a
bit is received for every bit sent, but with probability pit is the wrong bit. This channel is called
the Binary Symmetric Channel with crossover probability p. Assuming that it is important for the
receiver to receive what the sender is sending (say a new operating system), the sender is going to
have to do something fancier than just sending the bits. A natural thing to do would be to send
each bit twice. That way, if the receiver gets a 01 or 10, she will know that one of the two bits
was wrong. If we assume that there is also a channel going the other way, then she can request a
retransmission of the bit that was in error. However, even if the receiver gets a 11, there is still
1-1
