GEORGIA INSTITUTE OF TECHNOLOGY
School of Electrical and Computer Engineering
ECE 6258
Digital Image Processing
Fall 2003
Problem Set #4
Issued: Wednesday, October 1, 2003
Due (live): Friday, October 10, 2003
Due (video): Friday, October 24, 2003
Problem 4.1 (Huļ¬man Codes): Find a Huļ¬man code for the source shown below. The
resulting codes are not unique, but you should try to ļ¬nd a code whose longest code-
word is as short as possible.
Si:p(Si)
S1:0.20
S2:0.18
S3:0.12
S4:0.10
S5:0.10
S6:0.08
S7:0.06
S8:0.06
S9:0.06
S10:0.04
Problem 4.2 (Tunstall Codes): A Huļ¬man code accepts strings of symbols of equal
length and maps them to codewords of variable length. A run-length code, by way of
contrast, accepts input strings of variable length and outputs symbols of equal length.
The optimal code of the latter type is known as a Tunstall code. Tunstall codes are
duals of Huļ¬man codes. A Q-ary Tunstall code operating on strings of binary input
symbols can be found by the following procedure:
1. First form a tree with two leaves.
2. Grow the tree by extending the most probable leaf.
3. Repeat Step 2 until the tree has Qleaves.
4. The path from the root to the leaf corresponds to the input sequence that is
mapped to the symbol attached to the leaf.
(a) Design a Tunstall code for encoding the output of a zero-memory source S={0,1}
with p(0) = 0.75, p(1) = 0.25. The number of symbols to be encoded in Q=8.
Each symbol will be represented by a three-bit number.
(b) Compute the entropy of the source H(S).
(c) Compute the ratio of the average number of output bits per input bit.
(d) As an alternative, suppose a run-length code is used to encode the input strings
