EE465 Introduction to Digital Image Processing
Spring 2009 – Solution to HW3
1. (source entropy calculation)
1) This part gives the details of Example 3 in my PPT slides (#17)- i.e.,
how to calculate the entropy for a geometric source: Prob(X=n)=1/2n,
n=1,2,……
According to the definition of source entropy, we have
To calculate the above summation of infinite series, we can write
Therefore, H(X)=2bps
2) Prob(X=2n-1)=Prob(X=2n)=1/2n+1,n=1,2,……
2. (Prefix codes)
1) No, 10 is the prefix of 1010, 1000 and 10010.
2) Solution is not unique – it depends on how you label the binary
codeword tree. If we always label left branch by 1 and right branch by
0, then we have C={11,10,011,010,0011,0010,0001,0000};
∑∑
∞
=
∞
=
=−=
1
2
12
)
2
1
(log
2
1
)(
nnn
nn
n
XH
...
2
3
2
2
2
1
232
1
+++== ∑
∞
=nn
n
S...
2
4
2
3
2
2
12 32 ++++=S(1)
(2)
2...
2
1
2
1
2
1
1)1()2( 32 =++++=⇒− S
bps
n
XH
nnn
nn3
2
1
)
2
1
(log
2
1
2)(
1
1
2
11=
+
=−= ∑∑
∞
=
+
∞
=
+
1 0
