ECE563 Fall 2004 Homework 11
Date Assigned: 3 November 2004.
Date Due: 10 November 2004 in class.
Suggested Reading: Nothing in particular; this exercise tests your understanding of the
basic discrete memoryless channel setting. So, it might help to really understand the forward
and converse proofs of the main capacity theorem (Sections 8.7 and 8.9 of your text). You
can learn more about the geometric proof of the capacity of the AWGN channel, as discussed
in lecture, from Appendix B.5 in [2].
1. Sum Channel
Consider Kdiscrete memoryless channels with respective input and output alphabets
Xkand Ykand with respective information capacity Ck,1≤k≤K. Suppose that
both the input and output alphabets of the different channels are disjoint. The sum
channel associated to these channels is the channel whose input alphabet is the union
X:=
K
[
k=1
Xk,
of the individual input alphabets, the output alphabet is the union
Y:=
K
[
k=1
Yk,
of the respective output alphabets, and such that if input letter x∈ Xkis used the
output is y∈ Ykwith the channel transition probabilities corresponding to channel
k. In other words, at each time the transmitter may choose to use any one of the
Kchannels and transmit any symbol from the input alphabet of this channel - if the
transmitter chooses a letter from channel kthe receiver will see a symbol from the
output alphabet of channel kat that time.
(a) Show that the capacity of the sum channel is given by
C= log K
X
k=1
2Ck!,(1)
and find an expression for the optimal input probability distribution for the sum
channel in terms of the optimal input probability distributions for the individual
channels.
(b) Interpret the expression for Cin (1) as the average of the capacities of the indi-
vidual channels plus the information conveyed by the selection of the channel.
1
