Download Communication and Information Theory - Practice Problems with Solutions | ECE 460 and more Study notes Theories of Communication in PDF only on Docsity!
ECE 460: Communication and Information Theory
Prof. B.-P. Paris
Practice Problems
Solution
Below are solutions to the following problems:
Chapter 2 Problems: 9, 34, 38.
Chapter 3 Problems: 19, 20, 30.
Chapter 7 Problems: 17, 24, 25, 30, 33
- Problem 2.
- Since (a − b) 2 ≥ 0 we have that
ab ≤
a
2
b
2
with equality if a = b. Let
A =
[
∑n
i=
α
2 i
] 1
2
, B =
[
∑n
i=
β
2 i
] 1
2
Then substituting αi/A for a and βi/B for b in the previous inequality we
obtain αi
A
βi
B
α 2 i
A 2
β 2 i
B 2
with equality if
αi βi
A B
= k or αi = kβi for all i. Summing both sides
from i = 1 to n we obtain
n ∑
i=
αiβi
AB
n ∑
i=
α
2 i
A 2
n ∑
i=
β
2 i
B 2
2 A
2
n ∑
i=
α
2 i +^
2 B
2
n ∑
i=
β
2 i =^
2 A
2
A
2
2 B
2
B
2 = 1
Thus,
AB
n ∑
i=
αiβi ≤ 1 ⇒
n ∑
i=
αiβi ≤
[
n ∑
i=
α
2 i
] 1
2
[
n ∑
i=
β
2 i
] 1
2
Equality holds if αi = kβi, for i = 1,... , n.
- The second equation is trivial since |xiy ∗ i | = |xi||y ∗ i |. To see this write
xi and yi in polar coordinates as xi = ρx i e
jθx i (^) and y i =^ ρyi e
jθy i (^). Then,
|xiy
∗ i | = |ρx i ρy i e
j(θx i −θy i ) | = ρx i ρy i = |xi||yi| = |xi||y
∗ i |. We turn now to
prove the first inequality. Let zi be any complex with real and imaginary
components zi,R and zi,I respectively. Then,
n ∑
i=
zi
2
n ∑
i=
zi,R + j
n ∑
i=
zi,I
2
n ∑
i=
zi,R
n ∑
i=
zi,I
n ∑
i=
n ∑
m=
(zi,Rzm,R + zi,I zm,I )
Since (zi,Rzm,I − zm,Rzi,I ) 2 ≥ 0 we obtain
(zi,Rzm,R + zi,I zm,I )
2 ≤ (z
2 i,R +^ z
2 i,I )(z
2 m,R +^ z
2 m,I )
Using this inequality in the previous equation we get
n ∑
i=
zi
2
n ∑
i=
n ∑
m=
(zi,Rzm,R + zi,I zm,I )
n ∑
i=
n ∑
m=
(z
2 i,R
2 i,I
1 (^2) (z 2 m,R
2 m,I
1 2
n ∑
i=
(z
2 i,R +^ z
2 i,I )^
1 2
n ∑
m=
(z
2 m,R +^ z
2 m,I )^
1 2
n ∑
i=
(z
2 i,R +^ z
2 i,I )^
1 2
Thus
∣ ∣ ∣ ∣ ∣
n ∑
i=
zi
2
n ∑
i=
(z
2 i,R +^ z
2 i,I )^
1 2
or
n ∑
i=
zi
n ∑
i=
|zi|
The inequality now follows if we substitute zi = xiy ∗ i
. Equality is obtained
if
zi,R zi,I
zm,R zm,I
= k 1 or 6 zi = 6 zm = θ.
- From 2) we obtain
n ∑
i=
xiy
∗ i
2
n ∑
i=
|xi||yi|
But |xi|, |yi| are real positive numbers so from 1)
n ∑
i=
|xi||yi| ≤
[
n ∑
i=
|xi|
2
] 1
2
[
n ∑
i=
|yi|
2
] 1
2
Combining the two inequalities we get
n ∑
i=
xiy
∗ i
2
[
n ∑
i=
|xi|
2
] 1
2
[
n ∑
i=
|yi|
2
] 1
2
Squaring the previous inequality and integrating from −∞ to ∞ we obtain
−∞
|y(t)|
2 dt ≤ Eh
−∞
−∞
|x(t − τ )|
2 dτ dt
But by assumption
∞ −∞
∞ −∞ |x(t − τ )|
2 dτ dt, Eh are finite, so that the
energy of the output signal is finite.
Consider the LTI system with impulse response h(t) =
n=−∞ Π(t − 2 n).
The signal is periodic with period T = 2, and the power content of the
signal is PH =
1 2
. If the input to this system is the energy type signal
x(t) = Π(t), then
y(t) =
∞ ∑
n=−∞
Λ(t − 2 n)
which is a power type signal with power content PY =
1 2
- Problem 2.
x 1 (t) =
∞ ∑
n=−∞
n x(nTs)δ(t − nTs) = x(t)
∞ ∑
n=−∞
n δ(t − nTs)
= x(t)
[
∞ ∑
l=−∞
δ(t − 2 lTs) −
∞ ∑
l=−∞
δ(t − Ts − 2 lTs)
]
Thus
X 1 (f ) = X(f )?
[
2 Ts
∞ ∑
l=−∞
δ(f −
l
2 Ts
2 Ts
∞ ∑
l=−∞
δ(f −
l
2 Ts
)e
−j 2 πf Ts
]
2 Ts
∞ ∑
l=−∞
X(f −
l
2 Ts
2 Ts
∞ ∑
l=−∞
X(f −
l
2 Ts
)e
−j 2 π l 2 Ts Ts
2 Ts
∞ ∑
l=−∞
X(f −
l
2 Ts
2 Ts
∞ ∑
l=−∞
X(f −
l
2 Ts
l
Ts
∞ ∑
l=−∞
X(f −
2 Ts
l
Ts
- The spectrum of x(t) occupies the frequency band [−W, W ]. Suppose
that from the periodic spectrum X 1 (f ) we isolate Xk(f ) =
1 Ts X(f −
1 2 Ts
k Ts ), with a bandpass filter, and we use it to reconstruct x(t). Since Xk(f )
occupies the frequency band [2kW, 2(k+1)W ], then for all k, Xk(f ) cannot
cover the whole interval [−W, W ]. Thus at the output of the reconstruction
filter there will exist frequency components which are not present in the
input spectrum. Hence, the reconstruction filter has to be a time-varying
filter. To see this in the time domain, note that the original spectrum
has been shifted by f
′
1 2 Ts
. In order to bring the spectrum back to the
origin and reconstruct x(t) the sampled signal x 1 (t) has to be multiplied
by e
−j 2 π 1 2 Ts t^ = e−j^2 πW t. However the system described by
y(t) = e
j 2 πW t x(t)
is a time-varying system.
- Using a time-varying system we can reconstruct x(t) as follows. Use the
bandpass filter TsΠ(
f −W 2 W
) to extract the component X(f −
1 2 Ts
). Invert
X(f −
1 2 Ts ) and multiply the resultant signal by e
−j 2 πW t
. Thus
x(t) = e
−j 2 πW t F
− 1
[
TsΠ(
f − W
2 W
)X 1 (f )
]
- Problem 3.
y(t) = ax(t) + bx
2 (t)
= a(m(t) + cos(2πf 0 t)) + b(m(t) + cos(2πf 0 t))
2
= am(t) + bm
2 (t) + a cos(2πf 0 t)
+b cos
2 (2πf 0 t) + 2bm(t) cos(2πf 0 t)
- The filter should reject the low frequency components, the terms of
double frequency and pass only the signal with spectrum centered at f 0.
Thus the filter should be a BPF with center frequency f 0 and bandwidth
W such that f 0 − WM > f 0 −
W 2
> 2 WM where WM is the bandwidth of
the message signal m(t).
- The AM output signal can be written as
u(t) = a(1 +
2 b
a
m(t)) cos(2πf 0 t)
Since Am = max[|m(t)|] we conclude that the modulation index is
α =
2 bAm
a
- Problem 3.
- When USSB is employed the bandwidth of the modulated signal is the
same with the bandwidth of the message signal. Hence,
WUSSB = W = 10
4 Hz
At time t = T we have
y(T ) =
∫ T
0
si(τ )dτ +
∫ T
0
n(τ )dτ = ±
Eb
T
T +
∫ T
0
n(τ )dτ
The signal energy at the output of the integrator at t = T is
Es =
Eb
T
T
= EbT
whereas the noise power
Pn = E
[
T
0
T
0
n(τ )n(v)dτ dv
]
∫ T
0
∫ T
0
E[n(τ )n(v)]dτ dv
N 0
∫ T
0
∫ T
0
δ(τ − v)dτ dv =
N 0
T
Hence, the output SNR is
SNR =
Es
Pn
2 Eb
N 0
- The transfer function of the RC filter is
H(f ) =
1 + j 2 πRCf
Thus, the impulse response of the filter is
h(t) =
RC
e
− t RC (^) u − 1 (t)
and the output signal is given by
y(t) =
RC
∫ (^) t
−∞
r(τ )e
− t−τ RC (^) dτ
RC
∫ (^) t
−∞
(si(τ ) + n(τ ))e
− t−τ RC (^) dτ
RC
e
− t RC
t
0
si(τ )e
τ RC (^) dτ +
RC
e
− t RC
t
−∞
n(τ )e
τ RC (^) dτ
At time t = T we obtain
y(T ) =
RC
e
− T RC
T
0
si(τ )e
τ RC (^) dτ +
RC
e
− T RC
T
−∞
n(τ )e
τ RC (^) dτ
The signal energy at the output of the filter is
Es =
(RC)
2 e
− 2 T RC
∫ T
0
∫ T
0
si(τ )si(v)e
τ RC (^) e
v RC (^) dτ dv
(RC)
2
e
− 2 T RC
Eb
T
∫ T
0
e
τ RC (^) dτ
= e
− 2 T RC
Eb
T
e
T RC (^) − 1
Eb
T
1 − e
− T RC
The noise power at the output of the filter is
Pn =
(RC)
2
e
− 2 T RC
∫ T
−∞
∫ T
−∞
E[n(τ )n(v)]dτ dv
(RC)
2
e
− 2 T RC
T
−∞
T
−∞
N 0
δ(τ − v)e
τ +v RC (^) dτ dv
(RC)^2
e
− 2 T RC
T
−∞
N 0
e
2 τ RC (^) dτ
2 RC
e
− 2 T RC
N 0
e
2 T RC (^) =
2 RC
N 0
Hence,
SNR =
Es
Pn
4 EbRC
T N 0
1 − e
− T RC
- The value of RC that maximizes SNR, can be found by setting the
partial derivative of SNR with respect to RC equal to zero. Thus, if
a = RC, then
ϑSNR
ϑa
= 0 = (1 − e
− T a (^) ) −
T
a
e
− T a (^) = −e − T a
T
a
Solving this transcendental equation numerically for a, we obtain
T
a
= 1.26 =⇒ RC = a =
T
- Problem 7.
The energy of the two signals s 1 (t) and s 2 (t) is
Eb = A
2 T
If s 1 (t) is transmitted, then
- 5
0
r(t)dt =
- 5
0
[s 1 (t) + n(t)]dt = 1 +
- 5
0
n(t)dt
= 1 + n 1 ∫ (^2)
1
r(t)dt =
1
[s 1 (t) + n(t)]dt =
1
n(t)dt
= n 2
where n 1 is a zero-mean Gaussian random variable with variance
σ
2 n 1
= E
[∫
- 5
0
0
n(τ )n(v)dτ dv
]
N 0
0
dτ = 1. 5
and n 2 is is a zero-mean Gaussian random variable with variance
σ
2 n 2 =^ E
[∫
2
1
2
1
n(τ )n(v)dτ dv
]
N 0
2
1
dτ = 1
Thus, the vector representation of the received signal (at the output of
the integrators) is
r = [1 + n 1 , n 2 ]
Similarly we find that if s 2 (t) is transmitted, then
r = [0.5 + n 1 , 1 + n 2 ]
Suppose now that the detector bases its decisions on the rule
r 1 − r 2
s 1
s 2
T
The probability of error P (e|s 1 ) is obtained as
P (e|s 1 ) = P (r 1 − r 2 < T |s 1 )
= P (1 + n 1 − n 2 < T ) = P (n 1 − n 2 < T − 1)
= P (n < T )
where the random variable n = n 1 − n 2 is zero-mean Gaussian with vari-
ance
σ
2 n = σ
2 n 1
2 n 2 − 2 E[n 1 n 2 ]
= σ
2 n 1
2 n 2
1
N 0
dτ
= 1 .5 + 1 − 2 × 0 .5 = 1. 5
Hence,
P (e|s 1 ) =
2 πσ 2 n
∫ T − 1
−∞
e
− x^2 2 σ^2 n dx
Similarly we find that
P (e|s 2 ) = P (0.5 + n 1 − 1 − n 2 > T )
= P (n 1 − n 2 > T + 0.5)
2 πσ 2 n
∞
T +0. 5
e
− x^2 2 σ^2 n dx
The average probability of error is
P (e) =
P (e|s 1 ) +
P (e|s 2 )
2 πσ 2 n
T − 1
−∞
e
− x^2 2 σ^2 n dx +
2 πσ 2 n
∞
T +0. 5
e
− x^2 2 σ^2 n dx
To find the value of T that minimizes the probability of error, we set the
derivative of P (e) with respect to T equal to zero. Using the Leibnitz rule
for the differentiation of definite integrals, we obtain
ϑP (e)
ϑT
2 πσ^2 n
[
e
− (T −1)^2 2 σ^2 n − e
− (T +0.5)^2 2 σ n^2
]
or
(T − 1)
2 = (T + 0.5)
2 =⇒ T = 0. 25
Thus, the optimal decision rule is
r 1 − r 2
s 1
> <
s 2
- Problem 7.
- The dimensionality of the signal space is two. An orthonormal basis
set for the signal space is formed by the signals
ψ 1 (t) =
2 T
, 0 ≤ t <
T 2
0 , otherwise
ψ 2 (t) =
2 T
T 2
≤ t < T
0 , otherwise
- The optimal receiver is shown in the next figure
The output of the sampler at t =
T 2 and t = T is given by
r 1 = A
T
T
3 A
T
T
A
2 T
r 2 =
A
T
T
A
2 T
If the optimal receiver uses a threshold V to base its decisions, that is
r 1 − r 2
s 1
> <
s 2
V
then the probability of error P (e|s 1 ) is
P (e|s 1 ) = P (n 2 − n 1 > 2
A
2 T
− V ) = Q
A
2 T
8 N 0
V
N 0
If s 2 (t) is transmitted, then
r(t) = s 2 (t) +
s 2 (t −
T
) + n(t)
The output of the sampler at t =
T 2 and t = T is given by
r 1 = n 1
r 2 = A
T
T
3 A
T
T
A
2 T
The probability of error P (e|s 2 ) is
P (e|s 2 ) = P (n 1 − n 2 >
A
2 T
+ V ) = Q
A
2 T
8 N 0
V
N 0
Thus, the average probability of error is given by
P (e) =
P (e|s 1 ) +
P (e|s 2 )
Q
A
2 T
8 N 0
V
N 0
Q
A
2 T
8 N 0
V
N 0
The optimal value of V can be found by setting
ϑP (e) ϑV equal to zero. Using
Leibnitz rule to differentiate definite integrals, we obtain
ϑP (e)
ϑV
A^2 T
8 N 0
V
N 0
2
A^2 T
8 N 0
V
N 0
2
or by solving in terms of V
V = −
A
2 T
- Let a be fixed to some value between 0 and 1. Then, if we argue as in
part 5) we obtain
P (e|s 1 , a) = P (n 2 − n 1 > 2
A
2 T
− V (a))
P (e|s 2 , a) = P (n 1 − n 2 > (a + 2)
A
2 T
and the probability of error is
P (e|a) =
P (e|s 1 , a) +
P (e|s 2 , a)
For a given a, the optimal value of V (a) is found by setting
ϑP (e|a) ϑV (a)
equal
to zero. By doing so we find that
V (a) = −
a
A
2 T
The mean square estimation of V (a) is
V =
0
V (a)f (a)da = −
A^2 T
0
ada = −
A^2 T
