Download Properties of Finite Energy Digital Signals: Convolution, Fourier Transform, and Filters and more Study notes Health sciences in PDF only on Docsity!
Lecture 2
Introductory signal processing ideas
In this lecture we’ll look at some basic properties of finite energy digital signals, in
other words sequences x = {x n
n
in ℓ
2 (X) for some discrete set X which almost always
will be the set Z of all integers and so we will write ℓ
2 instead of the more formal ℓ
2 (Z);
if needed, we could restrict attention to sequences in which only finitely many of the x n
are non-zero, but that’s usually not necessary from a strictly mathematical point of view.
From a practical point of view, the values of the x n
would almost certainly be real, but it’s
actually more illuminating to allow complex-valued sequences, and so that’s what we shall
do. From a mathematical point of view what will be crucial is the fact that translation
n −→ n + m is defined on Z for all integers m, but to appreciate fully the mathematical
and signal processing ideas you need to keep track of how and where the space ℓ
2
is being
used. On some occasions it is a space of finite energy signals, while on others it is a space
of coefficients of elements of an inner product space with respect to some orthonormal
family. The interplay between mathematics and signal processing ideas will be a recurring
theme in this and subsequent lectures!
(2.1) Examples, decompositions. 1. The simplest such signal is the unit impulse
δ = (... , 0 , 1 , 0 ,... ) = {δ n
}, δ n
1 , n = 0,
0 , n 6 = 0,
i.e., δ = ε
(0) to use the notation of the previous lecture. Similarly, each of the ε
(n) is a
finite energy digital signal in which only one entry is non-zero.
- Each of
ϕ
(n)
=
ε
(2n)
(2n+1)
, ϕ˜
(n)
=
ε
(2n)
− ε
(2n+1)
is a finite energy signal such that E(ϕ
(n) ) = E( ϕ˜
(n) ) = 1.
- There are also oscillatory signals: fix a real number ξ 0
and define e ξ 0
by
(e ξ 0
n
= e
2 πinξ 0
.
For no value of ξ 0
does e ξ 0
have finite energy and hence does not belong to ℓ, however.
Nonetheless,if we fix integers k, N, 0 ≤ k < N , and define e w
by
(e w
n
= w
n
, w = e
2 πik/N
.
Then w is an N
th root of unity since w
N = e
2 πik = 1 for each choice of k. Again the
digital signal ew cannot have finite energy, but it has the important property that it is
N - periodic, so it belongs to ℓ
2 (Z N
To get started let’s look at a particularly simple signal
a = {... , 0 , 5 , 4 , 7 , 9 , 0 ,... } = 5 ε
(0)
(1)
(2)
(3)
having just 4 non-zero entries; calculations show that E(a) = 171. It admits a decompo-
sition
a =
into what we might call its coarse details, obtained by averaging pairs of consecutive terms,
and its fine details, obtained by ‘differencing’ pairs of consecutive terms. There are 8 non-
zero terms in this decomposition, but each value is repeated twice, so we should be able
to express this decomposition by 4 terms. To achieve this observe that we can write
ϕ
(0)
ϕ
(1)
and
ϕ˜
(0)
−
ϕ˜
(1)
,
so that
a =
ϕ
(0)
ϕ
(1)
ϕ˜
(0)
−
ϕ˜
(1)
is a more compact way of representing the coarse + fine detail decomposition of a, more
compact because we need only 4 coefficients. But is there some ‘clever’ way of expressing
how these 4 coefficients are obtained? Well, simple calculations show that
= (a, ϕ
(0)
),
= (a, ϕ
(1)
),
while
= (a, ϕ˜
(0)
), −
= (a, ϕ˜
(1)
).
Consequently,
a = (a, ϕ
(0)
) ϕ
(0)
(1)
) ϕ
(1)
coarse details
(0)
) ϕ˜
(0)
(1)
) ϕ˜
(1)
f ine details
since E(Sx) = E(x). Consequently, the convolution h ∗ x of a finite energy signal x
will itself have finite energy provided
n
|h n
| < ∞, i.e., when the coefficients of h are
absolutely convergent; in particular, the convolution x −→ h ∗ x will map finite energy
signals to finite energy ones if only finitely many h n
Now, the convolution operator x −→ h ∗ x will have an adjoint on ℓ
2
. But what form
will this adjoint take? Well, given sequences x = {x n
n
and y = {y n
n
, we see that
(h ∗ x, y) =
n
m
hn−m xm
yn
m
n
h n−m
y n
x m
= (x, h
∗
∗ y)
where the last convolution is defined by
h
∗
∗ y =
n
h n−m
y n
, i.e., h
∗
= { h −n
n
(2.3) Discrete-Time Fourier Transform. Given a sequence x = {x n
n
, its Discrete-
Time Fourier Transform (DT F T ), x → ̂x, is defined by
̂ x(ξ) =
n
xn e
− 2 πinξ
;
in signal analysis one usually writes X(ξ) instead of ̂x and we shall often follow this
convention. The sum makes good sense if only finitely many x n
= 0. In this case x̂ (ξ) can
be interpreted as the inner product
̂ x(ξ) = (x, e ξ
) = X(ξ)
of x with the oscillatory signal e ξ
defined in the previous section except that we have to
be careful because e ξ
does not have finite energy; in addition, because each exponential
function e
− 2 πinξ
has period 1, X(ξ) has period 1.
Definition. The Discrete-Time Fourier transform X(ξ) of a finite energy signal x will be
called the Frequency Response function of x.
It is often useful to think of these period 1 functions X(ξ) as functions on [−
1
2
1
2
]. Since
{ e
2 πinξ : −∞ < n < ∞ } is orthonormal in L
2 [−
1
2
1
2
], it follows that
E(X) =
1 / 2
− 1 / 2
X(ξ)
2
dξ
1 / 2
− 1 / 2
n
x n
e
− 2 πinξ
2
dξ =
n
|x n
2
= E(x).
Consequently, x → X(ξ) is energy-preserving as a mapping from ℓ
2
into L
2
[−
1
2
1
2
]. One
property of the (DT F T ) is that
Sx(ξ) =
n
x n− 1
e
− 2 πinξ
= e
− 2 πiξ
X(ξ),
i.e., the Discrete time Fourier transform maps delay into modulation which is a signal-
processing way of talking about pointwise multiplication by the oscillatory function e
− 2 πiξ
.
But a more crucial property is the following.
Theorem. The (DT F T ) maps convolution into pointwise multiplication, more precisely,
h ∗ x(ξ) =
h(ξ) x̂ (ξ) = H(ξ) X(ξ)
for all real ξ.
Proof : by definition
h ∗ x(ξ) =
n
m
hn−m xm
e
− 2 πinξ
n
m
h n−m
x m
e
− 2 πimξ
e
− 2 πi(n−m)ξ
,
which after simplification becomes
h ∗ x(ξ) =
n
h n−m
e
− 2 πi(n−m)ξ
m
x m
e
− 2 πimξ = H(ξ) X(ξ),
completing the proof.
Corollary. The Frequency Response function of the adjoint h
∗
= {h−n}n is given by
h
∗ (ξ) = H(ξ), H(ξ) =
n
h n
e
2 πinξ
.
Proof : by definition,
h
∗ (ξ) =
n
h −n
e
− 2 πinξ
=
n
h −n
e
2 πinξ
n
h n
e
− 2 πinξ
= H(ξ),
completing the proof.
(2.5) Sampling rate changes I. Down-sampling or sub-sampling a signal x = {xn}n
produces a new signal
(↓ 2)x = {x 2 n
n
by discarding all odd-indexed terms from x and re-indexing; clearly,
E((↓ 2)x) =
n
|x 2 n
2
≤
n
|x n
2
= E(x).
Notice that (↓ 2)x = (↓ 2)y irrespective of the values of their odd-indexed terms x 2 n+1, y 2 n+1;
thus different sequences may coincide after downsampling. On the other hand, in Fourier
terms,
1
2
X(
1
2
ξ) + X(
1
2
ξ +
1
2
1
2
n
x n
e
−πinξ
n
x n
e
−πin(ξ+1)
1
2
n
x n
e
−πinξ
(1 + (−1)
n
)
n
x 2 n
e
2 πinξ
.
Consequently,
(↓ 2)x(ξ) =
1
2
X(
1
2
ξ) + X(
1
2
ξ +
1
2
Notice that each of the functions X(
1
2
ξ), X(
1
2
ξ+
1
2
) has period 2, but when we add them we
end up with a period 1 function. As an illustration, consider the signal x whose Frequency
Response function is the period 1 function
X(ξ) = | 1 − 3 |ξ||, 0 ≤ |ξ| ≤
1
2
, X(ξ + 1) = X(ξ);
its graph is
1
2
3
2
1
2
1
2
Plotting the graphs of all three of
1
2
X(
1
2
ξ),
1
2
X(
1
2
ξ +
1
2
(↓ 2)x(ξ) =
1
2
X(
1
2
ξ) +
1
2
X(
1
2
ξ +
1
2
on the same axes we thus obtain
-4 -3 -2 -1 0 1 2 3 4 5 6 7
1
2
X(
1
2
ξ)
1
2
X(
1
2
ξ +
1
2
(↓ 2)x(ξ)
1
2
3
2
1
2
The figure makes clear that the graph of X(
1
2
ξ +
1
2
) is simply the graph of X(
1
2
ξ) shifted
in frequency by 1; signal processing language calls X(
1
2
ξ +
1
2
) an alias of X(
1
2
ξ).
(2.6) Sampling rate changes II. Up-sampling is the converse of down-sampling. Given
a signal y = {y n
n
, up-sampling produces a new signal
(↑ 2)y = {v n
v 2 n
= y n
v 2 n+
by inserting zeros between consecutive terms of y and relabelling. In mathematical terms,
up-sampling is the adjoint of down-sampling: indeed, on sequences x and y,
((↓ 2)x, y) =
n
x 2 n
y n
= (x, (↑ 2)y), x, y ∈ ℓ
2
.
More is true, in fact: since
E((↑ 2)y) =
n
|v 2 n
2
2
n
|y n
2
= E(y),
it is clearly energy-preserving. On the other hand, in terms of the (DT F S),
(↑ 2)y(ξ) =
n
y n
e
− 2 πi 2 nξ
= Y (2ξ),
where, as before,
H(ξ) =
ℓ
h ℓ
e
− 2 πiℓξ
.
is the Frequency Response function of the filter coefficients of H. But then, in view of
representation (††),
H(x) =
n
1 / 2
− 1 / 2
H(ξ) X(ξ) e
2 πinξ
dξ
ε
(n)
;
in other words, the action of H is to select or reject frequencies in a signal. Filters come
in many ‘flavors’ depending on how these frequencies are selected or rejected:
(a) ideal: H(ξ) takes only the values 0, 1;
(b) low-pass: |ξ| > a =⇒ H(ξ) = 0 for some 0 < a <
1
2
(c) high-pass: |ξ| < a =⇒ H(ξ) = 0 for some 0 < a <
1
2
Typical, but unrealistic, examples are
-5 -4 -3 -2 -1 0 1 2 3 4 5
0
1
2
3
4
1
2
1
2
high-pass
low-pass
Thus a low-pass filter keeps only ‘low’ frequencies in some band about the origin, while a
high-pass filter keeps only ‘high’ frequencies in some band not including the origin.
No FIR filters can be low or high pass in this sense defined above, however. Indeed, if
H is a FIR filter with filter coefficients h 0
, h 1
,... , h L− 1
for some L, say, then
H(z) = h 0
h 1
z
h L− 1
z
L− 1
P (z)
z
L− 1
for some polynomial P. So H(ξ) has at most finitely many zeros, hence cannot vanish
on any interval. The best that H(ξ) can do is vanish at a point, so we shall adopt the
following definition.
Definition. An FIR filter H will be said to ‘try hard’ to be a low-pass filter when H(0) 6 = 0
and H(±
1
2
) = 0; by contrast, H will be said to ‘try hard’ to be a high-pass filter when
H(0) = 0 and H(±
1
2
A key step in Daubechies construction of wavelets is to associate a ‘high-pass’ FIR filter
H to any ‘low-pass’ FIR filter. For suppose H has filter coefficients {h ℓ
ℓ
and Frequency
Response function
H(ξ) =
n
h n
e
− 2 πinξ
, H(0) 6 = 0 , H(±
1
2
Now define
H to be the FIR filter having filter coefficients {
h n
n
h n
n
h 1 −n
One of the problems for this lecture asks you to show that the corresponding Frequency
Response function
H(ξ) is given by
H(ξ) =
n
n
h 1 −n e
− 2 πinξ
= −e
− 2 πiξ
H(ξ +
1
2
in particular, therefore,
H(0) 6 = 0 =⇒
H(±
1
2
) 6 = 0 , H(±
1
2
H(0) = 0,
so
H tries hard to be a high-pass filter when H tries hard to be a low-pass filter.
Examples. Let’s look at three examples to make this clearer. Notice that in these exam-
ples we are allowing the number of filter coefficients to increase from 2 to 3, and then to
- There must be some pattern to this!!
- The Haar filters: h = {h n
n
and
h = {
h n
n
are causal FIR filters defined by
h n
1 √
2
, n = 0,
1 √
2
, n = 1,
0 , n 6 = 0, 1 ,
h n
1 √
2
, n = 0,
1 √
2
, n = 1,
0 , n 6 = 0, 1.
(I realize I’m using the same notation for different things - you have to work out the
precise meaning from context!! Shortly everything will settle down and h will mean just
one thing.) Simple calculations show that their respective Frequency Response functions
H(ξ) and
H(ξ) are given by
H(ξ) =
2 e
−πiξ
cos πξ,
H(ξ) =
2 i e
−πiξ
sin πξ.
Its frequency response function
G(ξ) =
2 e
− 2 πiξ
(cos πξ)
2
now has a double zero at ξ = ±
1
2
, and the associated filter
G,
˜gn =
1
2
√
2
, n = − 1 , 1 ,
1 √
2
, n = 0,
0 , otherwise,
has Frequency response function
G(ξ) =
2 (sin πξ)
2
.
Clearly
G is trying hard to be a high-pass filter having a double zero at the origin.
- Daubechies db2-filter: even at this early stage it’s impossible to resist introducing
the famous Daubechies-db2 filter coefficients - we do resist for the moment saying where
they come from, however! There are 4 coefficients and they are defined by
h 0 =
, h 1 =
, h 2 =
, h 3 =
with corresponding Frequency response function
H(ξ) =
2 e
− 3 πiξ
(cos πξ)
2
(cos πξ + i
3 sin πξ).
(To check this expression for H(ξ) it’s probably easiest to start with the given expression
and then use trig identities to write H(ξ) as a finite sum
3
n = 0
h n
e
− 2 πinξ .) Clearly db 2
tries hard to be a low-pass filter. Notice that the presence of the (cos πξ)
2
-term ensures
that H(ξ) has a zero of order 2 at ξ = ±
1
2
. This is the reason why it is often referred to
as the db2-filter; the Haar could well be called the db1-filter. Others refer to the db2-filter
as the D4-filter because it has 4 coefficients. We have followed Matlab in the choice of
notation because you will be making good use of its Wavelet toolbox!
(2.8) Filtering together with up/down sampling. Finally, let’s put these crucial
operations of filtering and up/down sampling together. Given an FIR filter H, consider
the operators
(‡) ↓ 2 ◦ H
∗
: x −→
ℓ
x ℓ
h ℓ− 2 n
n
, H ◦ ↑ 2 : x −→
ℓ
x ℓ
h n− 2 ℓ
n
on a sequence {x n
n
. What’s perhaps not clear is why we use ↓ 2 ◦ H
∗ instead of ↓ 2 ◦ H;
to see why, note that
(
H◦ ↑ 2
∗
∗
◦H
∗
= ↓ 2 ◦ H
∗
;
thus the first operator is just the adjoint of the second. More crucial perhaps, is the
question of just what the effect of these operators is on a signal. The S ◦ S
∗ example from
chapter 1 will provide the answer. In terms of Frequency Response functions
((↓ 2 ◦ H
∗
) x)
b
(ξ) =
1
2
H(
1
2
ξ) X(
1
2
ξ) + H(
1
2
ξ +
1
2
) X(
1
2
ξ +
1
2
while
((H ◦ ↑ 2) x)
b (ξ) = H(ξ) X(2ξ).
These results, though useful later, shed little light on the coarse+fine decomposition we
started out with. Patience! Let’s look first at the effect of these operators on sequences
themselves.
Example. Consider the case of the Haar filters
hn =
1 √
2
, n = 0,
1 √
2
, n = 1,
0 , n 6 = 0, 1 ,
hn =
1 √
2
, n = 0,
1 √
2
, n = 1,
0 , n 6 = 0, 1.
Now fix sequences a = {a n
n
, c = {c n
n
and recall the sequences {ϕ
(n) } n
, { ϕ˜
(n) } n
ϕ
(n)
=
ε
(2n)
(2n+1)
), ϕ˜
(n)
=
ε
(2n)
− ε
(2n+1)
),
defined in lecture 1 as well as being discussed earlier in this lecture. Since the filter coeffi-
cients are real, we can omit complex conjugates in (‡) and compute using the convolution
formulas in (‡):
(1) ↓ 2 ◦ H
∗
: (averaging operator)
(↓ 2 ◦ H
∗
)a =
n
a 2 n
ε
(n)
=
(a, ϕ
(n)
)
n
H
∗ : (difference operator)
H
∗
)a =
n
a 2 n
− a 2 n+
ε
(n)
=
(a, ϕ˜
(n)
)
n
(3) H ◦ ↑ 2: (spreading operator)
ℓ
c ℓ
h n− 2 ℓ
1 √
2
c m
, n = 2m,
1 √
2
c m
, n = 2m + 1,
(H ◦ ↑ 2)c =
n
c n
ϕ
(n)
;
Problems.
- The simplest FIR filter is the Lazy Filter
I : x −→ I(x) = x,
so-called because it does nothing to a signal; its filter coefficents {i n
n
are given by
i n
1 , n = 0,
0 , n 6 = 0.
In other words, I(x) is simply the convolution
I(x) = δ ∗ x = ε
(0)
∗ x
of x with the unit impulse δ. Notice that I ‘fills in’ the missing first example in the series
of filters given in (2.7) because it has only one non-zero filter coefficient. There really must
be a pattern to these filters (check problem 10(iv) also)!!
(i) Show that the Frequency Response function I(ξ) of I is given by I(ξ) = 1 for all ξ.
Why does this make good sense? What is the adjoint I
∗
of I? Does I succeeding
in trying hard to be low-pass or highpass?
(ii) Determine the associated FIR filter
I. What is its Frequency Response function
I(ξ)? What is its adjoint
I
∗ ?
(iii) Determine
(↓ 2 ◦ I
∗
)x, (I ◦ ↑ 2)x
for a sequence x.
(iv) Determine
I
∗
)x, (
I ◦ ↑ 2)x
for a sequence x.
(v) Determine the coarse + fine detail decomposition
(I ◦ ↑ 2) ◦ (↓ 2 ◦ I
∗
) + (
I ◦ ↑ 2) ◦ (↓ 2 ◦
I
∗
)
x
of a sequence x.
- Show that the adjoint, S
∗
, of the delay operator S is the advancing operator
S
∗
: {x n
n
−→ {x n+
n
translating the sequence in the opposite direction to S.
- Let x = {x n
n
be the sequence whose Frequency Response function is the period 1
extension of the function
X(ξ) = | 1 − 3 |ξ||, 0 ≤ |ξ| ≤
1
2
By using representation (††) in (2.3), find the sequence {x n
n
. (Hint: get rid of the outer
absolute value by using the fact that X is even, i.e., X(−ξ) = X(ξ); then get rid of the
inner absolute value by splitting the new integral into two parts.)
- As a further attempt to explain the appearance of the adjoints H
∗ and
H
∗ in the
coarse+fine decomposition of signals, show that
(H ◦ ↑ 2)
∗
= ↓ 2 ◦ H
∗
, (
H ◦ ↑ 2)
∗
= ↓ 2 ◦
H
∗
for any FIR filter H. In particular, therefore,
(H ◦ ↑ 2) ◦ (↓ 2 ◦ H
∗
)x = (H ◦ ↑ 2) ◦ (H ◦ ↑ 2)
∗
x
and
H ◦ ↑ 2) ◦ (↓ 2 ◦
H
∗
)x = (
H ◦ ↑ 2) ◦ (
H ◦ ↑ 2)
∗
x,
which are exactly the same as the synthesis/analysis mappings
S[f ] = (S ◦ S
∗
)f
associated in lecture 1 with an orthonormal family {φ n
n
in a general inner product space.
- Use problem 4 with H and
H the Haar filters to establish the results
(H ◦ ↑ 2) ◦ (↓ 2 ◦ H
∗
)a =
n
(a, ϕ
(n)
) ϕ
(n)
and
H ◦ ↑ 2) ◦ (↓ 2 ◦
H
∗
)a =
n
(a, ϕ˜
(n)
) ϕ˜
(n)
for the respective orthonormal families {ϕ
(n) } n
, { ϕ˜
(n) } n
stated in (2.8).
- Use some graphing facility to draw the graph of |H(ξ)| and |
H(ξ)| for the Daubechies
db2-filter.
- Show that the mapping H ◦ ↑ 2 is energy-preserving on ℓ
2
for a given FIR filter H
if and only if
(‡) |H(ξ)|
2
1
2
2
= 2.
(i) Deduce that if H ◦ ↑ 2 is energy-preserving on ℓ
2
, then so is
H ◦ ↑ 2.
(ii) Deduce that H ◦ ↑ 2 is energy-preserving on ℓ
2 when H is the Haar filter and the
db2-filter.
(iii) Does your proof of (ii) for the Haar and db2-filters suggest how one might construct
other filters H for which condition (‡) holds?
(iv) Use parts (i) and (ii) to show that the families {ϕ
(n) } n
and { ϕ˜
(n) } n
are orthonor-
mal in ℓ
2
, results established directly in lecture 1. Then use parts (i) and (ii) to
construct new orthonormal families in ℓ
2 from the db2 and
db2-filters.
