Download Fast Fourier Transform - Algorithm and Complexity Analysis - Lecture Slides and more Slides Computer Science in PDF only on Docsity!
Fast Fourier Transform
Jean Baptiste Joseph Fourier (1768-1830)
Fast Fourier Transform
Applications.^ n
Perhaps single algorithmic discovery that has had the greatestpractical impact in history. n^
Optics, acoustics, quantum physics, telecommunications, systemstheory, signal processing, speech recognition, data compression. n^
Progress in these areas limited by lack of fast algorithms. History.^ n
Cooley-Tukey (1965) revolutionized all of these areas. n^
Danielson-Lanczos (1942) efficient algorithm. n^
Runge-
König (1924) laid theoretical groundwork.
n^
Gauss (1805, 1866) describes similar algorithm. n^
Importance not realized until advent of digital computers.
3
Polynomials: Coefficient Representation
Degree n polynomial.Addition: O(n) ops.Evaluation: O(n) using Horner’s method.Multiplication (convolution): O(n
(^
n
n^
x a x a x a a x p
v
(^
n
n^
x b x b x b b x q
v
1 1 1 1 1 0 0
(^
−
−
−^
n
n
n^
x b a x b a b a x q x
p^
v
(^
1
2
2
1
0
v
v
−
−^
n
n^
a x a x a x a x a x p
2 2 1 1 0 0 1 1 0 0 0
(^
−
−
−
=^
−^
+ + ∑ + + + + = ×
n
n
n
j k
j
k j k^
x b a x b a x b a b a b a x q x
p^
v
v
Polynomials: Point-Value Representation
Degree n polynomial.^ n
Uniquely specified by knowing p(x) at n different values of x.
{
}^
where
(x,
(x
(x
1
(^1) - n
1 1
0 0
k
k
n^
x
p
y
y
y
y^
−
u
x
y = p(x)
x
j
y
j
Docsity.com
5
Polynomials: Point-Value Representation
Degree n polynomial.Addition: O(n).Multiplication: O(n), but need 2n points.Evaluation: O(n
2 ) using Lagrange’s formula.
{
}^
where
(x,
(x
(x
1
(^1) - n
1 1
0 0
k
k
n^
x
p
y
y
y
y^
−
u
{
} )
(x
(x
(x
1
1
(^1) - n 1 1 1 0 0 0
−
−^
n
n^
z y z y z y
u
∑
∏ ∏
− =^
≠ ≠
=^
1 0
n k^
k j
j
k k j
j
k^
x
x
x x y x p {
}
(x
(x
(x
1 2 1 2
(^1) - 2n
1 1 1 0 0 0
−
−^
n
n^
z y z y z y
u
{
}
z
where
(x
(x
(x
1
(^1) - n
1 1
0 0
k
k
n^
x
q
z
z
z^
−
u
Best of Both Worlds
Can we get "fast" multiplication and evaluation?
coefficient Yes! Convert back and forth between two representations. Representation
O(n
Multiplication
O(n) Evaluation
point-value
O(n)
O(n
FFT
O(n log n)
O(n log n)
(^1) - n
1 0
(^1) - n
1 0
b, b
a,
, a, a
b KK
(^2) - 2n
1 0
c
c
c
K
q(x
p(x
(^1) - 2n
1
0
(^1) - 2n
1
0
x q
x q
x p
x p
K K
r(x
(^1) - 2n
1
0
x r
x r^
K
O(n)
point-value multiplication
O(n
2 )
coefficient multiplication
O(n log n)
evaluation
FFT
interpolationinverse FFT
O(n log n)
7
Converting Between Representations: Naïve Solution
Evaluation (coefficient to point-value).^ n
Given a polynomial p(x) =a
0
x 2
1
+... + a
n-
x
n-
, choose n
distinct points {x
, x 0
,... , x 1
n-
} and compute y
k^
= p(x
), for each kk
using Horner’s method. n^
O(n
Interpolation (point-value to coefficient).^ n
Given n distinct points {x
, x 0
,... , x 1
n-
} and y
k^
= p(x
), computek
the coefficients {a
, a 0
,... , a 1
n-
} by solving the following linear
system of equations. n^
Note Vandermonde matrix is invertible iff x
k^
are distinct.
n^
O(n
−
−
−−
−
−
− − −
1 0 1 2
1 0 1 2
11
2
1
1
1 2
(^22)
2
1 1
(^21)
1
1 0
(^20)
0
n
n
nn
n
n
n n n
y y y y
a a a a
x
x
x
x
x
x
x
x
x
x
x
x
M
M
L
M
O
M
M
M
LLL
Fast Interpolation: Key Idea
Key idea: choose {x
, x 0
,... , x 1
n-
} to make computation easier!
n^
Set x
k^
= x
?j
−
−
−
−
−
− − −
1 0 1 2
0 1 2 1
1 0
2
1
1
1 2
22
2
1 1
21
1
1 0
20
0
n
n
n
n
n
n n n
y y y y
a a a a
x
x
x
x
x
x
x
x
x
x
x
x
M
M
L
M
O
M
M
M
LLL
Docsity.com
13
Divide-and-Conquer
Given degree n polynomial p(x) = a
0
x 1
1
2
x
2
+... + a
n-
x
n-
n^
Assume n is a power of 2, and let
ω
be the principal n
th
root of unity.
n^
Define even and odd polynomials:^ –
p
even
(x) := a
0
x 2
1
x 4
2
x 6 3
+... + a
n-
x
n/2 - 1
-^
p
odd
(x) := a
1
x 3
1
x 5
2
x 7 3
+... + a
n-
x
n/2 - 1
-^
p(x) = p
even
(x
2 ) + x p
odd
(x
n^
Reduces problem of^ –
evaluating degree n polynomial p(x) at
ω
ω
ω
n-
to^ –
evaluating two degree n/2 polynomials at: (
ω
ω
ω
n-
n^
L
p
even
(x) and p
odd
(x) only evaluated at n/2 complex (n/2)
th
roots
of unity.
FFT Algorithm
if (n == 1)
n
is a
power
of
return a
0
ω ←
e
2 π
i / n
(e
,e 0
,e 1
,...,e 2
n/2-
FFT(n/2,
a
,a 0
,a 2
,...,a 4
n-
(d
,d 0
,d 1
,...,d 2
n/2-
FFT(n/2,
a
,a 1
,a 3
,...,a 5
n-
for k = 0 to n/2 -
y
k^
e
k^
ω
k^
d
k
y
k+n/
e
k^
ω
k^
d
k
return (y
,y 0
,y 1
,...,y 2
n-
FFT (n, a
, a 0
, a 1
,... , a 2
n-
O(n) complex multipliesif we pre-compute
ω
k. )
log ( ) ( ) ( )
n n O n T n O n T n T
15
Recursion Treea
, a 0
, a 1
, a 2
, a 3
, a 4
, a 5
, a 6
7 a
, a 1
, a 3
, a 5
7
a
, a 0
, a 2
, a 4
6
a
, a 3
7
a
, a 1
5
a
, a 0
4
a^2
, a
6
a
0
a
4
a
2
a
6
a
1
a
5
a
3
a
7
"bit-reversed" order
Proof of Correctness
Proof of correctness. Need to show y
k^
= p(
ω
k^
for each k = 0,... , n-1, where
ω
is the principal
n
th
root of unity. n^
Base case. n = 1
⇒ ω
= 1. Algorithm returns y
0
= a
0
= a
ω 0
0
n^
Induction step. Assume algorithm correct for n / 2.^ –
let
ν
be the principal (n/2)
th
root of unity
-^
e
k^
= p
even
(ν
k^
) = p
even
(ω
2k
by Lemma 2
-^
d
k^
= p
odd
(ν
k^
) = p
odd
(ω
2k
by Lemma 2
-^
recall p(x) = p
even
(x
2 ) + x p
odd
(x
− ∑=
=^
1 0
n j
j k j
k^
a
p
ω ω ) ( ) (
(^
2
odd
2
even
k
k
kk k
k
k
p
p p
d
e
y
k
ω
ω
ω
ω ω
(^2) /
2
odd (^2) /
2
even
2
odd (^2) /
2
even
2
odd
2
even
(^2) /
n k
n k n k n k
k
n k
k
k
k
n k
p
p
p
p
p
p
p
d
e
y
k
k k
k
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω ω
Docsity.com
17
Best of Both Worlds
Can we get "fast" multiplication and evaluation?
coefficient Yes! Convert back and forth between two representations. Representation
O(n
Multiplication
O(n) Evaluation
point-value
O(n)
O(n
FFT
O(n log n)
O(n log n)
(^1) - n
1 0
(^1) - n
1 0
b, b
a,
, a, a^
b KK
(^2) - 2n
1 0
c
c
c
K
q(x
p(x
(^1) - 2n
1
0
(^1) - 2n
1
0
x q
x q
x p
x p
K K
r(x
(^1) - 2n
1
0
x r
x r^
K
O(n)
point-value multiplication
O(n
2 )
coefficient multiplication
O(n log n)
evaluation
FFT
interpolationinverse FFT
O(n log n)
Forward FFT: given {a
, a 0
,... , a 1
n-
} , compute {y
, y 0
,... , y 1
n-
Inverse FFT: given {y
, y 0
,... , y 1
n-
} compute {a
, a 0
,... , a 1
n-
− − − − − −
− − −
−^
1 0 1 2 3
) 1
)( 1
(
) 1
( 3
) 1
( 2
1
) 1
( 3
9
6
3
) 1
( 2
6
4
2
1
3
2
1
1 0 1 2 3
n n n n n n
n n n
n
a a a a a
y y y y y
M
L
M O M M M M
LLL L
M
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
− − − − − − −
− − −
−^
0 1 2 3 1
1 ) 1
)( 1
(
) 1
( 3
) 1
( 2
1
) 1
( 3
9
6
3
) 1
( 2
6
4
2
1
3
2
1
1 0 1 2 3
n n n n n n
n n n
n
y y y y y
a a a a a
M
L
M O M M M M
LLL L
M
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
Inverse FFT
19
Great news: same algorithm as FFT, except use
ω
as "principal" n
th
root of unity (and divide by n).
−
−
−
−
−
− − −
) 1
)( 1
(
) 1
( 3
) 1
( 2
) 1
(
) 1
( 3
9
6
3
) 1
( 2
6
4
2
) 1
(
3
2
1
n
n
n
n
n
n n n
n F
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
L
M O M M M M
LL L L
− − − − − − − − − −
−
−
−
−
−
−
−
−
−
− − − − − −
) 1
)( 1
(
) 1
( 3
) 1
( 2
) 1
(
) 1
( 3
9
6
3
) 1
( 2
6
4
2
) 1
(
3
2
1
1
n
n
n
n
n
n n n
n^
n
F
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
ω
L
M O M M M M
LL L L
Inverse FFT
Inverse FFT: Proof of Correctness
Summation lemma. Let
ω
be a primitive n
th
root of unity. Then
n^
If k is a multiple of n then
ω
k^
n^
Each n
th
root of unity
ω
k^
is a root of
x
n^
2
+... + x
n-
if
ω
k^
1 we have: 1 +
ω
k^
ω
k(2)
ω
k(n-1)
Claim: F
n^
and F
-1 n
are inverses.
^
−∑=
otherwise 0
mod 0
1 0
n
k n
n j
j k ω
(^
)
∑ ∑
− =
′ −
− =
′ −
′
−
otherwise 0
if
1 1
(^10)
)
(
(^10)
1
i i
n
n
F
F
n j
j i i
n j
ij
ij
ii
n n
ω
ω
ω
Docsity.com
