Download Dynamic Programming: Fibonacci Sequence and Edit Distance and more Study notes Computer Science in PDF only on Docsity!
CS 361, Lecture 25
Jared Saia
University of New Mexico
Today’s Outline
Intro Dynamic Programming
String Alignment
DP Intro
What is Dynamic Programming?tion and Reason in Common Sense (1905)it.” - George Santayana, The Life of Reason, Book I: Introduc- “Those who cannot remember the past are doomed to repeat
Dynamic
Programming
is
basically
“Divide
and
Conquer”
with memorization
Basic
Trick
is:
Don’t
solve
the
same
problem
more
than
once!
Fibonacci Example
Consider
the
following
procedure
for
computing
the
n -th
Fi-
Fib(n){bonacci number:
if (n<2)
return n;
else
return Fib(n-1) + Fib(n-2);
Analysis
Q: What is the runtime of Fib?
A: Except for recursive calls,
the entire algorithm takes a
constant number of steps.
If
T
n ) is the run time of the
algorithm on input
n , then we can say that:
T
T
T
n ) =
T
n (^) −
(^) 2) +
T
n (^) −
(^) 1) + 1
It’s easy to show by induction that
T
n ) = 2
F
n
This
is very bad!
Aside
Q: How can we solve
T
n ) exactly?
lecture to getA: We solved this recurrence using annihilaotrs in the last
T
n ) =
c 1 φ n
c 2 φˆ n
c 3 1 n
where
φ
=
1+
√ 5
2
and ˆ
φ
=
1 − √ 5
2
Aside II
If we solve for constants, we get that:
T
c 1 (^) +
(^) c 2
c 3
T
c 1 φ (^) +
(^) c 2 φˆ (^) +
(^) c 3
T
c 1 φ 2 +^
(^) c 2 φˆ 2 +^
(^) c 3
ination) gives: Solving this system of linear equations (using Gaussian elim-
c 1
= 1 +
c 2
= 1
c 3
=
Aside III
So our final solution is
T
n ) =
( 1 +
√ 5 ) φ n + ( 1
)
φ ˆ n
−^
(^) 1 = Θ(
φ n ) .
The Pattern
Formulate the problem recursively.
. Write down a formula
smaller subproblemsfor the whole problem as a simple combination of answers to
sidering the intermediate subproblems in the correct order.recurrence and works its way up to the final solution by con- Write an algorithm that starts with the base cases of yourBuild solutions to your recurrence from the bottom up.
problems. This is frequentlyNote: Dynamic Programs store the results of intermediate sub-
but not always
done with some type
of table.
Edit Distance
The
edit distance
between two words is the minimum number
required to transform one word into another.of letter insertions, letter deletions, and letter substitutions
For example,
the edit distance between
FOOD
and
MONEY
is at most four:
F
OOD
MOO
D
MON
∧ D
MONED
MONEY
String Alignment
Better way to display this process:
Place the words one above the other in a table
the second word for every deletionPut a gap in the first word for every insertion and a gap in
tutionsColumns with two different characters correspond to substi-
Then
the
number
of
editing
steps
is
just
the
number
of
columns that don’t contain the same character twice
Example
String Alignment for “FOOD” and “MONEY”: F
O
O
D
M
O
N
E
Y
the edit distance between “Food” and “Money”It’s not too hard to see that we can’t do better than four for
Example II
distance exactly. Example:Unfortunately, it can be more difficult to compute the edit A L G O R I T H M A L T R U I S T I C
Key Observation
remaining alignment must also be optimalIf we remove the last column in an optimal alignment, the
subalignment of all but the last column.Easy to prove by contradiction: Assume there is some better
Then we can just
a better overall alignment.paste the last column onto this better subalignment to get
Note:
The last column can be either:
- a blank on top
aligned with a character on bottomaligned with a blank on bottom or 3) a character on topaligned with a character on bottom, 2) a character on top
DP Solution
find a recursive definitionTo develop a DP algorithm for this problem, we first need to
Assume we have a
m
length string
A
and an
n
length string
B
Let
E
i, j
) be the edit distance between the first
i characters
of
A
and the first
j
characters of
B
Then what we want to find is
E
n, m
Recursive Definition
Say we want to compute
E
i, j
) for some
i
and
j
tion toFurther say that the “Recursion Fairy” can tell us the solu-
E
i ′ , j
), for all′
i ′ ≤
i ,
j ′ ≤
j ,
except
for
i ′ =
i
and
j ′ =
j
Q: Can we compute
E
i, j
) efficiently with help from the our
fairy friend?
Better Idea
We can build up a
m
×
n
table which contains all values of
E
i, j
in the 0-th row and 0-th columnWe start by filling in the base cases for this table: the entries
above, to the left and above and to the left.To fill in any other entry, we need to know the values directly
Thus we can fill in the table in the standard way:
left to
in each cell are always availableright and top down to ensure that the entries we need to fill
Example Table
are equalBold numbers indicate places where characters in the strings
Arrows represent predecessors that define each entry:
hori-
substitution.zontal arrow is deletion, vertical is insertion and diagonal is
itselfBold diagonal arrows are “free” substitutions of a letter for
example table, so there are three optimal edit sequences).ner gives an optimal alignment (there are three paths in thisAny path of arrows from the top left to the bottom right cor-
A L G O R I T H M
A 1 0 → 1 →
L
T
R 4 3 2 2 2 2 → 3 →
U 5 4 3 3 3 3 3 →
I 6 5 4 4 4 4 3 →
S 7 6 5 5 5 5 4 4 5 6
T 8 7 6 6 6 6 5 4
I 9 8 7 7 7 7 6 5 5 →
C
Analysis
Let
n
be the length of the first string and
m
the length of
the second string
Then there are Θ(
nm
) entries in the table, and it takes Θ(1)
time to fill each entry
This implies that the run time of the algorithm is Θ(
nm
Q: Can you find a faster algorithm?
