10/8/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Adam Smith
Algorithm Design and Analysis
LECTURE 19
Dynamic Programming
•
Knapsack problem
•
RNA Secondary Structure
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Dynamic Programming: Knapsack Problem and RNA Secondary Structure, Study notes of Computer Science
A lecture on dynamic programming, focusing on the knapsack problem and rna secondary structure. It covers the definition and examples of the knapsack problem, the greedy algorithm, and the bottom-up algorithm for the knapsack problem. The lecture also discusses the rna secondary structure problem, its definition, examples, subproblems, and dynamic programming over intervals.
Typology: Study notes
Pre 2010
1 / 20
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Dynamic Programming: Knapsack Problem and RNA Secondary Structure and more Study notes Computer Science in PDF only on Docsity!
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Adam Smith
Algorithm Design and Analysis
L
ECTURE
• Dynamic Programming
Knapsack problem
RNA Secondary Structure
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Review questions
Shortest-path(G,s,t) = shortest route from
s
to
t
Longest-path(G,s,t) = longest
simple
path
from
s
to
t
Question
: Do Shortest Path and Longest Path
poly-time dynamic programming algorithm?have optimal substructure that can be used for a
What if the graph is a DAG?
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Knapsack Problem
Given n objects and a "knapsack."
Item i weighs w
i
> 0 kilograms and has value v
i
Knapsack has capacity of W kilograms.
Goal: fill knapsack so as to maximize total value.
Ex: { 3, 4 } has value 40.
value
weight
W = 11
Greedy
: repeatedly add item with
maximum ratio
v
i
/ w
i
value = 35Example: { 5, 2, 1 } achieves only
greedy not optimal.
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Knapsack Problem
Given n objects and a "knapsack."
Item i weighs w
i
> 0 kilograms and has value v
i
Knapsack has capacity of W kilograms.
Goal: fill knapsack so as to maximize total value.
Ex: { 3, 4 } has value 40.
value
weight
W = 11
Greedy algorithm?
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Adding a new variable
Definition: OPT(i, w) = max profit subset of
items 1, …, i with weight limit w.
Case 1: OPT does not select item i.
OPT selects best of { 1, 2, …, i-1 } with weight limit w
Case 2: OPT selects item i.
new weight limit = w – w
i
OPT selects best of { 1, 2, …, i–1 } with new weight limit
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Bottom-up algorithm
Fill up an n-by-W array.
Input: n, W, w
1
,…,w
N,
v
1
,…,v
N
for
w = 0 to W
M[0, w] = 0
for
i = 1 to n
for
w = 1 to W
if
(w
i
> w)
M[i, w] = M[i-1, w]
else
M[i, w] = max {M[i-1, w], v
i
+ M[i-1, w-w
i
]}
return
M[n, W]
a proof of correctness?How do we make turn this into
lends itself directly to induction.Dynamic programming (and divide and conquer)
Base cases: OPT(i,0)=0, OPT(0,w)=0 (no items!).
formulaInductive step: explaining the correctness of recursive
If the following values are correctly computed:
OPT(0,w-1),OPT(1,w-1),…,OPT(n,w-1)
OPT(0,w),…,OPT(i-1,w)
Then the recursive formula computes OPT(i,w) correctly
Case 1: …, Case 2: … (from previous slide).
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
About proofs
mathematical statement’s truthProof is a rigorous argument about a
Should convince you
Should not feel like a shot in the dark
What makes a proof “good enough” is social
are too painful for human use).(Truly rigorous, machine-readable proofs exist but
In real life, you have to check your own work
Ideal: all problems should have grades 100% or 15%
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
10/8/
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Time and space complexity
Time and space:
(n W).
Not polynomial in input size!
"Pseudo-polynomial."
Decision version of Knapsack is NP-complete.
[KT, chapter 8]
Knapsack approximation algorithm.
There is a
value within 0.01% of optimum. [KT, section 11.8]poly-time algorithm that produces a solution with
RNA Secondary Structure Problem
RNA. String B = b
1 b 2 … b n
over alphabet { A, C, G, U }.
molecule.itself. This structure is essential for understanding behavior ofSecondary structure. RNA tends to loop back and form base pairs with
G
U
C
A G
A
A G
C
G
A
U
G
A
U
U
G A
A
C
A
C A
U
G
A
G
U
C
A
U
G C
G
C G
G C
Ex:
GUCGAUUGAGCGAAUGUAACAACGUGGCUACGGCGAGA
complementary base pairs: A-U, C-G
RNA Secondary Structure: Examples
Examples.
C
G
G
A C
G
U
U
U
A
A U G U G G C C A U G G
A C
G
U
U
A
A U G G G C A U C G G
A C
U
G
U
U
A
A G U U G G C C A U
sharp turn
crossing
ok
G
G
base pair
RNA Secondary Structure: Subproblems
First attempt. OPT(j) =
maximum number of base pairs in a secondary structure of the
substring b
b
…
b
j
.
Difficulty. Results in two sub-problems.
Finding secondary structure in: b
b
…
b
t-
.
Finding secondary structure in: b
t+
b
t+
…
b
n-
.
1
t
n
match b
t
and b
n
need more sub-problemsOPT(t-1)
Bottom Up Dynamic Programming Over Intervals
Time: O(n A. Do shortest intervals first.Q. What order to solve the sub-problems?
).
Space: O(n
).
Exercise: Find the secondary structure that achieves the max value.
RNA(b
1
,…,b
n
) { %Ignoring base cases
for
k = 5, 6, …, n-
for
i = 1, 2, …, n-k
Compute M[i, j]j = i + k
return
M[1, n]
}
using recurrence
i
j
Dynamic Programming Summary
Recipe.
Recursively define value of optimal solution.
Compute value of optimal solution.
Construct optimal solution from computed information.
Dynamic programming techniques.
Binary choice: weighted interval scheduling.
Multi-way choice: segmented least squares. [KT, section 6.3]
Adding a new variable: LCS, knapsack.
Dynamic programming over intervals: RNA secondary structure.
different intuitions.Top-down (memoization) vs. bottom-up: different people have
grammar has similar structureparsing algorithm for context-free