Dynamic Programming - Data Structures and Algorithm - Lecture Slides, Slides of Data Structures and Algorithms

Download Dynamic Programming - Data Structures and Algorithm - Lecture Slides and more Slides Data Structures and Algorithms in PDF only on Docsity!

Introduction to Dynamic Programming

Problem of the Day

Multisets are allowed to have repeated elements. A multiset of n items may thus have fewer than n! distinct permutations. For example, { 1 , 1 , 2 , 2 } has only six different permutations: { 1 , 1 , 2 , 2 }, { 1 , 2 , 1 , 2 }, { 1 , 2 , 2 , 1 }, { 2 , 1 , 1 , 2 }, { 2 , 1 , 2 , 1 }, and { 2 , 2 , 1 , 1 }. Design and implement an efficient algorithm for constructing all permutations of a multiset.

Greedy vs. Exhaustive Search

Greedy algorithms focus on making the best local choice at each decision point. In the absence of a correctness proof such greedy algorithms are very likely to fail. Dynamic programming gives us a way to design custom algorithms which systematically search all possibilities (thus guaranteeing correctness) while storing results to avoid recomputing (thus providing efficiency).

Recurrence Relations

A recurrence relation is an equation which is defined in terms of itself. They are useful because many natural functions are easily expressed as recurrences: Polynomials: an = an− 1 + 1, a 1 = 1 −→ an = n Exponentials: an = 2an− 1 , a 1 = 2 −→ an = 2n Weird: an = nan− 1 , a 1 = 1 −→ an = n! Computer programs can easily evaluate the value of a given recurrence even without the existence of a nice closed form.

How Slow?

Fn+1/Fn ≈ φ = (1 +

Thus Fn ≈ 1. 6 n. Since our recursion tree has 0 and 1 as leaves, computing Fn requires ≈ 1. 6 n^ calls!

What about Dynamic Programming?

We can calculate Fn in linear time by storing small values:

F 0 = 0 F 1 = 1 For i = 1 to n Fi = Fi− 1 + Fi− 2

Moral: we traded space for time.

Avoiding Recomputation by Storing Partial

Results

The trick to dynamic program is to see that the naive recursive algorithm repeatedly computes the same subproblems over and over and over again. If so, storing the answers to them in a table instead of recomputing can lead to an efficient algorithm. Thus we must first hunt for a correct recursive algorithm – later we can worry about speeding it up by using a results matrix.

Binomial Coefficient

The most important class of counting numbers are the binomial coefficients , where (nk) counts the number of ways to choose k things out of n possibilities.

• Committees – How many ways are there to form a k- member committee from n people? By definition, (nk).
• Paths Across a Grid – How many ways are there to travel from the upper-left corner of an n × m grid to the lower- right corner by walking only down and to the right? Every path must consist of n + m steps, n downward and m to the right, so there are (n^ +n^ m) such sets/paths.

Pascal’s Triangle

No doubt you played with this arrangement of numbers in high school. Each number is the sum of the two numbers directly above it:

1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1

Pascal’s Recurrence

A more stable way to compute binomial coefficients is using the recurrence relation implicit in the construction of Pascal’s triangle, namely, that

(nk) = (n k −− 11 ) + (n^ − k 1 )

It works because the nth element either appears or does not appear in one of the (nk) subsets of k elements.

Binomial Coefficient Implementation

long binomial coefficient(n,m) int n,m; (* compute n choose m ) { int i,j; ( counters ) long bc[MAXN][MAXN]; ( table of binomial coefficients *) for (i=0; i<=n; i++) bc[i][0] = 1; for (j=0; j<=n; j++) bc[j][j] = 1; for (i=1; i<=n; i++) for (j=1; j<i; j++) bc[i][j] = bc[i-1][j-1] + bc[i-1][j]; return( bc[n][m] ); }

Three Steps to Dynamic Programming

1. Formulate the answer as a recurrence relation or recursive algorithm.
2. Show that the number of different instances of your recurrence is bounded by a polynomial.
3. Specify an order of evaluation for the recurrence so you always have what you need.