Download Dynamic Programming: Techniques and Applications in Algorithm Design and more Cheat Sheet Computer science in PDF only on Docsity!
Chapter 6
In the previous chapters, we studied the technique of divide and conquer which gives efficient algorithms for many problems. In this chapter, we focus on another powerful technique for solving problems called dynamic programming (DP). 6.1 Divide and Conquer vs. DP As we saw in divide and conquer, the basic strategy in dynamic programming is again breaking problems into subproblems and combining their solutions to obtain the solution to the original problem. However, there are differences between the two strategies. While in divide and conquer, the problem is decomposed into independent and disjoint subproblems, in dynamic programming, it is almost always decomposed into independent and overlapping subproblems. Independent means the solution to one subproblem does not affect the solution to another subproblem. Thus the solution to each subproblem can be computed independently of one and another. Disjoint means that each subproblem is different; in many examples, the subproblems operate on different parts of the input and hence are naturally disjoint. For example, in mergesort, a divide and conquer algorithm for sorting (Section 5.1.2), the input array is partitioned into two (almost) equal and disjoint subarrays and the subproblem is solved in each subarray recursively. Since the subarrays are disjoint, the subproblems are disjoint. In other cases, the subproblems may share some input, but they are different subproblems. For example, in the linear-time selection algorithm (Section 5.2), the subproblem of finding the “median of medians” shares some input values with the other two subproblems one of either side of the pivot (note that only one of these will actually execute); however, the subproblems are different and hence disjoint. On the other hand, a key ingredient of dynamic programming is that the subproblems are overlapping, i.e., the same subproblem occurs in the solution of other subproblems. A main point of dynamic programming is to solve each subproblem only once (there is no point in solving the same subproblem again and again which is inefficient!). We will illustrate the technique with various examples throughout this chapter. Dynamic programming (DP) is generally a more difficult technique than divide and conquer. The main reason is that, in many applications of DP, the difficulty is in figuring out how to decompose a given problem into subproblems. Thus the nontrivial work in a
99
6.2. OPTIMIZATION PROBLEMS 100 DP solution is figuring out this decomposition. The best way to state the decomposition is by using a recursive formula. The recursive formula gives the solution of the problem in terms of the solutions of its subproblems. It is easy to show the correctness of the recursive formula by using induction. Translating the recursive formula into a pseudocode is also fairly straightforward — the only care that has to be taken is that each (distinct) subproblem has to be solved exactly once since the same
subproblem can arise in the solution of other subproblems, i.e., overlapping subproblems. Again, we will see how to do this translation with the help of various examples. The good news is that once we have the correct recursive formulation, an efficient implementation (that solves each subproblem only once) is rather standard and “mechanical” to write. However, coming with a correct recursive formulation (that leads to a correct and efficient algorithm) is usually an “art”. Nevertheless, there are a few “patterns” in coming with this recursive formulation which occurs repeatedly in many problems. Studying these patterns will help in designing DP algorithms. We will see these patterns in this chapter. 6.2 Optimization problems Dynamic programming is especially a powerful technique for solving optimization problems. In optimization problems (very much like in search problems that we saw in Chapter 2) the goal is to find (search for) an optimum (e.g., maximum or minimum) solution to a problem specified on an input domain. Usually the problem comes with some constraints and the goal is to find an optimal solution that satisfies the problem’s constraints. For example, consider the max-sum problem (see Exercise 5.18) stated as follows: Problem 6.1 ▶ Maxsum
Given an array A of size n containing positive and negative integers, deter- mine indices i and j, 1 ⩽ i ⩽ j ⩽ n, such that A[i] + A[i + 1] + · · · + A[j] is a
maximum. In other words, the goal is to find contiguous array values A[i], A[i + 1],... , A[j] such that their sum — A[i] + A[i + 1] + · · · + A[j] — is maximized. This is an optimization problem since we want to optimize, i.e., maximize, some value (i.e., the sum) over an input domain. The input domain for this is the set of all pair of indices i and j, 1 ⩽ i ⩽ j ⩽ n, with each pair giving rise to a sum. Thus there are n 2 such sums. Note that all these sums are “potential” solutions for the problem (the optimum is the one which is the maximum sum). These are called feasible solutions. Feasible solutions are those that satisfy the constraints of the problem. Here the constraint is that the array sum has to be over a contiguous subarray, i.e., it includes all array values between indices i and j (including A[i] and A[j]). It is easy to see that the problem can be solved by calculating all the possible n
2 array sums and taking the maximum among those. This takes at least
n 2 operations; actually, even more since to compute a particular sum A[i] + · · · + A[j] requires j − i addition operations. It is not difficult to calculate that the total number of addition operations over all the Θ(n 2 ) array sums is Θ(n
subproblems (since it takes at least constant time to solve a subproblem) and hence the running time with this decomposition is at least n 2
. If we are shooting for an O(n) time algorithm, the number of subproblems have to be O(n) as well. Generally, the smaller the total number of subproblems, the better. How to come with such a decomposition? This is the tricky part. One helpful idea in finding subproblems is to introduce auxiliary variables. Here is a first attempt keeping in mind that we are shooting for O(n) subproblems. Let M[i] be the solution to the max-sum problem considering only the first i elements of the array A. That is, we introduce the auxiliary variable i that restricts the
6.3. ILLUSTRATING THE DP TECHNIQUE USING THE MAXSUM PROBLEM array to only the first i elements. Clearly, the original problem is one of the subproblems, i.e., M[n]. Is this a good decomposition? To answer this, we need to check if we can write a recursive formulation for M[i] in terms of subproblems of smaller size, i.e., M[i − 1] and below. Can we write a recursive solution to M[i] based on M[i − 1]. Note that it is not enough to know just the value of M[i − 1], but we also need to know if the solution to M[i − 1] includes the last element, i.e., A[i − 1], or not. This is because, if A[i − 1] is not in the solution of M[i − 1], then A[i] cannot be included in the solution of M[i] if elements of A with index less than i − 1 are included; on the other hand, if A[i − 1] is included, then A[i] can be included in the solution of M[i]. Thus, in general, the solution of a subproblem M[i] depends on whether the last element, i.e., A[i] is in the solution or not. Hence, to write a recursion for M[i] we need to know whether A[i] is in the solution or not. This motivates the following subproblems. We consider the following two subproblems based on the above fact. Let Min[i] be the solution to the max-sum problem considering only the first i elements of the array A, i.e., the maximum sum considering the array A[1,... , i], with the condition that the solution includes the ith element, i.e., the last element A[i] is in the max-sum. Similarly, let Mout[i] be the solution to the max-sum problem considering the first i elements of the array A, i.e., the maximum sum considering the array A[1,... , i], with the condition that the solution does not include the ith element, i.e., the last element A[i] is not in the max-sum. The solution to the max-sum problem in the array A[1,... , i] is
M[i] = max
Min[i], Mout[i]
i.e., the maximum of the (only) two possibilities — Min[i] and Mout[i] — where the first includes A[i] and the second excludes it. The important point about the subproblems is that it includes the original problem as well which is simply M[n]. This is typical of DP, the decomposed subproblems typically include the original problem as one of the subproblems. Although we are not interested
in the solution of other subproblems, it turns out their solutions will help in building the solution to the original problem. The total number of subproblems is O(n), or 2n to be precise — Min[i] and Mout[i] for 1 ⩽ i ⩽ n. Step 4: Recursive formulation. We now come to the key step of expressing the solution of the problem recursively in terms of the solutions to its subproblems. This is called recursive formulation. Once we have the correct recursive formulation, it turns out it is easy to write an algorithm that implements the formulation (see next step). Recursive formulation is like writing a recurrence. There are one or more base cases which show the solution to base-case subproblems which are usually those that are of small 1Note that writing the recursive formulation is the next step in DP. However, it is quite obvious that these two steps go hand in hand; if we cannot easily write a recursive formula for our subproblems, then we try to modify it. 2A very naive way of formulating subproblems based on the fact that each element is in the solution or not is to look at all possible subsets of the array. There are 2
n such subsets. This is exponentially large and does not constitute an efficient decomposition since it ignores the key constraint that a feasible solution subset has to be contiguous. Even if we allow only for contiguous subsets to be contiguous, there are n 2 of them which is polynomial but still much bigger than O(n) that we are shooting for.
6.3. ILLUSTRATING THE DP TECHNIQUE USING THE MAXSUM PROBLEM size. Then the recurrence shows how to compute the solution of larger-sized subproblems in terms of smaller-sized subproblems. The advantage of recursive formulation is that its correctness can be quickly established by (as usual) mathematical induction. Let us illustrate recursive formulation by using the maxsum problem. As, mentioned above, M[i] = max{Min[i], Mout[i]}, since there are only two possibilities, either the last element (A[i]) is in the optimal solution or not. Note that M[n] gives the maxsum value for the whole array (which is the answer we want to find). The recursive formulation is writing a recurrence to compute Min[i] and Mout[i] recursively (for i > 1):
Min[i] = max
Min[i − 1] + A[i], A[i]
- Bound the (worst-case) time take to solve each subproblem — let it be T.
- Then the total time of DP is upper bounded by O(KT) because, there are at most K subproblems and each takes at most T time. Sometimes a better bound can be obtained by looking at the time taken by individual subproblems. Let the time taken to solve subproblem Si be Ti
, 1 ⩽ i ⩽ K. Then the overall time is PK i=1 Ti . Let us calculate the running time of the DP algorithm for the maxsum problem. The number of subproblems is 2n — Min[i] and Mout[i] for each i. The time to compute each is just constant, which follows from the recurrence. Hence, the total run time is O(n). Algorithm 22 MaxsumDP – Iterative Implementation of Maxsum
Input: An array A of length n Output: Maximum subarray sum
1: func MaxsumDP(A): 2: M[1] = A[1] 3: Min[1] = A[1] 4: Mout[1] = −∞ 5: for i = 2 to n: 6: Mout[i] = M[i − 1] 7: Min[i] = max(Min[i − 1] + A[i], A[i]) 8: M[i] = max(Min[i], Mout[i]) 9: return M[n]
6.3. ILLUSTRATING THE DP TECHNIQUE USING THE MAXSUM PROBLEM Algorithm 23 MaxsumRec – Naive Recursive Implementation of Maxsum
Input: An array A of length n Output: Maximum subarray sum
1: func MaxsumRec(A): 2: return max(MinRec(A), MoutRec(A)) 3: func MinRec(A): ▷ Maxsum including the ith element 4: if i == 1: 5: return A[1] 6: else: 7: return max(MinRec(A[1,... , i − 1]) + A[i]), A[i]) 8: func MoutRec(A): ▷ Maxsum excluding the ith element 9: if i == 1:
10: return −∞ 11: else: 12: return MaxsumRec(A[1,... , i − 1])
Algorithm 24 MaxsumRecLookup – Efficient implementation of the recursive formulation of the maxsum problem that uses the table lookup strategy.
Input: An array A of length n Output: Maxsum value 1: func MaxsumRecLookup(A): 2: for i = 1 to n: 3: M[i] = Min[i] = Mout[i] = −∞ ▷ initialization of arrays 4: Min[1] = A[1] 5: return max(MinRecLookup(n), MoutRecLookup(n)) 6: func MinRecLookup(i): 7: if Min[i] == −∞: ▷ Value has not yet been computed 8: include = MinRecLookup(i − 1) 9: Min[i] = max(include, A[i]) 10: return Min[i] 11: func MoutRecLookup(i): 12: if Mout[i] == −∞: ▷ Value has not yet been computed 13: Mout[i] = MaxsumRecLookup(A[1,... , i − 1]) 14: return Mout[i] Step 7: Converting the recursive formula into an algorithm. It is relatively straightforward to turn the above recursive formula into a pseudocode — either iterative or recursive. However, care has to be taken, especially in a recursive
6.3. ILLUSTRATING THE DP TECHNIQUE USING THE MAXSUM PROBLEM implementation so that each subproblem is executed exactly once. Otherwise, the running time of the implementation can be large (even exponential in the size of the problem). We will next show two different approaches for implementing DP algorithms:
- Iterative algorithm: Implementing the recursive formulation in an interative manner is what is known as dynamic programming. This is a bottom-up strategy, where we start from the solution of the base cases and then compute the solutions to larger-sized problems using smaller-sized problems. Let us illustrate this in the case of the maxsum problem. We first compute Min[1] and M[1] which are both A[1]. Note that Min[2] and Mout[2] depend only on Min[1] and M[1] and hence can be computed. Similarly Min[3] and Mout[3] can be computed and so on. Finally, we compute Min[n] and Mout[n]. The final answer is M[n] which is equal to the maximum of Min[n] and Mout[n]. This can be implemented by a for loop. The iterative DP algorithm is given in Algorithm 22. The running time of the iterative algorithm is clearly O(n) since there are are n − 1 iterations of the for loop and each iteration takes constant time. This is clearly an efficient implementation of the DP formulation as each subproblem is computed only once leading to an O(n) run
set S in[i] = True, otherwise we set S
in[i] = False. Algorithm 25 shows how to modify
the DP algorithm of Algorithm 22 to compute the additional array A[i]. To output the optimal solution we usually trace back the steps that we took to compute the optimum value, which in the case of the maxsum problem is M[n]. Note that M[n] is the maximum of Min[n] and Mout[n]. If the former is the maximum of the two, then A[n] is in the solution, otherwise, it is not. This information is stored in S[n]. If A[n] is in the solution, we look at whether A[n − 1] is in the solution — this can be inferred by looking at the S in[n]. We can then continue whether A[n − 2] is in the solution or not and so on. This is easier to do recursively, since we do this in a top-down fashion starting from the optimal value M[n]. The algorithm is given in Algorithm 26. Algorithm 25 MaxsumDPMod – This is the modified DP (iterative) implementation of the recursive formulation of the maxsum problem that also computes the S and S in
arrays which are used in outputting the optimal solution.
Input: An array A Output: Maxsum and arrays S and S in
1: func MaxsumDPMod(A): 2: M[1] = A[1] 3: Min[1] = A[1] 4: Mout[i] = −∞ 5: S[1] = in 6: for i = 2 to n: 7: Mout[i] = M[i − 1] 8: Min[i] = max{Min[i − 1] + A[i], A[i]} 9: M[i] = max(Min[i], Mout[i]) 10: if M[i] == Min[i]: 11: S[i] = in 12: else: 13: S[i] = out 14: if Min[i] == Min[i − 1] + A[i]: 15: S
in[i] = True
16: else:
17: S
in[i] = False 18: return M[n]
6.4. PROBLEM: MATRIX CHAIN MULTIPLICATION 108 Algorithm 26 MaxsumSol
Input: Array A, S, and S
in as in MaxsumDPMod Output: None. The optimal solution is printed backwards.
func MaxsumSol(A, S, S in):
if n ⩾ 1: if S[n] == in: print(A[n]) ▷ A[n] is included in the optimal solution if S in[n]: ▷ A[n − 1] is also included in the optimal solution MaxsumSol(A[1,... , n − 1], S[1,... , n − 1], Sin[1,... , n − 1]) else: ▷ S[n] = out MaxsumSol(A[1,... , n − 1], S[1,... , n − 1], Sin[1,... , n − 1]) We next further illustrate these ideas by looking at various problems. 6.4 Problem: Matrix Chain Multiplication We are given a chain of n matrices: < A1, A2,... , An > where matrix Ai has dimension pi−1 × pi , for i = 1, 2,... , n. We are interested in finding the “best” way to compute the product of the n matrices: Problem 6.2 ▶ Matrix Chain Multiplication Find a way to parenthesize the product A1A2... An which minimizes the number of multiplications. To understand the problem, we recall some basic facts about matrix multiplication:
- The number of columns of Ai
is equal to the number of rows of Ai+1 (for 1 ⩽ i ⩽ n−1).
This property is needed for a valid multiplication.
- Given 2 matrices A, B, with dimension m × n and n × p respectively, the number of element-wise multiplications needed to multiply the 2 matrices is mnp. Note that here we assume that multiplications are done in the standard way, i.e., the i, jth entry of the product matrix C = AB is computed as: cij =
/memoization or (2) bottom-up / iterative / DP. Key step in the recursive formulation In many DP problems, the key step in writing a correct recursive formulation is identifying the right subproblem(s). The subproblems may not be obvious. A useful idea is to introduce auxiliary parameters in the problem. For example, though we are interested in finding out the best way to paranthesize the entire matrix chain, by introducing the two auxiliary variables i and j, we identify the subproblem(s) of parenthesizing the subchain Ai
... Aj which proves useful in solving the desired problem. Note that the original problem is one of the subproblems, i.e., computing m[1, n]. Correctness of the recursive formulation
To show the correctness of the recursive formulation, as usual we we turn to mathe- matical induction.
- Base case (i = j): Trivially true.
- Recursive step (i < j): The induction hypothesis is that the recursive formulation is correct for all l such j − i < l. That is, the induction is on the length of the chain. Assuming the induction hypothesis, we prove that the formulation is correct for j − i = l, i.e., m[i, j] is the optimal cost of multiplying Ai ... Aj .
6.4. PROBLEM: MATRIX CHAIN MULTIPLICATION 110 Take the minimum of the j − i = l possible parenthesizations between Ai and Aj depending on where you split the product, i.e.,
(Ai
... Aj ) = (Ai ... Ak)(Ak+1... Aj ) We note that m[i, k] is the minimum cost of multiplying (Ai ... Ak) (by induction hypothesis), m[k + 1, j] is the minimum cost of multiplying A(Ak+1... Aj ) (again by induction hypothesis), and pi−1pkpj
is the cost of combining the two partial products. Hence, since the split is minimum, the overall cost m[i, j] is also optimal. Note that the above argument shows the optimal substructure property which is nothing but saying that the optimal solution to a problem consists of optimal solution to its subproblems. Here computing m[i, j] is the problem and that depends on m[i, k] and m[k + 1, j] (for all i ⩽ k < j), which are optimal solutions to subproblems. As we saw above, the proof is straightforward by using induction. The optimal substructure property
tells us that there is no need to consider suboptimal solutions to subproblems when solving a problem; only optimal solutions to subproblems are relevant. This drastically reduces the number of subproblem solutions to consider — leading to efficient algorithms. A Naive Algorithm The following is a naive algorithm that directly converts the recursive formula into a recursive algorithm. Exercise 6.3 asks you to show that the above algorithm takes exponential time. Algorithm 27 Cost
Input: A sequence of matrix dimensions: p = p0, p1,... , pn and indices i
and j
Output: The optimal cost to multiply Ai
... Aj
1: func Cost(p, i, j): 2: if i == j: 3: return 0 4: m[i, j] = ∞ 5: for k = i to j − 1: 6: q = Cost(p, i, k) + Cost(p, k + 1, j) + pi−1pkpj 7: if q < m[i, j]: 8: m[i, j] = q return m[i, j] Bottom-up Algorithm (DP/iterative algorithm) Here is the DP algorithm which is an iterative algorithm. Note that the DP algorithm builds the solution bottom up: it first computes 1 length chains (which is 0) and then successively computes chains of increasing length, i.e., lengths 2, 3,... , up to n. Each chain of length l depends on chains of length l − 1 or smaller. Hence computing bottom up works.
6.4. PROBLEM: MATRIX CHAIN MULTIPLICATION 111 Algorithm 28 MatrixCostDP – DP Matrix Cost Input: A sequence of matrix dimensions p Output: The optimal cost to multiply A1... An
1: func MatrixCostDP(p): 2: for i = 1 to n: 3: m[i, i] = 0 4: for l = 2 to n: 5: for i = 1 to n − l + 1: 6: j = i + l − 1 7: m[i, j] = ∞
the “best” alignment between the two sequences. Suppose we are given two DNA strings s and t as follows. s = GACGGATTAG t = GATCGGAATAG
The goal is to find the “best” way to “align” the two strings. We define what is meant by alignment next. Alignment An alignment is an insertion of spaces in arbitrary locations along the sequences so that they end up with the same size. A space in one sequence should not align with a space in the other. But spaces can be inserted at the beginning or at the end. An alignment is scored by a scoring scheme. We assume a given scoring matrix score where the entry score[x, y] gives the alignment score for characters x and y. For simplicity, we will assume that the score of any letter with space is -1. The score for an alignment is the sum of the scores of its aligned characters. A best alignment is one which receives the maximum score called the similarity — sim(s, t). Example We want to find the best way to align s = GACGGATTAG and t = GATCGGAATAG. Assume that the score for aligning two characters which are the same is 1, but aligning two different characters has score 0, while aligning a letter with a space has score -1. Then the best alignment of s and t is:
s = GA CGGATTAG t = GATCGGAATAG
Note that a space was introduced in s after the first two characters. The score for this alignment is 1 + 1 − 1 + 1 + 1 + 1 + 1 + 0 + 1 + 1 + 1 = 8. This is the maximum possible alignment score and hence this is a best alignment (there can be more than one best alignment). A Naive solution A naive approach is to look at all possible alignments of the two input strings, compute the score for each alignment and then take the maximum. This is very inefficient as
6.5. PROBLEM: ALIGNING TWO SEQUENCES 113 the number of possible alignments is at least exponential in the size of the strings. For example, if one string has 2n characters and the other string has n characters, one has to insert at least n gaps to align the two strings; the number of ways that one can insert n gaps among the n characters is at least 2n n
⩾ 2 n .
4 DP leads to a significantly more
efficient solution as described below. Recursive formulation As in the matrix multiplication problem, to compute the optimal alignment score we first write the recursive formulation. Again the key is to identify the right subproblems. Let string s be represented as an array (list) of characters s[1... m] and t as t[1... n]. We will use the subproblem(s) of finding the best alignment of the substrings s[1... i] and t[1... j], where 1 ⩽ i ⩽ m and 1 ⩽ j ⩽ n. Note that we only allow the right end to vary. One could have allowed “both” ends to vary, i.e., consider subproblems of aligning strings s[i... j] and t[k... l]. However this is not necessary and leads to an increased run time (though still polynomial) since the number of subproblems is more. We note that s[] and t[] define the empty string. The key idea behind the recursive formulation is that to compute sim(s[1... i], t[1... j]), we need to only look at aligning the last character of each of the two strings. There are only three possibilities: (1) aligning s[i] with t[j] (2) aligning gap with t[j] (3) aligning s[i] with gap. Thus we have the following recursive formula:
sim(s[1... i], t[1... j]) = max sim(s[1... i − 1], t[1... j − 1]) + score(s[i], t[j]) sim(s[1... i], t[1... j − 1]) + score(−, t[j]) sim(s[1... i − 1], t[1... j]) + score(s[i], −)
The base cases are:
sim(s[1... i], t[]) = −i sim(s[], t[1... j]) = −j sim(s[], t[]) = 0
The above base cases capture the three base case situations that arise from the three possibilities: (1) when t becomes empty, and s is not empty, in which case we have to align all the characters left in s with gaps (2) when s becomes empty, and t is not empty, in which case we have to align all the characters left in t with gaps (3) both become empty. Note that in the above base cases, we have used that aligning a gap with a character has score -1. The correctness of the recursive formula follows from mathematical induction which is left as an exercise. Dynamic Programming (Bottom-up) Algorithm We can convert the recursive formulation into a DP algorithm as follows in a fairly straightforward fashion. 4See Appendix G for the inequality used to infer this lower bound.
6.6. PROBLEM: 0/1 KNAPSACK 114 Algorithm 30 SimScoreDP
is 2 n , this leads to exponential running time. DP, again, leads to a significantly improved running time (that depends linearly on m) as explained below.
6.7. DP SUMMARY 115 Subproblems The key, again, for a correct recursive formulation is identifying the right subproblems. For this, as mentioned earlier, introducing auxiliary variables help. Let us introduce two new variables j, y as follows. Let KNAP(j, y) represent the problem: maximize P 1 ⩽i⩽j pixi ; subject to the constraint P
1 ⩽i⩽j wixi ⩽ y with the (integral) requirement that xi ∈ {0, 1}, 1 ⩽ i ⩽ j. The above is simply a restatement of the knapsack problem as an explicit maximization problem. In words, it is the subproblem that considers only the first j items with a knapsack capacity y. The Knapsack problem is simply KNAP(n, m). Note that we assume all weights and m are integers. Recursive formulation Let Pj (y) be the optimal solution to KNAP(j, y). Then, we can write:
Pj (y) = max{Pj−1(y), Pj−1(y − wj ) + pj} The base cases are: P0(y) = 0 for all y ⩾ 0 and Pi(y) = −∞ if y < 0. Correctness of the formulation The above formulation is correct because to compute Pj (y) we need to look at only two choices: either the aj th object is in the knapsack or not. If it is not included, then Pj (y) = Pj−1(y), since no weight has been added. If it is included, then Pj (y) = Pj−1(y − wj ) + pj ,
since we have to subtract the weight wj
from the knapsack capacity and include its profit.
The correctness of the formulation, as usual, can be established by induction. Again induction implies that the optimal substructure property holds. Pj (y) should contain within itself optimal solution to its subproblem(s) — which are Pj−1(y) (in the first case) and Pj−1(y − wj ) (in the second case). Of course, we take the maximum value of the two cases. DP algorithm A DP algorithm is straightforward from the above formulation. The desired optimal value is Pn(m) which we compute in a bottom up manner. That is, we first start from the base cases and compute P1(.), P2(.),... in this order. The time complexity is O(nm). Note that the running time depends on the number of
objects as well as the (magnitude) of the (integer) capacity. Hence this is not a polynomial time algorithm, since the running time is not a polynomial function of the input size which is log m (since m can be represented using log m bits), but rather a polynomial function of m. Such a running time is referred to as pseudo-polynomial and the algorithm is called a pseudo-polynomial algorithm. Such algorithms are fast when m is small; more importantly they can be converted to yield efficient polynomial time algorithms that give an approximately optimal solution. This is discussed in the next section. 6.7 DP Summary We summarize the common themes underlying the application of DP to various problems that we studied in this chapter.
6.8. WORKED EXERCISES 116 Key ingredients of DP For DP to apply for a problem — typically, an optimization problem — it should have two key ingredients:
- Optimal substructure property: The optimal solution is computed from optimal solutions to independent subproblems. Hence, instead of looking at all feasible solutions for the subproblems (which can be very inefficient), it is enough to look at only optimal solutions for the subproblems.
- Overlapping subproblems. Overlapping means the same subproblem occurs in the solution of many different subproblems. We compute the solution of each subproblem only once. Two algorithmic implementations As was illustrated by various examples, once we formulate the all-important recursive formulation of the problem, there are two algorithmic ways for solving DP-based problems.
- Iterative algorithm: Based on the recursive formula we write an iterative algorithm that solves the recursion in a bottom-up manner. That is, it first solves the base-case subproblems and then progressively solves subproblems of increasing size based on the the solutions to smaller-sized subproblems. Note that recursive formula shows how to compute a subproblem of size k from subproblems of smaller size. This iterative strategy is called dynamic programming which is a bottom-up strategy. (The term “programming” is actually a misnomer and has been used for historical reasons.)
- Recursive algorithm: An alternate way is to solve the problem recursively using the recursive formula directly, i.e., by writing a recusrive code. However, as we saw in the problems we studied, a naive recursive implementation can solve the same subproblems repeatedly and can be very inefficient. Hence we have to the strategy of table look up, also known as memoization. Compute the solution of each subproblem only once and store the result in a table. When the same subproblem is encountered subsequently, get the value by looking up the table. This strategy is called memoization or table look up — which is a top-down strategy, since it is a recursive stategy. 6.8 Worked Exercises Worked Exercise 6.1. Suppose you want to dial a telephone number consisting of n
