Download Dynamic Programming - Introduction to Operations Research - Lecture Notes and more Study notes Economics in PDF only on Docsity!
Dynamic Programming
1. Consider the Towers of Hanoi problem in which it takes 2 seconds to move one disc from one peg to another. How long (days) would it take to implement the dynamic programming solution in the case of a problem involving 10 discs?
Solution:
The DP solution to this problem is discussed in the lecture notes (pp. 9.23-9.28). It is also mentioned (in fact,
given in the 1993 Exam) that the DP solution requires 2n-1 moves. Thus, for n=10 discs, we shall need 210-
moves, which in turn will require 211-2 seconds, or (211-2)/(60x60x24) days to complete.
2. Write down the dynamic programming functional equation that we used in class to solve the knapsack problem and give the formal definition of f(s) as well as its physical interpretation. Then solve the following knapsack problem using this functional equation.
z *^ := max x
6 x 1 + 11 x 2 + 10 x 3 s. t. x 1 + 2 x 2 + 1.5 x 3 ≤ 3 x 1 , x 2 , x 3 ∈{0,1,2,3,}
You are expected to determine the value of z* as well as the optimal values of x1 ,x2 and x.
Solution:
The DP functional equation that we discussed in class is as follows:
f ( s ) = max n vn ≤ s
{ w n +^ f^ ( s^ −^ vn },^ s^ =^ 1,2,, V
with
f (0) = 0,
where
f ( s ):= max x
∑ (^) n^ N = 1 wnx (^) n , s = 0,1, 2,, V s. t ∑ (^) n^ N = 1 vn x^ n ≤^ s xn ∈{0,1,2,3,}
Thus, by definition f(s) is equal to the maximum weight of a knapsack of volume s given the unit weights and volumes specified by the vectors w and v respectively. To use this functional equation we need an item whose volume is equal to 1. If no such (real) item exits, we introduce a dummy item with 1 unit of volume and 0 unit of weight.
Since we also rely on the vector v to consist of positive integers, we multiply the functional constraint by 2 to obtain v=(2,4,3). In our case we have:
n 1 2 3 4 vn 2 4 3 1 wn 6 11 10 0
Solving the functional equation for s=0,1,2,...,6, in this order, we obtain.
f(0)=
f(1)=max {wn+f(1-vn ): vn≤1}= max {w 4 +f(1-v4)}=0 (Hence N(1)={4}).
f(2)=max {wn+f(2-vn ): vn≤2}= max {w 1 +f(2-v1),w 4 +f(2-v 4 )}
= max {6+f(2-2),0+f(2-1)}= max {6+0,0+0}= 6 (Hence N(2)={1}).
f(3)=max {wn+f(3-vn ): vn≤3}= max {w 1 +f(3-v1),w 3 +f(3-v 3 ),w 4 +f(3-v 4 )}
= max {6+f(3-2),10+f(3-3),0+f(3-1)} = max {6+0,10+0,0+6}
=10 (Hence N(3)={3}).
f(4)=max {wn+f(4-vn ): vn≤4} = max {w 1 +f(4-v 1 ),w 2 +f(4-v 2 ),w 3 +f(4- v3 ),w 4 +f(4-v4 )}
= max {6+f(4-2),11+f(4-4),10+f(4-3),0+f(4-1)}
= max {6+6,11+0,10+0,0+10}
=12 (Hence N(4)={1}).
f(5)=max {wn+f(5-vn ): vn≤5} = max {w 1 +f(5-v 1 ),w 2 +f(5-v 2 ),w 3 +f(5- v3 ),w 4 +f(5-v4 )}
= max {6+f(5-2),11+f(5-4),10+f(5-3),0+f(5-1)}
= max {6+10,11+0,10+6,0+12}
=16 (Hence N(5)={1,3}).
f(6)=max {wn+f(6-vn ): vn≤6} = max {w 1 +f(6-v 1 ),w 2 +f(6-v 2 ),w 3 +f(6-
v3 ),w 4 +f(6-v4 )}
= max {6+f(6-2),11+f(6-4),10+f(6-3),0+f(6-1)}
= max {6+12,11+6,10+10,0+16}
=20 (Hence N(6)={3}).
The recovery procedure yields the following:
s=
x=(0,0,0,0)
N(6)={3}
x=(0,0,1,0)
Thus, for s=13 the maximum weight is z=41 and the optimal solution is: x=(1,0,5).
4.. Write down a dynamic programming functional equation for generic knapsack problems of the form:
z *^ := max x z = 28 x 1 + 11 x 2 + 6 x 3
s. t. 4 x 1 + 6 x 2 + 3 x 3 = V x 1 , x 2 , x 3 ∈{0,1,2,3,}
where V is a given positive number. Then use the functional equation to compute optimal solutions for V= and V=7. In each case you are expected to determine the optimal value of z as well as the optimal values of x 1 ,
x2 and x 3.
Solution: Let f(s) = max weight of knapsack of volume s (completely full), s =0,1,2,...,V.
Then, assuming that we have an item with vj=1, the DP functional equation is as follows:
f(s) = max {wj + f(s-vj): vj<=s} , s=1,2,3,…,V
with f(0)=0.
Since we do not have an item with vj=1, we add an artificial variable, recalling that we have an “=” constraint. Thus, we set N=4, with v 4 =1 and w 4 =-M, where M is a very large number. Thus, the full range of items is as follows:
j 1 2 3 4 wj 28 11 6 -M vj 4 6 3 1
Solving for s=1,2,3, …, 7, - in this order - yields the following results:
f(1) = max {wj + f(1-vj): vj <= 1} = max {w 4 + f(1-v 4 )} = max {w 4 +f(1-1)} = max {w 4 + f(0)} = max {-M+0} = -M , N(1)={4}
f(2) = max {wj + f(2-vj): vj <= 2} = max {w 4 + f(2-v 4 )} = max {w 4 +f(2-1)} = max {w 4 + f(1)} = max {-M+(-M)} = -2M , N(1)={4}
f(3) = max {wj + f(3-vj): vj <= 3} = max { w 3 + f(3-v 3 ) , w 4 + f(3-v 4 )} = max { w 3 +f(3-3) , w 4 +f(3-1) } = max { w 3 + f(0) , w 4 + f(2) } = max { 6 + f(0) , -M+(-2M)} = max {6+0, -3M} = 6 , N(3)={3}
f(4) = max {wj + f(4-vj): vj <= 4} = max { w 1 + f(4-v 1 ) , w 3 + f(4-v 3 ) , w 4 + f(4-v 4 )} = max { w 1 +f(4-4) , w 3 +f(4-3) , w 4 +f(4-1) } = max { w 1 + f(0) , w 3 + f(1) , w 4 + f(3) } = max { w 1 + f(0) , w 3 + f(1) , w 4 + f(3) } = max { w 1 + 0 , w 3 + (-M) , w 4 + 6 } = max { 28 + 0 , 6 + (-M) , -M + 6 } = max { 28 , -M+6, -M+6} = 28, N(4)={1}
f(5) = max {wj + f(5-vj): vj <= 5} = max { w 1 + f(5-v 1 ) , w 3 + f(5-v 3 ) , w 4 + f(5-v 4 )} = max { w 1 +f(5-4) , w 3 +f(5-3) , w 4 +f(5-1) } = max { w 1 + f(1) , w 3 + f(2) , w 4 + f(4) } = max { w 1 + (-M) , w 3 + (-2M) , w 4 + 6 } = max { 28-M, 6-M , -M+ 6 } = 28-M, N(5)={1}
f(6) = max {wj + f(6-vj): vj <= 6} = max { w 1 + f(6-v 1 ) , w 2 + f(6-v 2 ) , w 3 + f(6-v 3 ) , w 4 + f(6-v 4 )} = max { w 1 +f(6-4) , w 2 +f(6-6) , w 3 +f(6-3) , w 4 +f(6-1) } = max { w 1 + f(2) , w 2 +f(0) , w 3 + f(3) , w 4 + f(5) } = max { w 1 + (-2M) , w 2 +0 , w 3 + 6 , w 4 + 28-M } = max { 28-2M, 11+0 , 6+6, -M+28-M } = max {28-2M, 11 , 12, 28-2M} = 12 , N(5)={3}
f(7) = max {wj + f(7-vj): vj <= 7} = max { w 1 + f(7-v 1 ) , w 2 + f(7-v 2 ) , w 3 + f(7-v 3 ) , w 4 + f(7-v 4 )} = max { w 1 +f(7-4) , w 2 +f(7-6) , w 3 +f(7-3) , w 4 +f(7-1) } = max { w 1 + f(3) , w 2 +f(1) , w 3 + f(4) , w 4 + f(6) } = max { w 1 + 6 , w 2 -M , w 3 + 28 , w 4 + 12 } = max { 28+6, 11-M , 6+28, -M+12} = max {34,11-M,34,-M+12}= 34, N(7)={1,3}
We therefore conclude that for V=5 there is no feasible solution (a positive artificial variable), and that for V= the optimal weight is equal to f(7)=34.
Recovery procedure:
X(0)^ =(0,0,0,0), s=7, N(7)={1,3) (we shall use j=1) X(1)^ =(1,0,0,0), s=s-v 1 =7-4=3, N(3)={3} X(2)^ =(1,0,1,0), s=s-v 3 =3-3=
Thus, the optimal solution is x=(1,0,1,0). This indeed yield a total weight equal to w 1 +w 3 =28+6=34. This mixture also satisfies the volume constraint: v 1 +v 3 = 4+3=7.
Remark:
Recovery of optimal solution:
x(0) = (0,0,0) , s(0) = 10 , K(10) = {1} x(1) = (1,0,0) , s(1) = 6 , K(6) = {1} x(2) = (2,0,0) , s(2) = 2 , K(2) = empty
Thus, the optimal solution is: x = (2,0,0) yielding z*=56.
The additional item type is dominated by the first item type so its availability will have no impact on the optimal solution.
6. Write down the generic knapsack problem we used in class and the dynamic programming functional equation associated with it. Modify this functional equation to handle the case where opt=min and use it to solve the following knapsack problem:
z *^ := min x z = 6 x 1 + 11 x 2 + 10 x 3
s. t. x 1 + 2 x 2 + 1.5 x 3 = 3 x 1 , x 2 , x 3 ∈{0,1,2,3,}
You are expected to determine the optimal value of z as well as the optimal values of x1 , x2 and x 3.
Solution:
The generic knapsack problem studied in class is as follows:
z * := max w (^) j j = 1
n
∑ x^ j
s. t
v (^) j j = 1
n
∑ x^ j ≤^ V
x (^) j ∈{0,1,2,...}
The functional equation is:
f ( s ) = max j v (^) j ≤ s
{ w (^) j + f ( s − v (^) j )}, s = 1,2,3,..., V
with f(s):=0.
To cope with opt=min, we simply change max to min in the functional equation. We shall multiply the volume constraint by 2 to make all the coefficients integers. The constraint is thus,
2x 1 + 4x 2 + 3x 3 = 6
so that v=(2,4,3) and v=(6,11,10). Because of the equality constraint, we first set f(0)=0, and f(s) = M, for s< volume of smallest item. In our case, f(1)=M. Hence we start with
f(0) = f(1)= M
and then we solve the functional equation for s=2,3,4,5,6.
The results are as follows:
f(2) =min {w(j)+f(2-v(j): j=1} = w(1) + f(2-v(1) ) = 6 + f(0)=6, K(2)={1}
f(3) =min {w(j)+f(3-v(j): j=1,3} = min {w(1) + f(3-v(1) ), w(3)+f(3-v(3)} = min{ 6 + M, 10+0} = 10, K(3)={3}
f(4) =min {w(j)+f(4-v(j): j=1,2,3} = min {w(1) + f(4-v(1), w(2)+f(4-w(2), w(3)+f(4-v(3)} = min{ 6 + 6, 11 + 0, 10+M} = 11, K(4)={2}
f(5) =min {w(j)+f(5-v(j): j=1,2,3} = min {w(1) + f(5-v(1)), w(2)+f(5-w(2)), w(3)+f(5-v(3))} = min{ 6 + 10, 11 + M, 10+6} = 16, K(5)={1,3}
f(6) =min {w(j)+f(6-v(j): j=1,2,3} = min {w(1) + f(6-v(1)), w(2)+f(6-w(2)), w(3)+f(6-v(3))} = min{ 6 + 11, 11 + 6, 10+10} = 17, K(6)={1,2}
Recovery:
x(0)^ =(0,0,0), s=6, K(6)={1,2}, 6-v1= x(1)^ =(0,1,0), s=4, K(4)={2}, 2-v2= x(2)^ =(1,1,0)
Thus, x=(1,1,0) and z=w(1)+w(2)=6 + 11 = 17.
7. Write down the generic dynamic programming functional equation for the Towers of Hanoi problem (2%) and then solve it for the case where there are 4 discs that must be moved from right to left (3%).
Solution :
The generic dynamic programming functional equation for the towers of Hanoi problem is as follows:
S(n,x,y) = S(n-1,x,not(x,y)),S(1,x,y),S(n-1,not(x,y),y)
Where S(n,x,y) is the solution to a problem requiring moving n discs from platform x to platform y and not(x,y) is the third platform (neither x nor y).
Thus, for our problem, n=4, x=R, y = L, so
S(4,R,L) = S(3,R,C),S(1,R,L),S(3,C,L)
Similarly,
S(3,R,C) = S(2,R,L),S(1,R,C),S(2,L,C)
S(3,C,L) = S(2,C,R),S(1,C,L),S(2,R,L)
S(2,R,L) = S(1,R,C),S(1,R,L),S(1,C,L)
i=B f(B, {C}) = min {d(B,C) + f(C,φ)} = d(B,C) + d(C,A) = 2 + 4 = 6 f(B, {D}) = min {d(B,D) + f(D,φ)} = d(B,D) + d(D,A) = 4 + 3 = 7 i=C f(C, {B}) = min {d(C,B) + f(B,φ)} = d(C,B) + d(B,A) = 3 + 3 = 6 f(C, {D}) = min {d(C,D) + f(D,φ)} = d(C,D) + d(D,A) = 2 + 3 = 5 i=D f(D, {B}) = min {d(D,B) + f(B,φ)} = d(D,B) + d(B,A) = 2 + 3 = 5 f(D, {C}) = min {d(D,C) + f(C,φ)} = d(D,C) + d(C,A) = 1 + 4 = 5
|s| =2: i=B f(B,{C,D}) = min {d(B,C) + f(C,{D}), d(B,D) + f(D,{C})} = min {2+5, 4+5} = 7 i=C f(C,{B,D}) = min {d(C,B) + f(B,{D}), d(C,D) + f(D,{B})} = min{3+7,2+5} = 7 i=D f(D,{B,C}) = min {d(D,B) + f(B,{C}), d(D,C) + f(C,{B})} =min {2+6,1+6}=
|s| = 3
f(A;{B,C,D}) = min {d(A,B)+f(B,{C,D} , d(A,C)+f(C,{B,D} , d(A,D)+f(D,{B,C}} = min {2+7 , 4+7, 3+ 7} = 9
Hence, the shortest tour (A,B,C,D) is of length 9.
9. Write down the generic functional equation of dynamic programming for the Towers of Hanoi problem (3%). Then use it to show that the solution it yields requires 2n-1 moves (where n is the number of discs).
Solution:
The generic DP functional equation of the Tower of Hanoi problems is as follows:
S(n,x,y) = S(n-1,x,xy), S(1,x,y), S(n-1,xy,y) , (xy := neither x nor y).
Thus, if we let f(n):= number of moves required for the solution of a problem with n discs, we have
f(n) = f(n-1)+ 1 + f(n-1) = 1 + 2f(n-1) (1)
Since f(1) =1, it follows that f(2) = 3, and therefore f(3) = 1+6 = 7. We can show, by induction on j that f(j) = 2 j-1.
Clearly, the inductive hypothesis is true for j=1 and j=2. So suppose that it is true for j=1,2,…,k. In particular, assume that f(k) = 2k-1. Then it follows from (1) and the inductive hypothesis that f(k+1) = 1 + 2(2k-1) = 1 + 2 k+1^ - 2 = 2 k+1^ - 1. Hence, the inductive hypothesis holds for j=k+1 as well and therefore it holds for all j=1,2,….
10. Write down the generic dynamic programming functional equation for the standard knapsack problem. Then solve it for the following specific knapsack problem:
z * := max x
7 x 1 + 15 x 2 + 10 x 3
s. t. 4 x 1 + 8 x 2 + 6 x 3 ≤ 13 x 1 , x 2 , x 3 ∈{0,1,2,...}
You are expected to determine the value of z* as well as the optimal values of the decision variables x 1 , x 2 and x 3. What would be the functional equation if the functional constraint is changed to 4 x 1 + 8 x 2 + 6 x 3 = 13 and
max is changed to min?
Solution.
The generic dynamic programming functional equation for the knapsack problem is
f ( s ) = max 1 ≤ j ≤ k { v (^) j + f ( s − w (^) j )}, w ≤ s ≤ W
with f(0) = 0, s<w
where w is the weight of the lightest item and W is the total (weight) capacity of the knapsack, and f(s) is the
maximum value of a knapsack of total weight capacity of s.
Solving this functional equation for the problem under consideration yields the following results:
s f(s) D*(s) 0 0 1 0 2 0 3 0 4 7 {1} 5 7 {1} 6 10 {3} 7 10 {3} 8 15 {2} 9 15 {2} 10 17 {1,3} 11 17 {1,3} 12 22 {1,2} 13 22 {1,2}
For example, consider s=13:
f(13) = max {v1 + f(13-w1), v2 + f(13-w2) , v3 + f(13-w3) }
= max {7+f(13-4),15+f(13-8),10+f(13-6)}
= min { 4 + 5 } = 9
f(3,{2}) = min { d(3,x) + f(x,{2}{x}): x in {2}}
= min { d(3,2) + f(2,{}) } = min { 4 + 3 } = 7
f(3,{4}) = min { d(3,x) + f(x,{4}{x}): x in {4}}
= min { d(3,4) + f(4,{}) } = min { 1 + 5 } = 6
f(4,{2}) = min { d(4,x) + f(x,{2}{x}): x in {2}}
= min { d(4,2) + f(2,{}) } = min { 3 + 3 } = 6
f(4,{3}) = min { d(4,x) + f(x,{3}{x}): x in {3}}
= min { d(4,3) + f(3,{}) } = min { 1 + 4 } = 5
f(2,{3,4}) = min { d(2,x) + f(x,{3,4}{x}): x in {3,4}}
= min { d(2,3) + f(3,{4}) , d(2,4) + f(4,{3}) } = min { 2 + 6 , 4 + 5 } = 8
f(3,{2,4}) = min { d(3,x) + f(x,{2,4}{x}): x in {2,4}}
= min { d(3,2) + f(2,{4}) , d(3,4) + f(4,{2}) } = min { 4 + 9 , 1 + 6 } = 7
f(4,{2,3}) = min { d(4,x) + f(x,{2,3}{x}): x in {2,3}}
= min { d(4,2) + f(2,{3}) , d(4,3) + f(3,{2}) } = min { 3 + 6 , 1 + 7 } = 8
f(1,{2,3,4}) = min { d(1,2) + f(2,{,3,4}) , d(1,3) + f(3,{2,4}) , d(1,4) + f(4,{2,3}) }
= min { 2 + 8 , 4 + 7 , 3 + 8 } = min { 10,11,11 } = 10 so the shortest tour takes 10 units of time.
Recovering the optimal decisions we obtain the optimal tour (1,2,3,4,1). Checking the length of this tour yields
2 + 2 + 1 + 5 =10 which is consistent with the value of f(1,{2,3,4})=10.
