Knapsack Problem - Introduction to Algorithms - Lecture Slides, Slides of Computer Science

Download Knapsack Problem - Introduction to Algorithms - Lecture Slides and more Slides Computer Science in PDF only on Docsity!

Algorithms

Dynamic programming

0-1 Knapsack problem

Review: Dynamic programming

• DP is a method for solving certain kind of problems
• DP can be applied when the solution of a problem includes solutions to subproblems
• We need to find a recursive formula for the solution
• We can recursively solve subproblems, starting from the trivial case, and save their solutions in memory
• In the end we’ll get the solution of the whole problem

Review: Longest Common Subsequence (LCS)

• Problem: how to find the longest pattern of characters that is common to two text strings X and Y
• Dynamic programming algorithm: solve subproblems until we get the final solution
• Subproblem: first find the LCS of prefixes of X and Y.
• this problem has optimal substructure : LCS of two prefixes is always a part of LCS of bigger strings

Review: Longest Common Subsequence (LCS) continued

• Define X (^) i , Y (^) j to be prefixes of X and Y of length i and j; m = |X|, n = |Y|
• We store the length of LCS(Xi , Y (^) j ) in c[i,j]
• Trivial cases: LCS(X 0 , Y (^) j ) and LCS(Xi , Y 0 ) is empty (so c[0,j] = c[i,0] = 0 )
• Recursive formula for c[i,j]:





 − −

− − + =

max( [ , 1 ], [ 1 , ]) otherwise

[ 1 , 1 ] 1 if [ ] [ ], [ , ] c i j c i j

c i j x i y j c i j

c[m,n] is the final solution

0-1 Knapsack problem

• Given a knapsack with maximum capacity W ,

and a set S consisting of n items

• Each item i has some weight w (^) i and benefit

value b (^) i (all w (^) i , b (^) i and W are integer values)

• Problem: How to pack the knapsack to achieve

maximum total value of packed items?

0-1 Knapsack problem: a picture

W = 20

w (^) i b^ i

9 10

5 8

4 5

3 4

2 3

Weight Benefit value

This is a knapsack Max weight: W = 20

Items

0-1 Knapsack problem: brute- force approach

Let’s first solve this problem with a straightforward algorithm

• Since there are n items, there are 2 n^ possible combinations of items.
• We go through all combinations and find the one with the most total value and with total weight less or equal to W
• Running time will be O(2 n)

0 - 1 Knapsack problem: brute- force approach

• Can we do better?
• Yes, with an algorithm based on dynamic programming
• We need to carefully identify the subproblems

Let’s try this: If items are labeled 1..n , then a subproblem would be to find an optimal solution for S (^) k = {items labeled 1, 2, .. k}

Defining a Subproblem

Max weight: W = 20 For S4 : Total weight: 14; total benefit: 20

w 1 = b 1 =

w 2

b 2 =

w 3 = b 3 =

w 4 = b 4 =4 w^ i b^ i

10

5 8

4 5

3 4

2 3

Weight Benefit

9

Item

4

3

2

1

5

S 4 S 5

w 1 = b 1 =

w 2

b 2 =

w 3 = b 3 =

w 4 = b 4 = For S 5 : Total weight: 20 total benefit: 26

Solution for S 4 is not part of the solution for S 5 !!!

?

Defining a Subproblem (continued)

• As we have seen, the solution for S 4 is not part of the solution for S 5
• So our definition of a subproblem is flawed and we need another one!
• Let’s add another parameter: w , which will represent the exact weight for each subset of items
• The subproblem then will be to compute B[k,w]

Recursive Formula

• The best subset of S (^) k that has the total weight w, either contains item k or not.
• First case: w (^) k>w. Item k can’t be part of the solution, since if it was, the total weight would be > w , which is unacceptable
• Second case: w (^) k <=w. Then the item k can be in the solution, and we choose the case with greater value





 − − − +

− >

max{ [ 1 , ], [ 1 , ] } else

[ 1 , ] if [ , ] k k

k B k w B k w w b

B k w w w B k w

0-1 Knapsack Algorithm

for w = 0 to W

B[0,w] = 0

for i = 0 to n

B[i,0] = 0 for w = 0 to W if wi <= w // item i can be part of the solution if b (^) i + B[i-1,w-w (^) i] > B[i-1,w] B[i,w] = b (^) i + B[i-1,w- w (^) i] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Example

Let’s run our algorithm on the

following data:

n = 4 (# of elements)

W = 5 (max weight)

Elements (weight, benefit):

Example (2)

for w = 0 to W B[0,w] = 0

0 0 0 0

0

0

W 0 1 2 3

4

5

i (^0 1 2 )

4