Download Dynamic Programming Solution for 0-1 Knapsack Problem and more Study notes Algorithms and Programming in PDF only on Docsity!
1
Dynamic programming
0-1 Knapsack problem
Dr. Steve Goddard
http://www.cse.unl.edu/~goddard/Courses/CSCE310J
CSCE 310J
Data Structures & Algorithms
2
Giving credit where credit is due:
» Most of slides for this lecture are based on slides created by Dr. David Luebke, University of Virginia. » Some slides are based on lecture notes created by Dr. Chuck Cusack, UNL. » I have modified them and added new slides.
CSCE 310J Data Structures & Algorithms
3
Another strategy for designing algorithms is
dynamic programming
» A metatechnique, not an algorithm (like divide & conquer) » The word “programming” is historical and predates computer programming
Use when problem breaks down into recurring
small subproblems
Dynamic Programming
4
Dynamic programming
It is used when the solution can be recursively
described in terms of solutions to subproblems
( optimal substructure ).
Algorithm finds solutions to subproblems and
stores them in memory for later use.
More efficient than “ brute-force methods ”, which
solve the same subproblems over and over again.
5
Summarizing the Concept of Dynamic Programming
Basic idea:
» Optimal substructure: optimal solution to problem consists of optimal solutions to subproblems » Overlapping subproblems: few subproblems in total, many recurring instances of each » Solve bottom-up, building a table of solved subproblems that are used to solve larger ones
Variations:
» “Table” could be 3-dimensional, triangular, a tree, etc.
6
Given some items, pack the knapsack to get
the maximum total value. Each item has some
weight and some value. Total weight that we can
carry is no more than some fixed number W.
So we must consider weights of items as well as
their value.
Item # Weight Value
Knapsack problem (Review)
7
Knapsack problem
There are two versions of the problem:
- “0-1 knapsack problem” and
- “Fractional knapsack problem”
1. Items are indivisible; you either take an item or
not. Solved with dynamic programming
2. Items are divisible: you can take any fraction of
an item. Solved with a greedy algorithm.
We have already seen this version
8
Given a knapsack with maximum capacity W , and
a set S consisting of n items
Each item i has some weight wi and benefit value
bi (all wi , bi and W are integer values)
Problem: How to pack the knapsack to achieve
maximum total value of packed items?
0-1 Knapsack problem
9
W = 20
wi bi
Weight Benefit value
This is a knapsack
Max weight: W = 20
Items
0-1 Knapsack problem: a
picture
10
Problem, in other words, is to find
∈ ∈
≤ i T
i iT
max bi subject to w W
0-1 Knapsack problem
The problem is called a “ 0-1 ” problem, because each item must be entirely accepted or rejected. In the “ Fractional Knapsack Problem ,” we can take fractions of items.
11
Let’s first solve this problem with a straightforward algorithm
Since there are n items, there are 2 n^ possible
combinations of items.
We go through all combinations and find the one
with maximum value and with total weight less or
equal to W
Running time will be O(2n)
0-1 Knapsack problem:
brute-force approach
12
Can we do better?
Yes, with an algorithm based on dynamic
programming
We need to carefully identify the subproblems
Let’s try this:
If items are labeled 1..n , then a subproblem
would be to find an optimal solution for
Sk = {items labeled 1, 2, .. k}
0-1 Knapsack problem:
brute-force approach
19
for w = 0 to W B[0,w] = 0 for i = 1 to n B[i,0] = 0 for i = 1 to n for w = 0 to W < the rest of the code > What is the running time of this algorithm?
O(W)
O(W)
Repeat n times
O(n*W) Remember that the brute-force algorithm takes O(2n)
Running time
20
Let’s run our algorithm on the
following data:
n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
Example
21
for w = 0 to W
B[0,w] = 0
i\W
Example (2)
22
for i = 1 to n
B[i,0] = 0
i\W
Example (3)
23
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Items:
i=
bi=
wi=
w=
w-wi =-
i\W
Example (4)
24
Items:
i\W i=
bi=
wi=
w=
w-wi =
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Example (5)
25
Items:
i\W i=
bi=
wi=
w=
w-wi =
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Example (6)
26
Items:
i\W i=
bi=
wi=
w=
w-wi =
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Example (7)
27
Items:
i\W i=
bi=
wi=
w=
w-wi =
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Example (8)
28
Items:
i\W i=
bi=
wi=
w=
w-wi =-
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Example (9)
29
Items:
i\W i=
bi=
wi=
w=
w-wi =-
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Example (10)
30
Items:
i\W i=
bi=
wi=
w=
w-wi =
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Example (11)
37
Items:
i\W i=
bi=
wi=
w= 5
w- wi=
if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w
Example (18)
38
Comments
This algorithm only finds the max possible value
that can be carried in the knapsack
» I.e., the value in B[n,W]
To know the items that make this maximum value,
an addition to this algorithm is necessary.
39
All of the information we need is in the table.
B [ n , W ] is the maximal value of items that can be
placed in the Knapsack.
Let i=n and k=W
if B [ i,k ] ≠ B [ i − 1,k ] then
mark the i th^ item as in the knapsack
i = i − 1 , k = k-wi
else
i = i − 1 // Assume the i th^ item is not in the knapsack
// Could it be in the optimally packed knapsack?
How to find actual Knapsack Items
40
Items:
i\W (^) i= k= 5 bi= wi= B [ i,k ] = 7 B [ i − 1,k ] =
i=n, k=W while i,k > 0 if B [ i,k ] ≠ B [ i − 1,k ] then mark the i th^ item as in the knapsack i = i − 1 , k = k-wi else i = i − 1
Finding the Items
41
Items:
i\W (^) i= k= 5 bi= wi= B [ i,k ] = 7 B [ i − 1,k ] =
i=n, k=W while i,k > 0 if B [ i,k ] ≠ B [ i − 1,k ] then mark the i th^ item as in the knapsack i = i − 1 , k = k-wi else i = i −− −− 1
Finding the Items (2)
42
Items:
i\W (^) i= k= 5 bi= wi= B [ i,k ] = 7 B [ i − 1,k ] =
i=n, k=W while i,k > 0 if B [ i,k ] ≠ B [ i − 1,k ] then mark the i th^ item as in the knapsack i = i − 1 , k = k-wi else i = i −−−− 1
Finding the Items (3)
43
Items:
i\W (^) i= k= 5 bi= wi= B [ i,k ] = 7 B [ i − 1,k ] = k − wi=
i=n, k=W while i,k > 0 if B [ i,k ] ≠ B [ i − 1,k ] then mark the i th^ item as in the knapsack i = i − 1 , k = k-wi else i = i −− −− 1
Finding the Items (4)
44
Items:
i\W (^) i= k= 2 bi= wi= B [ i,k ] = 3 B [ i − 1,k ] = k − wi=
i=n, k=W while i,k > 0 if B [ i,k ] ≠ B [ i − 1,k ] then mark the i th^ item as in the knapsack i = i − 1 , k = k-wi else i = i −−−− 1
Finding the Items (5)
45
Items:
i\W
3 3 3 3 0 3 4 4 7 0 3 4
i=n, k=W while i,k > 0 if B [ i,k ] ≠ B [ i − 1,k ] then mark the n th^ item as in the knapsack i = i − 1 , k = k-wi else i = i − 1
i= k= 0
The optimal knapsack should contain {1, 2}
Finding the Items (6)
46
Items:
i\W
3 3 3 3 0 3 4 4 7 0 3 4
i=n, k=W while i,k > 0 if B [ i,k ] ≠ B [ i − 1,k ] then mark the n th^ item as in the knapsack i = i − 1 , k = k-wi else i = i − 1
The optimal knapsack should contain {1, 2}
Finding the Items (7)
47
Review: The Knapsack Problem And Optimal Substructure
Both variations exhibit optimal substructure
To show this for the 0-1 problem, consider the
most valuable load weighing at most W pounds
» If we remove item j from the load, what do we know about the remaining load? » A: remainder must be the most valuable load weighing at most W - wj that thief could take, excluding item j
48
Solving The Knapsack
Problem
The optimal solution to the fractional knapsack
problem can be found with a greedy algorithm
» Do you recall how? » Greedy strategy: take in order of dollars/pound
The optimal solution to the 0-1 problem cannot be
found with the same greedy strategy
» Example: 3 items weighing 10, 20, and 30 pounds, knapsack can hold 50 pounds Suppose item 2 is worth $100. Assign values to the other items so that the greedy strategy will fail
