Download Knapsack Problem - Advanced Algorithms - Lecture Notes and more Study notes Advanced Algorithms in PDF only on Docsity!
Lecture 5: The Knapsack Problem
1 The Knapsack Problem Defined
Suppose we are trying to burgle someone’s house. In this house we find the set
S = a 1 , a 2 ,... , an− 1 , an a collection of objects s 1 , s 2 ,... , sn− 1 , sn their sizes p 1 , p 2 ,... , pn− 1 , pn their profits
that we would like to put in our knapsack of capacity B.
Our goal is to find a subset of these objects, whose total size is bounded by B and whose profit is maximized. We can view this in terms of indicator variables, where
xi =
1 if ai is chosen 0 otherwise
Then we need to find a setting of these xis to maximize
pixi. This would be a very simple linear program, but we note that we have an integer constraint, since no fence will pay you one third of the price if you bring them one third of a piccolo. So the Knapsack Problem is an integer program that is NP-Hard. We would like to find a polynomial time approximation, so that we can leave the house we are burgling before the residents get home.
2 Running example
We will refer to this example throughout the remainder of the notes.
Objects A B C D E Sizes 7 2 9 3 1 Profits 3 2 3 1 2
Knapsack size: B
3 A First Guess at Approximation
For each i, we consider the profit-to-size ratio
pi si
, and reorder the objects
such that (^) p 1 s 1
p 2 s 2
pn sn Referring back to the running example, this yields
Objects A B C D E Profits/Sizes 3 / 7 1 1 / 3 1 / 3 2 Order a 3 a 2 a 4 a 5 a 1
Intuitively, this allows us to see the “profit density”, or profit per unit size.
Algorithm 1 (Greedy): pick the first k objects greedily in this order until the next object ak+1 will not fit in the knapsack.
We can construct a simple example to show that this is not optimal:
100 1
((100 ∗ B) − 1)
B
Chosen by Algorithm 1 More profitable overall
Note that by setting the profit appropriately, we can generalize this example to make the approximation ratio arbitrarily bad.
4 Improving the Algorithm
We can modify the greedy algorithm to compensate for this issue:
5 How close can we get to OPT?
We now show that no algorithm gets within an additive factor of OPT unless P = N P. In the next section, we then show that we can get arbitrarily close to optimal by a multiplicative factor.
Theorem 5.1 If P 6 = N P , no polynomial time approximation algorithm can solve Knapsack with value P ∗^ − k for any fixed constant k.
Proof: By contradiction.
Assume there exists a polynomial time algorithm A with performance guar- antee k (an integer) for all instances of Knapsack. That is, for any Knapsack instance I, A(I) = P ∗^ − k.
We show that A can be used to construct a solution to Knapsack with value P ∗^ in polynomial time.
Suppose we have an instance of Knapsack:
I = {〈ai, pi, si〉} for i = 1... n
Let I′^ = {〈a′ i, p′ i, s′ i〉} where a′ i = ai, p′ 1 = pi · (k + 1) and s′ i = si.
Note: a feasible solution for I corresponds to a feasible solution for I′, and the value of a solution for I′^ is exactly (k+1) times the value of the corresponding solution for I.
Let M = A(I)
If we run A on I′, the solution satisfies the following:
|A(I′) − P ∗(I′)| ≤ k ⇒ |M · (k + 1) − P ∗(I) · (k + 1)| ≤ k
⇒ |M − P ∗(I)| ≤
k k + 1
Since
k k + 1
≤ 1 and M differs from P ∗(I) by an integral factor,
|M − P ∗(I)| ≤ 0
Which implies that M is an optimal solution for I, which we demonstrated was impossible above. ⇒⇐
6 Approximation Schemes
Definition 6.1 Let π be an optimization problem with objective function fπ and optimal solution S∗. We say that A is an approximation scheme for π if on input (I, ) where I is an instance of π and > 0 is a fixed error parameter, it outputs a solution S such that
fπ(I, S) ≤ (1 + ) · S∗^ if π is minimization problem fπ(I, S) ≥ (1 − ) · S∗^ if π is maximization problem
Definition 6.2 A is a PTAS (Polynomial Time Approximation Scheme) if it is an approximation scheme where for each fixed > 0 , its running time is polynomial in the size of instance I.
Notice: “polynomial time” can depend arbitrarily badly on :
n^3 · 22
2 1 or n^3 ·^2
(^1)
Definition 6.3 A is an FPTAS (Fully Polynomial Time Approxima- tion Scheme) if in the previous scheme you require that the running time of A be bounded by a polynomial in the size of I and 1 /.
Remark: If P 6 = N P , an FPTAS is the best you can do for an NP-Hard problem.
Profit p 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Count
i
Using the recurrence relation: A(i + 1, p) = min(A(i, p), A(i, p − pi+1) + si+1), it takes constant work to fill in each cell, thus costing O(n^2 · Pmax) to fill the table.
Profit p 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Count
i
8 What to Do When Profits are Too Large
When Pmax is bounded by a polynomial in the size of the input, then the above is a polynomial time algorithm for Knapsack. When it is not, we now present a (1 − )–approximation algorithm, by rounding the profit values.
Algorithm 3 (FPTAS):
- Given an > 0, let k =
· Pmax n
- For each ai, define p′ i =
⌊p i k
- Let I′^ = (〈a′ i, s′ i, p′ i〉) where a′ i = ai, s′ i = si, and p′ i as above.
- Using the dynamic programming method from the previous section, find the most profitable set S′.
- Output max(Smax, S′) where Smax is the smallest object of profit Pmax if Smax ≤ B otherwise 0.
Lemma 8.1 Let A denote the set output by Algorithm 3. Then Profit(A) ≥ (1 − ) · P ∗.
Proof: Let O denote the set with profit P ∗. For any ai, k · p′ i can be smaller than pi, but not by more than k.
⇒ Profit(O) − k · Profit′(O) ≤ n · k
Under the Profit′^ scheme, Profit′(S′) is OPTIMAL, i.e. Profit′(S′) ≥ Profit′(Y ) for all Y , and in particular Profit′(S′) ≥ Profit′(O).
Thus:
Profit(S′) ≥ k · Profit′(S′) ≥ k · Profit′(O) ≥ Profit(O) − n · k = P ∗^ − · Pmax
Since the algorithm also considers the most profitable element
Profit(A) ≥ Profit(Pmax) Profit(A) ≥ Profit(S′) ≥ P ∗^ − · Profit(A)s
Theorem 8.2 Algorithm 3 is an FPTAS for Knapsack.
Proof: By lemma, the solution P ∗^ is within (1 − ) of OPT.
The running time is O
n^2
Pmax k
= O
n^2
⌊n
, which is polynomial in
n and 1/.
