Download Randomized Rounding and Concentration Bounds in Set Cover Problem and more Study notes Advanced Algorithms in PDF only on Docsity!
CS787: Advanced Algorithms Scribe: Archit Gupta, Pavan Kuppili Lecturer: Shuchi Chawla Topic: Randomized rounding, concentration bounds Date: 10/10/
12.1 Set Cover
Let E = {e 1 , e 2 , .., en} be a set of n elements. Let S 1 , S 2 , .., Sm be subsets of E with associated costs c 1 , .., cm. As explained in a previous lecture, the problem of set cover is to choose 1 or more of the m subsets such that the union of them cover every element in E, and the objective is to find the set cover with the minimum cost. In a previous lecture, we have seen a greedy solution to the set cover which gave a log n approximation. In this lecture, we will see a linear programming solution with randomized rounding to achieve an O(logn) approximation.
12.1.1 ILP formulation of set cover
Let Xi be a random variable associated with each subset Si. Xi = 1, if Si is in the solution, and 0 otherwise.
The ILP formulation: Minimize
∑m i=1 ciXi^ s.t ∀e ∈ E,
i:e∈Si Xi^ ≥^ 1 - (1) ∀Xi, Xi ∈ { 0 , 1 } - (2)
The constraints in (1) ensure that every element is present in atleast one of the chosen subsets. The constraints in (2) just mean that every subset is either chosen or not. The objective function chooses a feasible solution with the minimum cost.
It can be noted that this problem formulation is a generalization of the vertex cover. If we map every edge to an element in E, and every vertex to one of the the subsets of E, the vertex cover is the same as the set cover problem with the additional constraint that every element e appears in exactly 2 subsets (corresponding to the 2 vertices it is incident on).
12.1.2 Relaxing to an LP problem
Instead of Xi ∈ { 0 , 1 }, relax the constraint so that Xi is in the range [0, 1]. The LP problem can be solved optimally in polynomial time. Now we need to round the solution back to an ILP solution.
12.1.3 Deterministic rounding
First, let us see the approximation we get by using a deterministic rounding scheme analogous to the one we used in the vertex cover problem.
Theorem 12.1.1 We have a deterministic rounding scheme which gives us an F-approximation
to the solution, where F is the maximum frequency of an element.
Proof: Let F be the maximum frequency of an element, or the maximum number of subsets an element appears in. Pick a threshold t = (^) F^1. If x 1 , .., xm is the solution to the LP, the ILP solution is all subsets for which xi ≥ (^) F^1.
This results in a feasible solution because in any of the constraints in (1) we have atmost F variables on the LHS, and atleast one of them should be ≥ (^) F^1 , so we will set atleast one of those variables to 1 during the rounding. So, every element will be covered. It can be noted here that the vertex cover has F = 2, and hence we used a threshold of 12 during the previous lecture. If the values of X 1 , .., Xm are x ′ 1 , .., x
′ m after rounding,^ ∀i, x
′ i ≤^ F xi^ [since if^ xi^ ≥^
1 F we round it to 1, and 0 otherwise] So the cost of the ILP solution ≤ F(cost of the LP solution). So, this deterministic rounding gives us an F approximation to the optimal solution.
12.1.4 Randomized rounding
Algorithm: Step 1. Solve the LP. Step 2 (Randomized rounding). ∀ i, pick Si independently with probability xi (where xi is the value of Xi in the LP solution). Step 3. Repeat step 2 until all elements are covered.
The intution behind the algorithm is that higher the value of xi in the LP solution, the higher the probability of Si being picked during the random rounding.
Theorem 12.1.2 With a probability (1 − (^) n^1 ), we get a 2logn approximation.
Proof:
Lemma 12.1.3 The expected cost in each iteration of step 2 is the cost of the LP solution.
Proof: Let Yi be a random variable. Yi = ci if Si is picked, and 0 otherwise. Let Y =
∑n i=1 Yi, E[Y] =
∑n i=1 E[Yi] [by linearity of expectation] =
∑n i=1 cixi = cost of the LP solution. This is a nice result as the expected cost of the ILP solution is exactly equal to the cost of the LP solution.
Lemma 12.1.4 The number of iterations of step 2 is 2logn with a high probability (whp).
holds:
Pr[X ≥ λ] = Pr[f (X) ≥ f (λ)] ≤
E[f (X)] f (λ)
12.2.2 Chebyshev’s Inequality
We now study a tighter bound called the Chebyshev’s bound. Let f (X) = (X − E[X])^2. Note that f is an increasing function if X > E[X]. We have,
Pr[|X − E[X]| ≤ λ] = Pr
[
(X − E[X])^2 ≤ λ^2
]
E
[
(X − E[X])^2
]
λ^2
σ^2 (X) λ^2
That is, the deviation of X from E[X] is a function of its variance(σ(X)). If the variance is small, then we have a tight bound.
Also not that with probability p, X ∈ E[X] ±
1 p σ(X)
12.2.3 Chernoff’s bound
We present Chernoff’s bound which is tighter compared to the Chebyshev and Markov bounds.
Let random variables X 1 , X 2 , ..., Xn be independent and identically distributed random vari- ables, where Xi ∈ [0, 1]. Note that the class of indicator random variables lie in this category.
Let X =
∑n i=1 Xi. Also^ E[X] =^
∑n i=1 E[Xi] =^ μ
For any δ > 0,
Pr[X ≥ (1 + δ)μ] ≤
eδ (1 + δ)1+δ
)μ (12.2.6)
It can also be shown for δ ≥ 0,
Pr[X ≥ (1 + δ)μ] ≤ e −
“ (^) δ 2 2+δ
” μ (12.2.7)
Note that the above probability resembles a gaussian/bell curve. When 0 ≤ δ ≤ 1
Pr[X ≥ (1 + δ)μ] ≤ e−^
δ^2 3 μ^ (12.2.8)
Pr[X ≤ (1 − δ)μ] ≤ e−^ δ 22 μ (12.2.9)
Example: Coin tossing to show utility of bounds. We toss n independent unbiased coins. We want to find a t such that: Pr[No. of times we get heads in n tosses ≥ t] ≤ 1 /n.
We use indicator random variables to solve this: If the ith coin toss is heads, let Xi = 1, oth- erwise Xi = 0. Let X =
∑n i=1 Xi. Here^ X^ is the total number of heads we get in^ n^ independent tosses. Note that μ = E[X] =
∑n i=1 E[Xi] =^ n/2, since^ E[Xi] = 1/2 (The probability of getting a heads).
To get bounds on Pr[X ≥ t], we first apply Markov’s inequality:
Pr[X ≥ t] ≤
E[X]
t
n/ 2 t
Pr[X ≥ t] = 1/n ⇒
n/ 2 t = 1/n ⇒ t = n^2 / 2
But, we don’t do any more than n coin tosses, so this bound is not useful. Note that Markov’s bound is weak.
Applying Chebyshev’s inequality:
Pr[X ≥ t] ≤ Pr[|X − μ| ≥ t − μ] ≤ σ^2 (X) (t − μ)^2
Evaluating σ(X) where Xi ∈ { 0 , 1 }, E[Xi] = 1/ 2
σ^2 (Xi) = E
[
(Xi − E[Xi])^2
]
Since this is a sum of independent random variables, the variances can be summed together:
σ^2 (X) =
∑^ n
i=
σ^2 (Xi) = n/ 4
Evaluating t,
Pr[X ≥ t] ≤ σ^2 (X) (t − μ)^2
= 1/n
⇒ t = μ + n/2 = n
Again this bound is quite weak.
Applying Chernoff’s bound:
Pr[X ≥ (1 + δ)μ] ≤ e−^
δ 32 μ = e−^
δ 62 n
Taking,
e−^
δ^2 6 n^ = 1/n
