Download Local Search-Approximations Algorithms-Lecture 07 Notes-Computer Science and more Study notes Approximation Algorithms in PDF only on Docsity!
CS880: Approximations Algorithms Scribe: Chi Man Liu Lecturer: Shuchi Chawla Topic: Local Search: Max-Cut, Facility Location Date: 2/13/
In previous lectures we saw how dynamic programming could be used to obtain PTAS for certain NP-hard problems. In this lecture we will discuss local search and look at approximation algorithms for two problems — Max-Cut and Facility Location.
7.1 Local Search
Local search is a heuristic technique for solving hard optimization problems and is widely used in practice. Suppose that a certain optimization problem can be formulated as finding a solution maximizing an objective function in the solution space. A local search algorithm for this problem would start from an arbitrary or carefully chosen solution, then iteratively move to other solutions by making small, local changes to the solution, each time increasing the objective function value, until a local optimum is found.
We view the solution space as a graph where vertices are the candidate solutions. An edge exists between two solutions if we can move from one to the other by making some small changes. A solution is a global optimum if its objective function value is maximized (or minimized, depending on the context) over all candidate solutions. A solution is a local optimum if none of its neighbor solutions has a greater (or smaller) objective function value.
When we apply local search to solving optimization problems we require that the solution graph has low (polynomially bounded) vertex degrees, so that at each vertex only polynomial time is needed to find a “good” neighbor. In order to show that the algorithm runs in polynomial time, we need to show that starting from any solution we can reach a local optimum in a polynomial number of steps. It doesn’t suffice to show that the diameter of the solution graph is polynomial. We will see a few different arguments including a potential function based argument. And lastly, to show that the algorithm is indeed a good approximation algorithm, we must prove that any local optimum is nearly as good as a global optimum.
In the following sections we will see how local search can be used as approximation algorithms for two problems. The first problem we consider is the Max-Cut problem. The second problem is the Facility Location problem for which the analysis is a bit more involved. We will defer the last part of the analysis to the next lecture.
7.2 Max-Cut
7.2.1 The Problem
We first review the notion of a cut in graph theory.
Definition 7.2.1 Let G = {V, E} be a weighted graph where each edge e ∈ E has a weight we. A
cut is a partition of V into two subsets S and S′^ = V \S. We simply denote a cut by either one of its subsets. The value of a cut S is
c(S) =
(s,s′)∈E s∈S,s′∈S′
ws,s′.
Now we can define the Max-Cut problem on weighted graphs.
Definition 7.2.2 (The Max-Cut Problem) Given a weighted graph G = {V, E} where each edge e ∈ E has a positive integral weight we, find a cut in G with maximum cut value.
While the Min-Cut problem is polynomial-time solvable by reducing it to Maximum Flow, the Max-Cut problem is known to be NP-hard. It has been shown that approximating Max-Cut to a factor of 17/16 is still NP-hard. In the following we present a simple 2-approximation local search algorithm for Max-Cut.
7.2.2 The Algorithm
The algorithm we are going to describe is a straightforward application of local search. The only thing we need to specify is the small changes we are allowed to make in each step. Since each candidate solution is a just a partition of the vertex set V , the following local step seems natural: Two partitions S 1 and S 2 are joined by an edge if and only if |S 1 \S 2 | ≤ 1 and |S 2 \S 1 | ≤ 1. Intuitively this means moving one vertex from one subset to the other. Thus we obtained the following algorithm:
- Start with an arbitrary partition (for example, ∅).
- Pick a vertex v ∈ V such that moving it across the partition would yield a greater cut value.
- Repeat step 2 until no such v exists.
Before we move on to prove that this algorithm is 2-approximate, let us first analyze its running time. Each step involves examining at most |V | vertices and selecting one that increases the cut value when moved. This process takes O(|V |^2 ) time. Since we assume that edges have positive integral weights, the cut value is increased by at least 1 after each iteration. The maximum possible cut value is
e∈E we, hence there are at most such number of iterations. The overall time complexity of this algorithm is thus O(|V |^2
e∈E we). For the special case of unweighted graphs (all weights equal to 1), the time complexity becomes O(|V |^4 ), which is strongly polynomial in the input size.
Note. There exist modifications to the algorithm that give a strongly polynomial running time, with an additional term in the approximation coefficient. An example of such modifications will be shown when we discuss the Facility Location problem.
We now proceed to prove that the above algorithm is 2-approximate.
Theorem 7.2.3 The local search algorithm described above gives a 2-approximation to the Max- Cut problem.
7.3 Facility Location
7.3.1 The Problem
Suppose there is a set of customers and a set of facilities serving these customers. Our goal is to open some facilities and assign each customer to the nearest opened facility. Each facility has an opening cost. Each customer has a routing cost, which is proportional to the distance (see the next paragraph) traveled. The Facilitiy Location problem asks for a subset of facilities to be opened such that the total cost (opening costs plus routing costs) is minimized. While the aforementioned “distance” could mean geographic distance, in general any metric would do. Recall the definition of a metric:
Definition 7.3.1 Let M be a set. A metric on M is a function d : M × M → R that satisfies the following three properties:
- d(x, y) ≥ 0 ; and d(x, y) = 0 iff x = y (non-negativity)
- d(x, y) = d(y, x) (symmetry)
- d(x, z) ≤ d(x, y) + d(y, z) (triangle inequality)
The formal definition of the the Facility Location problem is given below.
Definition 7.3.2 (The Facility Location Problem) Let X be a set of facilities and Y be a set of customers. Let c be a metric defined on X ∪ Y. For each i ∈ X, let fi be the cost of opening facility i. Let S be a subset of X. For each j ∈ Y , the routing cost of customer j with respect to S is rjS = mini∈S c(i, j). The total cost of S is C(S) = Cf (S) + Cr(S), where Cf (S) =
i∈S fi^ is the facility opening cost of S and Cr(S) =
j∈Y r S j is the routing cost of^ S. The^ Facility Location problem asks for a subset S∗^ ⊆ X that minimizes C(S∗) over all subsets of X.
The requirement that c is a metric seems arbitrary at first sight. However, if we let c be any general weight function, we will be able to reduce Set Cover to this relaxed Facility Location problem, implying that it is unlikely to be approximable within a constant factor (see lecture 3). By imposing the metric constraints, we are able to get a constant factor approximation algorithm for Facility Location.
The Facility Location problem arises very often in operations research. There are many variants of the problem, such as Capacitated Facility Location, in which each facility can only serve a certain number of customers. Another variant is the k-Median problem, in which facilities have no opening costs but at most k of them can be opened.
We remark that the best known approximation algorithm for Facility Location has an approximation coeffcient of 1.52 [3]. On the other hand, it has been shown that approximating this problem within a factor of 1.46 is NP-hard unless NP ⊆ DTIME(nlog log^ n) [4].
7.3.2 The Algorithm
We present a local search algorithm for approximating Facility Location within a factor of 5 + for any constant > 0 [5]. In fact, it can be shown that this algorithm is a (3 + )-approximation [6],
but the analysis is more involved and we will not go through that in class.
We start by specifying the solution graph. Each vertex in the graph represents a subset of facilities to be opened. At each step, we can add a facility to the subset, remove a facility from a subset, or swap a facility in the subset with one outside the subset. The algorithm is as follows:
- Start with an arbitrary subset of facilities.
- Carry out an operation (add, remove or swap) that leads to a decrease in the total cost by a factor of at least (1 − /p(n)), where p(n) is a polynomial to be determined later.
- Repeat step 2 until such an operation does not exist.
We first bound the running time of this algorithm. Let n = |X| + |Y |. For easier analysis we assume that all distances and costs are integers. Observe that the cost of any solution is bounded from above by some polynomial in n, c and f. Let q(n, c, f ) be such a polynomial. The cost of the solution after t iterations is at most (1 − /p(n))tq(n, c, f ). After p(n)/ iterations, the cost is at most q(n, c, f )/e since (1 − x)^1 /x^ ≈ 1 /e. Thus the algorithm halts after O(log q(n, c, f )p(n)/) steps, which is polynomial in 1/ and the size of the instance (in binary).
Theorem 7.3.3 says that any local optimum in the solution graph is nearly as good as any global optimum. Note that in the analysis, we will assume that in step 2 of the algorithm we carry out an operation as long as there is some improvement, even if the improvement is not by a factor of at least (1 − /p(n)). We will deal with this issue later on, and instead of a 5-approximation we will only get a (5+)-approximation. This theorem follows directly from Lemma 7.3.4 and Lemma 7.3. which we will see in a second.
Theorem 7.3.3 Let S∗^ be a global optimum and S be a local optimum. Then C(S) ≤ 5 C(S∗).
Remark. As mentioned at the beginning of this section, the bound on C(S) in Theorem 7.3.3 can be improved to 3C(S∗). We will not go into details.
For the proofs of the two lemmas we need a few notations. We define, for any j ∈ Y ,
- σ∗(j) = facility that customer j is assigned to in S∗
- σ(j) = facility that customer j is assigned to in S
- r∗(j) = routing cost for customer j in S∗
- r(j) = routing cost for customer j in S.
Lemma 7.3.4 Let S∗^ be a global optimum and S be a local optimum. Then Cr(S) ≤ C(S∗).
Proof: Let i∗^ ∈ S∗. Consider adding i∗^ to S. Since S is a local optimum, this operation does not decrease the total cost, and therefore
fi∗ +
j:σ∗(j)=i∗
(r∗(j) − r(j)) ≥ 0.
i
j
σ∗(j)
r∗ j
rj
i∗
Figure 7.3.2: Entities in Claim 7.3.6.
and noting that σ∗(j) ∈ S∗, we get c(i, i∗) ≤ c(i, σ∗(j)). By applying the triangle inequality again, c(i, σ∗(j)) ≤ c(j, i) + c(j, σ∗(j)) = rj + r∗ j. Combining the above, we get the following inequality:
c(j, i∗) − c(j, i) ≤ rj + r j∗.
Now we can substitute it into Equation 7.3.1 to obtain the desired result:
0 ≤ fi∗ − fi +
j:σ(j)=i
(rj + r∗ j )
≤ fi∗ − fi + Ri + R∗ i.
For a secondary facility i in S, we remove it and assign its customers to a nearby facility in S. Specifically, we assign them to the primary facility associated to σ∗(i).
Claim 7.3.7 If i ∈ S is secondary, then fi ≤ 2(Ri + R i∗ ).
Proof: Let i∗^ = σ∗(i) and i′^ = π(i∗). Consider removing i from S and assigning its customers to i′. Let j be a customer assigned to i in S, and consider the increase in its routing cost:
c(i′, j) − c(i, j) ≤ c(i, i∗) + c(i∗, i′) ≤ 2 c(i, i∗) ≤ 2 c(i, σ∗(j)) ≤ 2(rj + r∗ j )
The first and last inequalities follow from the triangle inequality; the second inequality follows from the fact that i′^ is the primary facility associated to i∗; the third inequality follows from the fact that i∗^ is the facility in S∗^ closest to i.
Summing over all customers of i, we have shown that the total increase in routing costs is no more than 2(Ri + R i∗ ). Since the increase in the total cost is non-negative, we get
−fi + 2(Ri + R∗ i ) ≥ 0.
i
j
rj
σ∗(j)
i∗
i′
r j∗
Figure 7.3.3: Entities in Claim 7.3.7.
From the above two claims we obtain
Cf (S) ≤ Cf (S∗) + 2
i∈S
(R∗ i + Ri)
= Cf (S∗) + 2Cr (S∗) + 2Cr(S) ≤ 2 C(S∗) + 2Cr(S).
Finally, by Lemma 7.3.4 we have
Cf (S) ≤ 2 C(S∗) + 2Cr(S) ≤ 4 C(S∗).
Finally, we will show that carrying out step 2 only when the cost decreases significantly increases the approximation factor by only a small amount. Our algorithm moves from one solution to another only if the total cost decreases by a factor of at least (1 − /p(n)), so it may not halt at a local optimum. Nevertheless, we can still bound the cost of the solution by slightly modifying the above proofs. In Lemma 7.3.4 and Lemma 7.3.5, we can replace 0 with (−/p(n))C(S) when we set up inequalities for the increases in cost. There are less than n^2 inequalities. Summing them up we get (1 − n^2 /p(n))C(S) ≤ 5 C(S∗). By choosing p(n) = n^2 , we have shown that C(S) is a (5 + )-approximation for C(S∗).
Recall that our analysis of the algorithm is not tight — the bound can be improved to C(S) ≤ 3 C(S∗) by reassigning customers more carefully in Lemma 7.3.5. In the following, we present an example showing that this is in fact the best we can get from this local search algorithm.
Suppose that there are k + 2 facilities and k + 1 customers “connected” in the way shown in Figure 7.3.4. (The distance between any two entities is the shortest path distance between them. Note that shortest path distances on a graph always form a metric.) The opening cost of facility xk+2 is 2k; other facilities have zero opening cost. The global optimum S∗^ = {i 1 ,... , ik+1} has
