Download Local Search Based Appoximation-Advanced Algorithms-Lecture 10 Notes-Computer Science and more Study notes Advanced Algorithms in PDF only on Docsity!
CS787: Advanced Algorithms Scribe: Yiying Zhang and Baris Aydinlioglu Lecturer: Shuchi Chawla Topic: Local Search Based Appoximation Date: October 03, 2007
Today we talk about an important technique of approximation algorithms: local search. We first give fundamental ideas of local search approximation. Then we look into two problems where local search based approximation algorithms apply, max-cut problem and facility location problem.
10.1 Fundamentals of Local Search
The basic idea of local search is to find a good local optima quickly enough to approximate the global optima. This idea is somewhat like hill climbing. Starting from an arbitrary point, we try to go down or climb up a little step and compare the objective value at the new point to that at the old point. We then choose the step that increase the objective value. In this manner, we finally reach a local maximal. Such algorithm is also known as ascending or descending algorithm.
We can also explain local search method using a graph. We consider a graph, whose nodes are feasible solutions and whose edges connect two “neighboring” feasible solutions. Two feasible solutions are neighbors if from either one we can obtain the other by a single “local” step. What constitutes a local step depends on the problem, as we will see through examples later. Note that we do not construct the entire graph, as its size would usually be exponential.
Once the structure of the graph is decided, in order to find a local optimum, we start with an arbitrary node in the graph of feasible solutions, and we look at its neighbors to see if there is a node with a better objective value. If there is, we find we move to that node feasible solution in the graph. We don’t necessarily look for the neighboring node with the best objective value; in most cases a better value is enough. We repeat this process until there is no neighbor with a better objective value. At that point we find the local optimum. See figure below.
Starting Node
Local Maximal
Figure 10.1.1: Graph Representation of Local Search
Now, the problem is how to design a good neighborhood structure for the graph. On one hand, we cannot allow for any node to be neighbor to too many other nodes. Otherwise in the extreme case we would get a complete graph and it would take exponential time to consider the neighbors of even a single node. On the other hand, we cannot have too few neighbors for a node, because we want any locally optimal node in the graph to have objective value within a factor of that of the global optimum.
In summary, we seek the following properties in a graph of feasible solutions:
- Neighborhood of every solution is small
- Local optima are nearly as good as global optima
- We can find some local optima in a polynomial number of steps
The third point is often proven by showing that at each step we improve the objective value by some large enough factor. If the original solution has objective value V and the local optimal solution has objective value V ∗, and at each step the objective value is improved by (1 + ≤), then we will have log(1+≤) V^ ∗ V steps. We will give more details in the following examples.
10.2 Max-Cut Problem
Problem Definition:
Given: Unweighted graph G = (V, E)
Goal: Determine a cut (S, S), where S 6 = V and S 6 = ∅, such that val(s) =
e∈E∩(S×S) we^ = |E ∩ (S × S)| is maximized.
S S
Figure 10.2.2: A cut (S, S)
This problem is known to be NP-hard, which may appear surprising since its minimization version is efficiently solvable.
10.3 Facility Location
Most of the difficulty in the study of local search approximation algorithms tends to be in the analysis activity rather than in the design activity. We now move on to a problem that is more representative of this fact: a simple algorithm, but an involved analysis.
10.3.1 Problem Definition
Consider the following prominent problem in Operations Research, named Facility Location. A company wants to distribute its product to various cities. There is a set I of potential locations where a storage facility can be opened and a fixed “facility cost” fi associated with opening a storage facility at location i ∈ I. (Sometimes we will say “facility i” to mean “facility at location i”). There also is a set J of cities to which the product needs to be sent, and a “routing cost” c(i, j) associated with transporting the goods from a facility at location i to city j. The goal is to pick a subset S of I so that the total cost of opening the facilities and transporting the goods is minimized.
In short, we want to minimize
C(S) = Cf (S) + Cr(S) , (10.3.1)
where
Cf (S) =
i∈S
fi , (10.3.2)
and
Cr(S) =
j∈J
min i∈S c(i, j). (10.3.3)
Notice that once a subset S of I is chosen, which facilities service which cities is automatically determined by the way Cr(S) is defined.
In the way we have stated it so far, this problem is a generalized version of SET COVER, which can be seen by the following reduction: Given a SET COVER instance (S, P), where P ⊂ 2 S^ , map each P ∈ P to a facility and each s ∈ S to a city, then set fP to 1 and set c(P, s) to either 0 or +∞ depending on if s ∈ P or not, respectively.
This reduction implies that the Facility Location problem would not allow a better approximation than what we can best do with SET COVER. In today’s lecture we want to study a constant fac- tor approximation to Facility Location, and for this purpose we impose some restrictions on the problem.
We restrict the costs c(i, j) to form a metric. A metric is a function that is symmetric and that satisfies the triangle equality. For the costs in our problem this means (respectively) that we have c(i, j) = c(j, i) for each i, j ∈ I ∪ J, and that we have c(i, j) ≤ c(i, k) + c(k, j) for all i, j, k ∈ I ∪ J. Notice that we have extended the notion of cost to in-between facilities and also cities. To make sense out of this, just think of costs as physical distances, and interpret equation (10.3.3) as each city being serviced by the nearest open facility.
10.3.2 A local search algorithm
Our local search algorithm works as follows. We pick any subset S of the set I of facilities. This gives us a feasible solution right away. We then search for the local neighborhood of the current feasible solution to see if there is a solution with a smaller objective value; if we find one we update our feasible solution to that one. We repeat this last step until we do not find a local neighbor that yields a reduction in the objective value. There are two types of “local steps” that we take in searching for a neighbor: i) remove/add a facility from/to the current feasible solution, or ii) swap a facility in our current solution with one that is not.
It is possible to consider variations on the local steps such as removing/adding more than one facility, or swapping d facilities out and bringing e facilities in, etc.. With more refined local steps, we can reach better local optima, but then we incur additional runtime due to the neighborhood graph getting denser. For our purposes the two local steps we listed above suffice, and indeed we show below that our algorithm with these two local steps give us a 5-factor approximation^1.
10.3.3 Analysis
Theorem 10.3.1 Let S be a local optimum that the above algorithm terminates with, and S∗^ be a global optimum. Then C(S) ≤ 5 · C(S∗).
We break up the proof of this theorem into two parts. First we show an upper bound on the routing cost of a local optimum, then we do the same for the facility cost.
Lemma 10.3. Cr(S) ≤ C(S∗).
Proof:
Consider adding some facility i∗^ in the globally optimal solution S∗^ to the locally optimal S to obtain S′. Clearly, C(S) ≥ C(S′) because our algorithm must have explored S′^ as a local step before concluding that S is locally optimal. If i∗^ was already in S then there is no change. So we have Cf (S′) − Cf (S) + Cr(S′) − Cr(S) ≥ 0.
Since S′^ = S ∪ {i∗}, we rewrite the above as
fi∗ + Cr(S′) − Cr(S) ≥ 0. (10.3.4)
Before moving on we introduce some notation for brevity. Given (a feasible solution) S ⊂ I, a facility location i, and a city j, we define the following:
— NS (i) = the set of cities that are serviced by facility i in S,^2
(^1) Actually, these two steps give us a 3-factor approximation, but we do not cover that analysis here. (^2) For reasons that will become clear later, a better way to think of the “facility i services city j in S” relation is “facility i is the closest to j among all facilities in S”. For brevity we will continue to speak of the “service” relation, but the latter interpretation lends itself better to generalizations and should be kept in mind.
which is the same thing as Cf (S∗) + Cr(S∗) − Cr(S) ≥ 0 ,
proving the lemma.
Lemma 10.3. Cf (S) ≤ 2 · C(S∗) + 2 · Cr(S) ≤ 4 · C(S∗).
Proof: We begin with some more notation. In the proof of the previous lemma we defined σ(j) to be a function of cities. Now we extend it to facility locations. Given a solution S, a facility location i, we define:
- σS^ (i) = argmini′∈S c(i, i′). In words, the nearest facility in S to i.
- σ∗(i) = σS ∗ (i).
The rest of the proof proceeds somewhat along the lines of the proof of the previous lemma. Therefore we omit the details when we employ the same type of arguments as before.
Consider swapping some facility i in the locally optimal solution S with σ∗(i), the nearest facility in S∗^ to i. Clearly, this gives us a solution S′^ with an objective function value larger than or equal to that of S, because S is locally optimal. Note that if σ∗(i) was already in S, there is no change.
With i swapped out and σ∗(i) swapped in, we now consider the following. If we “route” the goods in S′^ so that each city that was in NS (i) now gets served by σ∗(i), and not necessarily by the nearest open facility to it, then this would again yield a non-negative change in the objective function value of S′. As a result, the combined increase in the cost of this configuration, call it δ, would be
0 ≤ δ = fσ∗(i) − fi +
j∈Ns(i)
[c(j, σ∗(i)) − c(j, i)].
Recalling that the function c is a metric, we observe the following inequality regarding the term inside the summation in the last equation:
c(j, σ∗(i)) − c(j, i) ≤ c(i, σ∗(i)) ≤ c(i, σ∗(j)) ≤ c(i, j) + c(j, σ∗(j))
The first and last inequalities follow from the triangle inequality. The second inequality follows from the fact that in S∗, the distance between city i and its serving facility σ∗(i) is no larger than that between i any other facility in S∗, in particular σ∗(j). (Here we have used the σ∗^ function with both a city and a facility location as argument). We now have
0 ≤ δ ≤ fσ∗(i) − fi +
j∈NS (i)
[c(j, i) + c(j, σ∗(j))].
Expanding the summation on the right, we first get
j∈NS (i) c(j, i), which is the routing cost of the cities served by i in S; denote this by Ri. Second, we get
j∈NS (i) c(j, σ ∗(j)), which is the routing
cost of the cities served by i in S∗; denote this by R∗ i.
Now we can rewrite the last inequality as
0 ≤ δ ≤ fσ∗(i) − fi + Ri + R∗ i
Summing this equation over all i ∈ S , we obtain ∑
i∈S
fσ∗(i) − Cf (S) + Cr(S) + Cr(S∗) ≥ 0.
We are not quite done yet. The first sum is counting some facilities in S∗^ more than once and not counting some at all. We will finish this proof in the next lecture.
