Download Algorithms and Data Structures: A Comprehensive Guide with Exercises and Solutions and more Schemes and Mind Maps Design and Analysis of Algorithms in PDF only on Docsity!
COMP/MATH 3804 – Design and Analysis of Algorithms I
Fall 2015
FINAL EXAM SOLUTIONS
Question 1 (12%). Modify Euclid’s algorithm as follows.
function Newclid(a,b) if a<b: swap a and b if b=0: return a return Newclid(a-b,b)
Does Newclid(a,b) still compute gcd(a, b)? Briefly justify your answer. Show that Newclid(a,b) is exponentially slower than Euclid(a,b) on certain inputs.
Solution. Newclid(a,b) computes gcd(a, b): the proof of Euclid’s rule is based on the fact that gcd(a, b) = gcd(a − b, b), provided that a ≥ b. This is exactly what Newclid does. Moreover, Newclid terminates because the value of its input decreases at each recursive call. If b = 1, then Newclid(a,b) performs Θ(a) recursive steps, at least half of which take time Θ(log a), while Euclid(a,b) performs just one O((log a)^2 ) step. Therefore Newclid(a,1) is an exponential-time algorithm, while Euclid(a,1) runs in polynomial time.
Question 2 (12%). Suppose you have to choose among three algorithms to solve a problem:
- Algorithm A solves an instance of size n by recursively solving eight instances of size n/2, and then combining their solutions in time O(n^3 ).
- Algorithm B solves an instance of size n by recursively solving twenty instances of size n/3, and then combining their solutions in time O(n^2 ).
- Algorithm C solves an instance of size n by recursively solving two instances of size 2n, and then combining their solutions in time O(n).
Which one is preferable, and why?
Solution. Algorithm C does not even terminate, because it recursively calls itself on instances of increasing size. Hence it does not have a running time (i.e., its running time is “infinite”). By the Master theorem, Algorithm A’s running time is O(n^3 log n), and Algorithm B’s running time is O(nlog^3 20 ). Since log 3 20 < 3, Algorithm B is preferable.
Question 3 (12%). You are given an unsorted array of n distinct integers, along with m queries. Each query is an integer that you have to search in the array, reporting “found” or “not found”. Queries have to be processed in the order they are given. Assuming that m = b
nc, would you answer each query via a linear search of the unsorted array, or would you rather preliminarily sort
the array to speed up your searches? What if m = b
log nc instead? Briefly justify your answers.
Solution. Doing m linear searches requires Θ(mn) time. On the other hand, sorting the array takes Θ(n log n) time, and doing m binary searches on the sorted array requires Θ(m log n) time. Hence sorting is convenient (or equivalent to not sorting) if and only if m = Ω(log n). Therefore, if m = b
nc = Ω(log n), then sorting is convenient. If m = b
log nc 6 = Ω(log n), then sorting is not convenient.
Question 4 (15%). In the fast Fourier transform algorithm, why do we choose ω to be a primitive n-th root of unity (as opposed to any other n-th root of unity)?
Solution. Because, to compute the value representation of the input polynomial, we want to evaluate it at n distinct points. If ω is not a primitive n-th root of unity, then its powers do not generate all of the n-th roots of unity, but only a subset of them. This causes some repetitions among the first n powers of ω, which means that the polynomial gets evaluated at fewer than n points.
Question 5 (12%). Recall that, in a depth-first search of a directed graph, the vertex that receives the highest post number must lie in a source strongly connected component. Is it true that the vertex that receives the lowest post number lies in a sink strongly connected component? Give a short proof or provide a counterexample.
Solution. No. This graph is a counterexample: ({a, b, c}, {(a, b), (a, c), (b, a)}). If the depth-first search picks vertices in alphabetical order, then post(a) = 6, post(b) = 3, and post(c) = 5. Hence b has the lowest post number, but it does not lie in a sink strongly connected component.
Question 6 (10%). Recall that, in a depth-first search of a directed graph, for an edge (u, v) we distinguish three cases:
- if pre(u)<pre(v)<post(v)<post(u), then (u, v) is a forward edge;
- if pre(v)<pre(u)<post(u)<post(v), then (u, v) is a back edge;
- if pre(v)<post(v)<pre(u)<post(u), then (u, v) is a cross edge.
Why is the ordering pre(u)<post(u)<pre(v)<post(v) not possible for edge (u, v)?
Solution. The fact that pre(u) < pre(v) means that u is discovered before v. But post(u) < pre(v) implies that u is left for the last time before v is ever visited. But this contradicts the depth- first search algorithm, which visits every unvisited neighbor of u (hence also v) before abandoning u for the last time.
Question 7 (10%). In Dijkstra’s algorithm, why do we use a priority queue, as opposed to a regular first-in-first-out queue? Give an example of a graph with positive lengths on edges on which Dijkstra’s algorithm fails to compute shortest paths if a first-in-first-out queue is used.
Question 11 (15%). Recall that, in the Set Cover problem, we are given a universe set U of size n, along with m subsets S 1 , S 2 , · · · , Sm of U. The goal is to select the smallest number of these subsets, such that their union is all of U. Model Set Cover as an integer linear programming problem (i.e., a linear programming problem in which all variables are integers).
Solution. Let U = { 1 , · · · , n}. The linear programming problem has m integer variables x 1 , · · · , xm, where xi represents the subset Si: xi = 1 if Si is selected, and xi = 0 otherwise. The following linear programming specification is a direct translation of Set Cover:
minimize
∑m i=1 xi subject to
∑^ i:1∈Si^ xi^ ≥^1 i:2∈Si xi^ ≥^1 .. ∑. i:n∈Si xi^ ≥^1 0 ≤ x 1 ≤ 1 0 ≤ x 2 ≤ 1 .. . 0 ≤ xm ≤ 1
The objective function counts how many subsets have been selected, and the goal is to minimize it. The first n constraints express the fact that each element of U has to appear in at least one of the selected subsets. The last m inequalities set the bounds for the variables. This, paired with the assumption that the variables are integers, forces their values to be either 0 or 1.
Question 12 (12%). If we could solve an NP-complete problem in polynomial time, would all other problems in NP necessarily be solvable in polynomial time? Briefly justify your answer. If we could solve an NP-complete problem in time O(n^2015 ), would all other problems in NP neces- sarily be solvable in time O(n^2015 )? Briefly justify your answer.
Solution. First question: yes. Suppose that the NP-complete problem A is solvable in polynomial time, and let an NP problem B be given. Since A is NP-complete, any instance of B can be translated into an “equivalent” instance of A in polynomial time. Then the instance of A can be solved in polynomial time, by assumption. Composing these two algorithms yields a polynomial-time algorithm for B. Second question: no. Even if A is solvable in time O(n^2015 ), we still do not know anything about the running time of the reduction from B to A, except that it is polynomial. If such a polynomial is not O(n^2015 ), then composing the reduction and the algorithm for A does not yield an O(n^2015 ) algorithm. Hence, without making further assumptions, we cannot deduce that B can be solved in time O(n^2015 ).
Question 13 (18%). Let MAX SAT be the maximization problem of finding an assignment to the variables of a given Boolean formula (expressed in conjunctive normal form) such that the highest number of clauses evaluate to true. Consider the following approximation algorithm for MAX SAT:
function ApproxSAT(F) input: a Boolean formula F in conjunctive normal form output: an assignment to the variables of F A = assignment where every variable of F is true B = assignment where every variable of F is false if A satisfies more clauses of F than B does: return A else: return B
Prove that the approximation ratio of ApproxSAT is exactly 2.
Solution. Recall that the approximation ratio of an algorithm A for a maximization problem is defined as supI^ OPT( A(I)I ), where I is an instance of the problem. Referring to the ApproxSAT algorithm, observe that, if the assignment A does not make a clause true, then all the literals in the clause are negative. But in this case, B makes the clause true. Hence, each clause is satisfied by either A or B, implying that one of the two assignments satisfies at least half the clauses of the formula F. In particular, it satisfies at least half the maximum number of clauses of F that are simultaneously satisfiable, implying that ApproxSAT is at least a 2-approximation algorithm. To see why the approximation ratio of ApproxSAT is exactly 2, consider the Boolean formula (x) ∧ (¯y), which has two clauses, both of which are satisfied by the assignment x = true and y = false. The optimum here is 2, but A and B satisfy only one clause each.
Problem 1. Let an array of integers A = [a 1 , a 2 , · · · , an] be given. Suppose that there exists an (unknown) index k such that the subarray [a 1 , · · · , ak] is sorted in strictly increasing order, and the subarray [ak, · · · , an] is sorted in strictly decreasing order (i.e., if 1 ≤ i < j ≤ k then ai < aj , and if k ≤ i < j ≤ n then ai > aj ). Your goal is to determine k.
Part 1 (15%). Prove that any comparison-based algorithm that solves this problem has a running time of Ω(log n). [Hint: count the leaves of the comparison tree.]
Part 2 (25%). Describe in English (no pseudo-code) an optimum algorithm to solve this problem, and analyze its running time. [Hint: use divide and conquer.]
Solution. Part 1. Any such algorithm must return an index between 1 and n. Since ak can be located any- where in the array, all n possibilities can actually occur. Therefore, the choice tree of any comparison- based algorithm that solves this problem must have at least n leaves, implying that its height must be Ω(log n). The height of the tree is the number of comparisons that the algorithm performs in the worst case, which in turn is a lower bound to the total number of operations, and hence to the running time. It follows that the running time of such an algorithm must be Ω(log n). Part 2. We modify the binary search algorithm: intuitively, we find the maximum element ak by bisecting the array, looking at the “gradient” at the midpoint, and going “uphill”. If n ≤ 2, we trivially find k in constant time. If n ≥ 3, we set m = dn/ 2 e, and we inspect am− 1 , am, and am+1. If am− 1 < am > am+1, return m; if am− 1 > am > am+1, recurse on [a 1 , · · · , am− 1 ]; if am− 1 < am < am+1, recurse on [am+1, · · · , an]. The algorithm is correct because, by the properties of A, if am− 1 < am > am+1, then necessarily m = k. Also, if am− 1 , am, and am+1 are in descending order, then k < m (again by the properties of
Solution (Option 2). Preliminarily sort the array A in increasing order, in O(n log n) time. Then keep two indices i and j, initially pointing at the first and last element of the sorted array A, respec- tively. If ai + aj = k, return ai and aj ; if ai + aj < k, increase i by 1; if ai + aj > k, decrease j by 1. Repeat this step until two suitable elements are found, or until i = j. In the latter case, report that no two elements of A have sum k. Let us prove that the algorithm is correct. If no two elements of A have sum k, then obviously the algorithm will never find two such elements. Hence the difference between i and j will keep decreasing by 1 at every step, until i = j. When this happens, the algorithm correctly reports that the two desired elements do not exist. Suppose now that they do exist: ax + ay = k, with x < y. Suppose by contradiction that the algorithm never evaluates the sum ax + ay, and erroneously reports that no such elements exist. This only happens when i and j become equal, which means that i and j, collectively, have scanned the whole array. Therefore, at some point, either i has been equal to x, or j has been equal to y (because i ≤ j and x < y). Suppose without loss of generality that i = x occurs before j = y (the other case is symmetric, and note that they cannot occur both at the same time, because only one index changes at each step). At that point, since j has not been equal to y yet, it must be greater than y. But the array is sorted in increasing order, so the sum ai + aj must be greater than ax + ay = k (because ai = ax and aj > ay). Hence the algorithm decreases j by 1. This is repeated until j = y. Meanwhile i has not changed, implying that now ai = ax and aj = ay. So the sum ai + aj evaluates to k, and the two correct elements are returned, contradicting the assumption that they are never found. The running time of the algorithm is O(n log n) for the initial sorting phase, and O(n) for the final scan (because j − i decreases by 1 at each step, and is initially n − 1), yielding an overall running time of O(n log n).
Problem 4. We say that an undirected graph G = (V, E) is c-colorable if its vertices can be colored using a palette of c colors, in such a way that adjacent vertices get different colors. Formally, G is c-colorable if there exists a function f : V → { 1 , 2 , · · · , c}, called c-coloring, such that, for every {u, v} ∈ E, f (u) 6 = f (v).
Part 1 (20%). Describe in English (no pseudo-code) a linear-time algorithm that, given a connected graph G = (V, E) represented as an adjacency list, greedily colors its vertices in such a way that adjacent vertices get different colors. Moreover, if G is 2-colorable, your algorithm should not use more than two colors. Explain in intuitive terms why your algorithm is correct (i.e., why adjacent vertices get different colors, and why it returns a 2-coloring if the graph is 2-colorable), and show that it runs in time O(|V | + |E|). [Hint: do a breadth-first search.]
Part 2 (15%). Exhibit infinitely many 3-colorable graphs for which the algorithm you gave in Part 1 does not return a 3-coloring.
Part 3 (30%). Prove that deciding if a given graph is 3-colorable is NP-complete. [Hint: give a reduction from 3-SAT. Transform a Boolean formula ϕ into a graph G such that ϕ is satisfiable if and only if G is 3-colorable. Start by constructing a triangle, and label its vertices “True”, “False”, and “None”. This is the “palette triangle”. Then represent a variable of ϕ as a pair of adjacent vertices, and represent an OR gate as a triangle. Appropriately connect the palette triangle with the variable vertices and the OR gates in such a way that 3 -coloring the resulting graph is equivalent to finding a satisfying assignment to ϕ.]
Solution. Part 1. The algorithm is based on a breadth-first traversal, using a first-in-first-out queue. Intuitively, each time a vertex is visited, it is assigned a color that has not been assigned to any of its neighbors, trying to assign color 1 whenever possible (recall that colors are encoded as integers). The details are as follows. First, mark every vertex of G as unvisited and uncolored. Initialize the queue by inserting any vertex, and mark the vertex as visited. At each step, eject the first vertex v from the queue, and scan all its neighbors. When an unvisited neighbor of v is found, insert it into the queue and mark it as visited. Meanwhile, remember the largest color encountered so far among the neighbors of v, and also remember if color 1 has been encountered (ignoring the uncolored neighbors). At the end of the scan, if color 1 has not been found, then assign color 1 to v. If color 1 has been found, then assign v the largest color encountered, incremented by 1. Repeat until the queue is empty. This algorithm visits every vertex of G, due to the properties of breadth-first traversals, and because G is connected. Each time a vertex is processed, it is assigned a color that has not been assigned to any of its neighbors, yet. Therefore, eventually all vertices are colored, and no two neighbors have the same color. Suppose that G is 2-colorable, fix a 2-coloring f , and fix a starting vertex s. Without loss of generality, we may assume that f (s) = 1 (if not, invert all colors). It is easy to prove by induction that f (v) = 1 if and only if the distance between the vertices v and s is even. Again by induction, we can prove that our greedy algorithm, starting from s, computes exactly f , and therefore uses at most two colors. This is true for s, because none of the neighbors of s has a color when s is processed, and so by default s gets color 1. Now assume that our claim is true for all vertices at distance d from s, and let v be a vertex at distance d + 1. By the properties of the breadth-first search, when v is ejected from the queue, all the vertices at distance d from s have already been traversed, and colored. By the inductive hypothesis, all neighbors of v are either uncolored or have the same color, either 1 or 2. The greedy algorithm then assigns v the other color, which once again agrees with f. This holds for all vertices at distance d + 1 from s, which concludes our proof by induction. The running time of the algorithm is the same as the breadth-first search. Indeed, each time a neighbor of a vertex is examined, only a constant number of operations are performed (e.g., insert it into the queue, look up its color, check if it is 1, compare it with the maximum color seen so far, etc.). Similarly, the number of operations performed on each vertex (other than the scan of its neighbors) is constant (e.g., eject it from the queue, decide whether its color should be 1, etc.). Hence the running time is O(|V | + |E|). Part 2. The graph below is an example. Suppose that the greedy algorithm starts from vertex S, and then processes vertices in alphabetical order. The numbers are the colors that are assigned to each vertex. Despite the fact that the graph is 3-colorable, the greedy algorithm assigns color 4 to vertex E.
S
A
B
D
E
G F C
To construct infinitely many such graphs, extend the chain on the left.
Part 2 (25%). Let m =
n, and let the m subarrays have the same size (for simplicity, assume that m is an integer, at all levels of the recursion tree). The sorted subarrays are merged using a priority queue implemented as a binary heap (hence, insertions and deletions are performed in logarithmic time, with respect to the size of the queue). Prove that the running time of this variant is O(n · log n · log log n). [Hint: show that the recursion tree has O(log log n) levels.]
Part 3 (30%). Let m = 2, and let the two subarrays have sizes n/3 and 2n/3, respectively (for simplicity, assume these are always integers). The sorted subarrays are merged as usual. Prove that the running time of this variant is O(n log n). [Hint: use induction to show that T (n) ≤ C · n log n, for some constant C.]
Solution. Part 1. All the subarrays have size n/m. No matter in what order they are merged, m − 1 merges are performed between arrays of size less than n. Hence this process takes time O(mn), which is O(n), because m is a constant. Therefore the running time is T (n) = m · T (n/m) + O(n). By the Master theorem, T (n) = O(n log n). Part 2. At the k-th level of the recursion, the subarrays have size n^1 /^2 k
. Suppose that the base case has constant size c > 1. Then, the base level is reached when k satisfies the equation n^1 /^2
k = c. Solving for k yields k = log 2 logc n = log 2 (logc 2 · log 2 n) = log 2 logc 2 + log 2 log 2 n = O(log log n), because c is a constant. Hence the recursion tree has O(log log n) levels. At each level, each sub-problem is solved by merging some subarrays that are stored in a priority queue. At each step of the merging process, the subarray with the smallest first element is ejected from the priority queue, its first element is extracted, and the reduced sub-array is re-inserted into the queue, with a new priority given by its new first element. Since the number of sub-arrays cannot exceed n, each of these operations is done in time O(log n). Now observe that the sizes of all the sub-problems at each given level have sum exactly n. No matter how the n elements are distributed among the sub-problems, each element is first inserted into a queue and then extracted when its turn comes. So, processing each element takes time O(log n), and no element is processed twice in the same level. Therefore, the total amount of work that is done at each level is O(n log n) (this also includes splitting the sub-problems into smaller sub-problems, which collectively takes time O(n), at each level). In conclusion, there are O(log log n) levels, and the time spent at each level is O(n log n), implying that the total running time is O(n · log n · log log n). Part 3. Merging the two sub-arrays with the usual Merge procedure takes time O(n), regardless of their sizes. Hence the running time satisfies the recursion T (n) = T (n/3) + T (2n/3) + O(n). Let us prove by induction that T (n) ≤ C · n log 2 n, for some constant C, and for large-enough values of n. Note that, for all n smaller than any given constant, we have T (n) ≤ C′, where C′^ is a suitable constant. This implies that setting C ≥ C′^ works for all small-enough values of n. Let us now prove that T (n) ≤ C · n log 2 n holds for a given large-enough n, assuming that it holds for all the smaller values of n. n is taken to be sufficiently large, so that the term O(n) in the recursion is at most D · n, for some constant D (and no additive constant). Therefore, the recursion becomes
T (n) ≤ T (n/3) + T (2n/3) + Dn. Applying the inductive hypothesis to T (n/3) and T (2n/3), we get
T (n) ≤
C · n log 2 (n/3) +
C · n log 2 (2n/3) + Dn
C · n log 2 n −
C · n log 2 3
C · n log 2 n −
C · n log 2
= C · n log 2 n +
C · log 2 3 −
C · log 2 3 +
C + D
· n
= C · n log 2 n +
−C · log 2 3 +
C + D
· n.
If we want T (n) to be at most C · n log 2 n, we need the right-hand-side term to be negative. This condition can be expressed as
D − C · log 2 3 +
C ≤ 0 ,
which is equivalent to
C ≥
D
log 2 3 − 2 / 3
Setting C = 2D abundantly suffices. Since we must also have C ≥ C′, we may choose C = max{C′, 2 D}, which allows us to conclude that T (n) = O(n log n).
