Download 1 Introduction 2 Selection and more Lecture notes Reasoning in PDF only on Docsity!
CS161, Lecture 4 Median and Selection Scribe: Albert Chen (2015) Date: April 7, 2016
1 Introduction
Today, we introduce the problem of finding the kth smallest element in an unsorted array. First, we show it can be done in O(n log n) time via sorting and that any correct algorithm must run in Ω(n) time. However, it is not obvious that a linear-time selection algorithm exists. We present a linear-time select algorithm, with an intuition for why it has the desired properties to achieve O(n) running time.
2 Selection
Now for main part of this lecture. The selection problem is to find the kth smallest number in an array A.
Input: array A of n numbers, and an integer k ∈ { 1 , · · · , n}.
Output: the k-th smallest number in A.
One approach is to sort the numbers in ascending order, and then return the kth number in the sorted list. This takes O(n log n) time, since it takes O(n log n) time for the sort (e.g. by mergesort) and O(1) time to return kth number.
2.1 Minimum Element
As always, we ask if we can do better (i.e. faster in Big-Oh complexity). In the special case where k = 1, selection is the problem of finding the minimum element. We can do this in O(n) time by scanning through the array and keeping track of the minimum element so far. If the current element is smaller than the minimum so far, we update the minimum.
Algorithm 1: SelectMin(A) m ← ∞; n ← length(A); for i = 1 to n do if A(i) < m then m ← A(i);
return m;
In fact, this is the best running time we could hope for.
Definition 2.1. A deterministic algorithm is one which, given a fixed input, always performs the same operations (as opposed to an algorithm which uses randomness).
Claim 1. Any deterministic algorithm for finding the minimum has runtime Ω(n).
Proof of Claim ??. Intuitively, the claim holds because any algorithm for the minimum must look at all the elements, each of which could be the minumum. Suppose a correct deterministic algorithm does not look at A(i) for some i. Then the output cannot depend on A(i), so the algorithm returns the same value whether A(i) is the minimum element or the maximum element. Therefore the algorithm is not always correct, a contradiction. So there is no sublinear deterministic algorithm for finding the minimum.
So for k = 1, we have an algorithm which achives the best running time possible. By similar reasoning, this lower bound of Ω(n) applies to the general selection problem. So ideally we would like to have a linear-time selection algorithm in the general case.
2.2 Wishful Selection
In fact, a linear-time selection algorithm does exist. Before showing the linear time selection algorithm, it’s helpful to build some intuition on how to approach the problem. Suppose we have a black-box algorithm MagicMedian that can find the median in linear time (the median is the k = dn/ 2 eth smallest element). Then we could use MagicMedian as a subroutine to develop a linear-time selection algorithm. For clarity we’ll assume all elements are distinct from now on, but the idea generalizes easily.
Algorithm 2: WishfulSelect(A, k) m ← MagicMedian(A); n ← length(A); if k = dn/ 2 e then return m; A< ← {A(i) | A(i) < m}; A> ← {A(i) | A(i) > m}; if k < dn/ 2 e then return WishfulSelect(A<, k); else return WishfulSelect(A>, k − dn/ 2 e);
Each iteration, we use the median to partition the array into two halves, into all elements smaller than the median and all elements larger. Since the median is the dn/ 2 eth smallest element, we know which half contains the kth smallest element. If it’s the smaller half, the kth element is still the kth element; for the larger half we need to subtract dn/ 2 e because each element in the larger half is greater than the dn/ 2 e elements which are removed. We repeat, recursively working on a problem of smaller and smaller size until we find the kth element.
Claim 2. If MagicMedian runs in O(n) time, then WishfulSelect runs in O(n) time.
Proof. Intuitively, the running time is good because we remove half of the elements from consideration each iteration. It takes linear time for MagicMedian, linear time to create A< and A>, and constant time for the rest. Recusive calls are made on inputs of half the size. As a recurrence relation, T (n) ≤ cn + T (n/2). Expanding this, the runtime is T (n) ≤ cn + cn/2 + cn/4 + ... + c ≤ 2 cn, which is O(n).
Unfortunately, however, we do not know how to construct procedure MagicMedian without solving the original selection problem. Because of this, the WishfulSelect procedure doesn’t actually make sense.
2.3 Linear-Time Selection
Given a linear-time median algorithm, we can solve the selection problem in linear time (and vice versa). Unfortunately, we don’t have MagicMedian. But notice that as far as correctness goes, there was nothing special about partitioning around the median. We could use this same idea of partitioning and recursing on a smaller problem even if we partition around an arbitrary element. To get a good runtime, however, we need to guarantee that the subproblems get smaller quickly. In 1973, Blum, Floyd, Pratt, Rivest, and Tarjan came up with the Median of Medians algorithm. It is similar to the previous algorithm, but rather than partitioning around the exact median, uses a surrogate “median of medians”. The recursive calls are updated accordingly.
Now, |A>| = # of elements greater than q = (n − 1) − # elements smaller than q ≤ (n − 1) − 3 · (dg/ 2 e − 2) = n + 5 − 3 · dg/ 2 e ≤ n − 3 n/10 + 5 = 7n/10 + 5.
By symmetry, |A<| ≤ 7 n/10 + 5 as well.
The recursive call used to find the median of medians has input of size dn/ 5 e ≤ n/5 + 1. The other work in the algorithm takes linear time: constant time on each of dn/ 5 e groups for BruteForceMedian =⇒ linear time in total, O(n) time scanning A to make A< and A>, and O(n) time for finding the length of A and A<. The sum of this constant amount of linear steps is still linear.
Thus, we can write the full recurrence for the runtime,
T (n) ≤
c 1 n + T (n/5 + 1) + T (7n/10 + 5) if n > 5 c 2 if n ≤ 5.
This seems good at first glance. Each step does linear work in terms of its input size, and the total amount of linear work propagated to the next level is about n/5 + 7n/10 = 9n/10, so we expect to see an exponential decay in the runtime of each layer of the recursion tree, just as in WishfulSelect. In fact, in the last lecture we solved exactly the above recurrence and obtained T (n) = O(n) as claimed.
