Download Lower Bounds for Sorting Algorithms: Decision Trees and Counting Sort and more Slides Introduction to Computers in PDF only on Docsity!
Sorting revisited
I (^) In COMS12100 we looked at various sorting algorithms and analysed their time complexities. I (^) Bubble sort - Θ( n^2 ) I (^) Quicksort - Θ( n log n ) on average but Θ( n^2 ) in the worst case I (^) Merge Sort, Heap Sort - Θ( n log n ) in the worst case I (^) The question we want to answer is "is it possible to do any better?" I (^) What do we mean by “do better?" I (^) An algorithm gives an “upper bound" for a problem but tells us nothing about whether another faster algorithm might exist. I (^) We will look at how to prove a lower bound for sorting. I (^) Then we will show how to “beat" this lower bound.
Clifford, Harrow and Page
Sorting revisited
I (^) In COMS12100 we looked at various sorting algorithms and analysed their time complexities. I (^) Bubble sort - Θ( n^2 ) I (^) Quicksort - Θ( n log n ) on average but Θ( n^2 ) in the worst case I (^) Merge Sort, Heap Sort - Θ( n log n ) in the worst case I (^) The question we want to answer is "is it possible to do any better?" I (^) What do we mean by “do better?" I (^) An algorithm gives an “upper bound" for a problem but tells us nothing about whether another faster algorithm might exist. I (^) We will look at how to prove a lower bound for sorting. I (^) Then we will show how to “beat" this lower bound.
Clifford, Harrow and Page
Lower bounds for sorting
I (^) In order to prove a lower bound for a problem we need to somehow show that no algorithm can be faster than our bound in the worst case. This is different from seeing how fast a particular algorithm takes to run. I (^) To prove a lower bound we need to show that for any program P that correctly sorts all inputs, we can find an input such that P takes time ≥ cn log n for some c (and for all n larger than some constant). I (^) We have to define carefully which operations we want to count. We have to define the computational model. I (^) All the sorting algorithms we have seen so far have worked by comparing pairs of elements in the input. We will count only comparison operations and show that Ω(n log n) comparisons are always needed in the worst case. I (^) But is there any alternative to comparison sorting? We will see the answer to this question is yes.
Clifford, Harrow and Page
Lower bounds for sorting
I (^) In order to prove a lower bound for a problem we need to somehow show that no algorithm can be faster than our bound in the worst case. This is different from seeing how fast a particular algorithm takes to run. I (^) To prove a lower bound we need to show that for any program P that correctly sorts all inputs, we can find an input such that P takes time ≥ cn log n for some c (and for all n larger than some constant). I (^) We have to define carefully which operations we want to count. We have to define the computational model. I (^) All the sorting algorithms we have seen so far have worked by comparing pairs of elements in the input. We will count only comparison operations and show that Ω(n log n) comparisons are always needed in the worst case. I (^) But is there any alternative to comparison sorting? We will see the answer to this question is yes.
Clifford, Harrow and Page
Lower bounds for sorting
I (^) In order to prove a lower bound for a problem we need to somehow show that no algorithm can be faster than our bound in the worst case. This is different from seeing how fast a particular algorithm takes to run. I (^) To prove a lower bound we need to show that for any program P that correctly sorts all inputs, we can find an input such that P takes time ≥ cn log n for some c (and for all n larger than some constant). I (^) We have to define carefully which operations we want to count. We have to define the computational model. I (^) All the sorting algorithms we have seen so far have worked by comparing pairs of elements in the input. We will count only comparison operations and show that Ω(n log n) comparisons are always needed in the worst case. I (^) But is there any alternative to comparison sorting? We will see the answer to this question is yes.
Clifford, Harrow and Page
Decision Trees
Sort < a 1 ,... , a n > (assume w.l.o.g. that all a i are distinct)
Each internal node is labelled i : j for i, j ∈ { 1 , 2 ,... , n}
I (^) The left subtree shows subsequent comparisons if a i ≤ a j I (^) The right subtree shows subsequent comparisons if a i > a j
Clifford, Harrow and Page
Decision Trees
Sort < 7 , 4 , 6 >
Each internal node is labelled i : j for i, j ∈ { 1 , 2 ,... , n}
I (^) The left subtree shows subsequent comparisons if a i ≤ a j I (^) The right subtree shows subsequent comparisons if a i > a j
Clifford, Harrow and Page
Decision Trees
Sort < 7 , 4 , 6 >
Each internal node is labelled i : j for i, j ∈ { 1 , 2 ,... , n}
I (^) The left subtree shows subsequent comparisons if a i ≤ a j I (^) The right subtree shows subsequent comparisons if a i > a j
Clifford, Harrow and Page
Decision Tree Model
A decision tree can model the execution of any comparison sort. I (^) One tree for each input size n I (^) Tree contains all possible sequences of comparisons needed to sort the input I (^) The running time of the algorithm on a particular input is the length of the path taken I (^) Worst case is the height of the tree
Clifford, Harrow and Page
Lower Bound for Decision Tree Model
Theorem
Any decision tree that can sort n elements must have height Ω(n log n).
Proof.
The tree must contain at least n! leaves as there are n! different permutations to choose from. A binary tree of height h has ≤ 2 h^ leaves. Therefore n! ≤ number of leaves ≤ 2 h^.
h ≥ log n! ≥
n 2
log(
n 2
h ∈ Ω(n log n)
Clifford, Harrow and Page
Linear Time Sorting
Counting sort: No comparisons used at all Input: A[ 1 ,... , n] where A[j] ∈ { 1 , 2 ,... , k} Output: B[ 1 ,... , n], sorted and a permutation of A Auxiliary storage: C[ 0 ,... , k]
Clifford, Harrow and Page
Counting sort
for i ← 0 to k do C[i] ← 0; end for j ← 1 to n do C[A[j]] ← C[A[j]] + 1; end
. C[i] now contains the number of elements equal to i; for i ← 2 to k do C[i] ← C[i] + C[i − 1 ]; end . C[i] now contains the number of elements less than or equal to i; for j ← n downto 1 do B[C[A[j]]] ← A[j]; C[A[j]] ← C[A[j]] − 1; end
Clifford, Harrow and Page
Counting sort
for i ← 0 to k do C[i] ← 0; end for j ← 1 to n do C[A[j]] ← C[A[j]] + 1; end
. C[i] now contains the number of elements equal to i; for i ← 2 to k do C[i] ← C[i] + C[i − 1 ]; end . C[i] now contains the number of elements less than or equal to i; for j ← n downto 1 do B[C[A[j]]] ← A[j]; C[A[j]] ← C[A[j]] − 1; end
Clifford, Harrow and Page
Counting sort
for i ← 0 to k do C[i] ← 0; end for j ← 1 to n do C[A[j]] ← C[A[j]] + 1; end
. C[i] now contains the number of elements equal to i; for i ← 2 to k do C[i] ← C[i] + C[i − 1 ]; end . C[i] now contains the number of elements less than or equal to i; for j ← n downto 1 do B[C[A[j]]] ← A[j]; C[A[j]] ← C[A[j]] − 1; end
Clifford, Harrow and Page
