Sorting Algorithms: Linear-Time Algorithms and Lower Bounds, Slides of Computer Science

Download Sorting Algorithms: Linear-Time Algorithms and Lower Bounds and more Slides Computer Science in PDF only on Docsity!

Algorithms

Linear-Time Sorting Algorithms

Sorting So Far

 Insertion sort:

 Easy to code  Fast on small inputs (less than ~50 elements)  Fast on nearly-sorted inputs  O(n 2 ) worst case  O(n 2 ) average (equally-likely inputs) case  O(n 2 ) reverse-sorted case

Sorting So Far

 Heap sort:

 Uses the very useful heap data structure  Complete binary tree  Heap property: parent key > children’s keys  O(n lg n) worst case  Sorts in place  Fair amount of shuffling memory around

Sorting So Far

 Quick sort:

 Divide-and-conquer:  Partition array into two subarrays, recursively sort  All of first subarray < all of second subarray  No merge step needed!  O(n lg n) average case  Fast in practice  O(n 2 ) worst case  Naïve implementation: worst case on sorted input  Address this with randomized quicksort

Decision Trees

 Decision trees provide an abstraction of comparison sorts  A decision tree represents the comparisons made by a comparison sort. Every thing else ignored  (Draw examples on board)

 What do the leaves represent?

 How many leaves must there be?

Decision Trees

 Decision trees can model comparison sorts. For a given algorithm:  One tree for each n  Tree paths are all possible execution traces  What’s the longest path in a decision tree for insertion sort? For merge sort?

 What is the asymptotic height of any decision tree for sorting n elements?

 Answer: Ω( n lg n ) (now let’s prove it…)

Lower Bound For

Comparison Sorting

 So we have… n! ≤ 2 h

 Taking logarithms: lg ( n !) ≤ h

 Stirling’s approximation tells us:

 Thus:

n

e

n n  

  

! >  n

e

n h (^)  

  

≥ lg

Lower Bound For

Comparison Sorting

 So we have

 Thus the minimum height of a decision tree is Ω( n lg n )

( n n )

n n n e

e

n h

n

lg

lg lg

lg

= Ω

= −

 

  

≥ 

Sorting In Linear Time

 Counting sort

 No comparisons between elements!  But … depends on assumption about the numbers being sorted  We assume numbers are in the range 1.. k  The algorithm:  Input: A[1.. n ], where A[j] ∈ {1, 2, 3, …, k }  Output: B[1.. n ], sorted (notice: not sorting in place)  Also: Array C[1.. k ] for auxiliary storage

Counting Sort

1 CountingSort(A, B, k) 2 for i=1 to k 3 C[i]= 0; 4 for j=1 to n 5 C[A[j]] += 1; 6 for i=2 to k 7 C[i] = C[i] + C[i-1]; 8 for j=n downto 1 9 B[C[A[j]]] = A[j]; 10 C[A[j]] -= 1;

Work through example: A={4 1 3 4 3}, k = 4

Counting Sort

 Total time: O( n + k )

 Usually, k = O( n )  Thus counting sort runs in O( n ) time

 But sorting is Ω( n lg n )!

 No contradiction--this is not a comparison sort (in fact, there are no comparisons at all!)  Notice that this algorithm is stable

Counting Sort

 Cool! Why don’t we always use counting sort?

 Because it depends on range k of elements

 Could we use counting sort to sort 32 bit integers? Why or why not?

 Answer: no, k too large (2 32 = 4,294,967,296)

Radix Sort

 Intuitively, you might sort on the most significant digit, then the second msd, etc.

 Problem: lots of intermediate piles of cards (read: scratch arrays) to keep track of

 Key idea: sort the least significant digit first RadixSort(A, d) for i=1 to d StableSort(A) on digit i  Example: Fig 9.

Radix Sort

 Can we prove it will work?

 Sketch of an inductive argument (induction on the number of passes):  Assume lower-order digits {j: j<i}are sorted  Show that sorting next digit i leaves array correctly sorted  If two digits at position i are different, ordering numbers by that digit is correct (lower-order digits irrelevant)  If they are the same, numbers are already sorted on the lower-order digits. Since we use a stable sort, the numbers stay in the right order