Quicksort Algorithm: Complexity, Correctness, and Pivot Selection, Lecture notes of Data Structures and Algorithms

Download Quicksort Algorithm: Complexity, Correctness, and Pivot Selection and more Lecture notes Data Structures and Algorithms in PDF only on Docsity!

Algorithm Quicksort: Analysis of Complexity

Lecturer: Georgy Gimel’farb

COMPSCI 220 Algorithms and Data Structures

1 Algorithm quicksort

2 Correctness of quicksort

3 Quadratic worst-case time complexity

4 Linearithmic average-case time complexity

5 Choosing a better pivot

6 Partitioning algorithm

Basic Recursive Quicksort

If the size, n, of the list, is 0 or 1 , return the list. Otherwise: 1 Choose one of the items in the list as a pivot. 2 Next, partition the remaining items into two disjoint sublists, such that all items greater than the pivot follow it, and all elements less than the pivot precede it. 3 Finally, return the result of quicksort of the “head” sublist, followed by the pivot, followed by the result of quicksort of the “tail” sublist.

Partitioning: a[0],... , a[n − 1]

Recursive quicksort: a[0],... , a[i − 1]

Recursive quicksort: a[i + 1],... , a[n − 1]

Pivot: a[i]

a[∗] < pivot a[∗] ≥ pivot

Lemma 2.13 (Textbook): Quicksort is correct.

Proof: by math induction on the size n of the list.

• Basis. If n = 1, the algorithm is correct.
• (^) Hypothesis. It is correct on lists of size smaller than n.
• (^) Inductive step. After positioning, the pivot p at position i; i = 1,... , n − 1 , splits a list of size n into the head sublist of size i and the tail sublist of size n − 1 − i. - Elements of the head sublist are not greater than p. - Elements of the tail sublist are not smaller than p. - By the induction hypothesis, both the head and tail sublists are sorted correctly. - Therefore, the whole list of size n is sorted correctly.

Any implementation specifies what to do with items equal to the pivot.

Analysing Quicksort: The Worst Case T (n) ∈ Ω(n^2 )

Lemma 2.14 (Textbook): The worst-case time complexity of quicksort is Ω(n^2 ). Proof. The partitioning step: at least, n − 1 comparisons.

• At each next step for n ≥ 1 , the number of comparisons is one less, so that T (n) = T (n − 1) + (n − 1); T (1) = 0.
• “Telescoping” T (n) − T (n − 1) = n − 1 :

T (n)+T (n − 1)+T (n − 2)+.. .+T (3)+T (2) −T (n − 1)−T (n − 2)−.. .−T (3)−T (2)− T (1) = (n − 1) + (n − 2) +.. .+ 2 + 1 − 0 T (n)= (n − 1) + (n − 2) +.. .+ 2 + 1 = (n− 2 1)n

This yields that T (n) ∈ Ω(n^2 ).

Analysing Quicksort: The Average Case T (n) ∈ Θ(n log n)

For any pivot position i; i ∈ { 0 ,... , n − 1 }:

• (^) Time for partitioning an array : cn
• The head and tail subarrays contain i and n − 1 − i items, respectively: T (n) = cn + T (i) + T (n − 1 − i) Average running time for sorting (a more complex recurrence): T (n)= (^) n^1

∑n− 1 i=0 (T^ (i) +^ T^ (n^ −^1 −^ i) +^ cn) = (^) n^2 (T (0) + T (1) +... + T (n − 2) + T (n − 1)) + cn, or nT (n)= 2 (T (0) + T (1) +... + T (n − 2) + T (n − 1)) + cn^2 (n − 1)T (n − 1)= 2 (T (0) + T (1) +... + T (n − 2)) + c(n − 1)^2 nT (n) − (n − 1)T (n − 1) = 2T (n − 1) + 2cn − c ≈ 2 T (n − 1) + 2cn Thus, nT (n) ≈ (n + 1)T (n − 1) + 2cn, or T n^ (+1n) = T^ (n n− 1)+ (^) n^2 +1c

Implementations of Quicksort

Choices to be made for implementing the basic quicksort algorithm:

• How to implement the list?
• (^) How to choose the pivot?
• How to partition the list around the pivot?

Passive pivot choice – a fixed position in each sublist

• (^) Ω(n^2 ) running time for frequent in practice nearly sorted lists under the na¨ıve selection of the first or last position.
• (^) A more reasonable choice: the middle element of each sublist.
• Random inputs resulting in Ω(n^2 ) time are rather unlikely.
• (^) But still: vulnerability to an “algorithm complexity attack” with specially designed “worst-case” inputs.

Active Pivot Strategy

The best active pivot – the exact median of the list, dividing it into (almost) equal sized sublists, – is computationally inefficient.

The median-of-three strategy to approximate the true median The pivot p = median {a[ibeg], a[imid], a[iend]} where ibeg; iend, and imid =

⌊ (^) i beg+iend 2

refer to the first, last, and middlea elements, respectively, of a sublist, a[ibeg], a[ibeg + 1,... , a[iend]. abzc is an integer floor of the real value z.

An example: a = ( 45 , 25 , 15 , 31 , 75 , 80 , 60 , 20 , 19 )

median{ 45 , 75 , 19 } → 19 ≤ 45 ≤ 75] → 45

a = ((19, 25 , 15 , 31 , 20), 45 , (80, 60 , 75))

Partitioning Algorithm

Head End ↑L ↑R

1 Initialisation: 1 Start pointers L and R at the head of the list and at the end plus one, respectively. 2 Swap the pivot element, p, to the head of the list. 2 Iteration: while L < R, do: 1 Decrement R until it meets an element less than or equal to p. 2 Increment L until it meets an element greater than or equal to p. 3 Swap the elements pointed by L and R. 3 Once L = R, swap the pivot element with the element pointed to by L.

Example 2.17 (Textbook): Partitioning a List

Data to sort; pivot p = a[7] = 31 Description 25 8 2 91 15 50 20 31 70 65 Initial list L = 0; R = 10 31 8 2 91 15 50 20 25 70 65 Move pivot to head 31 8 2 91 15 50 20 25 70 65 Stop R 31 8 2 91 15 50 20 25 70 65 Stop L 31 8 2 25 15 50 20 91 70 65 Swap a[R] and a[L] 31 8 2 25 15 50 20 91 70 65 Stop R 31 8 2 25 15 50 20 91 70 65 Stop L 31 8 2 25 15 20 50 91 70 65 Swap a[R] and a[L] 31 8 2 25 15 20 50 91 70 65 Stop R = L 20 8 2 25 15 31 50 91 70 65 Swap a[L] with pivot Head (left) sublist ≤ p ≤ Tail (right) sublist

Pseudocode for Array-Based Quicksort

algorithm quickSort sorts the subarray a[l..r] Input: array a[0..n − 1]; array indices l, r begin if l < r then i ← pivot(a, l, r) return position of pivot j ← partition(a, l, r, i) return final position of pivot quickSort(a, l, j − 1) sort left subarray quickSort(a, j + 1, r) sort right subarray end if return a end