Download Divide and Conquer Algorithms: Recurrence Relations and Complexity Analysis - Prof. Robert and more Study notes Computer Science in PDF only on Docsity!
1
Divide and Conquer Algorithms,
Complexity Analysis of Recursive
Algorithms
Rosen Ch. 7.1: Recurrence Relations
Rosen Ch. 7.3: Divide & Conquer
Walls Ch. 10.2: Advanced Sorting Algorithms
2 Computation Time for Recursive Algorithms
Example: Compute the factorial function n!
int factorial(int n) { if n==0 return 1; else return factorial(n - 1) * n; } The number of operations required can be characterized by: f(n) = f(n-1)+ What series is generated by this recurrence relation? need an initial condition (corresponds to base case of recursion) f(0) = 1 f(n) = f(n-2) + 1 + 1 = f(n-3) + 1 + 1 + 1 = … 3 Recursive Algorithms
Example: Tower of Hanoi, move all disks to third peg without
ever placing a larger disk on a smaller one.
4 Recursive Algorithms
Example: Tower of Hanoi, move all disks to third peg without
ever placing a larger disk on a smaller one.
5 Recursive Algorithms
Example: Tower of Hanoi, move all disks to third peg without
ever placing a larger disk on a smaller one.
f(n) = f(n-1) + …
6 Recursive Algorithms
Example: Tower of Hanoi, move all disks to third peg without
ever placing a larger disk on a smaller one.
f(n) = f(n-1) + 1 + …
7 Recursive Algorithms
Example: Tower of Hanoi, move all disks to third peg without
ever placing a larger disk on a smaller one.
f(n) = f(n - 1) + 1 + f(n - 1)
f(n) = 2f(n - 1) + 1, f(1) = 1
8 Recursive Algorithms
Example: Tower of Hanoi, move all disks to third peg without
ever placing a larger disk on a smaller one.
f(n) = 2f(n - 1) + 1, f(1) = 1
Let's look at the code...
How to figure out an explicit formula for this sequence:
- Repeated substitution
- Theory of Recurrence Relations
Fibonacci’s Rabbits
Suppose a newly-born pair of
rabbits, one male, one female, are
put on an island.
A pair of rabbits doesn’t breed until 2 months old. Thereafter each pair produces another pair each month Rabbits never die.
How many pairs will there be after n
months?
image from: http://www.jimloy.com/algebra/fibo.htm (^1314)
Recurrence Examples
Suppose a string of decimal digits is a valid
codeword if it contains an even number of 0s.
How many such codewords?
Base case: a 1 =
Recurrence:
Extend valid string with digit != 0
Append 0 to invalid string of length n-
an = 9 an − 1 + ( 10 n −^1 − an − 1 ) = 8 an − 1 + 10 n −^1
15
Divide-and-Conquer
Basic idea: Take large problem and divide it into
smaller problems until problem is trivial, then
combine parts to make solution.
Recurrence relation for the number of steps required:
f(n) = a f(n / b) + g(n)
n/b - the size of the sub-problems solved
a - number of sub-problems
g(n) - steps necessary to combine solutions to sub-
problems
16
Example: Binary Search
public static int binSearch (int myArray[], int first, int last, int value) { // returns the index of value or -1 if not in the array int index; if (first > last) { index = -1; } else { int mno = (first + last)/2; if (value == myArray[ mno ]) { index = mno; } else if (value < myArray[ mno ]) { index = binSearch(myArray, first, mno-1, value); } else { index = binSearch(myArray, mno+1, last, value); } } return index; }
What are a, b, and g(n)?
f ( n ) = a ⋅ f ( n / b ) + g ( n )
17 Example: Finding Max in unsorted array Algorithm:
If n=1, then element is the max.
If n>1, divide sequence in half, find max 1 of each
and choose max from each half
What are a, b, and g(n)? Solve by repeated substitution € f ( n ) = a ⋅ f ( n / b ) + g ( n ) 18 Estimating big-O (Master Theorem) € Let f be an increasing function that satisfies f ( n ) = a ⋅ f ( n / b ) + c ⋅ nd whenever n = bk^ , where k is a positive integer, a ≥ 1 , b is an integer > 1, and c and d are real numbers with c positive and d nonnegative. Then f ( n ) = O (^) ( n d ) if a < bd O (^) ( n d^ log n ) if a = bd O (^) ( n log b^ a ) if a > bd
From section 7.3 in Rosen
19 Example: Binary Search using the Master Theorem f(n) = a f(n / b) + nd f(n) = O(?) € f ( n ) = O (^) ( n d ) if a < bd O (^) ( n d^ log n ) if a = bd O (^) ( n log b^ a ) if a > bd 20 Example: Max using the Master Theorem f(n) = a f(n / b) + nd f(n) = O(?) € f ( n ) = O (^) ( n d ) if a < bd O (^) ( n d^ log n ) if a = bd O (^) ( n log b^ a ) if a > bd
sortingDemo
http://www.cs.uwaterloo.ca/~bwbecker/sortingDemo/
25 26 Merge Sort: Space requirements Can divide be done in place?
Yes, easy
Can merge sort be done in-place?
Keeping the unmerged parts sorted
In O(n) time?
27 Merge using two arrays Data: Temp:
Step 1:^2^3 7 9 1^4^5 1 2 3 7 9 1 4 5 6 Step 2: 1 2 Step 3:^2 3^7^9 1^4^5 1 2 3 Step 4:^2 3^7^9 1 4^5^6
1 2 3 4 Sorting - Complex^^28
Merge using two arrays 2 3 7 9 1 4 5 6 1 2 3 4 Step 5:
Step 6: 2 3^7^9 1 4 5 1 2 3 4 5 6 Step 7: 2 3 7^9^1 4 5 1 2 3 4 5 6 7 Step 8: 2 3 7 9 1 4 5 6 1 2 3 4 5 6 7 9
29
O(n log(n)) mergeSort – Complexity
At depth i
work done
split merge
total work?
Total depth?
Total work?
30 mergeSort: Recurrence Analysis a = b = d = € O(?) f ( n ) = a ⋅ f ( n / b ) + cn
d
€ f ( n ) = O (^) ( n d ) if a < bd O (^) ( n d^ log n ) if a = bd O (^) ( n log b^ a ) if a > bd 31 Stable Sorting Algorithms
Suppose we are sorting a database of users
according to their name. Users can have identical
names.
A stable sorting algorithm maintains the relative
order of records with equal keys (i.e., sort key
values). Stability: whenever there are two records R
and S with the same key and R appears before S in
the original list, R will appear before S in the sorted
list.
Can we make mergeSort stable? How?
32 MergeSort – Summary Efficient Predictable performance Needs extra space
37 Quick Sort Code // move one item at a time until unknown region is empty for (int u = first + 1; u <= last;++u) { // u: index of first unknown // Invariant: theArray[first+1..lastS1] < pivot // move item from unknown to proper region if (theArray[u].compareTo(pivot) < 0) { // item from unknown belongs in S ++lastS1; swap(theArray,u,lastS1); } // else item from unknown belongs in S } // place pivot in proper position and return its location swap(theArray,first,lastS1); return lastS1; } // let's go look at the demo 38 When things go bad… Worst case
quicksort is O(n^2 ) when the smallest or largest
item is always chosen as the pivot (eg-s?)
39 quickSort – Algorithm Complexity
Depth of call tree?
O(log n) split roughly in half, best and average case
O(n) worst case
Work done at each depth
O(n)
Total Work
O(n log n) best and average case
O(n^2 ) worst case
40 quickSort: Best and average case a= b= d= € O() f ( n ) = a ⋅ f ( n / b ) + cn
d
€ f ( n ) = O (^) ( n d ) if a < bd O (^) ( n d^ log n ) if a = bd O (^) ( n log b^ a ) if a > bd
41 Strategies for Pivot Selection
First value
Worst case - if array is already sorted
Middle value
Better for sorted data, same as previous case for random;
worst case can still happen.
Median of 3 sample values
Worst case O(n^2 ) can still happen
but less likely Let's go look at the demo 42 Improvements Recursion incurs overhead
Dominates cost for small arrays
Hybrid sort algorithm
Use quicksort for large partitions
Use bubble, insertion or selection sort for smaller
arrays
43 Space requirement for quicksort How much memory does quicksort require? 44 How fast can we sort? Observation: all the sorting algorithms so far are comparison sorts
A comparison sort must do at least O(n)
comparisons ( why? )
We have an algorithm that works in O(n log n)
What about the gap between O(n) and O(n log n)
Theorem: all comparison sorts are Ω(n log n) MergeSort is therefore an “optimal” algorithm but in practice quicksort is faster
49 Correctness LSD Radix Sort
Inductive argument (on the number of passes):
Assume array sorted for (i-1) digits
Sorting next digit i makes array correctly sorted for i digits:
If two digits at position i are different, ordering numbers by that digit is correct (lower-order digits irrelevant) If they are the same, numbers are already sorted on the lower-order digits. Since we use a stable sort, the numbers stay in the right order 50 Radix Sort Radix sort is
Fast
Asymptotically fast (i.e., O( n ))
if the numbers are all in range 1..rk, where r is the radix,
we need k sweeps
Simple to code
Can we use it for
strings?
floats?
