Lecture Slides on Recurrences - Object-Oriented Programming | CS 2110, Study notes of Computer Science

Download Lecture Slides on Recurrences - Object-Oriented Programming | CS 2110 and more Study notes Computer Science in PDF only on Docsity!

Solving

Recurrences

Lecture 22 CS2110 – Fall 2008

Prelim tonight!

Analysis of Merge-Sort

Recurrence:

T(n) = c + d + e + f + 2T(n/2) + gn + h T(1) = i

First, simplify by dropping lower-order terms

Simplified recurrence:

T(n) = 2T(n/2) + cn

T(1) = d

How do we find the solution?

Solving Recurrences

 Unfortunately, solving recurrences is like solving

differential equations

 No general technique works for all recurrences

 Luckily, can get by with a few common patterns

 You will learn some more techniques in CS 2800

Solving Recurrences  Recurrences are important when using divide & conquer to design an algorithm  Solution techniques:  Can sometimes change variables to get a simpler recurrence  Make a guess, then prove the guess correct by induction  Build a recursion tree and use it to determine solution  Can use the Master Method  A “cookbook” scheme that handles many common recurrences To solve T(n) = aT(n/b) + f(n) compare f(n) with nlogba  Solution is T(n) = O(f(n)) if f(n) grows more rapidly  Solution is T(n) = O(nlogba) if nlogba^ grows more rapidly  Solution is T(n) = O(f(n) log n) if both grow at same rate  Not an exact statement of the theorem – f(n) must be “well- behaved”

Recurrence Examples  T(n) = T(n – 1) + 1 → T(n) = O(n) Linear Search  T(n) = T(n – 1) + n → T(n) = O(n 2 ) QuickSort worst-case  T(n) = T(n/2) + 1 → T(n) = O(log n) Binary Search  T(n) = T(n/2) + n → T(n) = O(n)  T(n) = 2 T(n/2) + n → T(n) = O(n log n) MergeSort  T(n) = 2 T(n – 1) → T(n) = O( n )

1/10,000 sec 1/2500 sec 1/400 sec 1/100 sec 9/100 sec 1/10 sec 3.2 sec 5.2 min 2.8 hr 28.1 days 1/1000 sec 1 sec 35.7 yr 400 trillion centuries a 75-digit number of centuries 2.8 hr 3.3 trillion years a 70-digit number of centuries a 185-digit number of centuries a 728-digit number of centuries 10 20 50 100 300 n n 2 n n 5 n 2

• The big bang was 15 billion years ago (5·10^17 secs)
  • Source: D. Harel, Algorithmics

How long would it take @ 1 instruction / μsec?

The Fibonacci Function  Mathematical definition: fib(0) = 0 fib(1) = 1 fib(n) = fib(n − 1) + fib(n − 2), n ≥ 2  Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, … int fib(int n) { if (n == 0 || n == 1) return n; else return fib(n-1) + fib(n-2); } Fibonacci (Leonardo Pisano) 1170−1240? Statue in Pisa, Italy Giovanni Paganucci 1863

The Fibonacci Recurrence int fib(int n) { if (n == 0 || n == 1) return n; else return fib(n-1) + fib(n-2); }

T(0) = c

T(1) = c

T(n) = T(n – 1) + T(n – 2) + c

 Solution is exponential in n

 But not quite O(

n

ϕ = (a+b)/b = b/a ϕ 2 = ϕ + 1 ϕ = = 1.618... 1 + √ 5 2 The Golden Ratio a b ratio of sum of sides (a+b) to longer side (b) = ratio of longer side (b) to shorter side (a)

Can We Do Better?

 Number of times loop is executed? n – 1

 Number of basic steps per loop? Constant

 Complexity of iterative algorithm = O(n)

 Much, much, much, much, much, better than O(ϕ

n

if (n <= 1) return n; int parent = 0; int current = 1; for (int i = 2; i ≤ n; i++) { int next = current + parent; parent = current; current = next; } return (current);

f n f n+ f n f n+ f n+ f n+  Let fn denote the n th (^) Fibonacci number  f 0 = 0  f 1 = 1  fn+2 = fn+1 + fn, n^ ≥^0  Note that^ , thus  Can compute the nth power of a matrix by repeated squaring in O(log n) time  Gives complexity O(log n)  Just a little cleverness got us from exponential to logarithmic! ...But We Can Do Even Better! 0 1 1 1 = 0 1 1 1 f 0 f 1 = n

s 1 + s 2 - s 4 + s 6 s 4 + s 5 s 6 + s 7 s 2 - s 3 + s 5 - s 7 a b c d e f g h Matrix Multiplication in Less Than O(n

) (Strassen's Algorithm)  Idea: naive 2 x 2 matrix multiplication takes 8 scalar multiplications, but we can do it in 7: where s 1 = (b - d)(g + h) s 5 = a(f - h) s 2 = (a + d)(e + h) s 6 = d(g - e) s 3 = (a - c)(e + f) s 7 = e(c + d) s 4 = h(a + b) =

S 1

• S 2

• S 4 + S 6 S 4 + S 5 S 6

• S 7 S 2

• S 3

• S 5

• S 7  Break 2 n+ x 2 n+ matrices up into 4 2 n x 2 n submatrices  Multiply them the same way where S 1 = (B - D)(G + H) S 5 = A(F - H) S 2 = (A + D)(E + H) S 6 = D(G - E) S 3 = (A - C)(E + F) S 7 = E(C + D) S 4 = H(A + B) E F G H A B C D Now Apply This Recursively – Divide and Conquer! =