Recurrence Relations and Master Theorem, Slides of Design and Analysis of Algorithms

Download Recurrence Relations and Master Theorem and more Slides Design and Analysis of Algorithms in PDF only on Docsity!

Recurrences

Objective

running time as recursive function

solve recurrence for order of growth

method: substitution

method: iteration/recursion tree

method: MASTER method

prerequisite:

- mathematical induction, recursive definitions

- arithmetic manipulations, series, products

Running time

will call it T(n) = number of computational steps

required to run the algorithm/program for input of

size n

we are interested in order of growth, not exact

values

- for example T(n) =^ Θ(n

2

) means quadratic running time

- T(n) = O(n logn) means T(n) grows not faster than CONSTnlog(n)

for simple problems, we know the answer right away

- example: finding MAX of an array

- solution: traverse the array, keep track of the max encountered

- running time: one step for each array element, so n steps for array

of size n; linear time T(n) = Θ(n)

Running time for complex problems

complex problems involve solving subproblems, usually

- init/prepare/preprocess, define subproblems

- solve subproblems

- put subproblems results together

thus T(n) cannot be computed straight forward

- instead, follow the subproblem decomposition

Running time for complex problems

many problems can be solved using a divide-and-

conquer strategy

- prepare, solve subproblems, combine results

running time can be written recursively

- T(n) = time(preparation) + time(subproblems) + time(combine)

- for MAX recursive: T(n) = 2*T(n/2) + O(1)

Running time for complex problems

many problems can be solved using a divide-and-

conquer strategy

- prepare, solve subproblems, combine results

running time can be written recursively

- T(n) = time(preparation) + time(subproblems) + time(combine)

- for MAX recursive: T(n) = 2*T(n/2) + O(1)

2 subproblems of size n/ max(array_Left) ; max(array_Right)

Recurrence examples

T(n) = 2T(n/2) + O(1)

T(n)= 2T(n/2) + O(n)

- 2 subproblems of size n/2 each, plus O(n) steps to combine results

T(n) = 4T(n/3) + n

- 4 subproblems of size n/3 each, plus n steps to combine results

T(n/4) + T(n/2) +n

- a subproblem of size n/4, another of size n/2; n

2

to combine

want to solve such recurrences, to obtain the order

of growth of function T

Substitution method

T(n) = 4T(n/2) + n

STEP1 : guess solution, order of growth T(n)= O(n

- that means there is a constant C and a starting value n^0 , such that

T(n)≤Cn

3

, for any n≥n 0

STEP2: verify by induction

- assume T(k)≤k

3

, for k<n

- induction step: prove that T(n)≤Cn

3

, using T(k)≤Ck

3

, for k<n

T (n) = 4 T (

n

) + n (1)

≤ 4 c

n

+ n (2)

c

n

3

+ n (3)

= cn

3

c

n

3

− n) (4)

≤ cn

3

; if

c

n

3

− n > 0 , choose c ≥ 2 (5)

Substitution method

Fallacious argument

- cant prove T(n)=O(n

2

) this way: need same constant steps 3-4-

- maybe its not true? Guess O(n

2

) was too low?

- or maybe we dont^ have the right proof idea

common trick: if math doesnt work out, make a

stronger assumption (subtract a lower degree term)

- assume instead T(k)≤C^1 k

2

-C 2 k, for k<n

- then prove T(n)≤C^1 n

2

-C 2 n, using induction

T (n) = 4 T ( n 2 ) + n (1) ≤ 4 c ￿ n 2 ￿ 2

• n (2) = cn 2
• n (3) = O(n 2 ) (4) ≤ cn 2 (5)

Substitution method

So we can prove T(n)=O(n

), but is that

asymptotically correct?

- maybe we can prove a lower upper bound, like O(nlogn)? NOPE

to make sure its the asymptote, prove its also the

lower bound

- T(n) =Ω(n

2

) or there is a different constant d s.t. T(n) ≥ dn

2

T (n) = 4 T (

n

) + n

c 1

n

2

− c 2

n

+ n

= c 1 n

2

− 2 c 2 n + n

= c 1 n

2

− c 2 n − (c 2 n − n)

≤ c 1 n

2

− c 2 n f or c 2 > 1

Iteration method T (n) = n + 4T ( n 2 ) = n + 4( n 2

• 4T ( n 4 )) = n + 2n + 4 2 T ( n 2 2 ) = n + 2n + 4 2 ( n 2 2
• 4T ( n 2 3 )) = n + 2n + 2 2 n + 4 3 T ( n 2 3 ) = .... = n + 2n + 2 2 n + ... + 2 k− 1 n + 4 k T ( n 2 k ) = k− 1 ￿ i= 2 i n + 4 k T ( n 2 k );

Iteration method T (n) = n + 4T ( n 2 ) = n + 4( n 2

• 4T ( n 4 )) = n + 2n + 4 2 T ( n 2 2 ) = n + 2n + 4 2 ( n 2 2
• 4T ( n 2 3 )) = n + 2n + 2 2 n + 4 3 T ( n 2 3 ) = .... = n + 2n + 2 2 n + ... + 2 k− 1 n + 4 k T ( n 2 k ) = k− 1 ￿ i= 2 i n + 4 k T ( n 2 k ); want k = log(n) ⇔ n 2 k = 1

Iteration method

math can be messy

- recap sum, product, series,

logarithms

- iteration method good for

guess, but usually

unreliable for an exact

result

- use iteration for guess, and

substitution for proofs

stopping condition

- T(...) = T(1), solve for k

T (n) = n + 4T ( n 2 ) = n + 4( n 2

• 4T ( n 4 )) = n + 2n + 4 2 T ( n 2 2 ) = n + 2n + 4 2 ( n 2 2
• 4T ( n 2 3 )) = n + 2n + 2 2 n + 4 3 T ( n 2 3 ) = .... = n + 2n + 2 2 n + ... + 2 k− 1 n + 4 k T ( n 2 k ) = k− 1 ￿ i= 2 i n + 4 k T ( n 2 k );

= n

log(n)− 1

i=

i

log(n)

T (1)

= n

log(n)

+ n

2

T (1)

= n(n − 1) + n

2

T (1) = Θ(n

2

want k = log(n) ⇔ n 2 k = 1

Iteration method: visual tree T (n) T ( n 2 ) T ( n 2 T ( ) n 2 ) T^ (^ n 2 ) T ( n 4 ) T ( n 4 ) T ( n 4 ) T ( n 4 ) T^ (^ n 4 ) T ( n 4 ) T ( n 4 ) T ( n 4 ) T^ (^ n 4 ) T ( n 4 ) T ( n 4 ) T ( n 4 ) T^ (^ n 4 ) T ( n 4 ) T ( n 4 ) T ( n 4 )

+n

+2n +4n