Download Lecture Notes on Recursive Function Analysis | CS 245 and more Study notes Data Structures and Algorithms in PDF only on Docsity!
03-0: Algorithm Analysis
for (i=1; i<=n*n; i++) for (j=0; j<i; j++) sum++;
03-1: Algorithm Analysis
for (i=1; i<=nn; i++) Executed nn times for (j=0; j<i; j++) Executed <= n*n times sum++; O(1)
Running Time: O(n^4 ) 03-2: Algorithm Analysis
for (i=1; i<=n*n; i++) for (j=0; j<i; j++) sum++;
Exact # of times sum++ is executed:
∑^ n^2
i=
i =
n^2 (n^2 + 1) 2
n^4 + n^2 2 ∈ Θ(n^4 )
03-3: Recursive Functions
long power(long x, long n) { if (n == 0) return 1; else return x * power(x, n-1); }
03-4: Recurrence Relations
T (n) = Time required to solve a problem of size n
Recurrence relations are used to determine the running time of recursive programs – recurrence relations them- selves are recursive
T (0) = time to solve problem of size 0
- Base Case T (n) = time to solve problem of size n
- Recursive Case 03-5: Recurrence Relations
long power(long x, long n) { if (n == 0) return 1; else return x * power(x, n-1); }
T (0) = c 1 for some constant c 1 T (n) = c 2 + T (n − 1) for some constant c 2 03-6: Solving Recurrence Relations T (0) = c 1 T (n) = T (n − 1) + c 2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c 2 03-7: Solving Recurrence Relations T (0) = c 1 T (n) = T (n − 1) + c 2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c 2 T (n − 1) = T (n − 2) + c 2 = T (n − 2) + c 2 + c 2 = T (n − 2) + 2c 2 03-8: Solving Recurrence Relations T (0) = c 1 T (n) = T (n − 1) + c 2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c 2 T (n − 1) = T (n − 2) + c 2 = T (n − 2) + c 2 + c 2 = T (n − 2) + 2c 2 T (n − 2) = T (n − 3) + c 2 = T (n − 3) + c 2 + 2c 2 = T (n − 3) + 3c 2 03-9: Solving Recurrence Relations T (0) = c 1 T (n) = T (n − 1) + c 2
If we knew T (n − 1), we could solve T (n).
T (n) = T (n − 1) + c 2 T (n − 1) = T (n − 2) + c 2 = T (n − 2) + c 2 + c 2 = T (n − 2) + 2c 2 T (n − 2) = T (n − 3) + c 2 = T (n − 3) + c 2 + 2c 2 = T (n − 3) + 3c 2 T (n − 3) = T (n − 4) + c 2 = T (n − 4) + 4c 2
03-10: Solving Recurrence Relations
T (0) = c 1 T (n) = T (n − 1) + c 2
If we knew T (n − 1), we could solve T (n).
03-17: Solving Recurrence Relations T (n) = T (n/2) + c 3 T (n/2) = T (n/4) + c 3 = T (n/4) + c 3 + c 3 = T (n/4)2c 3 T (n/4) = T (n/8) + c 3 = T (n/8) + c 3 + 2c 3 = T (n/8)3c 3 T (n/8) = T (n/16) + c 3 = T (n/16) + c 3 + 3c 3 = T (n/16) + 4c 3 03-18: Solving Recurrence Relations T (n) = T (n/2) + c 3 T (n/2) = T (n/4) + c 3 = T (n/4) + c 3 + c 3 = T (n/4)2c 3 T (n/4) = T (n/8) + c 3 = T (n/8) + c 3 + 2c 3 = T (n/8)3c 3 T (n/8) = T (n/16) + c 3 = T (n/16) + c 3 + 3c 3 = T (n/16) + 4c 3 T (n/16) = T (n/32) + c 3 = T (n/32) + c 3 + 4c 3 = T (n/32) + 5c 3 03-19: Solving Recurrence Relations T (n) = T (n/2) + c 3 T (n/2) = T (n/4) + c 3 = T (n/4) + c 3 + c 3 = T (n/4)2c 3 T (n/4) = T (n/8) + c 3 = T (n/8) + c 3 + 2c 3 = T (n/8)3c 3 T (n/8) = T (n/16) + c 3 = T (n/16) + c 3 + 3c 3 = T (n/16) + 4c 3 T (n/16) = T (n/32) + c 3 = T (n/32) + c 3 + 4c 3 = T (n/32) + 5c 3 =... = T (n/ 2 k) + kc 3 03-20: Solving Recurrence Relations T (0) = c 1 T (1) = c 2 T (n) = T (n/2) + c 3
T (n) = T (n/ 2 k) + kc 3
We want to get rid of T (n/ 2 k). Since we know T (1) ...
n/ 2 k^ = 1 n = 2 k lg n = k
T (n) = T (n/ 2 lg^ n) + lg nc 3 = T (1) + c 3 lg n = c 2 + c 3 lg n ∈ Θ(lg n)
03-21: Solving Recurrence Relations T (1) = c 2 T (n) = T (n/ 2 k) + kc 3
Set k = lg n:
T (n) = T (n/ 2 lg^ n) + (lg n)c 3 = T (n/n) + c 3 lg n = T (1) + c 3 lg n = c 2 + c 3 lg n ∈ Θ(lg n)
03-22: Power Modifications
long power(long x, long n) { if (n==0) return 1; if (n==1) return x; if ((n % 2) == 0) return power(xx, n/2); else return power(xx, n/2) * x; }
03-23: Power Modifications
long power(long x, long n) { if (n==0) return 1; if (n==1) return x; if ((n % 2) == 0) return power(power(x,2), n/2); else return power(power(x,2), n/2) * x; }
This version of power will not work. Why? 03-24: Power Modifications
long power(long x, long n) { if (n==0) return 1; if (n==1) return x; if ((n % 2) == 0) return power(power(x,n/2), 2); else return power(power(x,n/2), 2) * x; }
This version of power also will not work. Why? 03-25: Power Modifications
- We can also do this substitution visually, leads to Recursion Trees
- Consider:
T (n) = 2 T (n/2) + Cn T (1) = C 2 T (0) = C 2
03-30: Recursion Trees
- Start with the recursive definition
T(n) = Cn + 2T(n/2)
03-31: Recursion Trees
- Move the equation around a bit to get:
Cn
T(n/2) T(n/2)
T(n) =
- Repalce each occurance of T (n/2) with T (n/4) + T (n/4) + C(n/2)
03-32: Recursion Trees Cn
C(n/2) C(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
T(n) =
- Replace again, using T (n) = 2T (n/2) + Cn
03-33: Recursion Trees Cn
C(n/2) C(n/2)
C(n/4) C(n/4) C(n/4) C(n/4)
T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8)
T(n) =
- If we continue replacing ...
03-34: Recursion Trees
Cn
C(n/2) C(n/2)
C(n/4) C(n/4) C(n/4) C(n/4)
Cn
Cn
lg n Cn
2 lg n
C C C C C C C C C C C C C ... C C C C C C C
C(n/8) C(n/8) C(n/8) C(n/8) C(n/8) C(n/8) C(n/8) C(n/8)
= n leaves
Cn
Cn( lg n-1) + C n
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 C n 2 2 ∈ Θ(n lg n )
T(n) =
Totals for each level in the tree:
03-35: Recursion Trees
T (1) = C 1
T (n) = T (n − 1) + C 2
03-36: Recursion Trees
T (0) = C 1
T (1) = C 1
T (n) = T (n/2) + C 2
03-37: Substitution Method
- We can prove that a bound is correct using induction, this is the substituion method
T (1) = C 1
T (n) = T (n − 1) + C 2
Show: T (n) ∈ O(? ) 03-38: Substitution Method
- We can prove that a bound is correct using induction, this is the substituion method
T (1) = C 1
T (n) = T (n − 1) + C 2
T (n) ≥ C ∗ n for all n > n 0 , for some pair of constants C, n 0 03-44: Substitution Method T (1) = C 1 T (n) = T (n − 1) + C 2
Show: T (n) ∈ Ω(n), that is, T (n) ≥ C ∗ n
- Base case: T (1) = C 1 ≥ C ∗ 1 for some constant C
This is true as long as C ≤ C 1.
03-45: Substitution Method
T (1) = C 1
T (n) = T (n − 1) + C 2
Show: T (n) ∈ Ω(n), that is, T (n) ≥ C ∗ n
T (n) = T (n − 1) + C 2 Recurrence definition
03-46: Substitution Method
T (1) = C 1
T (n) = T (n − 1) + C 2
Show: T (n) ∈ Ω(n), that is, T (n) ≥ C ∗ n
T (n) = T (n − 1) + C 2 Recurrence definition ≥ C(n − 1) + C 2 Inductive hypothesis
03-47: Substitution Method
T (1) = C 1
T (n) = T (n − 1) + C 2
Show: T (n) ∈ Ω(n), that is, T (n) ≥ C ∗ n
T (n) = T (n − 1) + C 2 Recurrence definition ≥ C(n − 1) + C 2 Inductive hypothesis ≥ Cn + (C 2 − C) Algebra ≥ Cn If C ≤ C 2 This is true as long as C ≤ C 1. 03-48: Substitution Method
T (0) = C 2
T (1) = C 2
T (n) = 2 T (n/2) + C 1 n
Show: T (n) ∈ O(n lg n), that is, T (n) ≤ C ∗ n lg n 03-49: Substitution Method T (0) = C 2 T (1) = C 2 T (n) = 2 T (n/2) + C 1 n Show: T (n) ∈ O(n lg n), that is, T (n) ≤ C ∗ n lg n
- Base cases:
- T (0) = C 1 ≤ C ∗ 0 lg 0 for some constant C
- T (1) = C 1 ≤ C ∗ 1 lg 1 for some constant C
Hmmm.... 03-50: Substitution Method T (0) = C 2 T (1) = C 2 T (n) = 2 T (n/2) + C 1 n Show: T (n) ∈ O(n lg n), that is, T (n) ≤ C ∗ n lg n
- Only care about n > n 0. We can pick 2, 3 as base cases (why?)
- T (2) = C 1 ≤ C ∗ 2 lg 2 for some constant C
- T (3) = C 1 ≤ C ∗ 3 lg 3 for some constant C
03-51: Substitution Method T (0) = C 2 T (1) = C 2 T (n) = 2 T (n/2) + C 1 n T (n) = 2 T (n/2) + C 1 n Recurrence Definition 03-52: Substitution Method T (0) = C 2 T (1) = C 2 T (n) = 2 T (n/2) + C 1 n T (n) = 2 T (n/2) + C 1 n Recurrence Definition ≤ 2 C(n/2) lg(n/2) + C 1 n Inductive hypothesis 03-53: Substitution Method T (0) = C 2 T (1) = C 2 T (n) = 2 T (n/2) + C 1 n T (n) = 2 T (n/2) + C 1 n Recurrence Definition ≤ 2 C(n/2) lg(n/2) + C 1 n Inductive hypothesis ≤ Cn lg n/2 + C 1 n Algebra ≤ Cn lg n − Cn lg 2 + C 1 n Algebra ≤ Cn lg n − Cn + C 1 n Algebra 03-54: Substitution Method T (0) = C 2 T (1) = C 2 T (n) = 2 T (n/2) + C 1 n T (n) = 2 T (n/2) + C 1 n Recurrence Definition ≤ 2 C(n/2) lg(n/2) + C 1 n Inductive hypothesis ≤ Cn lg n/2 + C 1 n Algebra ≤ Cn lg n − Cn lg 2 + C 1 n Algebra ≤ Cn lg n − Cn + C 1 n Algebra ≤ Cn lg n If C > C 1
