1
Algorithmic Complexity 3
Fawzi Emad
Chau-Wen Tseng
Department of Computer Science
University of Maryland, College Park
Overview
Recurrence relations
Complexity categories
Comparing complexity
Types of complexity analysis
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Algorithmic Complexity: Understanding Recurrence Relations and Complexity Categories - Pro, Study notes of Computer Science
An overview of algorithmic complexity, focusing on recurrence relations and complexity categories. It covers the definition and components of recursive algorithms, the concept of base and recursive cases, and the use of recurrence relations to analyze the complexity of algorithms. The document also includes examples of recurrence relations and their solutions, as well as a discussion on comparing complexity and calculating asymptotic complexity.
Typology: Study notes
Pre 2010
1 / 14
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Algorithmic Complexity: Understanding Recurrence Relations and Complexity Categories - Pro and more Study notes Computer Science in PDF only on Docsity!
Algorithmic Complexity 3
Fawzi Emad
Chau-Wen Tseng
Department of Computer Science
University of Maryland, College Park
Overview
Recurrence relations
Complexity categories
Comparing complexity
Types of complexity analysis
Critical Section Example 7
Code (for input size n)
- DoWork (int n)
- if (n == 1)
- A
- else
- DoWork(n/2)
- DoWork(n/2)
Code execution
A ⇒ 1 times DoWork(n/2) ⇒ 2 times
Time(1) ⇒ 1 Time(n) = 2 × Time(n/2) + 1
critical
sections
Recursive Algorithms
Definition
An algorithm that calls itself
Components of a recursive algorithm
- Base cases Computation with no recursion
- Recursive cases Recursive calls Combining recursive results
Recurrence Relations
Base case
Value of function at some specified points Also called boundary values / boundary conditions
Base case example
T(1) = 0
T(1) = 1
T(2) = 1
T(2) = k
Solving Recurrence Equations
Back substitution (iteration method)
Iteratively substitute recurrence into formula
Example
T(1) = 5
T(n) = T(n-1) + 1 = ( T(n-2) + 1 ) + 1 = ( ( T(n-3) + 1 ) + 1 ) + 1 = ( ( ( T(n-4) + 1 ) + 1 ) + 1 ) + 1 = … = (…( T(1) + 1 ) + … ) + 1 = (…( 5 + 1 ) + … ) + 1 = n + 4
Example Recurrence Solutions
Examples
T(n) = T(n-1) + k ⇒ O(n) T(n) = T(n-1) + n ⇒ O(n^2 ) T(n) = T(n-1) + T(n-2) ⇒ O(n!) T(n) = T(n/2) + k ⇒ O(log(n)) T(n) = 2 × T(n/2) + k ⇒ O(n) T(n) = 2 × T(n-1) + k ⇒ O(2 n)
Many additional issues, solution methods
Take CMSC 351 – Introduction to Algorithms
Asymptotic Complexity Categories
Complexity Name Example
O(1) Constant Array access O(log(n)) Logarithmic Binary search O(n) Linear Largest element O(n log(n)) N log N Optimal sort O(n^2 ) Quadratic 2D Matrix addition O(n^3 ) Cubic 2D Matrix multiply O(nk) Polynomial Linear programming O(k n^ ) Exponential Integer programming
From smallest to largest
For size n, constant k > 1
Calculating Asymptotic Complexity
As n increases
Highest complexity term dominates Can ignore lower complexity terms
Examples
2 n + 100 ⇒ O(n) n log(n) + 10 n ⇒ O(nlog(n)) ½ n^2 + 100 n ⇒ O(n^2 ) n^3 + 100 n^2 ⇒ O(n^3 ) 1/100 2 n^ + 100 n^4 ⇒ O(2n^ )
Complexity Examples
2n + 100 ⇒ O(n)
0
100000
200000
300000
400000
500000
600000
700000
800000
(^1349120260533) (^10682118417582081611131602)
n nlog(n) 2 n + 100
Complexity Examples
½ n log(n) + 10 n ⇒ O(nlog(n))
0
100000
200000
300000
400000
500000
600000
700000
800000
(^22879178373756) (^150629755855115012256544252)
n nlog(n) 1/2 n log(n) + 10 n
Complexity Examples
½ n^2 + 100 n ⇒ O(n^2 )
0
100000
200000
300000
400000
500000
600000
700000
800000
(^22879) (^178373756150629755855) 115012256544252
nlog(n) n^2 1/2 n^2 + 100 n
Complexity Comparison Examples
log(n) vs. n½
1.001 n^ vs. n^1000
lim n→∞
f(n)
g(n)
lim n→∞
log(n)
n½^
lim n→∞
f(n)
g(n)
lim n→∞
1.001 n
n^1000
Not clear, use L’Hopital’s Rule
L’Hopital’s Rule
If ratio is indeterminate
0 / 0 or ∞ / ∞
Ratio of derivatives computes same value
Can simplify ratio by repeatedly taking
derivatives of numerator & denominator
lim n→∞
f(n)
g(n)
lim n→∞
f'(n)
g'(n)
=
Using L’Hopital’s Rule
1.001 n^ vs. n^1000
lim n→∞
f(n)
g(n)
lim n→∞
1.001 n
n^1000
lim n→∞
n 1.001 n-
1000 n^999
lim n→∞
n (n-1) 1.001 n-
1000 ×999 n^998
… ∞
Additional Complexity Measures
Upper bound
Big-O ⇒ Ο(…) Represents upper bound on # steps
Lower bound
Big-Omega ⇒ Ω(…) Represents lower bound on # steps
Combined bound
Big-Theta ⇒ Θ(…) Represents combined upper/lower bound on # steps Best possible asymptotic solution
NP Time Algorithm
Polynomial solution possible
If make correct guesses on how to proceed
Required for many fundamental problems
Boolean satisfiability Traveling salesman problem (TLP) Bin packing
Key to solving many optimization problems
Most efficient trip routes Most efficient schedule for employees Most efficient usage of resources
NP Time Algorithm
Properties of NP
Can be solved with exponential time Not proven to require exponential time Currently solve using heuristics
NP-complete problems
Representative of all NP problems Solution can be used to solve any NP problem Examples Boolean satisfiability Traveling salesman
P = NP?
Are NP problems solvable in polynomial time?
Prove P=NP Show polynomial time solution exists for any NP-complete problem Prove P≠NP Show no polynomial-time solution possible The expected answer
Important open problem in computer science
$1 million prize offered by Clay Math Institute
Algorithmic Complexity Summary
Asymptotic complexity
Fundamental measure of efficiency Independent of implementation & computer platform
Learned how to
Examine program Find critical sections Calculate complexity of algorithm Compare complexity