Download Asymptotic Notation: Understanding Function Growth and more Study notes Algorithms and Programming in PDF only on Docsity!
CS231 Algorithms Handout # Prof. Lyn Turbak January 31, 2001 Wellesley College
ASYMPTOTICS AND FUNCTIONS
Note: This handout summarizes highlights of CLR Chapter 2. See the book for more details.
--------------------------------------------------------------------------------------------------------------------- Motivation
Fine-grained bean counting exposes too much detail for comparing functions.
Want a course-grained way to compare functions that ignores constant factors and focuses on their relative growth in the limit as input sizes get large.
For example, consider:
n = 1 n = 1,000 n = 1,000,
p(n) = 100n + 1000
q(n) = 3n^2 + 2n + 1
r(n) = 0.1n^2
Sketch the above functions on the same set of axes:
How Do Your Functions Grow?
Asymptotic notation is a way of characterizing functions that facilitates comparing their growth in the limit of large inputs. Here is an informal summary of the notation:
Notation Pronunciation Loosely
f ∈ ω(g) f is way bigger than g^ f > g
f ∈ Ω(g) f is at least as big as g^ f^ ≥^ g
f ∈ Θ(g) f is about the same as g^ f = g
f ∈ O(g) f is at most as big as g^ f^ ≤^ g
f ∈ o(g) f is way smaller than g^ f < g
Notes:
- Each of ω(g), Ω(g), Θ(g), O(g), o(g) denotes a set of functions. Thus, ω(g) is the set of all
functions way bigger than g, Ω(g) is the set of all functions at least as big as g, etc.
- The notation f = ω(g) is really shorthand for f ∈ ω(g).
- The phrases “is at least O(...)” and “is at most Ω(...)” are non-sensical. “Is at least”
should be written Ω, and “is at most” should be written O.
Intuitively, what are the relationships between p, q, and r?
Formalizing o and ω
f ∈ o(g) if limn→∞ ^
f(n) g(n) =^0
f ∈ ω(g) if limn→∞ ^
f(n) g(n) =^ ∞
Examples:
Show p∈ o(r)
Show r∈ ω (r)
Formalizing O, Ω, and Θ
O(g) = {f | there exist positive constants c, n 0 such that
0 ≤ f(n) ≤ cg(n) for all n ≥ n 0 .}
Think of this as a game. Suppose you claim that f ∈ O(g). Then you select c and n 0 , but I
try to find a particular n that defeats your claim.
Ω(g) = {f | there exist positive constants c, n 0 such that
0 ≤ cg(n) ≤ f(n) for all n ≥ n 0 .}
Θ(g) = {f | there exist positive constants c 1 , c 2 , n 0 such that
0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n) for all n ≥ n 0 .}
Does anything Fall Between the Cracks?
The diagram on p. 3 implies that there are functions that are O(g) that are neither o(g) nor Θ (g),
and there are functions that are Ω(g) that are neither ω(g) nor Θ (g). Here's a concrete example:
f(n) = 1/n g(n) = n h(n) = nsin(n)
Show that h ∈ 0 (g) , but h ∉ o (g) and h ∉ Θ (g). (Similarly, h ∈ Ω (f), but h ∉ ω (g) and h ∉ Θ (g)).
Incomparable Functions
Not every two functions are comparable. Is k(n) = sqrt(n) related to h(n) above?
Exponentials
Notation:
- an^ = the product of n copies of a. - a-n^ =
1 an
Key Identities:
- am^ an^ = am+n^ {Special case: a^0 = 1.}
- (am^ )n^ = amn^ = (an^ )m
Examples:
(5^2 )^3 = 52 ⋅⋅ 53 = 52 ⋅+ 53 =
3 (^2) =
Let:
f(n) = 2n g(n) = 3n h(n) = 2cn k(n) = 2c+n
What symbols can fill the following blanks?
g ∈ ____ (f)
h ∈ ____ (f) (c < 1)
h ∈ ____ (f) (c = 1)
h ∈ ____ (f) (c > 1)
k ∈ ____ (f) (any c)
Asymptotics Involving Exponentials and Logarithms
How do log 2 n and log 3 n compare?
How do 2n^ and 3n^ compare?
Fact 1: if a > 0, limn→∞
an nb^
Fact 1 implies an^ ∈ ω(nb^ ). In other words: Any positive exponential grows faster than any polynomial.
Substituting lg n for n and 2a^ for a in Fact 1 yields:
Fact 2: if a > 0, limn→∞
na lgbn
Fact 2 implies na^ ∈ ω(lgb^ n). In other words: Any positive polynomial grows faster than any polylogarithmic function.
--------------------------------------------------------------------------------------------------------------------- Factorials
Definition: n! = 1⋅ 2 ⋅ 3 ... ⋅ n
Stirling's approximation: n! ≈ 2 πn
n e
n
Asymptotics derivable from Stirling's approximation:
- n! = o(nn^ )
- n! = ω(2n )
- lg(n!) = Θ(n lg n) ---------------------------------------------------------------------------------------------------------------------
