Asymptotic Notation - Algorithm - Study Notes, Study notes of Algorithms and Programming

Download Asymptotic Notation - Algorithm - Study Notes and more Study notes Algorithms and Programming in PDF only on Docsity!

Design and Analysis of Algorithms

Chapter 2

Asymptotic Notation

You may be asking that we continue to use the notation Θ() but have never defined it. Let’s remedy this now. Given any function g(n), we define Θ(g(n)) to be a set of functions that asymptotically equivalent to g(n). Formally:

Θ(g(n)) = {f(n) | there exist positive constants c 1 , c 2 and n 0 such that 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n) for all n ≥ n 0 }

This is written as “f(n) ∈ Θ(g(n))” That is, f(n) and g(n) are asymptotically equivalent. This means that they have essentially the same growth rates for large n. For example, functions like

• 4n^2 ,
• (8n^2 + 2n − 3 ),
• (n^2 /5 +

n − 10 log n)

• n(n − 3 )

are all asymptotically equivalent. As n becomes large, the dominant (fastest growing) term is some constant times n^2.

Consider the function f(n) = 8n^2 + 2n − 3

Our informal rule of keeping the largest term and ignoring the constant suggests that f(n) ∈ Θ(n^2 ). Let’s see why this bears out formally. We need to show two things for

f(n) = 8n^2 + 2n − 3

Lower bound f(n) = 8n^2 + 2n − 3 grows asymptotically at least as fast as n^2 ,

23

both the upper and lower bounds. The lower bound is satisfied because f(n) = 8n^2 + 2n − 3 does grow at least as fast asymptotically as n. But the upper bound is false. Upper bounds requires that there exist positive constants c 2 and n 0 such that f(n) ≤ c 2 n for all n ≥ n 0.

Informally we know that f(n) = 8n^2 + 2n − 3 will eventually exceed c 2 n no matter how large we make c 2. To see this, suppose we assume that constants c 2 and n 0 did exist such that 8n^2 + 2n − 3 ≤ c 2 n for all n ≥ n 0 Since this is true for all sufficiently large n then it must be true in the limit as n tends to infinity. If we divide both sides by n, we have

nlim→∞

8n + 2 −

n

≤ c 2.

It is easy to see that in the limit, the left side tends to ∞. So, no matter how large c 2 is, the statement is violated. Thus f(n) 6 ∈ Θ(n).

Let’s show that f(n) 6 ∈ Θ(n^3 ). The idea would be to show that the lower bound f(n) ≥ c 1 n^3 for all n ≥ n 0 is violated. (c 1 and n 0 are positive constants). Informally we know this to be true because any cubic function will overtake a quadratic.

If we divide both sides by n^3 :

lim n→∞

n

n^2

n^3

≥ c 1

The left side tends to 0. The only way to satisfy this is to set c 1 = 0. But by hypothesis, c 1 is positive. This means that f(n) 6 ∈ Θ(n^3 ).

The definition of Θ-notation relies on proving both a lower and upper asymptotic bound. Sometimes we only interested in proving one bound or the other. The O-notation is used to state only the asymptotic upper bounds.

O(g(n)) = {f(n) | there exist positive constants c and n 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n 0 }

The Ω-notation allows us to state only the asymptotic lower bounds.

Ω(g(n)) = {f(n) | there exist positive constants c and n 0 such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n 0 }

26 CHAPTER 2. ASYMPTOTIC NOTATION

The three notations:

Θ(g(n)) : 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n) O(g(n)) : 0 ≤ f(n) ≤ cg(n) Ω(g(n)) : 0 ≤ cg(n) ≤ f(n) for all n ≥ n 0

These definitions suggest an alternative way of showing the asymptotic behavior. We can use limits for define the asymptotic behavior. Limit rule for Θ-notation:

lim n→∞

f(n) g(n)

= c,

for some constant c > 0 (strictly positive but not infinity) then f(n) ∈ Θ(g(n)). Similarly, the limit rule for O-notation is

nlim→∞

f(n) g(n)

= c,

for some constant c ≥ 0 (nonnegative but not infinite) then f(n) ∈ O(g(n)) and limit rule for Ω-notation:

lim n→∞

f(n) g(n)

(either a strictly positive constant or infinity) then f(n) ∈ Ω(g(n)).

Here is a list of common asymptotic running times:

• Θ( 1 ): Constant time; can’t beat it!
• Θ(log n): Inserting into a balanced binary tree; time to find an item in a sorted array of length n using binary search.
• Θ(n): About the fastest that an algorithm can run.
• Θ(n log n): Best sorting algorithms.
• Θ(n^2 ), Θ(n^3 ): Polynomial time. These running times are acceptable when the exponent of n is small or n is not to large, e.g., n ≤ 1000.
• Θ( 2 n), Θ( 3 n): Exponential time. Acceptable only if n is small, e.g., n ≤ 50.
• Θ(n!), Θ(nn): Acceptable only for really small n, e.g. n ≤ 20.