Asymptotic Notation: Understanding Big O, Big Omega, and Big Theta, Slides of Data Structures and Algorithms

Download Asymptotic Notation: Understanding Big O, Big Omega, and Big Theta and more Slides Data Structures and Algorithms in PDF only on Docsity!

Asymptotic Notation

Problem of the Day

The knapsack problem is as follows: given a set of integers S = {s 1 , s 2 ,... , sn}, and a given target number T , find a subset of S which adds up exactly to T. For example, within S = { 1 , 2 , 5 , 9 , 10 } there is a subset which adds up to T = 22 but not T = 23. Find counterexamples to each of the following algorithms for the knapsack problem. That is, give an S and T such that the subset is selected using the algorithm does not leave the knapsack completely full, even though such a solution exists.

The RAM Model of Computation

Algorithms are an important and durable part of computer science because they can be studied in a machine/language independent way. This is because we use the RAM model of computation for all our analysis.

• Each “simple” operation (+, -, =, if, call) takes 1 step.
• Loops and subroutine calls are not simple operations. They depend upon the size of the data and the contents of a subroutine. “Sort” is not a single step operation.

• Each memory access takes exactly 1 step.

We measure the run time of an algorithm by counting the number of steps, where: This model is useful and accurate in the same sense as the flat-earth model (which is useful)!

Best-Case and Average-Case Complexity

The best case complexity of an algorithm is the function defined by the minimum number of steps taken on any instance of size n. The average-case complexity of the algorithm is the function defined by an average number of steps taken on any instance of size n. Each of these complexities defines a numerical function: time vs. size!

Exact Analysis is Hard!

Best, worst, and average are difficult to deal with precisely because the details are very complicated:

1 2 3 4......

It easier to talk about upper and lower bounds of the function. Asymptotic notation (O, Θ, Ω) are as well as we can practically deal with complexity functions. Docsity.com

O , Ω , and Θ

(c)

f(n) c2*g(n)

n0 n

cg(n) c1g(n)

f(n)

n0 n

f(n) c*g(n)

n0 n (a) (b)

The definitions imply a constant n 0 beyond which they are satisfied. We do not care about small values of n.

Formal Definition

• f (n) = O(g(n)) if there are positive constants n 0 and c such that to the right of n 0 , the value of f (n) always lies on or below c · g(n).
• f (n) = Ω(g(n)) if there are positive constants n 0 and c such that to the right of n 0 , the value of f (n) always lies on or above c · g(n).
• f (n) = Θ(g(n)) if there exist positive constants n 0 , c 1 , and c 2 such that to the right of n 0 , the value of f (n) always lies between c 1 · g(n) and c 2 · g(n) inclusive.

Big Omega Examples

3 n^2 − 100 n + 6 = Ω(n^2 ) because 2. 99 n^2 < 3 n^2 − 100 n + 6 3 n^2 − 100 n + 6 6 = Ω(n^3 ) because 3 n^2 − 100 n + 6 < n^3 3 n^2 − 100 n + 6 = Ω(n) because 1010

10 n < 3 n^2 − 100 + 6

Big Theta Examples

3 n^2 − 100 n + 6 = Θ(n^2 ) because O and Ω 3 n^2 − 100 n + 6 6 = Θ(n^3 ) because O only 3 n^2 − 100 n + 6 6 = Θ(n) because Ω only

Big Oh Multiplication by Constant

Multiplication by a constant does not change the asymptotics:

O(c · f (n)) → O(f (n)) Ω(c · f (n)) → Ω(f (n)) Θ(c · f (n)) → Θ(f (n))

Big Oh Multiplication by Function

But when both functions in a product are increasing, both are important:

O(f (n)) ∗ O(g(n)) → O(f (n) ∗ g(n)) Ω(f (n)) ∗ Ω(g(n)) → Ω(f (n) ∗ g(n)) Θ(f (n)) ∗ Θ(g(n)) → Θ(f (n) ∗ g(n))