Big-O Notation - Discrete Mathematics - Lecture Slides, Slides of Discrete Mathematics

Download Big-O Notation - Discrete Mathematics - Lecture Slides and more Slides Discrete Mathematics in PDF only on Docsity!

CSE115/ENGR160 Discrete Mathematics 03/01/

3.2 Growth of Functions

• Study number of operations used by algorithm
• For example, given n elements
  • Study the number of comparisons used by the linear and binary search
  • Estimate the number of comparisons used by the bubble sort and insertion sort
• Use big-O notation to study this irrespective of hardware

Big-O notation

• For instance, one algorithm uses 100n 2 +17n+ operations and the other uses n 3 operations
• Can figure out which is more efficient with big-O notation
• The first one is more efficient when n is large even though it uses more operations for smaller values of n, e.g., n=
• Related to big-Omega and big-Theta notations in algorithm design

Big-O notation

• Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers, we say f(x) is O(g(x)) if there are constants C and k such that |f(x)| ≤ C |g(x)| whenever x > k
• Read as f(x) is big-oh of g(x)

Example

• Show that f(x)=x^2 +2x+1 is O(x 2 )
• We observe that we can estimate size of f(x) when x>1 because x 2 >x and x^2 >1, 0≤ x^2 +2x+1 ≤ x^2 +2x 2 +x 2 =4x 2 when x>1. Thus, we can take C=4 and k=1 to show that f(x) is O(x^2 )
• Alternatively, when x>2, 2x≤x^2 and 1≤x^2

0≤ x^2 +2x+1 ≤ x^2 +x 2 +x 2 =3x 2 so C=3, and k=2 are also witnesses to relation f(x) is O(x^2 )

Example

Big-O notation

• When f(x) is O(g(x)) and h(x) is a function that has larger absolute values than g(x) does for sufficient large value of x
• It follows that f(x) is O(h(x))

|f(x)|≤C|g(x)| if x>k and if |h(x)|>|g(x)| for all x>k, then |f(x)|≤C|h(x)| if x>k

• When big-O notation is used, f(x) is O(g(x)) is chosen to be as small as possible (^) 10

Example

• Show that 7x 2 is O(x 3 )
• When x>7, 7x 2 <x 3 , So let C=1 and k=7, we see 7x 2 is O(x 3 )
• Alternatively, when x>1, 7x 2 <7x 3 and so that C=7 and k=1, we have the relationship 7x 2 is O(x 3 )

Example

• Previous example shows that 7x 2 is O(x 3 ). Is it also true that x 3 is also O(7x 2 )
• To show that, x 3 ≤7x^2 is equivalent to x≤7C whenever x>k
• No C exists for which x≤7C for all x>k
• Thus x 3 is not O(7x 2 )

Some important big-O results

• Let f(x)=a (^) nx n^ +a (^) n-1x n-1+…+a 1 x+a 0 , where a 0 , a 1 , …, a (^) n-1, a (^) n are all real numbers, then f(x) is O(x n^ )
• Using the triangle inequality, if x>

14

| ( )| ,where | | | | | | | |, 1

(| | | | | | | |)

(| | | |/ | |/ | |/ )

| | | | | | | |

( ) | |

1 1

1 1

1 1 1

1 1 1

1 1 1

≤ = + + + + =

≤ + + + +

= + + + +

≤ + + + +

= + + + +

−

−

− −

− −

− −

f x Cx C a a a a k

x a a a a

x a a x a x a x

a x a x a x a

f x a x a x ax a

n n n o

n n n o

n n n n o n

n n n n o

n n n n o











Example

• We know that n<2 n^ when n is a positive integer. Show that this implies n is O(2 n^ ) and use this to show that log n is O(n)
• n is O(2n^ ) by taking k=1 and C=
• Thus, log n<n (base 2), so log n is O(n)
• If we have logarithms to a different base b other than 2, we still have log (^) b n is O(n) as log (^) b n = log n/log b < n/log b when n is a positive integer. Take C=1/log b and k=

Growth of functions

Theorems

• Theorem 2: Suppose f 1 (x) is O(g 1 (x)) and f 2 (x) is O(g 2 (x)), - (f 1 +f 2 )(x) is O(max(|g 1 (x)|, |g 2 (x)|) - (f 1 f 2 )(x) is O(g 1 (x)g 2 (x)) |(f 1 f 2 )(x)|=|f 1 (x)||f 2 (x)|≤C 1 |g 1 (x)|C 2 |g 2 (x)| ≤C 1 C 2 |(g 1 g 2 )(x)|≤C|(g 1 g 2 )(x)| (C= C 1 C 2 , k=max(k 1 ,k 2 ))
• Corollary: f 1 (x) and f 2 (x) are both O(g(x)), then (f 1 +f 2 )(x) is O(g(x))

Example

• Big-O notation of f(n)=3n log(n!)+(n 2 +3)logn where n is a positive integer
• We know log(n!) is O(nlog n), so the first part is O(n 2 log n)
• As n^2 +3<2n^2 when n>2, it follows that n 2 +3 is O(n^2 ), and the second part is O(n 2 log n)
• So f(n) is O(n 2 log n)