Download Growth of Functions and Algorithm Complexity in Applied Discrete Mathematics and more Slides Discrete Mathematics in PDF only on Docsity!
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 1
Complexity
This means that algorithm B cannot be used for large inputs, while running algorithm A is still feasible.
So what is important is the growth of the complexity functions.
The growth of time and space complexity with increasing input size n is a suitable measure for the comparison of algorithms.
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 2
The Growth of Functions
The growth of functions is usually described using the big-O notation.
Definition: Let f and g be functions from the integers or the real numbers to the real numbers. We say that f(x) is O(g(x)) if there are constants C and k such that
|f(x)| ≤ C|g(x)|
whenever x > k.
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 3
The Growth of Functions
When we analyze the growth of complexity functions , f(x) and g(x) are always positive.
Therefore, we can simplify the big-O requirement to
f(x) ≤ C⋅g(x) whenever x > k.
If we want to show that f(x) is O(g(x)), we only need to find one pair (C, k) (which is never unique).
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 4
The Growth of Functions
The idea behind the big-O notation is to establish an upper boundary for the growth of a function f(x) for large x. This boundary is specified by a function g(x) that is usually much simpler than f(x). We accept the constant C in the requirement f(x) ≤ C⋅g(x) whenever x > k, because C does not grow with x. We are only interested in large x, so it is OK if f(x) > C⋅g(x) for x ≤ k.
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 5
The Growth of Functions
Example:
Show that f(x) = x^2 + 2x + 1 is O(x^2 ).
For x > 1 we have:
x^2 + 2x + 1 ≤ x^2 + 2x^2 + x^2 ⇒ x^2 + 2x + 1 ≤ 4x^2
Therefore, for C = 4 and k = 1:
f(x) ≤ Cx^2 whenever x > k.
⇒ f(x) is O(x^2 ).
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 6
The Growth of Functions
Question: If f(x) is O(x^2 ), is it also O(x^3 )?
Yes. x^3 grows faster than x^2 , so x^3 grows also faster than f(x).
Therefore, we always have to find the smallest simple function g(x) for which f(x) is O(g(x)).
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 7
The Growth of Functions
“Popular” functions g(n) are n log n, 1, 2n, n^2 , n!, n, n^3 , log n
Listed from slowest to fastest growth:
- 1
- log n
- n
- n log n
- n^2
- n^3
- 2n
- n!
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 8
The Growth of Functions
A problem that can be solved with polynomial worst- case complexity is called tractable.
Problems of higher complexity are called intractable.
Problems that no algorithm can solve are called unsolvable.
You will find out more about this in CS420.
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 9
Useful Rules for Big-O
For any polynomial f(x) = anxn^ + an-1xn-1^ + B + a 0 , where a 0 , a 1 , B, an are real numbers, f(x) is O(xn).
If f 1 (x) is O(g(x)) and f 2 (x) is O(g(x)), then (f 1 + f 2 )(x) is O(g(x)).
If f 1 (x) is O(g 1 (x)) and f 2 (x) is O(g 2 (x)), then (f 1 + f 2 )(x) is O(max(g 1 (x), g 2 (x)))
If f 1 (x) is O(g 1 (x)) and f 2 (x) is O(g 2 (x)), then (f 1 f 2 )(x) is O(g 1 (x) g 2 (x)). February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 10
Complexity Examples
What does the following algorithm compute? procedure who_knows(a 1 , a 2 , B, an: integers) who_knows := 0 for i := 1 to n- for j := i+1 to n if |ai – aj| > who_knows then who_knows := |ai – aj| {who_knows is the maximum difference between any two numbers in the input sequence} Comparisons: n-1 + n-2 + n-3 + B + 1 = (n – 1)n/2 = 0.5n^2 – 0.5n
Time complexity is O(n^2 ).
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 11
Complexity Examples
Another algorithm solving the same problem:
procedure max_diff(a 1 , a 2 , B, an: integers) min := a 1 max := a 1 for i := 2 to n if ai < min then min := ai else if ai > max then max := ai max_diff := max - min
Comparisons (worst case): 2n - 2
Time complexity is O(n).
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 12
Let us talk aboutB
Predicate Calculus
B so that you can understand
Assignment #1 better.
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 19
Quantification
Another example:
Let the universe of discourse be the real numbers.
What does ∀x∃y (x + y = 320) mean?
“For every x there exists a y so that x + y = 320.”
Is it true?
Is it true for the natural numbers?
yes
no
February 12, 2013 Applied Discrete Mathematics Week 3: Algorithms 20
Disproof by Counterexample
A counterexample to ∀x P(x) is an object c so that P(c) is false.
Statements such as ∀x (P(x) →→→→ Q(x)) can be disproved by simply providing a counterexample.
Statement: “All birds can fly.” Disproved by counterexample: Penguin.
