Understanding Algorithms: Properties, Examples, Complexity and Big-O Notation, Slides of Discrete Structures and Graph Theory

Download Understanding Algorithms: Properties, Examples, Complexity and Big-O Notation and more Slides Discrete Structures and Graph Theory in PDF only on Docsity!

Enough Mathematical Appetizers!

• Let us look at something more interesting:

• Algorithms

Algorithms

• What is an algorithm?
• An algorithm is a finite set of precise instructions for performing a computation or for solving a problem.
• This is a rather vague definition. You will get to know a more precise and mathematically useful definition when you attend CS420.
• But this one is good enough for now…

Algorithm Examples

• We will use a pseudocode to specify algorithms, which slightly reminds us of Basic and Pascal.
• Example: an algorithm that finds the maximum element in a finite sequence
• procedure max(a 1 , a 2 , …, a (^) n : integers)
• max := a (^1)
• for i := 2 to n
• if max < a (^) i then max := a (^) i
• {max is the largest element}

Algorithm Examples

• Another example: a linear search algorithm, that is, an algorithm that linearly searches a sequence for a particular element.
• procedure linear_search(x: integer; a 1 , a 2 , …, a (^) n : integers)
• i := 1
• while (i ≤ n and x ≠ a (^) i )
• i := i + 1
• if i ≤ n then location := i
• else location := 0
• {location is the subscript of the term that equals x, or is zero if x is not found}

Algorithm Examples

a c d f g h j l m o p r s u v x z

binary search for the letter ‘j’

center element

search interval

Algorithm Examples

a c d f g h j l m o p r s u v x z

binary search for the letter ‘j’

center element

search interval

Algorithm Examples

a c d f g h j l m o p r s u v x z

binary search for the letter ‘j’

center element

search interval

Algorithm Examples

a c d f g h j l m o p r s u v x z

binary search for the letter ‘j’

center element

search interval

found!

Complexity

• In general, we are not so much interested in the time and space complexity for small inputs.
• For example, while the difference in time complexity between linear and binary search is meaningless for a sequence with n = 10, it is gigantic for n = 2^30.

Complexity

• For example, let us assume two algorithms A and B that solve the same class of problems.
• The time complexity of A is 5,000n, the one for B is 1.1 n for an input with n elements.
• For n = 10, A requires 50,000 steps, but B only 3, so B seems to be superior to A.
• For n = 1000, however, A requires 5,000,000 steps, while B requires 2.5⋅ 10 41 steps.

Complexity

• Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B n 10 100 1, 1,000,

5,000n 50, 500, 5,000, 5 ⋅ 109

1.1 n 3

Complexity

• This means that algorithm B cannot be used for large inputs, while running algorithm A is still feasible.
• So what is important is the growth of the complexity functions.
• The growth of time and space complexity with increasing input size n is a suitable measure for the comparison of algorithms.

The Growth of Functions

• When we analyze the growth of complexity functions , f(x) and g(x) are always positive.
• Therefore, we can simplify the big-O requirement to
• f(x) ≤ C⋅g(x) whenever x > k.
• If we want to show that f(x) is O(g(x)), we only need to find one pair (C, k) (which is never unique).

The Growth of Functions

• The idea behind the big-O notation is to establish an upper boundary for the growth of a function f(x) for large x.
• This boundary is specified by a function g(x) that is usually much simpler than f(x).
• We accept the constant C in the requirement
• f(x) ≤ C⋅g(x) whenever x > k,
• because C does not grow with x.
• We are only interested in large x, so it is OK if f(x) > C⋅g(x) for x ≤ k.