Algorithm Analysis: Understanding the Asymptotic Notation and Complexity of Algorithms, Lecture notes of Algorithms and Programming

Download Algorithm Analysis: Understanding the Asymptotic Notation and Complexity of Algorithms and more Lecture notes Algorithms and Programming in PDF only on Docsity!

Week 1 Recap

› Stable matching problem.

› Gale-Shapley algorithm

• Runs in O(n^2 ) time
• Solutions are “optimal”

Stable Matching

› Question: For a given problem instance, there may be several stable matchings. Do all executions of Gale-Shapley yield the same stable matching? If so, which one?

› An instance with two stable matchings.

• X-A, Y-B, Z-C = (A-X, B-Y, C-Z)
• A-Y, B-X, C-Z.

Zeus

Yancey

Xavier

A

B

A

1 st

B

A

B

2 nd

C

C

C

3 rd

Clare

Bertha

Amy

X

X

Y

1 st

Y

Y

X

2 nd

Z

Z

Z

3 rd

Mathematics Refresher

Plotting Functions

Computational Tractability

"For me, great algorithms are the poetry of computation. Just like verse, they can be terse, allusive, dense, and even mysterious. But once unlocked, they cast a brilliant new light on some aspect of computing." - Francis Sullivan

A tractable problem is one that can be solved in a reasonable amount

of time; usually the distinction between tractable and intractable is

drawn at the boundary between problems that can be solved in an

amount of time that is polynomial ; those that require exponential time

are regarded as intractable.

There are two aspects of algorithmic performance:

› Time

› Instructions take time. › How fast does the algorithm perform? › What affects its runtime?

› Space

› Data structures take space › What kind of data structures can be used? › How does choice of data structure affect the runtime?

We will focus on time:

› How to estimate the time required for an algorithm

› How to reduce the time required

Algorithm Analysis

We will analyze algorithms by employing

mathematical techniques that analyze algorithms

independently of specific implementations,

computers, or data.

Algorithm Performance

› Worst-Case Analysis › The maximum amount of time that an algorithm require to solve a problem of size n. › This gives an upper bound for the time complexity of an algorithm. › Normally, we try to find worst-case behavior of an algorithm. › Best-Case Analysis › The minimum amount of time that an algorithm require to solve a problem of size n. › The best case behavior of an algorithm is NOT so useful. › Average-Case Analysis › The average amount of time that an algorithm require to solve a problem of size n. › Sometimes, it is difficult to find the average-case behavior of an algorithm. W orst-case analysis is more common than average-case analysis.

Kinds of Analyses

How many steps are taken in the execution of these codes?

if(x < y) { int temp = x; x = y; y = temp; }

It depends (more if x<y, less otherwise)! The worst case (when x<y): Num of steps = 1 + 1 + 1 + 1 = 4

Second Taste of Analysis (with condition)

How many steps are taken in the execution of this loop?

for (i = 0; i < 10 ; i++) { ar[i] = i; }

1  10 + 1

1  10

1  10

Number of steps = 1 + (1 ⋅ 10 + 1) + (1 ⋅ 10) + (1 ⋅ 10) = 32

1

Third Taste of Analysis (with loop)

Functions

Constant Logarithmic Poly Logarithmic Polynomial Exponential Factorial

5 5 log n (log n) 5 n^5 2 5n n!

(log n)^5 = log 5 n

Class of Functions

Polynomial

Linear Quadratic Cubic?

5n 5n 2 5n^3 5n^

Polynomial Functions

Worst-Case Analysis

› Worst case running time. Obtain bound on largest possible running time of algorithm on input of a given size N.

• Generally captures efficiency in practice.
• Draconian view, but hard to find effective alternative.

› Average case running time. Obtain bound on running time of algorithm on random input as a function of input size N.

• Hard (or impossible) to accurately model real instances by random distributions.
• Algorithm tuned for a certain distribution may perform poorly on other inputs.

Worst-Case Polynomial-Time

› Definition: An algorithm is efficient if its running time is polynomial.

› Justification: It really works in practice!

• Although 6.02 × 1023 × N^20 is poly-time, it is useless in practice.
• In practice, the poly-time algorithms that people develop almost always have low constants and low exponents.
• Breaking through the exponential barrier of brute force typically exposes some crucial structure of the problem.

› Exceptions.

• Some poly-time algorithms do have high constants and/or exponents, and are useless in practice.
• Some exponential-time (or worse) algorithms are widely used because the worst- case instances seem to be rare.

Unix grep