Analysis of Algorithm - Lecture 3 - Data Structures and Algorithm | CSC 3102, Study notes of Computer Science

Download Analysis of Algorithm - Lecture 3 - Data Structures and Algorithm | CSC 3102 and more Study notes Computer Science in PDF only on Docsity!

CSC 3102

Adv Data Structures and

Algorithm Analysis

Lecture 3

This lecture ….

• Analysis of algorithms: Growth

functions

• Quicksort

Experimental Studies

• (^) Write a program implementing the algorithm
• (^) Run the program with inputs of varying size and composition
• (^) Use a function, like the built-in clock() function, to get an accurate measure of the actual running time
• (^) Plot the results 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 0 50 100 Input Size Time (ms)

Limitations of Experiments

• It is necessary to implement the

algorithm, which may be difficult

• Results may not be indicative of the

running time on other inputs not

included in the experiment.

• In order to compare two algorithms,

the same hardware and software

environments must be used

Growth Rates

• (^) Growth rates of

functions:

• (^) Linear  n
• (^) Quadratic  n^2
• (^) Cubic  n^3
• (^) In a log-log chart,

the slope of the

line corresponds

to the growth rate

of the function

1E- 1 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E- 1 1E+1 1E+3 1E+5 1E+7 1E+ T ( n ) n Cubic Quadratic Linear

Recall that for input size n,

• Insertion Sort ’s running time is:

An

2

+ Bn + C, (A,B,C are constants)

• Merge Sort ’s running time is:

Dn log n + En + F, (D,E,F are constants)

• To compare their running times for large

n, we can just focus on the dominating

term (the term that grows fastest when n increases)

– An^2 vs Dn log n

Dominating Term

Constant Factors

• (^) The growth rate is not affected by - (^) constant factors or - (^) lower-order terms
• (^) Examples
  • (^102) n  105 is a linear function
  • (^105) n^2  108 n is a quadratic function 1E- 1 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E+ 1E- 1 1E+1 1E+3 1E+5 1E+7 1E+ T ( n ) n Quadratic Quadratic Linear Linear

Definition: Given a function g(n), we denote

(g(n)) to be the set of functions

{ f(n) | there exists positive constants

c and n 0 such that

0 ≤ f(n) ≤ c g(n) for

all n ≥ n 0 }

Big-O notation

Rough Meaning: (g(n)) includes all functions

that are upper bounded by g(n)

Big-Oh Example

• (^) Example: the function n^2 is not O ( n ) - (^) n^2  cn - (^) n  c - (^) The above inequality cannot be satisfied since c must be a constant 1 10 100 1, 10, 100, 1,000, 1 10 100 1,

n

n^ 100n 10n n

More Big-Oh Examples

7n-

7n-2 is O(n) need c > 0 and n 0  1 such that 7n-2  c•n for n  n 0 this is true for c = 7 and n 0 = 1

3n

3

+ 20n

2

3n 3

• 20n 2
• 5 is O(n 3 ) need c > 0 and n 0  1 such that 3n 3
• 20n 2
• 5  c•n 3 for n  n 0 this is true for c = 4 and n 0 = 21

3 log n + log log n

3 log n + log log n is O(log n) need c > 0 and n 0  1 such that 3 log n + log log n  c•log n for n  n 0 this is true for c = 4 and n 0 = 2

17

Big-Oh and Growth Rate

• (^) The big-Oh notation gives an upper bound on the growth rate of a function
• (^) The statement “ f ( n ) is O ( g ( n ))” means that the growth rate of f ( n ) is no more than the growth rate of g ( n )
• (^) We can use the big-Oh notation to rank functions according to their growth rate f ( n ) is O ( g ( n )) g ( n ) is O ( f ( n )) g ( n ) grows more Yes No f ( n ) grows more No Yes Same growth Yes Yes

Big-Oh

Conventions

• If is f ( n ) a polynomial of degree d , then f ( n )

is O ( nd ), i.e.,

1.Drop lower-order terms

2.Drop constant factors

• Use the smallest possible class of functions

• (^) Say “ 2 n is O ( n )” instead of “ 2 n is O ( n^2 )”

• Use the simplest expression of the class

• (^) Say “ 3 n  5 is O ( n )” instead of “ 3 n  5 is O (3 n )”

Intuition for

Asymptotic Notation

Big-Oh  (^) f(n) is O(g(n)) if f(n) is asymptotically less than or equal to g(n) big-Omega  (^) f(n) is (g(n)) if f(n) is asymptotically greater than or equal to g(n) big-Theta  (^) f(n) is (g(n)) if f(n) is asymptotically equal to g(n) little-oh  (^) f(n) is o(g(n)) if f(n) is asymptotically strictly less than g(n) little-omega  (^) f(n) is (g(n)) if is asymptotically strictly greater than g(n)

Example Uses of the

Relatives of Big-Oh

f(n) is (g(n)) if, for any constant c > 0, there is an integer constant n 0  0 such that f(n)  c•g(n) for n  n 0 need 5n 0 2  c•n 0  given c, the n 0 that satisfies this is n 0  c/5  0  (^) 5n^2 is  (n) f(n) is (g(n)) if there is a constant c > 0 and an integer constant n 0  1 such that f(n)  c•g(n) for n  n 0 let c = 1 and n 0 = 1  (^) 5n^2 is  (n) f(n) is (g(n)) if there is a constant c > 0 and an integer constant n 0  1 such that f(n)  c•g(n) for n  n 0 let c = 5 and n 0 = 1  (^) 5n^2 is  (n^2 )