Asymptotic Analysis: Understanding the Growth Rate of Algorithms and Data Structures, Lecture notes of Data Structures and Algorithms

Download Asymptotic Analysis: Understanding the Growth Rate of Algorithms and Data Structures and more Lecture notes Data Structures and Algorithms in PDF only on Docsity!

MORE ON ASYMPTOTIC

ANALYSIS

Asymptotic Analysis:

 estimates the time or space required by a program as function of the input size  We analyze:  (^) time required for an algorithm  (^) the space required for a data structure.

Best, Worst and Average Cases:

 Largest-value sequential search algorithm - always examines every array value.  However, for some algorithms, different inputs require different amounts of time. E.g. Sequential search for K in an array of n integers:  (^) Begin at first element in array and look at each element in turn until K is found Best case: Find at first position. Cost is 1 compare. Worst case: Find at last position. Cost is n compares. Average case:( n +1)/2 compares if we assume the element with value K is equally likely to be in any position in the array. 4

Asymptotic Analysis: Upper Bounds  Upper bound - indicates the upper or highest growth rate that an algorithm can have.  measured on the best-case, average-case, or worst-case inputs  Big-Oh notation:  (^) If the upper bound for an algorithm’s growth rate (say, the worst case) is f(n), then this algorithm is “in the set O(f(n))in the worst case” (or just “in O(f(n))in the worst case”).

Big-oh Notation (cont.) Big-oh notation indicates an upper bound. T(n) grows at a rate no faster than f (n)  …“is in O(f(n))” or “ Є O(f(n))”, no strict equality actually Example: If T ( n ) = 3 n 2 then T ( n ) is in O( n 2 ). Wish tightest upper bound: While T ( n ) = 3 n 2 is in O( n 3 ), we prefer O( n 2 ).

Big-Oh Examples Example 1: Finding value X in an array (average cost, assuming equal probability of appearing in any position). T ( n ) = c s n /2. For all values of n > 1, c s n /2 <= c s n. Therefore, by the definition, T ( n ) is in O( n ) for n 0

1 and c = c s

A Common Misunderstanding “The best case for my algorithm is n =1 because that is the fastest.” WRONG! Big-oh refers to a growth rate as n grows to . It states a claim about the greatest amount of some resource (usually time) that is required by an algorithm for some class of inputs of size n (typically the worst such input, the average of all possible inputs, or the best such input). Best case is defined as which input of size n is cheapest among all inputs of size n.

Lower Bound  describes the least amount of a resource that an algorithm needs for some class of input.  Like big-Oh notation, this is a measure of the algorithm’s growth rate.  Measured for some particular class of inputs: the worst-, average-, or best-case input of size n.

Big-Omega Example T ( n ) = c 1 n 2

• c 2 n. c 1 n 2
• c 2 n >= c 1 n 2 for all n > 1. T ( n ) >= cn 2 for c = c 1 and n 0

Therefore, T ( n ) is in ( n 2 ) by the definition. We want the greatest lower bound.

Theta Notation When big-Oh and meet, we indicate this by using (big-Theta) notation. Definition: An algorithm is said to be ( h ( n )) if it is in O( h ( n )) and it is in ( h ( n )). For polynomial equations on T(n), we always have . There is no uncertainty, since once we have the equation, we have a “complete” analysis.

Simplifying Rules (1) If f ( n ) is in O( g ( n )) and g ( n ) is in O( h ( n )), then f ( n ) is in O( h ( n )). [if some function g(n) is an upper bound for your cost function, then any upper bound for g(n) is also an upper bound for your cost function.] (2) If f ( n ) is in O( kg ( n )) for any constant k > 0, then f ( n ) is in O( g ( n )). [Ignore constants] (3) If f 1 ( n ) is in O( g 1 ( n )) and f 2 ( n ) is in O( g 2 ( n )), then ( f 1 + f 2 )( n ) is in O(max( g 1 ( n ), g 2 ( n ))). [Given two parts of a program run in sequence consider only the more expensive part. Drop lower order terms] (4) If f 1 ( n ) is in O( g 1 ( n )) and f 2 ( n ) is in O( g 2 ( n )) then f 1 ( n ) f 2 ( n ) is in O( g 1 ( n ) g 2 ( n )). [If some action is repeated some number of times, and each repetition has the same cost, then the total cost is the cost of the action multiplied by the number of times that the action takes place. Useful for analyzing loops]

Simplifying Rules (cont.) Taking the first three rules, ignore all constants and all lower-order terms to determine the asymptotic growth rate for any cost function. The higher-order terms soon swamp the lower- order terms in their contribution to the total cost as n becomes larger. Example: if T(n) = 3n^4 +5n^2 , then T(n) is in O(n^4 ). n^2 contributes relatively little to the total cost.

Running Time Examples (2)

Example 3: sum = 0; for (i=1; i<=n; j++) //first for loop for (j=1; j<=i; i++)// is double loop sum++; for (k=0; k<n; k++)//second for loop A[k] = k; [First statement is c 1 =>(1). Double for loop :

• sum++ requires constant time,c 2
• inner for loop is executed i times, by rule (4), c 2 i
• outer for loop is executed n times, but each time the cost of the inner loop is different ,c 2 i with i changing each time. -Thus, the total cost of the loop is c 2 times the sum of the integers 1 through n: => (n^2 ). Final for loop is c 3 n = (n) time (n). Result:  (n^2 ) .] (^19)

Running Time Examples (3) Example 4: Compare. sum1 = 0; for (i=1; i<=n; i++) for (j=1; j<=n; j++) sum1++; sum2 = 0; for (i=1; i<=n; i++) for (j=1; j<=i; j++) sum2++; [First loop, sum is n^2. Second loop, sum is (n+1)(n)/2. Both are (n^2 ).] In the first double loop, the inner for loop always executes n times In the second double loop, the inner for loop always executes i times