Asymptotic Analysis: Understanding Algorithm Efficiency through Common Functions, Lecture notes of Design and Analysis of Algorithms

Download Asymptotic Analysis: Understanding Algorithm Efficiency through Common Functions and more Lecture notes Design and Analysis of Algorithms in PDF only on Docsity!

Lecture 03:

Asymptotic Analysis

Ms. Andleeb Yousaf Khan Spring 2017

Analysis of Algorithm

 (^) A data structure is a systematic way of organizing and accessing data, and an algorithm is a step-by-step procedure for performing some task in a finite amount of time. To classify some data structures and algorithms as "good", we must have precise ways of analyzing them.  (^) Analyzing the efficiency of a program involves characterizing the running time and space usage of algorithms and data structure operations. Particularly, the running time is a natural measure of goodness, since time is precious.

Common Functions used in Analysis

The Constant Function f ( n ) = C

 (^) For any argument n , the constant function f ( n ) assigns the value C. It doesn't matter what the input size n is, f ( n ) will always be equal to the constant value C. The most fundamental constant function is f ( n ) = 1, and this is the typical constant function that is used in the book.  (^) The constant function is useful in algorithm analysis, because it characterizes the number of steps needed to do a basic operation on a computer, like adding two numbers, assigning a value to some variable, or comparing two numbers. Executing one instruction a fixed number of times also needs constant time only.  (^) Constant algorithm does not depend on the input size.  (^) Examples : arithmetic calculation, comparison, variable declaration, assignment statement, invoking a method or function.

The Linear Function f ( n ) = n

 (^) Another simple yet important function. Given an input value n , the linear function f assigns the value n itself.  (^) This function arises in an algorithm analysis any time we do a single basic operation for each of n elements. For example, comparing a number x to each element of an array of size n will require n comparisons. The linear function also represents the best running time we hope to achieve for any algorithm that processes a collection of n inputs.  (^) Whenever n doubles, so does the running time.  (^) Example: print out the elements of an array of size n.

The N-Log-N Function f ( n )

= n log n

 (^) This function grows a little faster than the linear function and a lot slower than the quadratic function ( n 2). If we can improve the running time of solving some problem from quadratic to N-Log-N, we will have an algorithm that runs much faster in general. It scales to a huge problem, since whenever n doubles, the running time more than doubles.  Example: merge sort, which will be discussed later.

The Cubic Function and Other

Polynomials

 (^) The cubic function f ( n ) = n 3.^ This function appears less frequently in the context of the algorithm analysis than the constant, linear, and quadratic functions. It's practical for use only on small problems. Whenever n doubles, the running time increases eightfold.  Example:^ n^ by^ n^ matrix multiplication.  The functions we have learned so far can be viewed as all being part of a larger class of functions, the^ polynomials. A polynomial function is a function of the form: a nx n+a n-1x n-1+……………………..a 2x 2+a 1x+a  where a 0, a 1, ... , an are constants, called the coefficients of the polynomial, and an != 0. Integer n , which indicates the highest power in the polynomial, is called the degree of the polynomial.

The Exponential Function f ( n )

= b

n

 (^) In this function, b is a positive constant, called the base , and the argument n is the exponent. In the algorithm analysis, the most common base for the exponential function is b = 2. For instance, if we have a loop that starts by performing one operation and then doubles the number of operations performed with each iteration, then the number of operations performed in the n th iteration is 2 n.  Exponential algorithm is usually not appropriate for practical use.  Example: Towers of the Hanoi.

Comparing Growth Rates

 (^) Ideally, we would like data structure operations to run in times proportional to the constant or logarithm function, and we would like our algorithms to run in linear or n -log- n time. Algorithms with quadratic or cubic running times are less practical, but algorithms with exponential running times are infeasible for all but the smallest sized inputs.

Comparing Growth Rates

Example

 (^) Associate a "cost" with each statement.  (^) Find the "total cost“ by finding the total number of times each statement is executed. Algorithm 1 Cost arr[0] = 0; c 1 arr[1] = 0; c 1 arr[1] = 0; c 1 … … arr[N-1] = 0; c 1

---

= c1+c1+…+c1 =c1 x N Algorithm 2 Cost for( i=0; i<N; i++) c 2 arr[i] = 0; c 1

---

= (N+1)c2+Nc1 = (c2+c1)* N + c

Another Example

Algorithm 3 Cost sum = 0; c 1 for(i=0; i<N; i++) c 2 for(j=0; j<N; j++) c 2 sum += arr[i] [j]; c 3

---

= c 1+c2(N+1)+c2N(N+1)+c3N 2

Counting Primitive Operations  (^) Line 1 is one op (assigning a value).  (^) Loop initializing is one op (assigning a value).  (^) Line 3 is five ops per iteration (mult, add, 2 array refs, assign).  (^) Line 3 is executed n times; total is 5n.  (^) Loop incrimination is two ops (an addition and an assignment)  (^) Loop incrimination is done n times; total is 2n.  (^) Loop termination test is one op (a comparison i<n) each time.  (^) Loop termination is done n+1 times (n successes, one failure); total is n+1.  (^) Return is one op.  (^) The total is thus 1+1+5n+2n+(n+1)+1 = 8n+4.

Primitive Operations

 (^) By inspecting the pseudocode, we can determine the maximum number of primitive operations executed by an algorithm as a function of the input size. Think about the worst case, best case, and average case.