Algorithm analysis and time complexities, Lecture notes of Data Structures and Algorithms

Download Algorithm analysis and time complexities and more Lecture notes Data Structures and Algorithms in PDF only on Docsity!

Lecture 2: Algorithm Analysis

Kamilla Klonowska

Outlines

• What to analyze and why to analyze algorithms?
• Big-Oh notation and computation
• How to check the algorithm analysis?
• Exercises and practices on algorithm analysis
• Chapter 2

Running Time

• Check the following program fragments.

• Guess which one takes longer time? Why?

//fragment 1

for ( int i = 0; i< n ; i ++) sum ++;

//fragment 2

for ( int i = 0; i < n; i ++) for ( int j = 0; j < n; j ++) sum ++;

Running Time

• Running time of an algorithm is related to
  • The size of problem, or amount of data, input size N
  • Speed of the computer
  • Architecture of the computer
  • Compiler used (optimized or not)
• Algorithm analysis
  • find out the asymptotic relation between running time and size N
  • T ( N )

Running Time for moderate inputs

Figure 2. Plot ( N vs. time) of various algorithms

Numerical calculation

n log 2 n n nlog 2 n n 2 2 n* 2 1 2 2 4 4 16 4 16 64 256 6.5 × 104 64 6 64 384 4096 1.84 × 10 19 1.84 × 10 19 μsec = 2.14 × 10 8 days = 5845 centuries

Big-Oh Notation

• Definition T ( N ) = O ( f ( N ) ) if there are positive integers c and n 0 such that T ( N ) ≤ c f ( N ) when N ≥ n 0
• How to read it? T ( N ) = O ( f ( N ) ): T of N is Big-Oh of f ( N )
• How to understand? The growth rate of T ( N ) is less than or equal to that of f ( N ).

Graphical Explanation

• T ( N ) = O ( f ( N ) ) if there are positive integers c and n 0 such that T ( N ) ≤ c f ( N ) when N ≥ n 0
• Curves for T(N) and c f ( N ) intersect at N = n 0.
• For all points to the right of the vertical line (at N = n 0 ), T ( N ) ≤ c f ( N )
• Asymptotic, when N is very large

Examples with T ( N ) = 100 N

• Given T ( N ) = 100 N
  • Let f ( N ) = N 3
  • Let c = 1 and n 0 = 10
  • T ( N ) = 100 N ≤ cf ( N ) = N 3 when N ≥ 10
• Conclusion:
  • T ( N ) = 100 N = O ( f ( N ) ) = O ( N 3 )
  • The growth rate of 100 N is less than or equal to that of N 3

Examples with T ( N ) = 100 N

• Given T ( N ) = 100 N
  • Let f ( N ) = N 2
  • Let c = 1 and n 0 = 100
  • T ( N ) = 100 N ≤ cf ( N ) = N 2 when N ≥ 100
• Conclusion
  • T ( N ) = 100 N = O ( f ( N )) = O ( N 2 )
  • The growth rate of 100 N is less than or equal to that of N 2

Examples with T ( N ) = 100 N

• Given T ( N ) = 100 N
  • T ( N ) = O ( N 3 )
  • T ( N ) = O ( N 2 )
  • T ( N ) = O ( N )  T ( N ) = O ( N ) is the best answer, it is more precise

Computations on T ( N ) and problem sizes

• T ( N ) = cN 3
• T (10 N ) = c (10 N ) 3
• T (10 N ) = 1000 c N 3 = 1000 T ( N )  if the problem size is 10 timed, the running time will be 1000 timed!
• T ( N ) = c N 2
• T (10 N ) = c (10 N ) 2
• T (10 N ) = 100 c N 2 = 100 T ( N )  If the problem size is 10 timed, the running time will be 100 timed!
• T ( N ) = c N log N
  • T (10 N ) = c (10 N ) log(10 N )
  • T (10 N ) = 10 c N log(10 N ) = 10 c N log N + 10 c N log10 = 10 T ( N ) + c´ N , where c´ = 10 c log

Some big-oh styles

• If c is a constant , it can be thrown away: T ( N ) = O ( c ) = O (1) T ( N ) = O ( cN ) = O ( N )
• Do not include lower order terms: T ( N ) = O ( N 2 + N ) = O ( N 2 )
• Examples O (1000 N ) = O ( N ) O ( N 2 + 3 N + 2) = O ( N 2 ) O (3 N 3 + 6 N 2 - 4 N + 2) = O (3 N 3 ) = O ( N 3 )

Ex. 2.

• Suppose T 1 ( N ) = O ( f ( N )) and T 2 ( N ) = O ( f ( N )). Which of the following are true? a) T 1 ( N ) + T 2 ( N ) = O ( f ( N )) b) T 1 ( N ) – T 2 ( N ) = o ( f ( N )) c) T 1 ( N ) / T 2 ( N ) = O (1) d) T 1 ( N ) = O ( T 2 ( N )) Think always about worst-case scenario, ex. T 1 ( N ) = N and T 2 ( N ) = 1.