Insertion sort analysis, Thesis of Advanced Computer Programming

Download Insertion sort analysis and more Thesis Advanced Computer Programming in PDF only on Docsity!

Time Complexity of Algorithms

(Asymptotic Notations)

• The level in difficulty in solving mathematically posed problems as measured by - The time (time complexity) - memory space required - (space complexity)

What is Complexity?

A Comparison of Growth-Rate

Functions

Design & Analysis of Algorithms 4

Properties of Growth-Rate

Functions

1. We can ignore low-order terms in an algorithm’s growth-rate function.
   • If an algorithm is O(n 3 +4n 2 +3n), it is also O(n 3 ).
   • We only use the higher-order term as algorithm’s growth-rate function.
2. We can ignore a multiplicative constant in the higher-order term of an algorithm’s growth-rate function.
   • If an algorithm is O(5n 3 ), it is also O(n 3 ).
3. O(f(n)) + O(g(n)) = O(f(n)+g(n))
   • We can combine growth-rate functions.
   • If an algorithm is O(n 3 ) + O(4n 2 ), it is also O(n 3 +4n 2 )  So, it is O(n 3 ).
   • Similar rules hold for multiplication. Design & Analysis of Algorithms 5

• Algorithm analysis means predicting resources such as
  • computational time
  • memory
• Worst case analysis
  • Provides an upper bound on running time
  • An absolute guarantee
• Average case analysis
  • Provides the expected running time
  • Very useful, but treat with care: what is “average”?
    • Random (equally likely) inputs
    • Real-life inputs

Complexity Analysis

Asymptotic Notations Properties

• Categorize algorithms based on asymptotic growth rate e.g. linear, quadratic, exponential
• Ignore small constant and small inputs
• Estimate upper bound and lower bound on growth rate of time complexity function
• Describe running time of algorithm as n grows to . Limitations
• not always useful for analysis on fixed-size inputs.
• All results are for sufficiently large inputs. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

                       upper boundfor  . meansfunction isan asymptotically 0 ,forall :thereexistpositiveconstants and such that Fora given function 0 ,denotedby thesetof functions, f n f n g n g n f n cg n n n g n f n c n g n g n o o         

Big-Oh Notation (O)

Intuitively : Set of all functions whose rate of growth is the same as or lower than that of g ( n ). We may write f(n) = O(g(n)) OR f(n)  O(g(n)) If f, g: N  R

, then we can define Big-Oh as

g ( n ) is an asymptotic upper bound for f ( n ).

Big-Oh Notation

 c > 0,  n 0  0 and  n  n 0 , 0  f(n)  c.g(n) f(n)  O(g(n))

Examples

Example 2: Prove that n 2  O(n 2 ) Proof: Assume that f(n) = n 2 , and g(n) = n 2

Now we have to show that f(n)  O(g(n))

Since f(n) ≤ c.g(n)  n 2 ≤ c.n 2  1 ≤ c, take, c = 1, n 0

Then n 2 ≤ c.n 2 for c = 1 and n  1 Hence, 2n 2

 O(n

2 ), where c = 1 and n 0

Examples

Examples

Example 3: Prove that 1000.n 2

• 1000.n  O(n 2 ) Proof: Assume that f(n) = 1000.n 2
• 1000.n, and g(n) = n 2 We have to find existence of c and n 0 such that 0 ≤ f(n) ≤ c.g(n)  n  n 0 1000.n 2
• 1000.n ≤ c.n 2 = 1001.n 2 , for c = 1001 1000.n 2
• 1000.n ≤ 1001.n 2 1000.n ≤ n 2  n 2  1000.n  n 2

• 1000.n  0  n (n-1000)  0, this true for n  1000 f(n) ≤ c.g(n)  n  n 0 and c = 1001 Hence f(n)  O(g(n)) for c = 1001 and n 0

Examples

                       lowerbound for  . ,meansthat function isanasymptotically 0 forall :thereexistpositiveconstants and such that Fora given function denoteby thesetof functions, f n f n g n g n cg n f n n n g n f n c n g n g n o o        

Big-Omega Notation ()

Intuitively : Set of all functions whose rate of growth is the same as or higher than that of g ( n ). We may write f(n) = (g(n)) OR f(n)  (g(n)) If f, g: N  R

, then we can define Big-Omega as

Big-Omega Notation

g ( n ) is an asymptotically lower bound for f ( n ).  c > 0,  n 0  0 ,  n  n 0 , f(n)  c.g(n) f(n)   (g(n))

Examples

Example 2: Prove that 5.n + 10  (n)

Proof:

Assume that f(n) = 5.n + 10, and g(n) = n

f(n)  (g(n))?

We have to find the existence of c and n

0

s.t.

c.g(n) ≤ f(n)  n  n

0

c.n ≤ 5.n + 10 

c ≤ 5 + 10/n (divide by n)

if we take, c = 5 and n

0

= 1 then

c.n ≤ 5.n + 10  n  n

0

And hence f(n)  (g(n)), for c = 5 and n

0

Examples

Examples

Example 3: Prove that 100.n + 5  (n

2

Proof:

Let f(n) = 100.n + 5, and g(n) = n 2 Assume that f(n)  (g(n))? Now if f(n)  (g(n)) then there exist c and n 0 s.t. c.g(n) ≤ f(n)  n  n 0

c.n 2 ≤ 100.n + 5  c.n ≤ 100 + 5/n  n ≤ 100/c, for a very large n, which is not possible And hence f(n)  (g(n))

Examples