Discrete Mathematics Lecture 13: Algorithmic Complexity and Orders of Growth - Prof. Kemal, Study notes of Discrete Mathematics

Download Discrete Mathematics Lecture 13: Algorithmic Complexity and Orders of Growth - Prof. Kemal and more Study notes Discrete Mathematics in PDF only on Docsity!

Discrete Mathematics

Kemal Akkaya

DISCRETE MATHEMATICS

Lecture 13 Dr. Kemal Akkaya

Department of Computer Science

Discrete Mathematics

Kemal Akkaya

§3.3: Algorithmic Complexity^ z

The

algorithmic complexity

of a computation is,

most generally, a measure of how

difficult

it is to

perform the computation. z That is, it measures some aspect of the

cost

of

computation (in a general sense of “cost”).^ z^ Amount of resources required to do a computation. z Some of the most common complexitymeasures:^ z^ “Time” complexity: # of operations or steps required^ z^ “Space” complexity: # of memory bits required

Discrete Mathematics

Kemal Akkaya

Complexity & Orders of Growth^ z

Suppose algorithm A has worst-case timecomplexity (w.c.t.c., or just

time

)^ f (

n ) for

inputs of length

n , while algorithm B (for

the same task) takes time

g (

n ).

z^ Suppose that

f ∈

O(

g ).

z^ Which algorithm will be

fastest

on all

sufficiently-large, worst-case inputs?^ z^ f(n)

is at most order of

g(n),

and hence faster

Discrete Mathematics

Kemal Akkaya

Example 1: Max algorithm^ z

Problem:

Find the

simplest form

of the

exact

order of growth (

Θ) of the

worst-

case

time complexity (w.c.t.c.) of the max

algorithm, assuming that each line of code takes some constant time everytime it is executed (with possibly differenttimes for different lines of code).

Discrete Mathematics

7

Kemal Akkaya

Complexity analysis,

cont.

Now, what is the simplest form of the exact(

Θ) order of growth of

t ( n

)( )( ) (^1) ( ) (^1) ( )( ) (^1) (

) (^1) ( ) 1 ( ) (^1) ( ) (^1) ( ) (^1) (

) (^1) (

) (

)(

2

4

2

3 2

1

n n

n

n

t t t t nt

n i n i

Θ= Θ+ Θ= Θ Θ+ Θ=

Θ −

Θ= Θ+ ⎞ ⎟ ⎠ ⎛^ ⎜ ⎝

Θ

Θ=

⎞ +⎟ ⎠

⎛^ ⎜ ⎝

=

Discrete Mathematics

Kemal Akkaya

Example 2: Linear Search^ procedure

linear search

( x

: integer,

a ,^1

a^2

an

: distinct integers)

i^ :=

t^1

while

( i^

≤^ n

∧^

x^ ≠

a ) i

t^2

i^ :=

i^ + 1

t^3

if^ i

≤^

n^ then

location

i^

t^4

else

location

t^5

return

location

t^6

Discrete Mathematics

Kemal Akkaya

Review §2.3: Complexity^ z

Algorithmic complexity =

cost

of computation.

z^ Focus on

time

complexity for our course.

z^ Although space & energy are also important. z^ Characterize complexity as a function of inputsize: Worst-case, best-case, or average-case. z^ Use orders-of-growth notation to conciselysummarize the growth properties of complexityfunctions.

Discrete Mathematics

11

Kemal Akkaya

Example 3: Binary Search^ procedure

binary search

( x :integer

, a^1

,^ a^2

, …,

a^ : distinct integers, sorted smallest to n largest) i^ :=

1 j^ :=

n while

i < j

begin m^ :=

⎣( i +

j )/

⎦

if^ x

a^ m

then

i^ :=

m +

else

j^ :=

m

end if^ x

=^ a

then i

location

:=

i^ else

location

:=

0

return

location

Θ(1)

Θ(1)

Θ(1)

Key question: How many loop iterations?

Discrete Mathematics

Kemal Akkaya

Names for some orders ofgrowth^ z

Θ(1)

Constant

z^ Θ

(log

nc )Logarithmic (same order

∀ c

)

z^ Θ

(log

c^ n

)Polylogarithmic

z^ Θ

( n )

Linear

z^ Θ

c ( n )^

Polynomial (for any

c )

z^ Θ

n ( c )^

Exponential (for

c >1)

z^ Θ

( n !)

Factorial

(With

c^ a constant.)^ NP Problems

Discrete Mathematics

Kemal Akkaya

Key Things to Know^ z

Definitions of algorithmic complexity,time complexity, worst-case timecomplexity. z Names of specific orders of growth ofcomplexity. z How to analyze the worst case, bestcase, or average case order of growth oftime complexity for simple algorithms.