Download Analysis of Algorithms - Lecture Slides | CSCE 310 and more Study notes Algorithms and Programming in PDF only on Docsity!
Design and Analysis of Algorithms - Chapter 2 1
http://www.cse.unl.edu/~goddard/Courses/CSCE310J
Analysis of Algorithms
Dr. Steve Goddard
CSCE 310J: Data Structures & Algorithms
Design and Analysis of Algorithms - Chapter 2 2
I Giving credit where credit is due:
- Most of the lecture notes are based on the slides from the Textbook’s companion website - http://www.aw.com/cssuport/
- Several slides are from William Spears of the University of Wyoming
- I have modified them and added new slides
CSCE 310J: Data Structures & Algorithms
Design and Analysis of Algorithms - Chapter 2 3
Analysis of Algorithms
I Issues:
- Correctness
- Time efficiency
- Space efficiency
- Optimality
I Approaches:
- Theoretical analysis
- Empirical analysis
Design and Analysis of Algorithms - Chapter 2 4
Theoretical analysis of time efficiency
Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size
I Basic operation: the operation that contributes most towards the running time of the algorithm.
T ( n ) ≈ copC ( n ) running time (^) execution time for basic operation
Number of times basic operation is executed
input size
Design and Analysis of Algorithms - Chapter 2 5
Input size and basic operation examples
Problem Input size measure Basic operation
Visiting a vertex or Graph problem #vertices and/or edges traversing an edge
Floating point Compute a n multiplication n
Floating point Dimensions of matrices multiplication Multiply two matrices of floating point numbers
Search for key in list of items n Number of items in list n Key comparison
Design and Analysis of Algorithms - Chapter 2 6
Empirical analysis of time efficiency
I Select a specific (typical) sample of inputs
I Use physical unit of time (e.g., milliseconds)
OR
I Count actual number of basic operations
I Analyze the empirical data
Design and Analysis of Algorithms - Chapter 2 7
Best-case, average-case, worst-case
For some algorithms efficiency depends on type of input:
I Worst case: W( n ) – maximum over inputs of size n
I Best case: B( n ) – minimum over inputs of size n
I Average case: A( n ) – “average” over inputs of size n
- Number of times the basic operation will be executed on typical input
- NOT the average of worst and best case
- Expected number of basic operations repetitions considered as a random variable under some assumption about the probability distribution of all possible inputs of size n Design and Analysis of Algorithms - Chapter 2 8
Example: Sequential search
I Problem: Given a list of n elements and a search key K , find an element equal to K , if any. I Algorithm: Scan the list and compare its successive elements with K until either a matching element is found ( successful search ) of the list is exhausted ( unsuccessful search ) I Worst case
I Best case
I Average case
Design and Analysis of Algorithms - Chapter 2 9
Types of formulas for basic operation count
I Exact formula e.g., C( n ) = n ( n -1)/
I Formula indicating order of growth with specific multiplicative constant e.g., C( n ) ≈ 0.5 n^2
I Formula indicating order of growth with unknown multiplicative constant e.g., C( n ) ≈ cn^2
Design and Analysis of Algorithms - Chapter 2 10
Order of growth
I Most important: Order of growth within a constant multiple as n →∞
I Example:
- How much faster will algorithm run on computer that is twice as fast?
- How much longer does it take to solve problem of double input size?
I See table 2.
Design and Analysis of Algorithms - Chapter 2 11
Table 2.
Design and Analysis of Algorithms - Chapter 2 12
Asymptotic growth rate
I A way of comparing functions that ignores constant factors and small input sizes
I O( g ( n )): class of functions f ( n ) that grow no faster than g ( n )
I Θ ( g ( n )): class of functions f ( n ) that grow at same rate as g ( n )
I Ω( g ( n )): class of functions f ( n ) that grow at least as fast as g ( n )
see figures 2.1, 2.2, 2.
Design and Analysis of Algorithms - Chapter 2 19
Another Example
Thisapproaches0,so log.
topandbottomtoget:( 1 / )/. 5 2 /
log / Nowtakethederivativeofthe
Thenlim log /
Let log and.
- 5
. 5. 5 . 5
5
5
N N o(N )
N N N
NN.
f(N)/g(N) N NN
f(N) N N g(N) N
N
−
→∞
Design and Analysis of Algorithms - Chapter 2 20
Establishing rate of growth: Method 2 – using definition
I f ( n ) is O( g ( n )) if order of growth of f ( n ) ≤ order of growth of g ( n ) (within constant multiple) I There exist positive constant c and non-negative integer n 0 such that
f ( n ) ≤ c g ( n ) for every n ≥ n 0 Examples: I 10 n is O(2 n^2 )
I 5 n +20 is O(10 n )
Design and Analysis of Algorithms - Chapter 2 21
Basic Asymptotic Efficiency classes
n! factorial
2 n exponential
n^3 cubic
n^2 quadratic
n log n n log n
n linear
log n logarithmic
1 constant
Design and Analysis of Algorithms - Chapter 2 22
Time efficiency of nonrecursive algorithms
Steps in mathematical analysis of nonrecursive algorithms:
I Decide on parameter n indicating input size
I Identify algorithm’s basic operation
I Determine worst , average , and best case for input of size n
I Set up summation for C(n) reflecting algorithm’s loop structure
I Simplify summation using standard formulas (see Appendix A)
Design and Analysis of Algorithms - Chapter 2 23
Examples:
I Matrix multiplication
I Selection sort
I Insertion sort
I Mystery Algorithm
Design and Analysis of Algorithms - Chapter 2 24
Matrix multipliacation
Design and Analysis of Algorithms - Chapter 2 25
Selection sort
Design and Analysis of Algorithms - Chapter 2 26
Insertion sort
Design and Analysis of Algorithms - Chapter 2 27
Mystery algorithm
for i := 1 to n - 1 do
max := i ; for j := i + 1 to n do if |A[ j, i ]| > |A[ max, i ]| then max := j ; for k := i to n + 1 do swap A[ i, k ] with A[ max, k ]; for j := i + 1 to n do for k := n + 1 downto i do A[ j, k ] := A[ j, k ] - A[ i, k ] * A[ j, i ] / A[ i, i ] ;
Design and Analysis of Algorithms - Chapter 2 28
Programming with Recursion
I Recursion is similar to a proof by induction:
- There must be a base (trivial) case.
- The recursion is assumed to hold for all k < N.
- The N th case is built from the k < N cases.
Design and Analysis of Algorithms - Chapter 2 29
Asside: Recall Proof by Induction
I Proof by (strong) induction:
- Show theorem true for trivial case(s). Then,
assuming theorem true up to case N , show true
for N+1. Thus true for all N.
Design and Analysis of Algorithms - Chapter 2 30
Proof that T(N) >= F(N)
( 1 ) ( ) ( 1 ) 1 ( 1 )
Nowprovefor the 1 case:
Assumetheoremholdsforall , 1
and ( 1 ) ( ) ( 1 ).
Weknowthat ( 1 ) ( ) ( 1 )
2 4 2 2
1 1 1 1
Basecases: 0 1 0 1
≤ ≤
= ≥ =
= ≥ =
= ≥ =
TN TN TN F(N) F(N-) F N
N
k k N
FN FN FN
TN TN TN
T( ) F( ).
T() F().
T( ) F( ) ,
Design and Analysis of Algorithms - Chapter 2 37
Logarithms
I Definition: XA^ = B if and only if log XB = A ( x is
the “base” of the logarithm).
- Example: 10 2 = 100 means log 10 100 = 2.
I Theorems
log (^) X AB = logX A + logX B log (^) X A/B = log (^) X A – log (^) X B log (^) X AB^ = B logX A
Design and Analysis of Algorithms - Chapter 2 38
Logarithms…
I Theorem: log (^) A B = log (^) C B / log (^) CA
Proof: Let X = log CB, Y = log CA, and Z = log AB. By
the definition of logarithm: CX^ = B, CY^ = A, and
AZ^ = B.
Thus CX^ = B = AZ^ = CYZ^ , X = YZ, Z = X/Y.
Design and Analysis of Algorithms - Chapter 2 39
Logarithms…
I The notation for logs can be confusing. There are
two alternatives:
- log 2 (x) = log (log x) or
- log 2 (x) = (log x) 2
I Generally, we use the 2 nd^ definition.
I Note: log 2 (x) is not (log x 2 )
I Note: log is not a multiplicative entity, it is a
function.
Design and Analysis of Algorithms - Chapter 2 40
Series
I Proof by Gauss when 9 years old (?!):
1
= (^) ∑ =
N N
S i
N
i
S N N
S N N N
S N N N
Design and Analysis of Algorithms - Chapter 2 41
Finite Series
1
∑
NN N
S i
N
i
Design and Analysis of Algorithms - Chapter 2 42
Finite Series
I Proof:
∑
N
i
k k
k
N
S i
1
1
1 2 3 4 …… N N+
1 k
2 k
3 k
N k
1
1
1
1 1
|
=
∑ (^) ∫
k k
N
k
x
i xdx
k
N k N
i
N k k
Design and Analysis of Algorithms - Chapter 2 43
Finite Series
Proof:
=
∑
A
A
S A
N N
i
i
... 1 ... 1
( 1 ... )( 1 )
2 1 2 1
2
N
A A A A A A A
A A A A
0
=
∑
N
N
i
S i
Design and Analysis of Algorithms - Chapter 2 44
Finite Series
Proof:
1
=
∑
A
A A
S A
N N
i
i
A A A A A A A
A A A A N N N
N
2
... ...
( ... )( 1 )
1
=
∑
N N
i
S i
Design and Analysis of Algorithms - Chapter 2 45
General Rules for Sums
i i
i
ik k i
i
nk
imk
i
n
im
ik
i
i i
i
i
i i
i i i
i
n
im
n
im
ax x ax
a a
ca c a
a b a b
c c cn m
∑ ∑
∑ ∑
∑ ∑
∑ ∑ ∑
∑ ∑
=
=
=
= = − +
= =+
= = ( )
1 ( 1 )
