Asymptotic Analysis in CSE 332: Data Structures - Understanding Algorithm Complexity, Study Guides, Projects, Research of Design

An introduction to asymptotic analysis in the context of CSE 332: Data Structures. It covers the importance of analyzing algorithm performance, the concept of correctness, and the use of proof by induction. The document also discusses how to measure performance, focusing on time complexity, and presents examples of linear and binary search algorithms. Lastly, it explains the concept of solving recurrence relations.

Typology: Study Guides, Projects, Research

2021/2022

Uploaded on 09/12/2022

dreamingofyou
dreamingofyou 🇬🇧

4.3

(16)

233 documents

1 / 8

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
1
CSE 332: Data Structures
Asymptotic Analysis
Richard Anderson, Steve Seitz
Winter 2014
2
Announcements
Homework requires you get the textbook
(either E2 or E3)
Go to Thursdays sections
Homework #1 out on today (Wednesday)
Due at the beginning of class
next Wednesday(Jan 17).
3
Algorithm Analysis
Correctness:
Does the algorithm do what is intended.
Performance:
Speed time complexity
Memory space complexity
Why analyze?
To make good design decisions
Enable you to look at an algorithm (or code)
and identify the bottlenecks, etc.
4
Correctness
Correctness of an algorithm is established
by proof. Common approaches:
(Dis)proof by counterexample
Proof by contradiction
Proof by induction
Especially useful in recursive
algorithms
5
Proof by Induction
Base Case: The algorithm is correct for
a base case or two by inspection.
Inductive Hypothesis (n=k): Assume
that the algorithm works correctly for
the first k cases.
Inductive Step (n=k+1): Given the
hypothesis above, show that the k+1
case will be calculated correctly.
6
Recursive algorithm for sum
Write a recursive function to find the sum of
the first n integers stored in array v.
sum(int array v, int n) returns int
if n = 0 then
sum = 0
else
sum = nth number + sum of first n-1 numbers
return sum
pf3
pf4
pf5
pf8

Partial preview of the text

Download Asymptotic Analysis in CSE 332: Data Structures - Understanding Algorithm Complexity and more Study Guides, Projects, Research Design in PDF only on Docsity!

CSE 332: Data Structures

Asymptotic Analysis

Richard Anderson, Steve Seitz

Winter 2014

2

Announcements

• Homework requires you get the textbook

(either E2 or E3)

• Go to Thursdays sections

• Homework #1 out on today (Wednesday)

  • Due at the beginning of class

next Wednesday (Jan 17).

3

Algorithm Analysis

  • Correctness:
    • Does the algorithm do what is intended.
  • Performance:
    • Speed time complexity
    • Memory space complexity

• Why analyze?

  • To make good design decisions
  • Enable you to look at an algorithm (or code)

and identify the bottlenecks, etc.

4

Correctness

Correctness of an algorithm is established

by proof. Common approaches:

– (Dis)proof by counterexample

– Proof by contradiction

– Proof by induction

• Especially useful in recursive

algorithms

5

Proof by Induction

• Base Case: The algorithm is correct for

a base case or two by inspection.

• Inductive Hypothesis (n=k): Assume

that the algorithm works correctly for

the first k cases.

• Inductive Step (n=k+1): Given the

hypothesis above, show that the k+

case will be calculated correctly.

6

Recursive algorithm for sum

  • Write a recursive function to find the sum of

the first n integers stored in array v.

sum(int array v, int n) returns int if n = 0 then sum = 0 else sum = nth number + sum of first n-1 numbers return sum

7

Program Correctness by Induction

• Base Case:

• Inductive Hypothesis (n=k):

• Inductive Step (n=k+1):

8

How to measure performance?

9

We will focus on analyzing time complexity.

First, we have some “rules” to help measure

how long it takes to do things:

Second, we will be interested in best and

worst case performance. 9

Analyzing Performance

Basic operations

Consecutive statements

Conditionals

Loops

Function calls

Recursive functions

Constant time

Sum of times

Test, plus larger branch cost

Sum of iterations

Cost of function body

Solve recurrence relation…

10

Complexity cases

We’ll start by focusing on two cases.

Problem size N

  • Worst-case complexity : max # steps

algorithm takes on “most challenging” input

of size N

  • Best-case complexity: min # steps

algorithm takes on “easiest” input of size N

11

Exercise - Searching

bool ArrayContains (int array[], int n, int key){

// Insert your algorithm here

What algorithm would you choose to implement this code snippet?^12

Linear Search Analysis

bool LinearArrayContains (int array[], int n, int key ) { for( int i = 0; i < n; i++ ) { if( array[i] == key ) // Found it! return true; } return false; }

Best Case:

Worst Case:

19

Fast Computer vs. Slow Computer

20

Fast Computer vs. Smart Programmer

(small data)

21

Fast Computer vs. Smart Programmer

(big data)

22

Asymptotic Analysis

  • Consider only the order of the running time
    • A valuable tool when the input gets “large”
    • Ignores the effects of different machines or different implementations of same algorithm

23

Asymptotic Analysis

• To find the asymptotic runtime, throw

away the constants and low-order

terms

  • Linear search is
  • Binary search is

Remember: the “fastest” algorithm has the slowest growing function for its runtime

T worstLS ( n ) 3 n  3  O ( n )

T ( n ) 7  log 2 n  9 O (log n )

BS

worst   

24

Asymptotic Analysis

Eliminate low order terms

  • 4n + 5 
  • 0.5 n log n + 2n + 7 
  • n^3 + 3 2n^ + 8n 

Eliminate coefficients

  • 4n 
  • 0.5 n log n 
  • 3 2n^ =>

25

Properties of Logs

Basic:

  • AlogAB^ = B
  • logAA =

Independent of base:

  • log(AB) =
  • log(A/B) =
  • log(AB) =
  • log((AB)C) =

Changing base  multiply by constant

  • For example: log 2 x = 3.22 log 10 x
  • More generally
  • Means we can ignore the base for

asymptotic analysis

(since we’re ignoring constant multipliers)

26

Properties of Logs

n

A

n B

B

A log log

log 

27

Another example

  • Eliminate

low-order

terms

  • Eliminate

constant

coefficients

16 n^3 log 8 (10n^2 ) + 100n^2

28

Comparing functions

  • f(n) is an upper bound for h(n)

if h(n) ≤ f(n) for all n

This is too strict – we mostly care about large n

Still too strict if we want to ignore scale factors

29

Definition of Order Notation

• h(n) є O(f(n)) Big-O “Order”

if there exist positive constants c and n 0

such that h(n) ≤ c f(n) for all n ≥ n 0

O(f(n)) defines a class (set) of functions

30

Order Notation: Intuition

Although not yet apparent, as n gets “sufficiently

large”, a ( n ) will be “greater than or equal to” b ( n )

a ( n ) = n^3 + 2 n^2

b ( n ) = 100 n^2 + 1000

37

Big-O: Common Names

  • constant: O(1)
  • logarithmic: O(log n) (logkn, log n^2  O(log n))
  • linear: O(n)
  • log-linear: O(n log n)
  • quadratic: O(n^2 )
  • cubic: O(n^3 )
  • polynomial: O(nk) (k is a constant)
  • exponential: O(cn) (c is a constant > 1)

38

Asymptotic Lower Bounds

  • ( g ( n ) ) is the set of all functions

asymptotically greater than or equal to g ( n )

  • h ( n )  ( g( n ) ) iff

There exist c >0 and n 0 >0 such that h ( n )  c

g( n ) for all n  n 0

39

Asymptotic Tight Bound

  • ( f ( n ) ) is the set of all functions

asymptotically equal to f ( n )

  • h ( n )  ( f ( n ) ) iff h ( n )  O( f ( n ) ) and h ( n )  ( f ( n ) ) - This is equivalent to: lim ( )/ ( ) n  h n f nc  0

40

Full Set of Asymptotic Bounds

  • O( f ( n ) ) is the set of all functions

asymptotically less than or equal to f ( n )

  • o( f ( n ) ) is the set of all functions

asymptotically strictly less than f ( n )

  • ( g ( n ) ) is the set of all functions

asymptotically greater than or equal to g ( n )

  • ( g ( n ) ) is the set of all functions

asymptotically strictly greater than g ( n )

  • ( f ( n ) ) is the set of all functions

asymptotically equal to f ( n )

41

  • h ( n )  O( f ( n ) ) iff There exist c >0 and n 0 >0 such that h ( n )  c f ( n ) for all nn 0
  • h ( n )  o( f ( n )) iff There exists an n 0 >0 such that h ( n ) < c f ( n ) for all c >0 and nn 0 - This is equivalent to:
  • h ( n )  ( g( n ) ) iff There exist c >0 and n 0 >0 such that h ( n )  c g( n ) for all nn 0
  • h ( n )  ( g( n ) ) iff There exists an n 0 >0 such that h ( n ) > c g( n ) for all c >0 and nn 0 - This is equivalent to:
  • h ( n )  ( f ( n ) ) iff h ( n )  O( f ( n ) ) and h ( n )  ( f ( n ) ) - This is equivalent to:

Formal Definitions

lim ( )/ ( ) n  h n f n  0

lim ( )/ n  h n g n ( )  

lim ( )/ ( ) n  h n f nc  0 42

Big-Omega et al. Intuitively

Asymptotic Notation Mathematics

Relation

O 

o <

43

Complexity cases (revisited)

Problem size N

  • Worst-case complexity : max # steps

algorithm takes on “most challenging” input

of size N

  • Best-case complexity: min # steps

algorithm takes on “easiest” input of size N

  • Average-case complexity : avg # steps

algorithm takes on random inputs of size N

  • Amortized complexity : max total # steps

algorithm takes on M “most challenging”

consecutive inputs of size N , divided by M

(i.e., divide the max total by M ).

44

Bounds vs. Cases

Two orthogonal axes:

  • Bound Flavor
    • Upper bound (O, o)
    • Lower bound (, )
    • Asymptotically tight ()
  • Analysis Case
    • Worst Case (Adversary), T worst( n )
    • Average Case, T avg( n )
    • Best Case, T best( n )
    • Amortized, T amort( n )

One can estimate the bounds for any given case.

45

Bounds vs. Cases

46

Pros and Cons

of Asymptotic Analysis

47

Big-Oh Caveats

  • Asymptotic complexity (Big-Oh) considers only large n
    • You can “abuse” it to be misled about trade-offs
    • Example: n 1/10^ vs. log n
      • Asymptotically n 1/10^ grows more quickly
      • But the “cross-over” point is around 5 * 10^17
      • So n 1/10^ better for almost any real problem
  • Comparing O() for small n values can be misleading
    • Quicksort: O(nlogn)
    • Insertion Sort: O(n^2 )
    • Yet in reality Insertion Sort is faster for small n
    • We’ll learn about these sorts later