Space Complexity - Data Structures and Algorithm - Lecture Slides, Slides of Data Structures and Algorithms

Download Space Complexity - Data Structures and Algorithm - Lecture Slides and more Slides Data Structures and Algorithms in PDF only on Docsity!

Data Structures &

Algorithm Analysis

Last Time

• Steps in problem solving

• Algorithm analysis

– Space complexity

– Time complexity

– Pseudo-code

Asymptotic Notation

• Goal: to simplify analysis by getting rid of

unneeded information (like “rounding”

• We want to say in a formal way 3n 2 ≈ n^2

• The “Big-Oh” Notation:

– given functions f(n) and g(n), we say that f(n) is

O( g(n) ) if and only if there are positive constants c

and n 0 such that f(n)≤ c g(n) for n ≥ n 0

Graphic Illustration

• f(n) = 2n+

• Conf. def:

– Need to find a

function g(n) and

a const. c such as

f(n) < cg(n)

• g(n) = n and c = 4

•  f(n) is O(n)

• The order of f(n)

is n

g ( n ) = n

c g ( n ) = 4 n

n

f ( n ) = 2 n + 6

Properties of Big-Oh

• If f(n) is O(g(n)) then af(n) is O(g(n)) for any a.
• If f(n) is O(g(n)) and h(n) is O(g’(n)) then f(n)+h(n) is O(g(n)+g’(n))
• If f(n) is O(g(n)) and h(n) is O(g’(n)) then f(n)h(n) is O(g(n)g’(n))
• If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) is O(h(n))
• If f(n) is a polynomial of degree d , then f(n) is O(n d)
• nx^ = O(a n), for any fixed x > 0 and a > 1
  • An algorithm of order n to a certain power is better than an algorithm of order a ( >
    1. to the power of n
• log nx^ is O(log n), fox x > 0 – how?
• log x^ n is O(ny^ ) for x > 0 and y > 0
  • An algorithm of order log n (to a certain power) is better than an algorithm of n raised to a power y.

Asymptotic analysis - terminology

• Special classes of algorithms:

logarithmic : O(log n) linear : O(n) quadratic : O(n 2 ) polynomial : O(n k), k ≥ 1 exponential : O(a n), n > 1

• Polynomial vs. exponential?
• Logarithmic vs. polynomial?

“Relatives” of Big-Oh

• “Relatives” of the Big-Oh
  • Ω (f(n)): Big Omega – asymptotic lower bound
  • Θ (f(n)): Big Theta – asymptotic tight bound
• Big-Omega – think of it as the inverse of O(n)
  • g(n) is Ω (f(n)) if f(n) is O(g(n))
• Big-Theta – combine both Big-Oh and Big-Omega
  • f(n) is Θ (g(n)) if f(n) is O(g(n)) and g(n) is Ω (f(n))
• Make the difference:
  • 3n+3 is O(n) and is Θ (n)
  • 3n+3 is O(n 2 ) but is not Θ (n 2 )

More “relatives”

• Little-oh – f(n) is o(g(n)) if for any c>0 there is

n0 such that f(n) < c(g(n)) for n > n0.

• Little-omega

• Little-theta

• 2n+3 is o(n 2 )

• 2n + 3 is o(n)?

Example (cont’d)

Algorithm prefixAverages2(X):

Input: An n-element array X of numbers.

Output: An n -element array A of numbers such that A[i] is the

average of elements X[0], ... , X[i].

Let A be an array of n numbers. s ← 0 for i ← 0 to n do s ← s + X[ i ] A[ i ] ← s/( i + 1) return array A

Back to the original question

• Which solution would you choose?
  • O(n 2 ) vs. O(n)
• Some math …

– properties of logarithms:

logb(xy) = logbx + logby logb (x/y) = logbx - logby logbxa = alogbx logba= logxa/logx b

– properties of exponentials:

a (b+c)^ = a b^ a c a bc^ = (a b^ ) c a b^ /a c^ = a (b-c) b = a logab b c^ = a c*logab Docsity.com

Analyzing recursive algorithms

function foo (param A, param B) {

statement 1;

statement 2;

if (termination condition) {

return;

foo(A’, B’);

Solving recursive equations by repeated substitution

T(n) = T(n/2) + c substitute for T(n/2)

= T(n/4) + c + c substitute for T(n/4)

= T(n/8) + c + c + c

= T(n/2 3 ) + 3c in more compact form

= T(n/2 k^ ) + kc “inductive leap”

T(n) = T(n/2logn^ ) + clogn “choose k = logn”

= T(n/n) + clogn = T(1) + clogn = b + clogn = θ(logn)

Problem

• Running time for finding a number in a sorted

array

[binary search]

– Pseudo-code

– Running time analysis

ADT

• ADT = Abstract Data Types

• A logical view of the data objects together

with specifications of the operations required

to create and manipulate them.

• Describe an algorithm – pseudo-code

• Describe a data structure – ADT