Lecture Notes on Amotized Analysis in Algorithms, Slides of Design and Analysis of Algorithms

Download Lecture Notes on Amotized Analysis in Algorithms and more Slides Design and Analysis of Algorithms in PDF only on Docsity!

Amortized Analysis

Analyzing Calls to a Data Structure

• Some algorithms involve repeated calls to one or more

data structures

• Example: Heap sort

• repeatedly insert keys into a priority queue (heap)
• repeatedly remove the smallest key from the heap

• When analyzing the running time of the overall

algorithm, we need to sum up the time spent in all the

calls to the data structure

• When different calls take different times, how can we

accurately calculate the total time?

Amortized Analysis

• Purpose is to accurately compute the total time spent

in executing a sequence of operations on a data

structure

Basic Idea:

In amortized analysis, the time required to perform a sequence

of operations is averaged over all the operations performed.

 No involvement of probability

 Average performance on a sequence of operations,

even if some operation is expensive

 Guarantee average performance of each operation

among the sequence in worst case

Methods of Amortized Analysis

• Aggregate Method: we determine an upper

bound T(n) on the total sequence of n operations.

The cost of each will then be T(n)/n.

• Accounting Method: we charge the operations.

We overcharge some operations early and use

them to as prepaid charge later for other

operations.

• Potential Method: we maintain credit as

potential energy associated with the structure as

a whole.

Example #1: Stack operations

Operations are:

– Push(S,x)

– Pop(S)

– Multipop(S,k) - pop the top k elements

Implement with either array or linked list

– time for Push is O(1)

– time for Pop is O(1)

– time for Multipop is O(min(|S|,k))

Sequence of n push, pop, Multipop

operations

Worst-case cost of Multipop is O(n)

Have n operations

Therefore, worst-case cost of sequence is

O(n

2

)

Example #2: k-Bit Counter A

• k-bit Binary Counter: A[0..k1]

K-





1

0

[ ] 2

k

i

i

x A i

INCREMENT(A)

1. i  0
2. while i < length[A] and A[i] = 1
3. do A[i]  0 reset a bit
4. i  i + 1
5. if i < length[A]

6. then A[i]  1 set a bit

Aggregate Method for k-Bit Counter

• Worst-case time for an increment is O(k),

occurs when all k bits are flipped

• But in a sequence of n operations, not all of

them will cause all k bits to flip

• bit 0 flips with every increment
• bit 1 flips with every 2nd increment
• bit 2 flips with every 4th increment …
• bit k flips with every 2

k

-th increment

Accounting Method

• Assign a cost, called the "amortized cost", to

each operation

• Assignment must ensure that the sum of all

the amortized costs in a sequence is at least

the sum of all the actual costs

– remember, we want an upper bound on the total

cost of the sequence

• How to ensure this property?

Accounting Method

• Every operation is assigned an amortized cost
• For each operation in the sequence:
  • if amortized cost > actual cost then store extra as a credit with

an object in the data structure

• if amortized cost < actual cost then use the stored credits to

make up the difference

• If c

i

is the actual cost and c

i

^ is the amortized cost then,

we must ensure that for all n :

• Thus, the total amortized costs provide an upper bound

on the total true costs.

Accounting Method: Stack Example

3 ops:

Push(S,x) Pop(S) Multi-pop(S,k)

•Assigned

cost:

2 0 0

•Actual cost: 1 1 min(|S|,k)

Note: Push(S,x) pays for possible later pop of x.

Accounting Method for Stack

• When pushing an object, pay Rs.

Rs.1 pays for the push

Rs.1 is prepayment for it being popped by

either pop or Multipop

Since each object has Rs.1, which is credit,

the credit can never go negative

Therefore, total amortized cost = O(n), is an

upper bound on total actual cost

CPSC 411, Fall 2008: Set 6 19

Accounting Method for k-Bit Counter

 When Incrementing,

• Amortized cost for line 3 = Rs.
• Amortized cost for line 6 = Rs.

 Amortized cost for INCREMENT(A) = Rs.

 Amortized cost for n INCREMENT(A) = Rs.2n =O(n)