Lower Bounds for Comparison Sorts: Analysis of Sorting Algorithms, Lecture notes of Design and Analysis of Algorithms

Download Lower Bounds for Comparison Sorts: Analysis of Sorting Algorithms and more Lecture notes Design and Analysis of Algorithms in PDF only on Docsity!

AlgorithmsChapter 8

Sorting in Linear Time

Assistant Professor: Ching‐Chi Lin

[email protected]

Department of Computer Science and Engineering

National Taiwan Ocean University

Outline `^ Lower

bounds

for^ sorting

`^ Counting

sort

`^ Radix

sort

`^ Bucket

sort

2

Lower bounds for sorting `^ Lower

bounds

`^ Ω(

n ) to

examine

all^ the

input.

`^ All

sorts

seen

so^ far

are^ Ω

( n lg n

`^ We’ll

show

that

Ω( n

lg n ) is

a^ lower

bound

for^ comparison

sorts.

`^ Decision

tree

`^ Abstraction

of^ any

comparison

sort.

`^ A^ full

binary

tree.

`^ Represents

comparisons

made

by

`^ a^ specific

sorting

algorithm

`^ on^

inputs

of^ a^ given

size.

`^ Control,

data

movement,

and^

all^ other

aspects

of^ the

algorithm

are^ ignored. 4

`^ For

insertion

sort

on^3

elements:

`^ How

many

leaves

on^ the

decision

tree?

`^ There

are^ ≥

n!^ leaves,

because

every

permutation

appears

at^ least

once. Decision tree^5

1:

1:

2:

1: 2: 〈1,2,3〉

〈1,3,2〉

〈3,1,2〉

〈2,1,3〉

〈2,3,1〉

〈3,2,1〉

≤ ≤

≤

≤ A [1]^ >^ A [2] ≤

A [1]^ >^

A [2] A [1]^ >^

A [3]

A [1]^ ≤^

A [2] A [2]^ >^

A [3]

A [1]^ ≤^

A [2]

A [1]^ ≤^

A [2]^ ≤^

A [3]

compare

A [1]^ to

A [2]

Properties of decision trees

2/

`^ Theorem

1 Any

decision

tree

that

sorts

n^ elements

has

height

Ω( n

lg n ).

Proof

`^ l^ ≥

n !,^ where

l^ =^ #

of^ leaves.

`^ By

lemma

1,^ n!

≤^ l^ ≤

h^ 2 or

h^ 2 ≥^ n !.

`^ Take

logs:

h^ ≥^ lg(

n !).

`^ Use

Stirling’s approximation:

n!^ > (

nn/e )

h^ > lg(

nn / e ) = n lg( n / e

=^ n lg

n^ −^ n

lg^ e =^ Ω( n

lg n ). 7

Properties of decision trees

3/

`^ Corollary

1 Heapsort and

merge

sort

are^

asymptotically

optimal

comparison

sorts.

Proof

`^ The

O ( n lg

n )^ upper

bounds

on^ the

running

times

for^ heapsort

and^ merge

sort^

match

the^ Ω

( n lg n

)^ worst

‐case

lower

bound

from

Theorem

8

Counting sort `^ Non

‐comparison

sorts.

`^ Depends

on^ a

key

assumption:

numbers

to^ be

sorted

are

integers

in^ {

,^^1 ,^.

.^.^ ,^ k

`^ Input:

A [^

.^.^ n ],

where

A [^ j^ ]

^1 ,^.^.

.^ ,^ k }

for^ j

=^1 ,^

2 ,^.^.^

.^ ,^ n.

Array

A^ and

values

n^ and

k^ are

given

as^ parameters.

`^ Output:

B [^.

.^ n ],^

sorted.

B^ is^ assumed

to^ be

already

allocated

and

is^ given

as^ a^

parameter.

`^ Auxiliary

storage:

C [^.

.^ k ].

`^ Worst

‐case

running

time:

Θ( n

+ k ).

10

The C

OUNTING

‐SORT

procedure

C^ OUNTING

‐SORT^ (

A ,^ B ,^ k

)

1.^

for^ i^ ←

0 to^ k

2.^

do^ C [ i

]^ ←^0

3.^

for^ j^ ←

1 to^ length

[ A ]

4.^

do^ C [ A

[ j ]]^ ←

C [ A [ j ]]

+^1

5.^

/*^ C [ i ]

now^ contains

the^ number

of^ elements

equal

to^ i.^ */

6.^

for^ i^ ←

1 to^ k

7.^

do^ C [ i

]^ ←^ C

[ i ]^ +^ C

[ i^ −^ 1]

8.^

/*^ C [ i ]

now^ contains

the^ number

of^ elements

less^ than

or^ equal

to^ i.^ */

9.^

for^ j^ ←

length

[ A ]^ downto

^1

10.^

do^ B [ C

[ A [ j ]]]

←^ A [

j ]

11.^

C [ A [ j ]]

←^ C [

A [ j ]]^ −

1

11

Θ( k ) Θ( n ) Θ( k ) Θ( n )

`^ The

running

time:

Θ( n

+ k ).

Properties of counting sort `^ A^ sorting

algorithm

is^ said

to^ be

stable

if^ keys

with

same

value

appear

in^ same

order

in^ output

as^ they

did^

in^ input.

`^ Counting

sort

is^ stable

because

of^ how

the^

last^ loop

works.

`^ Counting

sort

will^ be

used

in^ radix

sort.

13

Outline `^ Lower

bounds

for^ sorting

`^ Counting

sort

`^ Radix

sort

`^ Bucket

sort

14

Correctness of radix sort `^ Proof:

By^ induction

on^ number

of^ passes

( i^ in^

pseudocode).

^ Basis:^^

i^ =^ 1.

There

is^ only

one^

digit,

so^ sorting

on^ that

digit

sorts

the array.

`^ Inductive

step: `^ Assume

digits

… ,^ i^ −

1 are

sorted.

`^ Show

that

a^ stable

sort^

on^ digit

i^ leaves

digits

… ,^ i^ sorted:

`^ If^2

digits

in^ position

i^ are^

different,

ordering

by^ position

i^ is^ correct,

and^ positions

1,…,^

i^ −^1 are

irrelevant.

`^ If^2

digits

in^ position

i^ are^

equal,

numbers

are^ already

in^ the

right

order

(by^ inductive

hypothesis).

The^ stable

sort^

on^ digit

i^ leaves

them

in^ the

right

order.

16

Time complexity of radix sort `^ Assume

that

we^ use

counting

sort

as^ the

intermediate

sort.

`^ When

each

digit

is^ in

the^

range

0 to

k −1,

each

pass

over

n^ d ‐digit

number

takes

time

Θ( n

+^ k ).

`^ There

are^

d^ passes,

so^ the

total

time

for^ radix

sort

is^ Θ

( d ( n^

+^ k )).

`^ If^

k^ =^ O

( n ),^ time

=^ Θ(

dn ).

`^ Lemma

2:^ Given

n^ d ‐digit

numbers

in^ which

each

digit

can^

take^

on

up^ to

k^ possible

values,

R^ ADIXSORT

correctly

sorts

these

numbers

in

Θ( d ( n

+^ k ))

time. 17

Break each key into digits

2/

`^ Recall

that

the^

running

time

is^ Θ

(( b / r

)( n^ +

r 2 )).

`^ How

to^ choose

r?

`^ Balance

b / r^ and

n^ +^2

r.

`^ If^

b^ <^ ⎣

lg n ⎦,

then

choosing

r^ =^ b

yields

a^ running

time

of

( b / b )(

n^ +^2

r )^ =^ Θ

( n ).

`^ If^

b^ ≥^ ⎣

lg n ⎦,

then choosing

r^ ≈^ lg

n^ gives

us

`^ If^ r

lg^

n ,^ then

r^ 2 term

in^ the

numerator

increases

faster

than

the^ r

term

in^ the

denominator. `^ If^ r

< lg n

,^ then

b / r^ term

increases,

and^

n^ +^2

r^ term

remains

at^ Θ

( n ).

19

).lg ( )) ( lg (^

bn n nn b n

θ

θ^

=

The main reason `^ How

does

radix

sort

violate

the^

ground

rules

for^ a

comparison

sort? `^ Using

counting

sort^

allows

us^ to

gain

information

about

keys

by

means

other

than

directly

comparing

2 keys.

`^ Used

keys

as^ array

indices.

20