Parallel Algorithms and Sorting in Parallel Programming, Thesis of Parallel Computing and Programming

Download Parallel Algorithms and Sorting in Parallel Programming and more Thesis Parallel Computing and Programming in PDF only on Docsity!

Parallel Algorithm & Sorting in Parallel

Programming

Contents:

1.1 Parallel algorithm

1.2 A Network for sorting

1.3 Sorting on a linear array

1.4 Sorting on the CRCW Model

1.5 Sorting on the CREW Model

1.6 Sorting on the EREW Model

Parallel Sorting:-



The fundamental operation of comparison-

based sorting is compare-exchange.



The lower bound on any comparison-based

sort of n numbers is Θ (nn log n).



The sorted list is partitioned with the

property that each partitioned list is sorted

and each element in processor P

i

's list is less

than that in P

j

's list if i < j

Sorting: Parallel Compare Exchange Operation

A parallel compare-exchange operation. Processes P

i

and P

j

send their elements to each other. Process P

i

keeps min{a

i

,a

j

}, and P

j

keeps max{a

i

, a

j

Cont…



Average optimal sequential complexity: O(n log n)



Parallel efficiency limitations



Partitions are unbalanced



A single processor performs the initial

partitioning

Example of quicksort



Let S = (6,5 ,9,2,4,3,5 , 1, 7,5,8 ).

T he first call to procedure Q U I C K S O R T

produces 5 as the median element of S, and hence

S1 = {2,4,3,1,5,5} and

S2 = {6,9,7,8,5}.

Note that S1 = 6 and S2= 5. A recursive call to Q U I

C K S O R T with S, as input produces the two

subsequences {2,1,3} and {4,5,5}. The second call

with S, as input produces {6,5,7}an d {9,8}. Further

recursive calls complete the sorting of these

sequences.

COMPLEXITY OF QUICKSORT

For some constant c, we can express the running

time of procedure

QUICKSORT as

= O(n log n),

1.2 A NETWORK FOR SORTING



It is rather straightforward to use a collection of

merging networks



to build a sorting network for the sequence S = {s1,

s2,... , sn), where n is a power of 2. The idea is the

following.



In a first stage, a rank of n/2 comparators is used to

create n/2 sorted sequences each of length 2.



In a second stage, pairs of these are now merged into

sorted sequences of length 4 using a rank of (2,2)-

merging networks. Again, in a

FI G : ODD EVEN SORTI NG NE TWO RK FO R SEQ UENCE OF EI GHT E LEM ENTS

The odd-even sorting network is impractical for large input

sequences :

(i) The network is extremely fast. It can sort a sequence of

length 2^20 within, on the order of, (20)2 time units.

This is to be contrasted with the time required by

procedure QUICKSORT, which would be in excess of

20 million time units.[(log n)^2]

(ii) The number of comparators is too high. Again for n =

2^20, the network would need on the order of 400 million

comparators.[n (log n)^2]

(iii) The architecture is highly irregular and the wires linking

the comparators have lengths that vary with n.

Odd-Even Transposition Sort



Variation of bubble sort.



Operates in two alternating phases, even phase

and odd phase.



Even phase

Even-numbered processes exchange numbers

with their right neighbour.



Odd phase

Odd-numbered processes exchange numbers with

their right neighbour.



Odd-Even Transposition Sort -

example

Parallel time complexity: T

par

= O( n ) (for P=n)

MERGE SPLIT:-

Now consider the second approach. If N processors,

where N < n,

Assume that each of the N processors in the linear array

holds a subsequence of S of length n/N.

The comparison-exchange operations of procedure

ODD-EVEN TRANSPOSITION are now replaced with

merge-split operations on subsequences.

Let Si denote the subsequence held by processor Pi.

Initially, the Si are random subsequences of S.

Sorting sequence of twelve elements using procedure

MERGE SPILIT:-