PRAM: Parallel Random Access Machine - Lecture Notes, Study notes of Introduction to Sociology

Download PRAM: Parallel Random Access Machine - Lecture Notes and more Study notes Introduction to Sociology in PDF only on Docsity!

PRAM

(Parallel Random Access Machine)

Lecture Overview

„ Why do we need a model

„ PRAM

„ Some PRAM algorithms

„ Analysis

A Parallel Machine Model

„ What is a machine model?

„ Describes a “machine”

„ Puts a value to the operations on the

machine

„ Why do we need a model?

„ Makes it easy to reason algorithms

„ Achieve complexity bounds

„ Analyzes maximum parallelism

RAM (Random Access Machine)

„ Unbounded number of local memory cells

„ Each memory cell can hold an integer of

unbounded size

„ Instruction set included – simple operations,

data operations, comparator, branches

„ All operations take unit time

„ Time complexity = number of instructions

executed

„ Space complexity = number of memory

cells used

PRAM (Parallel RAM) (contd.)

P 0 P 1 P 2 P N

Shared Memory Cells

Strengths of PRAM

„ Natural extension of RAM

„ It is simple and easy to understand

„ Removes communication and

synchronization issues

„ Can be used as a benchmark

„ If algorithm fails on PRAM then too bad

„ It is useful to reason (specially with

increasing shared memory machines)

Memory Classifications

„ PRAMs are classified based on their Read/Write

abilities

„ Exclusive Read (ER) – all processors can

simultaneously read from distinct memory locations

„ Exclusive Write (EW) – all processors can

simultaneously write to distinct memory locations

„ Concurrent Read (CR) – All processors can

simultaneously read from any memory location

„ Concurrent Write (CW) – All processors can write to

any memory location

Concurrent Write (CW)

„ What value gets written finally?

„ Priority CW – processors have priority

based on which write value is decided

„ Common CW – multiple processors can

simultaneously write only if values are

same

„ Arbitrary/Random CW – any one of the

values are randomly chosen

Parallel Addition

P 0 P 1 P 2 P 3

P 0 P 2

P 0

Step 1

Step 2

Step 3

Parallel Addition

„ log(n) steps=time needed

„ n/2 processors needed

„ Speed-up = n/log(n)

„ Efficiency = 1/log(n)

„ Applicable for other operations too

„ +, *, <, >, == etc.

Example 2 (complicating things)

„ p processor PRAM withn numbers (p ≤ n)

„ Doesx exist within then numbers?

„ P 0 containsx and finally P 0 has to know

„ Algorithm

„ Inform everyone whatx is

„ Every processor checks [n/p] numbers and

sets a flag

„ Check if any of the flags are set to 1

Example 2 (complicating things)

„ Inform everyone whatx is

„ Every processor checks [n/p] numbers

and sets a flag

„ Check if any of the flags are set to 1

„ log(p)

„ n/p

„ log(p)

„ 1

„ n/p

„ log(p)

„ 1

„ n/p

„ 1

EREW CREW CRCW

(common)

Some variants of PRAM

„ Bounded shared memory PRAM,

PRAM(m)

„ Global memory segmented into modules

„ Any given step, onlym memory accesses

can be serviced

„ Bounded number of processor PRAM

„ Any problem that can be solved for ap

processor PRAM int steps can be solved in

ap’ processor PRAM int’=O(tp/p’) steps

Some variants of PRAM

„ LPRAM

„ Charge a cost of L units to access global

memory

„ Any algorithm that runs properly in ap

processor PRAM can run in this model with

a loss of factor L

„ BPRAM

„ Charge L units to access first message

„ B units for each subsequent message

Conclusions

„ We need some model to reason,

compare, analyze and design algorithms

„ PRAM is simple and easy to understand

„ Rich set of theoretical results

„ Over-simplistic and often not realistic

„ Made way to models like BSP, LogP