Streamlt Parallelizing Compiler, Lecture Slides - Assembly Programming, Slides of Assembly Language Programming

Download Streamlt Parallelizing Compiler, Lecture Slides - Assembly Programming and more Slides Assembly Language Programming in PDF only on Docsity!

Prof. Saman Amarasinghe, MIT.

1

6.189 IAP 2007 MIT

6.189 IAP 2007

Lecture 12

StreamIt Parallelizing Compiler

2

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

Common Machine Language ●

Represent common properties of architectures

„

Necessary for performance

●

Abstract away differences in architectures

„

Necessary for portability

●

Cannot be too complex

„

Must keep in mind the typical programmer

●

C and Fortran were the common machine languages foruniprocessors

„

Imperative languages are not the correct abstraction forparallel architectures.

●

What is the correct abstraction for parallel multicoremachines?

4

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

Explicit Parallelism ●

Programmer controls details of parallelism!

●

Granularity decisions:

„

if too small, lots of synchronization and thread creation

„

if too large, bad locality

●

Load balancing decisions

„

Create balanced parallel sections (not data-parallel)

●

Locality decisions

„

Sharing and communication structure

●

Synchronization decisions

„

barriers, atomicity, critical sections, order, flushing

●

For mass adoption, we need a better paradigm:

„

Where the parallelism is natural

„

Exposes the necessary information to the compiler

5

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

Unburden the Programmer ●

Move these decisions to compiler!

„

Granularity

„

Load Balancing

„

Locality

„

Synchronization

●

Hard to do in traditional languages

„

Can a novel language help?

7

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

Outline ●

Why we need New Languages?

●

Static Schedule

●

Three Types of Parallelism

●

Exploiting Parallelism

8

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

push=

Steady-State Schedule ●

All data pop/push rates are constant

●

Can find a Steady-State Schedule

„

of items in the buffers are the same before and theafter executing the schedule

„

There exist a unique minimum steady state schedule

●

Schedule = { }

pop= push=

pop=

A

B

C

10

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

push=

push=

Steady-State Schedule ●

All data pop/push rates are constant

●

Can find a Steady-State Schedule

„

of items in the buffers are the same before and theafter executing the schedule

„

There exist a unique minimum steady state schedule

●

Schedule = { A, A }

pop= push=

pop=

A

B

C

11

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

push=

Steady-State Schedule ●

All data pop/push rates are constant

●

Can find a Steady-State Schedule

„

of items in the buffers are the same before and theafter executing the schedule

„

There exist a unique minimum steady state schedule

●

Schedule = { A, A, B }

pop= push=

pop=

pop= push=

A

B

C

13

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

push=

Steady-State Schedule

pop= push=

pop=

pop= push=

A

B

C

●

All data pop/push rates are constant

●

Can find a Steady-State Schedule

„

of items in the buffers are the same before and theafter executing the schedule

„

There exist a unique minimum steady state schedule

●

Schedule = { A, A, B, A, B }

14

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

push=

Steady-State Schedule ●

All data pop/push rates are constant

●

Can find a Steady-State Schedule

„

of items in the buffers are the same before and theafter executing the schedule

„

There exist a unique minimum steady state schedule

●

Schedule = { A, A, B, A, B, C }

pop= push=

pop=

pop=

A

B

C

16

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

Initialization Schedule ●

When peek > pop, buffer cannot be empty afterfiring a filter

●

Buffers are not empty at the beginning/end of thesteady state schedule

●

Need to fill the buffers before starting the steadystate execution

peek=

pop= peek=4push=

pop= push=

17

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

Outline ●

Why we need New Languages?

●

Static Schedule

●

Three Types of Parallelism

●

Exploiting Parallelism

19

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

Types of Parallelism

Task Parallelism

„

Parallelism explicit in algorithm

„

Between filters

without

producer/consumer

relationship

Data Parallelism

„

Between iterations of a

stateless

filter

„

Place within scatter/gather pair (

fission

)

„

Can’t parallelize filters with state

Pipeline Parallelism

„

Between producers and consumers

„

Stateful

filters can be parallelized

Scatter Gather

Scatter

Gather

Task

Pipeline

Data

Data Parallel

20

6.189 IAP 2007 MIT

Prof. Saman Amarasinghe, MIT.

Types of Parallelism

Traditionally: Task Parallelism

„

Thread (fork/join) parallelism

Data Parallelism

„

Data parallel loop (

forall

)

Pipeline Parallelism

„

Usually exploited in hardware

Scatter Gather

Scatter

Gather

Task

Pipeline

Data