Parallel Execution & Compilers: Dependence Analysis & Enhanced Parallelization, Slides of Assembly Language Programming

Download Parallel Execution & Compilers: Dependence Analysis & Enhanced Parallelization and more Slides Assembly Language Programming in PDF only on Docsity!

Prof. Saman Amarasinghe, MIT.

1

6.189 IAP 2007Lecture 11Parallelizing Compilers

2

Outline ●^ Parallel Execution ●^ Parallelizing Compilers ●^ Dependence Analysis ●^ Increasing Parallelization Opportunities ●^ Generation of Parallel Loops ●^ Communication Code GenerationProf. Saman Amarasinghe, MIT.

4

Why Loops? ●^ 90% of the execution time in 10% of the code^ „^ Mostly in loops ●^ If parallel, can get good performance^ „^ Load balancing ●^ Relatively easy to analyzeProf. Saman Amarasinghe, MIT.

5

Programmer Defined Parallel Loop ●^ FORALL^ „^ No “loop carrieddependences”^ „^ Fully parallel Prof. Saman Amarasinghe, MIT.

●^ FORACROSS^ „^ Some “loop carrieddependences”

7

Parallel Execution ●^ Example^ FORPAR I = 0 to NA[I] = A[I] + 1 ●^ Block Distribution: Program gets mapped into^ Iters = ceiling(N/NUMPROC);FOR P = 0 to NUMPROC-1FOR I = PIters to MIN((P+1)Iters, N)A[I] = A[I] + 1 ●^ Code that fork a function^ Iters = ceiling(N/NUMPROC);ParallelExecute(func1);^ …^ void func1(integer myPid){ FOR I = myPidIters to MIN((myPid+1)Iters, N)A[I] = A[I] + 1} Prof. Saman Amarasinghe, MIT.

8

Outline ●^ Parallel Execution ●^ Parallelizing Compilers ●^ Dependence Analysis ●^ Increasing Parallelization Opportunities ●^ Generation of Parallel Loops ●^ Communication Code GenerationProf. Saman Amarasinghe, MIT.

10

Iteration Space ●^ N deep loops^ Æ^ Prof. Saman Amarasinghe, MIT.

n-dimensional discretecartesian space „ Normalized loops: assume step size = 1 FOR^ I^ =^0 to^

(^6) FOR J = I to^7 ●^ Iterations are represented ascoordinates in iteration space^ „^ i̅^ = [i, i, i^1

,…, i] 3 n^

0 1 2^3

5 6 7 ^ J

(^012) I Æ 3 4 5 6

11

Iteration Space ●^ N deep loops^ Æ^ Prof. Saman Amarasinghe, MIT.

n-dimensional discretecartesian space „ Normalized loops: assume step size = 1 FOR^ I^ =^0 to^

(^6) FOR J = I to^7 ●^ Iterations are represented ascoordinates in iteration space ●^ Sequential execution order of iterations^ Î^ Lexicographic order[0,0], [0,1], [0,2], …, [0,6], [0,7],[1,1], [1,2], …, [1,6], [1,7],[2,2], …, [2,6], [2,7],

………[6,6], [6,7],

0 1 2^3

5 6 7 ^ J

(^012) I Æ 3 4 5 6

13

Iteration Space ●^ N deep loops^ Æ^ Prof. Saman Amarasinghe, MIT.

n-dimensional discretecartesian space „ Normalized loops: assume step size = 1 FOR^ I^ =^0 to^

(^6) FOR J = I to^7 ●^ An affine loop nest^ „^ Loop bounds are integer linear functions ofconstants, loop constant variables andouter loop indexes^ „^ Array accesses are integer linear functionsof constants, loop constant variables andloop indexes

0 1 2^3

5 6 7 ^ J

(^012) I Æ 3 4 5 6

14

Iteration Space ●^ N deep loops^ Æ^ Prof. Saman Amarasinghe, MIT.

n-dimensional discretecartesian space „ Normalized loops: assume step size = 1 FOR^ I^ =^0 to^

(^6) FOR J = I to^7 ●^ Affine loop nest

Æ^ Iteration space as aset of liner inequalities 0 ≤ I I ≤ (^6) I ≤ J J ≤ 7

0 1 2^3

5 6 7 ^ J

(^012) I Æ 3 4 5 6

16

Dependences ●^ True dependence^ a^ = =^ a ●^ Anti dependence^ =^ a a^ = ●^ Output dependence^ a^ = a^ = ●^ Definition:Data dependence exists for a dynamic instance i and j iff^ „^ either i or j is a write operation^ „^ i and j refer to the same variable^ „^ i executes before j ●^ How about array accesses within loops?Prof. Saman Amarasinghe, MIT.

17

Outline ●^ Parallel Execution ●^ Parallelizing Compilers ●^ Dependence Analysis ●^ Increasing Parallelization Opportunities ●^ Generation of Parallel Loops ●^ Communication Code GenerationProf. Saman Amarasinghe, MIT.

19

Array Accesses in a loop Prof. Saman Amarasinghe, MIT.

FOR I = 0 to 5A[I] = A[I] + 1 0 1 2^3 4

0 1 2^3

Iteration Space^5

Data Space

= A[I]A[I]= A[I]A[I]= A[I]A[I]= A[I]A[I]= A[I]A[I]= A[I]A[I]

20

Array Accesses in a loop Prof. Saman Amarasinghe, MIT.

FOR I = 0 to 5A[I+1] = A[I] + 1 0 1 2^3 4

0 1 2^3

Iteration Space^5

Data Space

= A[I]A[I+1]= A[I]A[I+1]= A[I]A[I+1]= A[I]A[I+1]= A[I]A[I+1]= A[I]A[I+1]