Algorithm Structure II - Lecture Slides | COSC 4397, Study notes of Computer Science

Download Algorithm Structure II - Lecture Slides | COSC 4397 and more Study notes Computer Science in PDF only on Docsity!

COSC 4397 – Parallel ComputationEdgar Gabriel

COSC 4397

Parallel ComputationAlgorithm structure (II)

Edgar GabrielSpring 2006

2

COSC 4397 – Parallel ComputationEdgar Gabriel

Algorithm structure

-^

Task parallelism:– Implemented by Task queues– Task distribution vs. work stealing

-^

Divide and Conquer for recursive problems– Split problem into sub-problems until a lower limit in the problem

size has been reached

• Solve the sub-problem– Merge the results of the sub-problems into the final result

Organize by data

decomposition

Organize by tasks^ Task Parallelism Divide and Conquer

Geometric decomposition Recursive data

Organize by flow of

data Pipeline Event-basedcoordination

4

COSC 4397 – Parallel ComputationEdgar Gabriel

Task Parallelism using work stealing

Worker Process 1

Worker Process 2

5

COSC 4397 – Parallel ComputationEdgar Gabriel

Divide and Conquer algorithms

split

split Base solve

Merge

split Base solve

Merge

Merge

7

COSC 4397 – Parallel ComputationEdgar Gabriel

Numerical differentiation –forward difference formula

-^

From the definition of derivativesone can derive an approximationfor the 1st derivative

-^

The same formula can be obtained from the Taylorseries, e.g.

h

x f h x f x

f^

h

lim ) (^

0

′^

→

h

x f h x f x

f^

(^

2

ξ f h x f h x f h x

f^

f h

h

x

hf

h x f

x f^

x

y

h x^ n

x^ nh

) ( xf

) (^ xn ′ f

8

COSC 4397 – Parallel ComputationEdgar Gabriel

Central Difference Formula

-^

A better formula is derived if looking at the following twoterms

-^

Subtracting equation (5:2)from (5:1) leads to–

h

is quadratic in the error term

) (

! 3

) ( 2 ) ( ) ( )

(^

1

3

2

ξ f h x f h x

fh

x f

h x f^

′′′

′′

′

=

) (

! 3

) ( 2 ) ( ) ( )

(^

2

3

2

ξ f h x f h x

fh

x f

h x f^

′′′

−

′′

′

−

=

−

[...] 12 )]

(

)

( [ (^12) ) (

(^2) h

h x f h x f h x

f^

−

−

−

= ′^

x

y

h x^ n

h

) ( xf

) (^ xn ′ f

h x^ n

−

h

(5:1) (5:2)

10

Numerical differentiation - summary^ COSC 4397 – Parallel ComputationEdgar Gabriel

-^

Forward difference formula:

-^

Central difference formula for the 1

st

derivative:

-^

Central difference formula for the 2

nd

derivative:

h

x f h x f x

f^

(^

)]

[

(^

h x f h x f h x

f^

)]

( ) ( 2 ) ( [ 1 )

(^

2

h x f x f h x f h x

f^

11

Differential equations - terminology^ COSC 4397 – Parallel ComputationEdgar Gabriel

•^

Differential equations: equations containing the derivative of afunction as a variable– An ordinary differential equation (ODE) only contains

functions of one independent variable

• A partial differential equation (PDE) contains functions of

multiple independent variables and their partial derivatives

•^

The

order

of a differential equation is that of the highest

derivative that it contains

-^

The goal is to find a function

y(t)

whose derivatives fulfill the

given differential equations, e.g.

(^

) 1 (

) (^

−

n

n^

y y y y t f t y

13

COSC 4397 – Parallel ComputationEdgar Gabriel

Finite Differences Approach (II)

-^

For simplicity, lets assume the points are equally spaced

-^

A two point boundary value problem becomes then

-^

Equation (x:1) leads to a system of equations

-^

Solving the system of linear equations gives the solutionof the ODE at the distinct points

ih a

xi

n

i^

) 1

(

−^ +

=^

n

a b

h

y^0

1

1

1

1

2

−

−

+^

i

i

i

i

i^

y y h y x f y y y h

+^1 n y

(x:1)

1

1 0

,^

• n

n^

x x

x x

14

COSC 4397 – Parallel ComputationEdgar Gabriel

Example (I)

-^

Solve the following two point boundary value problemusing the finite difference method with h=0.

-^

Since h=0.2, the mesh points are

-^

Thus,

y

1

• y

4

are unknown

y

2 2

x

dy dx

dx

y d

(^

y

5 4 3 2 1 0

x x x x x x

(^

0

0

x y

y^

(^

5

5

x y

y

x

16

COSC 4397 – Parallel ComputationEdgar Gabriel

Example (III)

1

2

1

0

x

y

y

y^

2

1

y y : 1 = i

2

1

y

y^

i^

3

2

1

y

y

y^

i^

4

3

2

y y y : 4 =

i^

4

3

y

y

    ^ −

y^1 y^2 y^3 y^4

or

i

A

y^

b

17

COSC 4397 – Parallel ComputationEdgar Gabriel

1

ˆ^0

−

i T

i^

r r

Solving

Ay=b

using Bi-CGSTAB

0

0

Ay

b

r^

Given

A,b

and an initial guess

y

0

Given

such that ˆ^ r

ˆ^

0 0

r r^

T

1 0

0

0

0

p

v^ for i = 1,2,…

1

1

−

−

=

i

i i

α ω

ρ ρ

β

(^

1 1

1

1

− −

−

−^

i i

i

i

i^

v

p

r

p

i

i^

Ap

v^

i i Tv ρ ˆ r 0

α

=

i

i^

v

r

s

−^1 As

t^

t t

s Tt T

i^

= ω

s

p

y

y^

i

i

i

i

−^1

t

s

r^

i

i

Scalar product

Matrix-vectormultiplication

19

COSC 4397 – Parallel ComputationEdgar Gabriel

2

1

x

x^

Matrix-vector product in parallel

1 2 3 4

1 2 3 4

rhsrhsrhs rhs

x x x x

3

2

1

x

x

x^

3

4

3

2

rhs

x

x

x^

4

4

3

rhs

x

x^

1 rhs =

2 rhs =

Process 0Process 1

Process 0 needs

x

3

Process 1 needs

x

4

20

Matrix vector product in parallel (II)^ COSC 4397 – Parallel ComputationEdgar Gabriel

-^

Introduction of ghost cells

-^

Looking at the source code, e.g …

-^

…since the vector used in the matrix vector multiplicationchanges every iteration, you always have to update theghost cells before doing the calculation

1 Process zero x

2 x^

3 x

Process one

4 x

3 x^

2 x )

(^

1 1

1

1

− −

−

−^

i i

i

i

i^

v

p

r

p

i

i^

Ap

v^

… …