Distributed Systems: Consensus and Agreement in Software Engineering, Study notes of Software Engineering

Download Distributed Systems: Consensus and Agreement in Software Engineering and more Study notes Software Engineering in PDF only on Docsity!

Distributed Software Development

Consensus and Agreement Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science — University of San Francisco – p. 1/

Previously on CS 682

Time is a big challenge in systems that don’t share aclock.

Insight: we often don’t need to know the exact timethat events occur.

Instead, we need to know the order in which theyhappened.

Department of Computer Science — University of San Francisco – p. 2/

Happens before

The

happens before

relation is denoted

Happens before is defined:

If

e k i , e l i

and

k < l

, then

e k i

e l i

(sequentially ordered events in the same process)

If

e i

send

m

and

e j

receive

m

, then

e i

e j

(send must come before receive)

If

e

e ′^

and

e ′

e ′′

, then

e

e ′′

(transitivity)

If

e

e ′^

and

e ′^

e

, then we say that

e

and

e ′

are

concurrent. (

e

e ′

These events are unrelated, and could occur in eitherorder.

Department of Computer Science — University of San Francisco – p. 4/

Happens before

Happens before provides a partial ordering over theglobal history.

H,

We call this a distributed computation.

A distributed computation can be represented with aspace-time diagram.

Department of Computer Science — University of San Francisco – p. 5/

Space-time diagram

Arrows indicate messages sent between processes.

Causal relation between events is easy to detect

Is there a directed path between events?

• e 1 1

e 4 3

• e 2 1

e 1 3 Department of Computer Science — University of San Francisco – p. 7/

Monitoring a distributed computation

Recall that we want to know what the global state ofthe system is at some point in time.

Active monitoring won’t work

Updates from different processes may arrive out oforder.

We need to restrict our monitor to looking at consistent cuts

A cut is consistent if, for all events

e

and

e ′

e

C

and

e ′^

e

e ′^

C

In other words, we retain causal ordering andpreserve the ’happens before’ relation.

Department of Computer Science — University of San Francisco – p. 8/

Why does this work?

If we assume a delay of

δ

, at time

t

, all messages

sent before

t

δ

have arrived.

By processing them in increasing order, causality ispreserved.

• e

e ′^

RC

e

< RC

e ′

But we don’t

have

a global clock!!

Department of Computer Science — University of San Francisco – p. 10/

Logical clocks

Each process maintains a logical clock. (

LC

Maps events to natural numbers. (0,1,2,3,...).

In the initial state, all LCs are 0.

Each message

m

contains a timestamp indicating the

logical clock of the sending process.

After each event, the logical clock for a process isupdated as follows:

LC

e

LC

if

e

is a local or send event.

LC

e

max

LC, T S

m

if

e

receive

m

The LC is updated to be greater than both theprevious clock and the timestamp.

Department of Computer Science — University of San Francisco – p. 11/

Logical clocks

• Notice that logical clock values are increasing with respect to causal precedence. - Whenever e → e ′ ,^ LC ( e ) < LC ( e ′)

The monitor can process messages according to their logical clocks to always have aconsistent global state.

Are we done?

Maybe. If we have synchronous communication, this is great.

In an asynchronous model, this delivery rule lacks liveness .

Without a bound on message delay, we can never be sure that we won’t haveanother message with a lower logical clock.

We can’t detect gaps in the clock sequence.

Example: we’ll wait forever for the nonexistent message for e 3 from p 1 .

We can solve this by waiting to deliver e 3 until we’ve received messages with higher LCs from all other processes. Department of Computer Science — University of San Francisco – p. 13/

Causal delivery

Recall that FIFO only refersto messages sent by thesame process.

causal delivery

is a more

general property which saysthat if

send

m 1

send

m 2

then

deliver

m 1

m 2

when

different processes aresending

m 1

and

m 2

Logical clocks aren’t enoughto give us causal delivery.

1 p 2 p 3 p

Department of Computer Science — University of San Francisco – p. 14/

Vector Clock example

p 1 p 2 p 3 (1,0,0) (0,1,0) (0,0,1) (2,1,0) (1,0,2)(1,0,3) (1,0,4) (3,1,3) (1,2,4) (4,1,3) (4,3,4) (1,0,5) (5,1,6) (5,1,3) (6,1,3) Department of Computer Science — University of San Francisco – p. 16/

Vector clocks

The monitor can then process events in order ofascending vector clocks.

This ensures causality.

Two clocks are inconsistent if

c 1

[

i

]

< c 2

[

i

]

and

c 1

[

i

]

c 2

[

j

]

If a cut contains no inconsistent clocks, it isconsistent.

Vector clocks allow the implementation of causaldelivery.

Department of Computer Science — University of San Francisco – p. 17/

Applying causal delivery

Causal delivery gives us almost all of the functionalitythat we need from a global clock.

We can build on top of this to solve more complexcoordination problems.

Coordination often requires not only that allprocesses agree on state, but that all processes canensure that every other process sees the same state.

Department of Computer Science — University of San Francisco – p. 19/

Consensus and agreement

A fundamental problem in distributed systems isgetting a set of processes or nodes to agree on oneor more values.

Is a procedure continuing or aborted?

What value is stored in a distributed database?

Which process is serving as coordinator?

Has a node failed?

There are a set of related problems that require a setof processes to coordinate their states or actions.

Department of Computer Science — University of San Francisco – p. 20/