Download Distributed Software Development: Replication - Lecture Notes and more Study notes Software Engineering in PDF only on Docsity!
Distributed Software Development
Replication
Chris Brooks
Department of Computer Science
University of San Francisco
Department of Computer Science — University of San Francisco – p. 1/
??
Previously on cs682: Causal delivery
•^
causal delivery
says that if
send
(m
send
(m
, then
deliver
(m
(m
when different processes are sending
m
1
and
m
•^
Logical clocks aren’t enough to give us causal delivery.
Department of Computer Science — University of San Francisco – p. 2/
??
Previously on cs682: Vector clocks
•^
Solution: keep a “logical clock” for each process.
-^
these are stored in a vector
V C
◦^
Assumes number of processes in known and fixed.
-^
Update rule:^ ◦
V C
(e
)[
i] =
V C
[i] + 1
for send and internal.
◦^
V C
(e
max
(V C, T S
(m
for receive; then
V C
(e
)[
i] =
V C
[i] + 1
•^
On receive, the vector clock takes the max on acomponent-by-component basis, then updates the local clock.
Department of Computer Science — University of San Fra
Previously on cs682: Vector Clock example
p^1 p^2 p^3
(1,0,0) (0,1,0) (0,0,1)
(2,1,0) (1,0,2)(1,0,3) (1,0,4)
(3,1,3)
(1,2,4)
(4,1,3)
(4,3,4) (1,0,5)
(5,1,6)
(5,1,3)
(6,1,3)
Department of Computer Science — University of San Francisco – p. 4/
??
Previously on cs682: Consensus
•^
If we don’t have reliable communication, consensus isimpossible, even without failures.
-^
With reliable communication, we can solve consensus for crashfailures.
-^
In asynchronous systems, it is impossible to guarantee that wewill reach consensus, even in the presence of a single crashfailure.
-^
This means that we can’t do:^ ◦
Asynchronous Byzantine generals ◦ Asynchronous totally ordered multicast
Department of Computer Science — University of San Francisco – p. 5/
??
Replication
•^
Replication is the maintenance of copies of data at multiplecomputers.
-^
Enhances a service by providing:^ ◦
Fault tolerance ◦ Improved performance ◦ Increased availability ◦ Information redundance
Department of Computer Science — University of San Fra
Why is replication useful?
•^
Increased performance.^ ◦
By moving data closer to a client, latency is reduced. ◦ Web caching, proxy servers are an example of this.
-^
Performance is improved most effectively with immutable data.^ ◦
If the client is going to change the data and send it back,performance gains are reduced.
Department of Computer Science — University of San Francisco – p. 7/
??
Why is replication useful?
•^
Increased availability.
-^
Many services need to be highly available
-^
Replication provides a way of overcoming server failures.
-^
If a server will fail with probability
p
, then we can determine how
many servers are needed to provide a given level of service:^ ◦
Avail
p
n
•^
For example, if a server has a 5% chance of failure (i.i.d) over agiven time period, and we want 99.9% availability, we need atleast 4 replicas.
Department of Computer Science — University of San Francisco – p. 8/
??
Client-side replication
•^
Note that replication is not limited to servers.
-^
Multiple clients may need to replicate data.^ ◦
Shared code or documents being edited. ◦ Meeting scheduling ◦ Conferencing or whiteboard software.
Department of Computer Science — University of San Fra
Fault tolerance
•^
Highly available data may not be correct data.^ ◦
For example, in the presence of network outages.
-^
Fault tolerance guarantees correct behavior in the presence ofa given number of faults.
-^
Similar to availability, but a coordination element is alsorequired.
-^
We may also want to ensure against corruption of data.
Department of Computer Science — University of San Francisco – p. 10/
??
Outline
•^
Passive Replication^ ◦
What problems must be solved for this?
-^
Active Replication^ ◦
What problems must be solved for this?
-^
Lazy Replication^ ◦
What problems must be solved for this?
Department of Computer Science — University of San Francisco – p. 11/
??
Client Requirements
•^
Single logical copy.
-^
Multiple physical copies.
-^
Consistency^ ◦
The details of this will depend on the application.
Department of Computer Science — University of San Fran
Group Communication
•^
Updating replicas is a
group communication
problem.
•^
If groups can change dynamically, then a group membershipservice is needed.
-^
This keeps track of the processes that are currently in a group.
Department of Computer Science — University of San Francisco – p. 19/
??
Group membership
•^
A group membership process needs to:^ ◦
Provide an interface for membership changes, creation, anddestruction. ◦ Provide a failure detector ◦ Notify members of membership changes. ◦ Perform group address expansion. When a message is sentvia multicast, it is sent to the group address. Themembership process must then deliver that message to allprocesses.
Department of Computer Science — University of San Francisco – p. 20/
??
View-synchronous communication
•^
View-synchronous communication is an extension of reliablemulticast.
-^
It uses a
view
, which is a list of all processes currently in the
group.
-^
When membership changes, a new view is sent to all members.
-^
All messages that originate before a new view must bedelivered before that new view is delivered.^ ◦
Provides a cut in the message timeline. ◦ Can also be thought of as providing a state transition.
View-synchronous communication
•^
View-synchronous communication guarantees that if oneprocess delivers a message within the context of a view, allprocesses deliver that message with that view’s context.
-^
Change of view is treated as another timestamped message.
-^
Allows us to break the timeline into states, or sequences ofviews.
-^
All processes agree on what has happened in that view.
Department of Computer Science — University of San Francisco – p. 22/
??
Comments on passive replication
•^
n^
replicas are needed to tolerate
n
crash failures.
•^
Very easy from the front end’s point of view; all that is needed isto communicate with a single server, and possible changeservers if the primary fails.
-^
Problem: view-synchronous communication requires significantoverhead.^ ◦
This can lead to latency in transmitting data back to theclient.
Department of Computer Science — University of San Francisco – p. 23/
??
Failures
•^
Passive replication deals nicely with crash failures.
-^
If a backup crashes, there’s no problem.
-^
if the primary crashes, it is replaced by a new primary.View-synchronous group communication allows the replicas toagree as to what operations have been performed.
-^
The new primary picks up at that point.
Active Replication
Client
Client
RM RM RM
FrontEnd
FrontEnd
•^
In active replication, allreplica managers are equal
-^
Front ends multicastrequests to all replicamanagers.
-^
All managers processrequest independently
-^
If a replica crashes, otherscontinue as normal.
Department of Computer Science — University of San Francisco – p. 25/
??
Active Replication
•^
Sequence of events:1. Front end attaches a unique identifier to request and
multicasts it to all RMs using totally ordered reliablemulticast.
- Group communication system delivers messages to all RMs
in the same order.
- Each replica manager evaluates the request.4. Since totally ordered multicast is used for communication,
no agreement phase is needed.
- Each replica returns its response to the front end.
Department of Computer Science — University of San Francisco – p. 26/
??
Active Replication
•^
When can the front end return a response to the client?
-^
Depends on what sorts of failures you want to tolerate.^ ◦
Crash failures: can return the first response. ◦ Byzantine failures: Must collect
2 3
Active replication
•^
Notice that active replication assumes totally ordered reliablemulticast.
-^
Recall that this is equivalent to the consensus problem.
-^
We need either:^ ◦
A synchronous system ◦ Failure detectors
-^
As with passive replication, totally ordered multicast may causelatencies.
Department of Computer Science — University of San Francisco – p. 28/
??
Failures
•^
Failure is totally transparent here.
-^
Nothing new is needed if a replica manager crashes.
-^
Too many Byzantine failures can cause the usual problems.
Department of Computer Science — University of San Francisco – p. 29/
??
Lazy Replication
•^
Active and passive replication provide fault tolerance andsequential consistency.
-^
They do this by immediately synchronizing before returningdata to the client.
-^
This is great for fault tolerance, but not for availability.^ ◦
Some applications may not tolerate latency.
-^
We can trade latency for sequential consistency to get a highlyavailable service.
Updating with lazy replication
•^
If the query’s timestamp precedes the replica manager’s, theupdate can proceed immediately.
-^
Otherwise, the update is queued until the causally-previousmessages arrive.
-^
Once this happens, the replica manager logs the update andupdates its timestamp.
Department of Computer Science — University of San Francisco – p. 37/
??
Gossiping
•^
Replicas synchronize with each other through
gossip
messages
•^
A gossip message contains a log of past updates and atimestamp.
-^
When a gossip message is received, the receiver must:^ ◦
Merge the log with its own. (the log may reflect otherupdates it hasn’t seen) ◦ Apply stable updates ◦ Remove entries from the log that are known to have beenapplied everywhere.
Department of Computer Science — University of San Francisco – p. 38/
??
Gossiping
•^
Update frequency (how often gossip messages are sent) isapplication-dependent.^ ◦
Bulletin board: minutes, or less. ◦ Inventorying systems: possibly less frequently. ◦ May also depend on network availability. (for example,meeting schedulers on PDAs)
-^
Who updates are sent to can also be tuned to fit the application.^ ◦
Can choose randomly ◦ Can choose the replica who is ’farthest behind’ ◦ Can use a fixed topology.
-^
Trade off amount of communication against latency and effectsof failure.
Comments on lazy replication
•^
Provides high availability; clients usually receive an immediateresponse.
-^
Works even when the network is partitioned.
-^
Not appropriate for applications where replicas must be tightlysynchronized, such as video conferencing.
-^
Scalability is an issue, due to number of messages and size oftimestamp.
Department of Computer Science — University of San Francisco – p. 40/
??
Failures
•^
The gossip architecture can also deal nicely with failure.
-^
Front ends can connect to more than one RM.
-^
It’s possible for a transaction to be lost if it is submitted to an RMwho immediately crashes before sending it on to any other RM.
Department of Computer Science — University of San Francisco – p. 41/
??
Summary
•^
Replication is the maintenance of copies of data at multiplecomputers.^ ◦
Provides fault tolerance, increased performance, highavailability.
-^
Passive replication: a single RM interacts with clients, andsynchronizes with slaves.
-^
Active replication: All RMs communicate with clients andsynchronize with totally ordered multicast.
-^
Lazy replication: RMs immediately return responses to clientsand synchronize later.
