Fault Tolerance - Distributed Operating Systems - Lecture Slides, Slides of Operating Systems

Download Fault Tolerance - Distributed Operating Systems - Lecture Slides and more Slides Operating Systems in PDF only on Docsity!

Fault Tolerance

Part I Introduction

Part II Process Resilience

Part III Reliable Communication

Part IV Distributed Commit

Part V Recovery

Fault Tolerance

• A DS should be fault-tolerant
  • Should be able to continue functioning in the presence of faults
• Fault tolerance is related to dependability

Availability & Reliability (1)

• Availability : A measurement of whether a system is ready to be used immediately - System is up and running at any given moment
• Reliability : A measurement of whether a system can run continuously without failure - System continues to function for a long period of time

Availability & Reliability (2)

• A system goes down 1ms/hr has an availability of more than 99.99%, but is unreliable
• A system that never crashes but is shut down for a week once every year is 100% reliable but only 98% available

Faults

• A system fails when it cannot meet its promises

(specifications)

• An error is part of a system state that may lead to a

failure

• A fault is the cause of the error
• Fault-Tolerance : the system can provide services even

in the presence of faults

• Faults can be:
  • Transient (appear once and disappear)
  • Intermittent (appear-disappear-reappear behavior)
    • A loose contact on a connector intermittent fault
  • Permanent (appear and persist until repaired)

Failure Models Type of failure Description Crash failure A server halts, but is working correctly until it halts Omission failure Receive omission Send omission A server fails to respond to incoming requests A server fails to receive incoming messages A server fails to send messages Timing failure A server's response lies outside the specified time interval Response failure Value failure State transition failure The server's response is incorrect The value of the response is wrong The server deviates from the correct flow of control Arbitrary failure (Byzantine failure) A server may produce arbitrary responses at arbitrary times

Example – Redundancy in Circuits (1)

Example – Redundancy in Circuits (2) Triple modular redundancy.

Process Resilience

• Mask process failures by replication
• Organize processes into groups, a message sent to a group is delivered to all members
• If a member fails, another should fill in

Flat Groups versus Hierarchical Groups

a) Communication in a flat group.

b) Communication in a simple hierarchical group

Agreement

• Need agreement in DS:
  • Leader, commit, synchronize
• Distributed Agreement algorithm : all non-faulty processes achieve consensus in a finite number of steps
• Perfect processes, faulty channels: two-army
• Faulty processes, perfect channels: Byzantine generals

Two-Army Problem

Byzantine Generals - Example (1) The Byzantine generals problem for 3 loyal generals and1 traitor. a) The generals announce the time to launch the attack (by messages marked by their ids). b) The vectors that each general assembles based on (a) c) The vectors that each general receives in step 3, where every general passes his vector from (b) to every other general.

Byzantine Generals – Example (2)

The same as in previous slide, except now

with 2 loyal generals and one traitor.