Distributed Systems: Concepts, Design Goals, and Scaling Techniques, Zusammenfassungen von Computer Vision

It is the summary of Distributed Application. What is distributed and how it has been utilised. What are the best practices

Art: Zusammenfassungen

2020/2021

Hochgeladen am 19.06.2022

mohammad-afwan
mohammad-afwan 🇩🇪

5 dokumente

1 / 5

Toggle sidebar

Diese Seite wird in der Vorschau nicht angezeigt

Lass dir nichts Wichtiges entgehen!

bg1
Exercise 4
Task 1
Distributed System
A distributed system is a collection of autonomous computing elements that appears to its
users as a single coherent system.
A distributed system is a collection of computing elements each being able to behave
independently of the other. A computing element, which we will generally refer to as a
node, can be either a hardware device or a software process.
A second feature is that users (be they people or applications) believe they are dealing with
a single system.
This means that one way or another the autonomous nodes need to collaborate.
Establishing this collaboration lies at the heart of developing distributed systems.
Nodes are programmed to achieve common goals, which are realized by exchanging
messages with each other.
A node reacts to incoming messages, which are then processed and, in turn, lead to further
communication through message passing.
An important observation is that, because of dealing with independent nodes, each one will
have its notion of time. We cannot always assume that there is something like a global clock.
Practice shows that a distributed system is often organized as an overlay network [Tarkoma,
2010].
A node is typically a software process equipped with a list of other processes it can directly
send messages to. It may also be the case that a neighbour needs to be first looked up to.
Message passing is then done through TCP/IP or UDP channels.
Each node contains a list of the node to whom to connect directly
As mentioned, a distributed system should appear as a single coherent system. Achieving a
single-system view is often asking too much, for which reason, in our definition of a
distributed system, we have opted for something weaker, namely that it appears to be
coherent.
Offering a single coherent view is often challenging enough.
Data is stored should be of no concern, and neither should it matter that the system may be
replicating data to enhance performance. This so-called distribution transparency.
Unix-like operating systems in which resources are accessed through a unifying file-system
interface, effectively hiding the differences between files, storage devices, and main
memory, but also networks.
To assist the development of distributed applications, distributed systems are often
organized to have a separate layer of software that is logically placed on top of the
respective operating systems of the computers that are part of the system.
Each application is offered the same interface. The distributed system provides the means
for components of a single distributed application to communicate with each other, but also
to let different applications communicate.
In a sense, middleware is the same to a distributed system as what an operating system is to
a computer: a manager of resources offering its applications to efficiently share and deploy
those resources across a network. Next to resource management, it offers services that can
also be found in most operating systems
The main difference with their operating-system equivalents is that middleware services are
offered in a networked environment.
Middleware can also be viewed as a container of commonly used components and functions
that now no longer must be implemented by applications separately.
pf3
pf4
pf5

Unvollständige Textvorschau

Nur auf Docsity: Lade Distributed Systems: Concepts, Design Goals, and Scaling Techniques und mehr Zusammenfassungen als PDF für Computer Vision herunter!

Exercise 4

Task 1

❖ Distributed System

➢ A distributed system is a collection of autonomous computing elements that appears to its users as a single coherent system. ➢ A distributed system is a collection of computing elements each being able to behave independently of the other. A computing element, which we will generally refer to as a node, can be either a hardware device or a software process. ➢ A second feature is that users (be they people or applications) believe they are dealing with a single system. ➢ This means that one way or another the autonomous nodes need to collaborate. Establishing this collaboration lies at the heart of developing distributed systems. ➢ Nodes are programmed to achieve common goals, which are realized by exchanging messages with each other. ➢ A node reacts to incoming messages, which are then processed and, in turn, lead to further communication through message passing. ➢ An important observation is that, because of dealing with independent nodes, each one will have its notion of time. We cannot always assume that there is something like a global clock. ➢ Practice shows that a distributed system is often organized as an overlay network [Tarkoma, 2010]. ➢ A node is typically a software process equipped with a list of other processes it can directly send messages to. It may also be the case that a neighbour needs to be first looked up to. Message passing is then done through TCP/IP or UDP channels. ➢ Each node contains a list of the node to whom to connect directly ➢ As mentioned, a distributed system should appear as a single coherent system. Achieving a single-system view is often asking too much, for which reason, in our definition of a distributed system, we have opted for something weaker, namely that it appears to be coherent. ➢ Offering a single coherent view is often challenging enough. ➢ Data is stored should be of no concern, and neither should it matter that the system may be replicating data to enhance performance. This so-called distribution transparency. ➢ Unix-like operating systems in which resources are accessed through a unifying file-system interface, effectively hiding the differences between files, storage devices, and main memory, but also networks. ➢ To assist the development of distributed applications, distributed systems are often organized to have a separate layer of software that is logically placed on top of the respective operating systems of the computers that are part of the system. ➢ Each application is offered the same interface. The distributed system provides the means for components of a single distributed application to communicate with each other, but also to let different applications communicate. ➢ In a sense, middleware is the same to a distributed system as what an operating system is to a computer: a manager of resources offering its applications to efficiently share and deploy those resources across a network. Next to resource management, it offers services that can also be found in most operating systems ➢ The main difference with their operating-system equivalents is that middleware services are offered in a networked environment. ➢ Middleware can also be viewed as a container of commonly used components and functions that now no longer must be implemented by applications separately.

➢ Communication: ▪ A common communication service is the so-called Remote Procedure Call (RPC). Allows an application to invoke a function that is implemented and executed on a remote computer as if it was locally available. ➢ Transactions: ▪ Many applications make use of multiple services that are distributed among several computers. Middleware generally offers special support for executing such services in an all-or-nothing fashion, commonly referred to as an atomic transaction.

❖ Design goals

➢ Resources can be virtually anything, but typical examples include peripherals, storage facilities, data, files, services, and networks, to name just a few. There are many reasons for wanting to share resources. One obvious reason is that of economics. For example, it is cheaper to have a single high-end reliable storage facility be shared than having to buy and maintain storage for each user separately. ➢ Connecting users and resources also makes it easier to collaborate and exchange information, as is illustrated by the success of the Internet with its simple protocols for exchanging files, mail, documents, audio, and video. ➢ An important goal of a distributed system is to hide the fact that its processes and resources are physically distributed across multiple computers possibly separated by large distances. In other words, it tries to make the distribution of processes and resources transparent, that is, invisible, to end-users and applications. ➢ Access transparency deals with hiding differences in data representation and the way that objects can be accessed. At a basic level, we want to hide differences in machine architectures, but more important is that we reach an agreement on how data is to be represented by different machines and operating systems. ➢ Location transparency refers to the fact that users cannot tell where an object is physically located in the system. ➢ Naming plays an important role in achieving location transparency. In particular, location transparency can often be achieved by assigning only logical names to resources, that is, names in which the location of a resource is not secretly encoded. ➢ Where relocation transparency refers to being moved by the distributed system, migration transparency is offered by a distributed system when it supports the mobility of processes and resources initiated by users, without affecting ongoing communication and operations. ➢ Replication transparency deals with hiding the fact that several copies of a resource exist, or that several processes are operating in some form of lockstep mode so that one can take over when another fails. To hide replication from users, all replicas must have the same name. ➢ A distributed system must provide failure transparency. This means that a user or application does not notice that some piece of the system fails to work properly and that the system subsequently (and automatically) recovers from that failure. Masking failures is one of the hardest issues in distributed systems and is even impossible when certain realistic assumptions are made. ➢ Distribution transparency is generally considered preferable for any distributed system, there are situations in which attempting to blindly hide all distribution aspects from users is not a good idea. ➢ There is also a trade-off between a high degree of transparency and the performance of a system. ➢ An open distributed system is essentially a system that offers components that can easily be used by or integrated into other systems. ➢ A collection of relatively small and easily replaceable or adaptable components.

▪ There is one serious drawback to caching and replication that may adversely affect scalability. Because we now have multiple copies of a resource, modifying one copy makes that copy different from the others. Consequently, caching and replication lead to consistency problems. ➢ Discussion. ▪ When considering these scaling techniques, one could argue that size scalability is the least problematic from a technical point of view. In many cases, increasing the capacity of a machine will save the day, although perhaps there is a high monetary cost to pay. Geographical scalability is a much tougher problem as network latencies are naturally bound from below. Therefore, we may be forced to copy data to locations close to where clients are, leading to problems of maintaining copies consistent. ➢ Pitfalls ▪ Distributed systems differ from traditional software because components are dispersed across a network. Not taking this dispersion into account during design time is what makes so many systems needlessly complex and results in flaws that need to be patched later. ▪ Following false assumptions that many make when developing a distributed application for the first time:

  • The network is reliable
  • The network is secure
  • The network is homogeneous
  • The topology does not change
  • Latency is zero
  • Bandwidth is infinite
  • Transport cost is zero
  • There is one administrator Task 2

❖ Various transparency properties are given in fig.1.2, p.8. Please discuss each of these

properties by giving examples.

❖ Give examples of applications (not the same as in the book, of course!) that benefit

(and how) from scaling

**- CPU

  • I / O
  • Network** ➢ Access - Hide differences in data representation and how an object is accessed ▪ Regardless of how resource access and representation have to be performed on each computing entity, the users of a distributed system should always access resources in a single, uniform way. Example: SQL Queries ➢ Location - Hide where an object is located ▪ The user can query on any system(s) or fragment(s) of a system as if they were stored locally on the user’s site. The fact that the system or its fragments are stored at the remote site in the distributed database system, should be completely oblivious to the end-user. ➢ Relocation - Hide that an object may be moved to another location while in use ▪ A resource move while in use, should not be noticeable to the end-user. ➢ Migration - Hide that an object may move to another location

▪ Users should not be aware of whether a resource or computing entity possesses the ability to move to a different physical or logical location. ➢ Replication - Hide that an object is replicated ▪ Whenever a user updates a data item, the update is reflected in all the copies of the data. However, this operation should not be known to the user. This is concurrency transparency. ➢ Concurrency - Hide that an object may be shared by several independent users ▪ Whenever a user updates a data item, the update is reflected in all the copies of the data. However, this operation should not be known to the user. This is concurrency transparency. ➢ Failure - Hide the failure and recovery of an object ▪ In the event of an individual node failure, the system should ensure that the failed nodes' operations continue at other sites. This feature is referred to as Failure Transparency. ❖ Benefits from CPU Scaling ➢ You have a web application that currently runs on two instances, and you want the CPU utilization of the Auto Scaling group to stay at around 50 per cent when the load on the application changes. This gives you extra capacity to handle traffic spikes without maintaining an excessive number of idle resources. ❖ Benefits from Network Scaling ➢ As a business grows, so do its networking requirements. Businesses rely on the network infrastructure to provide mission-critical services. Network outages can result in lost revenue and lost customers. Network designers must design and build an enterprise network that is scalable and highly available. ❖ Benefits from I/O Scaling ➢ This means the rate at which a process progresses is limited by the speed of the I/O subsystem. A task that processes data from a disk, for example, counting the number of lines in a file is likely to be I/O bound.