Multiprocessors and Interconnection Networks in Parallel Computing, Slides of Operating Systems

An overview of multiprocessors and multicomputers, focusing on their architecture, interconnection networks, and communication methods. Topics include shared memory systems, remote procedure calls, processor allocation, and scheduling. Examples of multiprocessors and multicomputers are also provided.

Typology: Slides

2012/2013

Uploaded on 04/23/2013

ashakiran
ashakiran 🇮🇳

4.5

(27)

261 documents

1 / 104

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Communication Models for Parallel Computer
Architectures
Two distinct models have been proposed for how
CPUs in a parallel computer system should
communicate.
In the first model, all CPUs share a common physical
memory.
This kind of system is called a multiprocessor or shared memory
system.
In the second design, each CPU has its own private
memory.
Such a design is called a multicomputer or distributed memory
system.
Docsity.com
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16
pf17
pf18
pf19
pf1a
pf1b
pf1c
pf1d
pf1e
pf1f
pf20
pf21
pf22
pf23
pf24
pf25
pf26
pf27
pf28
pf29
pf2a
pf2b
pf2c
pf2d
pf2e
pf2f
pf30
pf31
pf32
pf33
pf34
pf35
pf36
pf37
pf38
pf39
pf3a
pf3b
pf3c
pf3d
pf3e
pf3f
pf40
pf41
pf42
pf43
pf44
pf45
pf46
pf47
pf48
pf49
pf4a
pf4b
pf4c
pf4d
pf4e
pf4f
pf50
pf51
pf52
pf53
pf54
pf55
pf56
pf57
pf58
pf59
pf5a
pf5b
pf5c
pf5d
pf5e
pf5f
pf60
pf61
pf62
pf63
pf64

Partial preview of the text

Download Multiprocessors and Interconnection Networks in Parallel Computing and more Slides Operating Systems in PDF only on Docsity!

Communication Models for Parallel Computer

Architectures

  • Two distinct models have been proposed for how

CPUs in a parallel computer system should

communicate.

  • In the first model, all CPUs share a common physical

memory.

  • This kind of system is called a multiprocessor or shared memory system.
  • In the second design, each CPU has its own private

memory.

  • Such a design is called a multicomputer or distributed memory system.

Multiprocessors

  • Consider a program to find all of the objects in a bit-map

image.

  • One copy of the image is kept in memory.
  • Each CPU runs a single process which inspects one section of the image.
  • Some objects occupy multiple sections, so it is essential that each process have access to the entire image.
  • Example multiprocessors include:
  • Sun Enterprise 1000
  • Sequent NUMA-Q
  • SGI Origin 2000
  • HP/Convex Exemplar

Multicomputers

  • In a multicomputer solving the same problem, each CPU

has a section of the image in its local memory.

  • If the CPUs need to follow an object across the border, they must request the information from a neighboring CPU. - This is done via message passing.
  • Programming multicomputers is more difficult than

programming multiprocessors, but they are more scalable.

  • Building a multicomputer with 10,000 CPUs is straightforward.

Multicomputers

Shared Memory

  • Each machine has its own virtual memory and its own page table.
  • When a CPU does a LOAD or STORE on a page it does not have, a trap to the OS occurs.
  • The OS locates the page and asks the CPU currently holding it to unmap the page and send it over the interconnection network.
  • When it arrives, the page is mapped in and the faulting instruction restarted.
  • A third possibility is to have a user-level runtime system

implement a form of shared memory.

Shared Memory

Interconnection Networks

• Multicomputers are held together by

interconnection networks which move packets

between CPUs and memory.

  • The CPUs and memory modules of multiprocessors are

also interconnected.

  • Interconnection networks consist of:
    • CPUs
    • Memory modules
    • Interfaces
    • Links
    • Switches

Interconnection Networks

– The links are the physical channels over which bits

move. They can be

  • electrical or optical fiber
  • serial or parallel
  • simplex, half-duplex, or full duplex

– The switches are devices with multiple input ports

and multiple output ports.

  • When a packet arrives at an input port on a switch

some bits are used to select the output port to which

the packet is sent.

Switching

  • An interconnection network consists of switches and wires

connecting them.

  • The following slide shows an example.
    • Each switch has four input ports and four output ports.
    • In addition each switch has some CPUs and interconnect circuitry.
    • The job of the switch is to accept packets arriving on any input port and send each one out on the correct output port.
    • Each output port is connected to an input port of another switch by a parallel or serial line.

Switching

Switching

Communication Methods

  • When a program is split up into pieces, the pieces

(processes) often need to communicate with one another.

  • This communication can be done in one of two ways:
    • shared variables
    • explicit message passing
    • Logical sharing of variables is possible even on a multicomputer.
    • Message passing is easy to implement on a multiprocessor by simply copying from the sender to the receiver.

Message Passing Modes

• Messaging systems can be either persistent or

transient

– Are messages retained when the senders and/or

receivers stop executing?

• Can also be either synchronous or

asynchronous

– Blocking vs. non-blocking

Persistent Communication

  • Persistent communication of letters back in the days of the

Pony Express.