Download Interconnection Networks for Parallel Computers-Parallel Processing-Lecture Slides and more Slides Parallel Computing and Programming in PDF only on Docsity!
Interconnection Networks
for Parallel Computers
-^
Interconnection networks carry data betweenprocessors and to memory.
-^
Interconnects are made of switches and links (wires,fiber).
-^
Interconnects are classified as Shared Busses, Staticor Dynamic.
-^
Static networks consist of point-to-pointcommunication links among processing nodes andare also referred to as
direct
networks.
-^
Dynamic networks are built using switches andcommunication links. Dynamic networks are alsoreferred to as
indirect
networks.
docsity.com
Static and Dynamic
Interconnection Networks
Static network
Indirect network
Switching element
Processing node
Network interface/switch
P^ P
P^
P
P
P P
P
Classification of interconnection networks: (a) a static
network; and (b) a dynamic network.
docsity.com
Interconnection Networks:
Network Interfaces
-^
Processors talk to the network via a networkinterface.
-^
The network interface may hang off the I/O bus or thememory bus.
-^
In a physical sense, this distinguishes a cluster froma tightly coupled multicomputer.
-^
The relative speeds of the I/O and memory busesimpact the performance of the network.
docsity.com
Network Topologies
- A variety of network topologies have
been proposed and implemented.
- These topologies tradeoff
performance for cost.
- Commercial machines often
implement hybrids of multipletopologies for reasons of packaging,cost, and available components.
docsity.com
Network Topologies: Buses
Cache /
Local Memory
Cache /
Local Memory
Shared Memory
Data
Processor 0
Address
Data
Shared Memory
Processor 0
Processor 1
(a) (b) Address
Processor 1
Bus-based interconnects (a) with no local caches; (b) with local memory/caches.
Since much of the data accessed by processors is local to the processor, a
local memory can improve the performance of bus-based machines. Example??
docsity.com
Network Topologies: Crossbars
A crossbar network uses an
p×b
grid of switches to
connect
p
inputs to b outputs in a non-blocking manner.
Memory Banks
b−
5 4 3 2 1 0
Processing Elements
(^0123456) p−
element A switching
docsity.com
Static Network Topologies
- Completely Connected• Star• Tree• Linear Array & Ring• 2-D mesh & Torus• Hypercube• k-d mesh/cube
docsity.com
Network Topologies:
Completely Connected Network
- Each processor is connected to every
other processor.
- The number of links in the network scales
as
O(p
- While the performance scales very well,
the hardware complexity is not realizablefor large values of
p
- In this sense, these networks are static
counterparts of crossbars.
docsity.com
Network Topologies:
Star Connected Network
- Every node is connected only to a
common node at the center.
- Distance between any pair of nodes is 2.
However, the central node becomes abottleneck.
- In this sense, star connected networks
are static counterparts of buses.
docsity.com
Network Topologies:
Linear Arrays
Linear arrays: (a) with no wraparound links; (b) with
wraparound link.
(a)
(b)
docsity.com
Network Topologies:Tree-Based Networks
Complete binary tree networks: (a) static tree network (b)dynamic tree network
(a)
(b)
Processing nodesSwitching nodes
docsity.com
Network Topologies: Tree Properties
2 [(logp+1)
-1] for p node
complete binary tree.
- Links higher up the tree potentially carry
more traffic than those at the lower levels.
- For this reason, a variant called a fat-tree,
fattens the links as we go up the tree.
- Trees can be laid out in 2D with no wire
crossings. This is an attractive property oftrees.
docsity.com
Network Topologies:
Hypercubes
Construction of hypercubes from hypercubes of lower dimension.
0 1
00 01
10 11
000
010
001
011
100
110 111
101
0000
0100
0001
0011
0101
0110
0010
0111
1100
1110 1111
1011
1001
1000
1101
1010
0-D hypercube
1-D hypercube
2-D hypercube
3-D hypercube
4-D hypercube
docsity.com
Network Topologies:
Properties of Hypercubes
- The distance between any two nodes is
at most
log p
log p
neighbors.
- The distance between two nodes is
given by the number of bit positions atwhich the two nodes differ.
- Routing algorithm for Hypercube.
docsity.com
