CS575 Parallel Processing
Lecture three: Interconnection Networks
Wim Bohm, CSU
Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 license.
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Dynamic Interconnection Network in the Parallel Processing | CS 575, Study notes of Computer Science
Material Type: Notes; Professor: Bohm; Class: Parallel Processing; Subject: Computer Science; University: Colorado State University; Term: Unknown 1989;
Typology: Study notes
Pre 2010
1 / 21
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Dynamic Interconnection Network in the Parallel Processing | CS 575 and more Study notes Computer Science in PDF only on Docsity!
CS575 Parallel Processing^ Lecture three: Interconnection NetworksWim Bohm, CSU Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 license.
Interconnection networks^ ^ Connect^ ^ Processors, memories, I/O devices^ ^ Dynamic interconnection networks^ ^ Connect any to any using switches or busses^ CS575^ lecture 3
2
^ Connect any to any using switches or busses ^ Two types of switches^ ^ On / off: 1 input, 1 output^ ^ Pass through / cross over: 2 inputs, 2 outputs Static interconnection networks ^ Connect point to point using “wires”
Bus^ ^ Connects processors, memories, I/O devices^ ^ Master: can issue a request to get the bus^ ^ Slave: can respond to a request, one bus is granted^ If there are multiple masters, we nee an arbiter^ CS575^ lecture 3
4
^ If there are multiple masters, we nee an arbiter Sequential ^ Only one communication at the time ^ Bottleneck Cache coherence problems ^ Memory and cache can get out of sync
Crossbar vs bus^ ^ Crossbar^ ^ Scalable in performance^ ^ Not scalable in hardware complexity^ CS575^ lecture 3
5
^ Bus^ ^ Not scalable in performance^ ^ Scalable in hardware complexity ^ Compromise: multistage network
Static Interconnection Networks^ ^ Fixed wires (channels) between devices^ ^ Many topologies^ ^ Completely connected^ CS575^ lecture 3
Completely connected^ ^ (n(n-1))/2 channels^ ^ Static counterpart of crossbar^7
^ Star^ ^ One central PE for message passing^ ^ Static counterpart of bus ^ Multistage network with PE at each switch
More topologies^ ^ Necklace or ring^ ^ Mesh^ ^ 2D^ ^ 3D^ CS575^ lecture 3
8
^ 3D Trees ^ Fat tree Hypercuben^ ^2 nodes in nD hypercube ^ n links per node in nD hypercube ^ Addressing: 1 bit per dimension
Measures of static networks^ ^ Diameter^ ^ Maximal shortest path between two nodes^ ^ Ring:^ p/2,^ hypercube: log(p)^ 2D wraparound mesh: 2^
sqrt(p)/2 2D wraparound mesh: 2 sqrt(p)/2 CS575 lecture 3^10
^ Connectivity^ ^ Measure of multiplicity of paths between nodes^ ^ Arc connectivity^ ^ Minimum #arcs to be removed to create two disconnectednetworks^ ^ Ring: 2, hypercube: log(p), mesh: 2, wraparound mesh: 4
More measures^ ^ Bisection width^ ^ Minimal #arcs to be removed to partition the network in two equalhalves^ ^ Ring: 2, complete binary tree: 1,
2D mesh: sqrt(p) Question: bisection width of a hypercube? Question: bisection width of a hypercube? CS575 lecture 3^11
^ Channel width^ ^ #bits communicated simultaneously over channel ^ Channel rate / bandwidth^ ^ Peak communication rate (#bits/second) ^ Bisection bandwidth^ ^ Bisection width * channel bandwidth
Meshes and Hyper cubes^ ^ Mesh^ ^ Buildable, cheaper than hyper cubes^ ^ Many applications map naturally^ Cut through works well in meshes^ CS575^ lecture 3
13
^ Cut through works well in meshes ^ Many commercial systems based on it Hyper cube ^ Recursive structure nice for algorithm design ^ Often same O complexity as PRAMs ^ Often hypercube algorithm also good for othertopologies, so good starting point
Embedding^ ^ Relationship between two networks^ ^ Studied by mapping one into the other^ ^ Why? Discuss.^ G(V,E)^ ^ G’(V’,E’)^ CS575^ lecture 3
14
^ G(V,E)^ ^ G’(V’,E’)^ ^ graph G, G’, vertices V, V’, edges E, E’^ ^ Map E^ E’, V^ ^ V’ ^ congestion^ of k: k (>1) e-s to one e’ ^ dilation^ of k: 1 e to k e’-s ^ expansion: |V| / |V’| ^ Often we want^ congestion=dilation=expansion=
ring to hypercube cont’ 0 00 000 G(0,1) = 0
i^ →G(i,dim)
1 01 001 G(1,1) = 1^11 011 CS^
11 011 x 10 010 G(i,x+1) = 0||G(i,x)^ i<2 x+1x^110 = 1||G(2^ -i-1,x) i>=2^111 (|| implies concatenation)^101100 lecture 3^16
2D Mesh into hypercube^ ^ Note 2D Mesh^ ^ Rows: rings^ ^ Cols:^ rings^ CS575^ lecture 3
Cols:^ rings^17
r^ s^ 2 * 2wraparound mesh into 2
r+s^ cube
^ Map node(i,j) onto node G(i,r)||G(j,s) ^ Row coincides with sub cube ^ Column coincides with sub cube ^ S.t. if adjacent in mesh then adjacent in cube
Routing Mechanisms^ ^ Determine all source^
destination paths
^ Minimal: a shortest path ^ Deterministic: one path per (src,dst) pair^ CS575^ lecture 3
19