Download Matrix Operations on GPUs: Inner Products and Matrix-Matrix Multiplication and more Study notes Computer Science in PDF only on Docsity!
CIS 700/010: Matrix Operations I
Suresh Venkatasubramanian
Scribed by Kennedy Behrman
March 3, 2005
1 Matrix operations
There are three basic matrix operations that would be part of any GPU matrix toolkit:
- The inner product of two vectors c = a · b.
- Matrix-vector operations: y = Ax.
- Matrix-Matrix operations: C = A + B, D = AB, E = A−^1.
A number of problems can be solved once one has these basic operations (especially in physical simulations). This is one of the most studied problems on the GPU.
2 The Inner Product
Consider the inner product c = a · b, which we rewrite as c =
∑n i=1 aibi.
2.1 Technique 1: Small memory
Each vector is stored in a 1D texture. In the ith^ rendering pass, we render a single point at coordinates (0,0), having a single texture coordinate i. The fragment program uses i to index into the two textures and returns the value s + ai + bi, where s is the running sum maintained over the previous i − 1
passes. Note that since we cannot read and write the location where s is stored in a single pass, we use a ping-pong trick to maintain s. This procedure takes n passes, and requires only a fixed number of texture locations (excluding the storage for a and b).
2.2 Technique 2: Fewer passes
The second technique uses more working memory (n units), but requires fewer passes. We write a and b as 2D textures (2D textures allow for more storage, since the dimension of a texture is typically bounded, and are better optimized by the rasterizer). We now multiply the contents of the textures, storing the result in a third texture c. This can be done with a simple fragment program that takes the fragment coordinates and looks up the a and b textures, returning their product. We render a single quad in order to activate the fragment program.
c =
a 0 b 0 a 1 b 1 ... ... aibi ... ... ... anbn
Finally, all the numbers in c must be summed together. This can be done in log n passes, using a standard reduce operation. This procedure takes only log n passes, but requires 3n units of texture memory.
3 Matrix-Matrix operations
We can store matrices as 2D textures. Addition is trivial.
3.1 Multiplication
3.2 Technique 1: The Basic Approach
”Fast matrix multiplies using graphics hardware” by Larsen and McAllister”[2] Express multiplication of two matrices as dot product of vectors of matrix rows and columns. That is to compute some cell cij of matrix C, we take the dot product of row i of matrix A with column j of matrix B:
output ← c + ax 2 ∗ b 2 y pass k output ← c + axk ∗ bky
Cxy = Σnk=1axkbky 1)uses n passes 2)space N = n^2
3.3 Technique 2: A Speedup
”Dense Matrix Multiplication” by ´Ad´am Morav´anszky To make it faster: Instead of making one computation per pass, compute multiple additions per pass in fragment program:
Pass 1 becomes: output ← ax 1 b 1 y + ax 2 b 2 y + ax 3 b 3 y + ax 4 b 4 y Must consider that there is a tradeoff between the length of fragment program vs. the number of passes.
3.4 Technique 3: Using All Channels
”Cache and Bandwidth Aware Matrix Multiplication on the GPU”, by Hall, Carr and Hart”[1].
[ A 11 A 12 A 21 A 22
] [
B 11 B 12
B 21 B 22
]
We have been using only the red component, propose storing across all colors:
[
A 11 R A 12 G
A 21 B A 22 A
] [
B 11 R B 12 G
B 21 B B 22 A
]
[
A 11 B 11 + A 12 B 12 A 11 B 21 + A 12 B 22
A 21 B 11 + A 22 B 12 A 22 B 21 + A 22 B 22
]
divide into 2:
[
A 11 B 11 + A 12 B 12
A 21 B 11 + A 22 B 12
]
[
A 22 B 21 + A 22 B 22
A 11 B 21 + A 12 B 22
]
ArrbbBrgrg + AggaaBbaba Basically a swizzle operation to speed things up. Closing thought: The basic idea here is using the inner product calculation in parellel.
References
[1] Hall, J., Carr, N., and Hart, J. C. Cache and bandwidth aware matrix multiplication on the gpu. Tech. Rep. UIUCDCS-R-2003-2328-1, UIUC, 2003.
[2] Larsen, E., and McAllister, D. Fast matrix multiplies using graph- ics hardware. In Supercomputing ’01: Proceedings of the 2001 ACM/IEEE conference on Supercomputing (CDROM) (2001), ACM Press, pp. 55–55.
