Sparse Matrices: Storage and Computation Efficiency through Sparse Representation, Essays (university) of Digital Image Processing

Download Sparse Matrices: Storage and Computation Efficiency through Sparse Representation and more Essays (university) Digital Image Processing in PDF only on Docsity!

SPARSE

MATRICES

Sparse Matrices :

• (^) Special class of matrices which have majority of their elements equal to ZERO.
• (^) This Property allows MATLAB to

 STORAGE :

store only non zero elements of the matrix, together with their indices, this implies

that less memory can be used

COMPUTING TIME :

By eliminating operations on ZERO elements, computation time can be reduced.

Logically, designing an algorithm that computes only on non-zero elements

• (^) Storing non-zero elements with triples : Row , Column , Value
• (^) Sparse matrices representation can be done in 2 ways  (^) Array representation  (^) Linked List representation

Array representation :

• (^) 2 D array is used to represent a sparse matrices as 3 rows o (^) Row – index value of row , where non-zero element is located o (^) Column – index value of column , where non-zero element is located o (^) Value – index value of non-zero element is located where row & column index

Linked List representation

• (^) There are three structures required in Linked List representation
  1. Head node
  2. Row node
  3. Column node Total row number Total column number Total non zero value pointer row number pointer for next row Pointer for next column column number value Pointer for next number in same row

• (^) Example
• c0 c1 c2 c r r r r 4 4 6 0

Row node Column node

0 1 1 1 2 0 2 3 1 3 X 1 1 2 X 0 4 2 2 3 X (^1 1 33 1) X 1 2 X

Head node

X- represents no non zero element

Categorization of Sparse representation techniques :

• (^) w.r.t different viewpoints
• (^) Different methods with motivations, ideas and concerns
• (^) Variety of strategies

Based on

atoms/samples

NAÏVE Based

DICTIONARY

LEARNING

Based on

availability of

labels of atoms

Supervised

learning

Semi-

supervised

Unsupervised

learning

Sparse constraints

Structured

constraint

Space constraint

1) l 0-norm minimization 2) lp -norm (0 < p < 1) minimization;

3. l 1 -norm minimization
4. l 2,1-norm minimization
5. sparse representation with l 2 -norm minimization Representation of results in sparse representation can be dominated by optimizer Different norm minimizations : Framework :

• (^) Exploit the linear representation of samples to represent the required solution
• (^) Representation coefficients of these samples
• (^) Utilize the solution to reconstruct the desired results

Generating Signals in Sparseland

K M N A fixed Dictionary

• (^) Every column in D (dictionary) is a prototype signal (Atom).
• The vector  is generated randomly with few non-zeros in random locations and random values. A sparse & random vector N D  α x

Transforms in Sparse:

• (^) Assume that x is known to emerge from M
• (^) We desire simplicity, independence, and expressiveness.
• (^) How about “Given x, find the α that generated it in M ”?

Known In Order to Transform : We need to solve an under-determined linear system of equations:

• (^) We will measure sparsity using the L 0 norm , L 1 norm, L 2 norm
• (^) Among all (infinitely many) possible solutions we want the sparsest !! α  D α  x 0 α

M Signal’s Transform in Sparseland Multiply by D A sparse & random vector

• (^) How effective are those ways?
  • (^) How would we get D?
• (^) Are there practical ways to get?

α

s.t. x D

Min

0 αˆ x  D α α  ˆα^ ^ α

General Formalisms

• (^) L 0 minimization
• (^) L 0 constrained optimization
• (^) L 1 minimization
• (^) L 1 constrained optimization Sparse and Overcomplete Representations 20    st X  D Min .. 0    st X  D Min .. 1 st C Min X   0 2 2 ..    D st C Min X   1 2 2 ..    D