Compressive Sensing - Matrix Computation - Lecture Slides, Slides of Advanced Computer Architecture

Download Compressive Sensing - Matrix Computation - Lecture Slides and more Slides Advanced Computer Architecture in PDF only on Docsity!

• Lecture

Overview

Compressive sensing Applications

Compressible signals

Consider a real-valued finite-length one-dimensional, discrete-time signal x ∈ IRN Any signal in IRN^ can be represented in terms of basis N × 1 vectors {ψi }Ni= For simplicity, assume orthonormal basis, and x is represented as

x =

∑^ N

i=

si ψi , or x = Ψs (1)

where s is the N × 1 vector of weighted coefficients, si = 〈x, ψi 〉 = ψ> i x The signal x is K -sparse if it is a linear combination of only K basis vectors, i.e., , only K of the si coefficients are nonzero and N − K are zero (of great interest when K  N) The signal is compressible if (1) has only a few large coefficients and many small coefficients

Transform coding

The fact that compressible signals are well approximated by K -sparse representations forms the foundation of transform coding In data acquisition (e.g., digital cameras) transform coding plays a central role: (^1) full N-sample signal x is acquired (^2) complete set of transform coefficients {si } is computed via s = Ψ>x (^3) K largest coefficients are located and the N − K smallest coefficients are discarded (^4) K values and locations of the largest coefficients re encoded JPEG: exploits sparse representation based on discrete cosine transform (DCT) JPEG 2000: exploits sparse representation based on discrete wavelet transform (DWT)

Compressive sensing: Two principles

Sparsity: I (^) information rate of a continuous time signal may be much smaller than suggested by its bandwidth I (^) discrete time signal depends on a number of degree of freedom which is much smaller than its length I (^) many natural signals are sparse or compressible Incoherence: I (^) extends the duality between time and frequency domains I (^) objects having a sparse representation in Ψ must be spread out in the domain in which they are sampled, just as a Dirac or spike in the time domain is spread out in the frequency domain I (^) unlike the signal of interest, the sampling/sensing waveforms have a extremely dense representation in Ψ

Compressive sensing

Consider a general linear measurement process that computes M < N inner products between x and a collection of vectors {φj }Mj=1 as in yj = 〈x, φj 〉 Arrange the measurements yj in an M × 1 vector y and the measurement vectors φ> j as rows in an M × N matrix Φ, we have

y = Φx = ΦΨs = Θs

where Θ = ΦΨ is an M × N matrix Nonadaptive measurement process, i.e., Φ is fixed and does not depend on x I (^) need to find a stable measurement matrix Φ such that the salient information in any K -sparse or compressible signals is not damaged by dimensionality reduction from x ∈ IRN^ to y ∈ IRM^ (M < N) I (^) a reconstruction algorithm to recover x from only M ≈ K measurements y

Designing a stable measurement matrix (cont’d)

Restricted isometry property (RIP): A sufficient condition for a stable solution for both K -sparse and compressible signals is that Θ satisfies (2) for an arbitrary 3K -sparse vector v Incoherence: A related condition that requires the row {φi } of Φ cannot sparsely represent the columns {φi } of Ψ, and vice versa Direct construction of Φ such that Θ = ΦΨ has the RIP requires verifying (2) for each of the

(N

K

possible combinations However, both the RIP and incoherence can be achieved with high probability simply by selecting Φ as a random matrix For example, let the matrix elements φj,i be independently and identically distributed (iid) random variables form a Gaussian probability density function with zero mean and variance 1/N Then the measurements y are merely M different randomly weighted linear combinations of the elements of x

Gaussian random measurements

The measurement matrix Φ is incoherent with the basis Ψ = I of data spikes with high probability More specifically, an M × N iid Gaussian matrix Θ = ΦI = Φ can be shown to have the RIP with high probability if M ≥ c K log(N/K ) with c a small constant Thus, K -sparse and compressible signals of length N can be recovered from only M ≥ c K log(N/K )  N random Gaussian measurements The matrix Φ is universal in the sense that Θ = ΦΨ will be iid Gaussian and thus have the RIP with high probability regardless of choice of orthonormal basis Ψ

Optimization problems

Minimum ` 2 -norm reconstruction ˆs = arg min ‖s′‖ 2

Θs′^ = y

which can be solved with closed form solution, ˆs = Θ>(ΘΘ>)−^1 y, but almost never find a K -sparse solution Minimum ` 0 -norm reconstruction ˆs = arg min ‖s′‖ 0

Θs′^ = y which can recover a K -sparse signals with only M = K + 1 iid Gaussian measurements, but it is both numerically unstable and NP-complete Minimum ` 1 -norm reconstruction ˆs = arg min ‖s′‖ 1 Θs′^ = y which can recover K -sparse signals and closely approximate compressible signals with high probability using only M ≤ c K log(N/K ) iid Gaussian measurements via convex optimization

Applications

Single pixel, compressive digital camera that directly acquires M random linear measurements without first collecting the N pixel values Use digital micromirror device consisting of an array of N tiny mirrors where each one can be independently oriented To collect measurements, a random number generator sets the mirror orientations in a pseudorandom pattern to create the measurement φj and the voltage at the photodiode equals yj , the inner product between φj and x The process repeats M times to obtain y

[Baraniuk 07]