Householder Transformation - Matrix Computation - Lecture Slides, Slides of Advanced Computer Architecture

Download Householder Transformation - Matrix Computation - Lecture Slides and more Slides Advanced Computer Architecture in PDF only on Docsity!

• Lecture

Overview

QR decomposition by Householder transformation

QR decomposition by Givens rotation

Geometry of elementary projectors

For u, x ∈ IR

n , s.t. ‖u‖ = 1

Orthogonal projectors onto span{u} and u

⊥ are

Pu =

uu

>

u>u

, and Pu⊥ = I −

uu

>

u>u

For u 6 = 0, the Householder transformation or the elementary reflector

about u

⊥ is

R = I − 2

uu

>

u>u

or

R = I − 2 uu

>

when ‖u‖ = 1, and

R = R

> = R

− 1

Triangularization by introducing zeros

The matrix Qk is chosen to introduce zeros below the diagonal in the

k-th column while preserving all the zeros previously introduced

      × × ×

× × ×

× × ×

× × ×

× × ×

Q 1

Q 2

× × ×

Q 3

× × ×

× ×

A Q 1 A Q 2 Q 1 A Q 3 Q 2 Q 1 A

Qk operates on row k,... , m (changed entries are denoted by

boldface or  and blank entries are zero)

At beginning of step k, there is a block of zeros in the first k − 1

columns of these rows

The application of Qk forms linear combinations of these rows, and

the linear combination of the zero entries remain zero

After n steps, all the entries below the diagonal have been eliminated

and Qn · · · Q 2 Q 1 A = R is upper triangular

Householder transformation (cont’d)

To introduce zeros into k-th column (x ∈ IR

m−k+ ), the Householder

transformation F should

x =

×

×

×

F

−→ F x =

‖x‖

0

. . .

0

= ‖x‖e 1 = αe 1

The reflector F will reflect the space IR

m−k+ across the hyperplane

H orthogonal to u = ‖x‖e 1 − x

A hyperplane is characterized by a vector u = ‖x‖e 1 − x

Householder transformation (cont’d)

Every point x ∈ IR

m is mapped to a mirror point

F x = (I − 2

uu

>

u>u

)x = x − 2 u(

u

> x

u>u

and hence

F = (I − 2

uu

>

u

> u

Will fix the +/- sign in the next slide

Householder QR factorization

Algorithm:

for k = 1 to n do

x = Ak:m,k

uk = sign(x 1 )‖x‖ 2 e 1 + x

uk =

uk ‖uk ‖ 2

Ak:m,k:n = (I − 2 uk u

> k )Ak:m,k:n

end for

Recall

Qk =

[

I 0

0 F

]

Upon completion, A has been reduced to upper triangular form, i.e.,

R in A = QR

Q

> = Qn · · · Q 2 Q 1 or Q = Q

> 1 Q

> 2 · · ·^ Q

> n

QR decomposition with Householder transformation

Want to compute QR decomposition A with Householder

transformation

A =

Need to find a reflector for first column of A, x = [12, 6 , −4]

> to

‖x‖e 1 = [14, 0 , 0]

>

u=‖x‖e 1 − x = [2, − 6 , 4]

> = 2[1, − 3 , 2]

>

F 1 =I − 2

uu> u>u

 , F

1 A^ =

Next need to zero out A 32 and apply the same process to

A

′

[

]

Givens rotations

Givens rotation: orthogonal transform to zero out elements selectively

G (i, k, θ) =

0 · · · c · · · s · · · 0

. . .

0 · · · −s · · · c · · · 0

. . .

i

k

i k

where c = cos(θ) and s = sin(θ) for some θ

Pre-multiply G (i, k, θ) amounts to a counterclockwise rotation θ in

the (i, k) coordinate plane, y = G (i, k, θ)x

yj =

cxi − sxk j = i

sxi + cxk j = k

xj j 6 = i, k

Givens rotations (cont’d)

Can zero out yk = sxi + cxk = 0 by setting

c =

xi √

x

2 i +^ x

2 k

, s =

−xk √

x

2 i +^ x

2 k

, θ = arctan(xk /xi )

QR decomposition can be computed by a series of Givens rotations

Each rotation zeros an element in the subdiagonal of the matrix,

forming R matrix, Q = G 1... Gn forms the orthogonal Q matrix

Useful for zero out few elements off diagonal (e.g., sparse matrix)

Example

A =

Want to zero out A 31 = −4 with rotation vector (6, −4) to point

along the x-axis, i.e., θ = arctan(− 4 /6)

Gram-Schmidt, Householder and Givens

Householder QR is numerically more stable

Gram-Schmidt computes orthonormal basis incrementally

Givens rotation is more useful for zero out few selective elements

Eigendecomposition

Also known as spectral decomposition

A is a square matrix

A = QDQ

− 1

where Q is a square matrix whose columns are eigenvector and D is a

diagonal matrix whose elements are the corresponding eigenvalues

With eigendecomposition

AQ = QD

Aqi = λi qi

where λi and qi are eigenvalues and eigenvectors of

Ax = λx

The eigenvectors are usually normalized but not necessarily

If A can be eigendecomposed with all non-zero eigenvalues

A

− 1 = QD

− 1 Q

− 1