Matrix Norm - Matrix Computation - Lecture Slides, Slides of Advanced Computer Architecture

Download Matrix Norm - Matrix Computation - Lecture Slides and more Slides Advanced Computer Architecture in PDF only on Docsity!

• Lecture

Overview

Basic definition: matrix norm, range, rank, null space, matrix inverse Elementary analytical ad topological properties

Matrix norm (cont’d)

p-norms: ‖A‖p = sup x 6 = 0

‖Ax‖p ‖x‖p Note that matrix p-norms are defined in terms of vector p-norms. It is clear that ‖A‖p is the p-norm of the largest vector obtained by applying A to a unit p-norm vector

‖A‖p = sup x 6 = 0

∥ A

x ‖x‖p

p

= max ‖x‖p =

‖Ax‖p

When A is non-singular,

min ‖x‖p =

‖Ax‖p =

‖A−^1 ‖p Frobenius norm and p-norms define families norms that ‖AB‖p ≤ ‖A‖p ‖B‖p A ∈ IRm×n, B ∈ IRn×q For every A ∈ IRm×n^ and x ∈ IRn, we have ‖Ax‖p ≤ ‖A‖p ‖x‖p.

Matrix norm (cont’d)

More generally, for any vector norm ‖ · ‖α on IRn^ and ‖ · ‖β on IRm, we have ‖Ax‖β ≤ ‖A‖α,β ‖x‖α where ‖A‖α,β is a matrix norm defined by

‖A‖α,β = sup x 6 = 0

‖Ax‖β ‖x‖α

We say that ‖ · ‖α,β is subordinate to the vector norms ‖ · ‖α and ‖ · ‖β. Since the set {x ∈ IRn^ : ‖x‖α = 1} is compact and ‖ · ‖β is continuous, it follows that

‖A‖α,β = max ‖x‖α=

‖Ax‖β = ‖Ax∗‖β

for some x∗^ ∈ IRn^ having unit α-norm.

Matrix 2-norm

Theorem

If A ∈ IRm×n, then there exists a unit 2-norm z ∈ IRn^ such that A>Az = μ^2 z where μ = ‖A‖ 2.

Proof.

Suppose z ∈ Rn^ is a unit vector such that ‖Az‖ 2 = ‖A‖ 2. Since z maximizes the function

g (x) =

‖Ax‖^22 ‖x‖^22

x>A>Ax x>x

, it follows that with by setting gradient ∇g (z) = 0 ,

∂g (z) ∂zi

[

(z>z)

∑n j=1(A

A)ij zj − (z>A>Az)zi^ ]^ /(z>z)^2

In vector notation, A>Az = (z>A>Az)z. The theorem follows by setting μ = ‖Az‖ 2.

Matrix 2-norm (cont’d)

It implies that ‖A‖^22 is a zero of the polynomial p(λ) = det(A>A − λI ), i.e.,

‖A‖ 2 = max ‖x‖ 2 = ‖Ax‖ 2 =

λmax

where λmax is the largest eigenvalue. ‖A‖ 2 is the square root of the largest eigenvalue of A>A. When A is non-singular

‖A−^1 ‖ 2 =

min‖x‖ 2 =1 ‖Ax‖ 2

λmin

where λmin is the smallest eigenvalue of A>A.

Matrix 2-norm (cont’d)

Computation of matrix 2-norm is iterative and more complicated than that of the matrix 1-norm or ∞-norm. The order of magnitude of ‖A‖ 2 can be computed easily.

Corollary

If A ∈ IRm×n, then ‖A‖ 2 ≤

‖A‖ 1 ‖A‖∞

Proof.

If z 6 = 0 is such that A>Az = μ^2 z with μ = ‖A‖ 2 , then μ^2 ‖z‖ 1 = ‖A>Az‖ 1 ≤ ‖A>‖ 1 ‖A‖ 1 ‖z‖ 1 = ‖A‖∞‖A‖ 1 ‖z‖ 1

Range, null space, and rank

Range: The range of A ∈ IRm×n^ is

ran(A) = {y ∈ IRm^ : y = Ax, x ∈ IRn}

Null space: null(A) = {x ∈ IRn^ : Ax = 0 } If A = [a 1 ,... , an], then

ran(A) = span{a 1 ,... , an}

Rank: the number of linear independent columns of A.

rank(A) = dim(ran(A))

A is rank deficient if rank(A) < min{m, n}. If A ∈ IRm×n, then

dim(null(A)) + rank(A) = n