1/33
The Secular Equation
My First Encounter with
Prof. Gene H. Golub (1932 – 2007)
Walter Gander, ETH and HKBU
International Workshop on Matrix Computations
Gene Golub Memorial Day 2018
Hangzhou
April 20 – 24, 2018
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Gene Golub's Encounters with the Secular Equation, Study notes of Computational Methods
This document recounts the story of Professor Gene Golub's encounter with the Secular Equation during his academic career. It includes his discovery of the minimal norm solution for rank deficient matrices and his work on least squares problems with quadratic constraints. The document also mentions his collaboration with other researchers and his influential papers on the topic.
Typology: Study notes
2021/2022
1 / 34
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Gene Golub's Encounters with the Secular Equation and more Study notes Computational Methods in PDF only on Docsity!
The Secular Equation
My First Encounter with Prof. Gene H. Golub (1932 – 2007)
Walter Gander, ETH and HKBU
International Workshop on Matrix Computations Gene Golub Memorial Day 2018 Hangzhou April 20 – 24, 2018
Least Squares with Rank Deficient Matrix
- Consider the least squares problem ‖Ax − b‖^2 = min with A ∈ Rm×n, m > n, rank(A) = r < n.
- Many solutions, want minimal norm solution xmin
- Today xmin is computed most conveniently by the SVD
- decompose A = U ΣV>^ with σ 1 ≥ σ 2 ≥ · · · ≥ σn ≥ 0
- determine rank r, σr 6 = 0, σk = 0, k = r + 1,... , n
- form Ur := U (:, 1 : r), Vr := V (:, 1 : r), Σr := Σ(1 : r, 1 : r)
- =⇒ xmin = Vr Σ− r 1 U> r b
Example: A ∈ R^40 ×^8 , rank(A) = 3 SolByExtra
m=40; A=magic(m); n=m/5; A=A(:,1:n); b=A*rand(n,1); Solutions : Matlab A\b Using SVD Extrapolation Warning: Rank deficient with rank r = 3 4 iterations 1.846673326583244 0.457727535991772 0. 2.459131966980111 0.747175086297768 0. 0 0.694218248476673 0. 0 0.616598049455061 0. 0 0.669554887276158 0. 0 0.535347735013380 0. 0 0.482390897192287 0. 0.725632546879197 0.828425400739452 0.
Norm of solutions: 3.159758693196030 1.812127976894189 1. Norm of residuals: 1.0e-10 * 0.055694948577711 0.136651389778460 0.
My Way to Stanford
- In the audience of my 1974-talk were
Peter Henrici Rudolf Kalman 1923 – 1987 1930 – 2016
- Kalman gave me one of his papers containing a proof (using only the Penrose Equations) that the pseudo-inverse is unique.
- Henrici encouraged me to apply for a NSF grant to continue the research with the “master of least squares algorithms”: Gene Golub.
Gene gave me one of his papers:
Secular Equation – One of the Favorite Topics of Gene
- Conference on Computational Methods with Applications, August 19 - 25, 2007, Harrachov, Czech Republic a.
- Gene’s talk is available on-line:
ahttp://www.cs.cas.cz/~harrachov
One of the Many Examples in Gene’s Talk
Secular Equation Represented by BSVD
- BSVD (generalized SVD, also GSVD) for pair of matrices Am×n, Cp×n: U>AX = DA = diag(α 1 ,... , αn), αi ≥ 0 V>CX = DC = diag(γ 1 ,... , γq ), γi ≥ 0 , q = min(n, p) where U m×m^ and V p×p^ orthogonal and Xn×n^ nonsingular.
- If γ 1 ≥... ≥ γr >γr+1 =... = γq = 0 then μi = α
(^2) i γ i^2 ,^ i^ = 1,... , r^ are the eigenvalues of generalised EV-Problem A>Ax = μC>Cx.
- With c := U>b and e := V>d the secular equation becomes
f (λ) =
∑^ r 1=
α^2 i
( (^) γ ici −^ αiei α^2 i + λγ i^2
) 2
∑^ p i=r+
e^2 i = δ^2
f has at most r poles for λ = −μi and f (λ) = δ^2 at most 2 r solutions
Characterization of the Solution
If (x 1 , λ 1 ) and (x 2 , λ 2 ) are solutions of the normal equations, then
Thm 1
‖Ax 2 − b‖^2 − ‖Ax 1 − b‖^2 = λ^1 − 2 λ^2 ‖C(x 1 − x 2 )‖^2.
If λ 1 > λ 2 =⇒ ‖Ax 1 − b‖ < ‖Ax 2 − b‖ =⇒ the largest solution λ determines solution
Thm 2
− λ^1 + 2 λ^2 ‖C(x 1 − x 2 )‖^2 = ‖A(x 1 − x 2 )‖^2. =⇒ λ 1 + λ 2 < 0 =⇒ At most one λ > 0
Inequality Constraint
‖Ax − b‖^2 = min subject to ‖Cx − d‖^2 ≤ δ^2
- M = {x | ‖Ax − b‖ = min}
- If ‖Cx∗^ − d‖ ≤ δ for some x∗^ ∈ M then x∗^ is a solution. Constraint is not active.
- If {x | ‖Cx − d‖ ≤ δ} ∩ M = ∅ then constraint is active, solution on boundary: ‖Cx − d‖^2 = δ^2 (a) solve secular equation f (λ) = δ^2 for the only λ∗^ > 0 (b) x(λ∗) is the solution. One of the typical applications is from Christian Reinsch, “Smoothing by Spline Functions”, 1967.
Example 1 ‖Ax − b‖ = min s.t. ‖Cx − d‖ ≤ 10 Bsp
A b 0.73980.8930 0.52440.7545 -4.4414-5. 0.02590.1376 0.16980.6727 -0.7691-2. 0.42410.7646 0.61870.0068 -1.1464-4.
C d -1.6443-0.0263 -1.9204-0.3913 2.26503. -1.9660 -0.2804 2.
‖Ax − b‖ = const, ‖Cx − d‖ = 10 f (λ) = ‖Cx(λ) − d‖
active constraint, λi = [− 0. 7857 , 0 .0772], poles= −μi = [− 0. 4582 , − 0 .2935]
Example 2 ‖Ax − b‖ = min s.t. ‖Cx − d‖ = 10 Bsp
A b 0.58590.1907 0.63090.8920 -3.9636-3. 0.50340.0509 0.67340.6853 -1.9904-5. 0.05610.3352 0.69570.7998 -1.4789-3.
C d -0.5194-1.4917 -0.9237-0.1797 3.74133. -0.3088 -1.2986 2.
‖Ax − b‖ = const, ‖Cx − d‖ = 10 f (λ) = ‖Cx(λ) − d‖ λi = [− 1. 6157 , − 0. 9211 , − 0. 1827 , − 0 .0962], poles = −μi = [− 1. 2686 , − 0 .1393]
Example 3 secular equation with one (double) pole Bsp
A b 0.92000.9800 0.99000.8000 2.90002. 0.04000.8500 0.81000.8700 1.66002. 0.86000.1700 0.93000.2400 2.72000. 0.23000.7900 0.05000.0600 0.33000. 0.10000.1100 0.12000.1800 0.34000.
C = A
d 0.81470. 0.12700. 0.63240. 0.27850. 0.95750.
λi = [− 1. 4057 , − 0 .5943], −μi = [− 1. 0000 , − 1 .0000]