Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Notes from a seminar on tikhonov regularization and inverse theory taught by thomas shores at the department of mathematics in a university. The notes cover topics such as the generalized singular value decomposition (gsvd), tikhonov regularization, iterative methods, error bounds, and regularization techniques using bounds as constraints.
Typology: Study notes
1 / 42
This page cannot be seen from the preview
Don't miss anything!
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques
Instructor: Thomas Shores Department of Mathematics
Lecture 23, April 6, 2006 AvH 10
Theorem Let G be an m × n matrix and L a p × n matrix. Then there exist m × m orthogonal U, p × p orthogonal V and n × n nonsingular matrix X with m ≥ n ≥ min {p, n} = q such that
UT^ GX = diag {λ 1 , λ 2 ,... , λn} = Λ = Λm,n V T^ LX = diag {μ 1 , μ 2 ,... , μq } = M = Mp,n ΛT^ Λ + MT^ M = 1.
Also 0 ≤ λ 1 ≤ λ 2 · · · ≤ λn ≤ 1 and 1 ≥ μ 1 ≥ μ 2 · · · ≥ μq ≥ 0. The numbers γi = λi /μi , i = 1 ,... , rank (L) ≡ r are called the generalized singular values of G and L and 0 ≤ γ 1 ≤ γ 2 · · · ≤ γr.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
The Experiment: Place sensors at vertical depths zj , j = 1 ,... , n, in a borehole, then: Generate a seizmic wave at ground level, t = 0. Measure arrival times dj = t (zj ), j = 1 ,... , n. Now try to recover the slowness function s (z), given
t (z) =
∫ (^) z
0
s (ξ) d ξ =
0
s (ξ) H (z − ξ) d ξ
It should be easy: s (z) = t′^ (z). Hmmm.....or is it?
Do Example 5.4-5.5 from the CD.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
The Experiment: Place sensors at vertical depths zj , j = 1 ,... , n, in a borehole, then: Generate a seizmic wave at ground level, t = 0. Measure arrival times dj = t (zj ), j = 1 ,... , n. Now try to recover the slowness function s (z), given
t (z) =
∫ (^) z
0
s (ξ) d ξ =
0
s (ξ) H (z − ξ) d ξ
It should be easy: s (z) = t′^ (z). Hmmm.....or is it?
Do Example 5.4-5.5 from the CD.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
The Experiment: Place sensors at vertical depths zj , j = 1 ,... , n, in a borehole, then: Generate a seizmic wave at ground level, t = 0. Measure arrival times dj = t (zj ), j = 1 ,... , n. Now try to recover the slowness function s (z), given
t (z) =
∫ (^) z
0
s (ξ) d ξ =
0
s (ξ) H (z − ξ) d ξ
It should be easy: s (z) = t′^ (z). Hmmm.....or is it?
Do Example 5.4-5.5 from the CD.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
The Experiment: Place sensors at vertical depths zj , j = 1 ,... , n, in a borehole, then: Generate a seizmic wave at ground level, t = 0. Measure arrival times dj = t (zj ), j = 1 ,... , n. Now try to recover the slowness function s (z), given
t (z) =
∫ (^) z
0
s (ξ) d ξ =
0
s (ξ) H (z − ξ) d ξ
It should be easy: s (z) = t′^ (z). Hmmm.....or is it?
Do Example 5.4-5.5 from the CD.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
Model Resolution Matrix: As usual, Rm,α,L = G \G. We can show this is XFX −^1.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
Model Resolution Matrix: As usual, Rm,α,L = G \G. We can show this is XFX −^1.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
TGSVD: We have seen this idea before. Simply apply it to formula above, remembering that the generalized singular values are reverse ordered. Formula becomes
mα,L =
∑^ p
j=k
γ j^2 γ^2 j + α^2
UTj d
cj
Xj +
∑^ n
j=p+ 1
UTj d
Xj
Key question: where to start k.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
Basic Idea: Comes from statistical leave-one-out cross validation. Leave out one data point and use model to predict it. Sum these up and choose regularization parameter α that minimizes the sum of the squares of the predictive errors
V 0 (α) =
m
∑^ m
k= 1
Gm α,[k]L
k
− dk
One can show a good approximation is
V 0 (α) =
m ‖G mα − d‖ 2 Tr (I − GG )^2
Example 5.6-7 gives a nice illustration of the ideas. Use the CD Instructor: Thomas Shores Department of Mathematicsscript to explore it. Change the startup le path to Math 4/896: Seminar in Mathematics Topic: Inverse Theo
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
Basic Idea: Comes from statistical leave-one-out cross validation. Leave out one data point and use model to predict it. Sum these up and choose regularization parameter α that minimizes the sum of the squares of the predictive errors
V 0 (α) =
m
∑^ m
k= 1
Gm α,[k]L
k
− dk
One can show a good approximation is
V 0 (α) =
m ‖G mα − d‖ 2 Tr (I − GG )^2
Example 5.6-7 gives a nice illustration of the ideas. Use the CD Instructor: Thomas Shores Department of Mathematicsscript to explore it. Change the startup le path to Math 4/896: Seminar in Mathematics Topic: Inverse Theo
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
Basic Idea: Comes from statistical leave-one-out cross validation. Leave out one data point and use model to predict it. Sum these up and choose regularization parameter α that minimizes the sum of the squares of the predictive errors
V 0 (α) =
m
∑^ m
k= 1
Gm α,[k]L
k
− dk
One can show a good approximation is
V 0 (α) =
m ‖G mα − d‖ 2 Tr (I − GG )^2
Example 5.6-7 gives a nice illustration of the ideas. Use the CD Instructor: Thomas Shores Department of Mathematicsscript to explore it. Change the startup le path to Math 4/896: Seminar in Mathematics Topic: Inverse Theo
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
Error Estimates: They exist, even in the hard cases where there is error in both G and d. In the simpler case, G known exactly, they take the form
‖mα − m˜α‖ 2 ‖mα‖ 2
≤ κα
∥d^ −^ ˜d
2 ‖G mα‖ 2
where κα is inversely proportional to α.
Chapter 6: Iterative Methods A Brief DiscussionChapter 5: Tikhonov Regularization Chapter 7: Additional Regularization Techniques Error Bounds
Error Estimates: They exist, even in the hard cases where there is error in both G and d. In the simpler case, G known exactly, they take the form
‖mα − m˜α‖ 2 ‖mα‖ 2
≤ κα
∥d^ −^ ˜d
2 ‖G mα‖ 2
where κα is inversely proportional to α.
