Download NLR: Non-Linear Regression with Levenberg-Marquardt Algorithm and more Study notes Mathematical Statistics in PDF only on Docsity!
1
NLR produces the least square estimates of the parameters for models that are not linear in their parameters. Unlike in other procedures, the weight function is not treated as a case replicate in NLR.
Model
Consider the model
f = f 1 x ,Θ 6
where Θ is a p × 1 parameter vector, x is a vector of independent variables, and f is a function of x and Θ.
Goal
Find the least square estimate Θ ∗^ of Θ such that Θ ∗^ minimizes the objective function
F (^) 1 6Θ = R WR ′ (1)
where
∗
R
W
R R
R y f f f x i n W W
n i i i i i n
1
1
K
K
K
1 6
1 6
Diag
and n is the number of cases. For case i, yi is the observed dependent variable, xi is the vector of observed independent variables, Wi is the weight function which can be a function of Θ.
The gradient of F at Θ (^) j is defined as
∇ F = 2 J ′⋅j WR
where J ⋅ j is the jth column of the n × pJacobian matrix J whose (^) 1 6i j, th element J (^) ij is defined by
J
R
W
W f ij
i i
i j
i j
Estimation
The modified Levenberg-Marquardt algorithm that was proposed by Moré (1977) and is contained in MINPACK is used in NLR to solve equation (1).
Given an initial value Θ 1 6^0 for Θ , the algorithm is as follows:
At stage k + 1 , k=0 1 2, , , K
f (^) i f k i
1 6 (^) = 1 6F
Θ (^) , R (^) i y f k i i
1 6 1 6k
= − , Fk = F Θ1 6k^ , and J 1 6k^ = J Θ1 6k
- Choose an appropriate non-negative scalar such that
F 4 Θ k^ + h (^) k 9
Statistics
When iteration terminates, the following statistics are printed.
Parameter Estimates and Standard Errors
The asymptotic standard error of Θ ∗j^ is estimated by the square root of the jth diagonal element a (^) jj of A , where
A = J W J
′
∗ ∗ ∗ ∗
F −
n p
4 Θ 9 1
and J ∗^ and W ∗^ are the Jacobian matrix J and weight function W evaluated at Θ ∗^ , respectively.
Asymptotic 95% Confidence Interval for Θ j
Θ ∗j ± t 1 0 975. ,n −p a (^6) ii
Asymptotic Correlation Matrix of the Parameter Estimates
C = D −^ 1 2^ AD −1 2
where
D = Diag 4 a 11, K,a (^) pp 9
and aii is the ith diagonal element of A.
Analysis of Variance Table
Source df Sum of Squares
Residual n^ −^ p
F 4 Θ ∗ 9
Regression p^ SS (^) uncorrected − F 4 Θ ∗ 9
Uncorrected Total n^ SSuncorrected
Corrected Total n − (^1) SS y Wi i
n uncorrected −^
∗
=
∑
2
1
4 Θ 9
where
SS W y
y W y W
i i
n i
i i
n i i i
n
uncorrected =
∗
∗
∗
Q
Q Q
3 8
3 8 3 8
1
2
1 1
References
Moré, J. J. 1977. The Levenberg-Marquardt algorithm: implementation and theory in numerical analysis. In: Lecture Notes in Mathematics 630, G. A. Watson, ed. Berlin: Springer-Verlag. 105–116.
