Scarica Model Identification Formulario - PoliMi | prof. Garatti e più Schemi e mappe concettuali in PDF di Model identification solo su Docsity!
Time series
Stochastic process
Full description of a stochastic process
too complicated
Does not consider the probability of all
occurrences, but gives an indication of the
behaviors of different realizations
Mean function
Covariance function
White noise
Complete uncorrelation with itself at different
time instants
MA(n) process
Linear combination of the past n values of e(t)
Always stationary
AR processes are MA(inf) with coefficients determined by the AR coefficients and recursively applying
the AR equation
AR(m) process
Linear combination of its m past values
An AR(m) is well defined if its MA inf form is
well defined
Weak description of SP :=
m y
(t , s)
γ y
(t , τ , s)
Stationary stochastic process
Probabilistic properties are time translation
invariant
- γ y
- | γ y
( τ ) | ≤ γ y
- γ y
( τ ) = γ y
(− τ )
process
Linear combination of the past values of e(t)
If , is well defined
M A(∞)
∞
i= 1
c
2
i
< ∞ y (t )
- function that relates v and s in full detail
- probability of every realization
S (t , s) ∀t
P s = s i
Let SSP, then it’s a White Noise with mean and
variance if the following conditions hold
e (t ) μ
λ
2
m e
= μ , ∀t
γ e
( 0 ) = λ
2
γ e
( τ ≠ 0 ) = 0
γ y
(t , τ , s) = E [
( v (t , s) − m v
(t , s)) ) (v (t − τ , s) − m v
(t − τ , s)) ]
portion of the real world that we interact with and that
generates a quantity of interest y (t )
Let , it is said MA infinity process
the signal
e (t ) ∼ W N (0, λ
2 )
y (t ) = c 0
e (t ) + … + c i
e (t ) + … =
∞
i= 0
c i
e (t − i )
Let and the recursive equation
the AR(m) process is defined as the steady state
solution obtained by taking the initial condition
and letting the initial time tend to
e (t ) ∼ W N (0, λ
2 )
y (t ) = a 1
y (t − 1 ) + … + a m
y (t − m) + e (t )
y (t 0
y (t 0
m y
= m
γ y
(t 1
, t 2
) depends on t 1
− t 2
= τ only
⇒ γ y
( τ ) = E
[
(y (t ) − m y
)(y (t − τ ) − m y
]
Let e (t ) ∼ W N (0, λ , it is said MA process the signal
2 )
y (t ) = c 0
e (t ) + c 1
e (t ) + … + c n
e (t )
m y
= E [s (t , s)] =
s
v (t , s)P (d s)
Infinite sequence of random variable s all
defined on the same probabilistic space
Includes all possible realizations
v (s, t )
Well definition theorem 2
Covariance of depolarized process
Spectrum of SSP
Given SSP, which is filtered through an ideal pass- Kinchine-Wiener Theorem
band filter, and is the output process
y (t )
˜y (t )
y
( ω ¯) = lim
δ → 0
γ y˜
There is just one stationary output which corresponds to
the steady state process. If is a.s. all possible
outputs obtained from all possible initializations of the
digital filter asymptotically and exponentially tend to
the steady state solution
y (t ) W (z )
ARMAX (d,p,n,m)
with
p : X part order
k : exogenous input delay
Let , it is said ARMA the steady
state solution of
e (t ) ∼ W N (0, λ
2 )
y (t ) = a 1
y (t − 1 ) + … + a m
y (t − m) + c 0
e (t ) + … + c n
e (t − n)
- is an SSP
- is asymptotically stable ( all poles
strictly inside the unitary circle)
e (t )
W (t ) | p i
γ y
λ
2
1 + a
2
γ y
( τ ) = a γ y
( τ − 1 ) = a
| τ | γ y
Yule Walker equations
For AR(1)
y (t ) = a y (t − 1 ) + e (t ), e (t ) ∼ W N (0, λ
2 )
Well definition theorem 1
y (t ) ARMA is well defined if and only if
Spectrum of SP
Time discrete Fourier transform of the
covariance function
is
- Periodic
- Real function
- Positive
- Symmetric (even)
y
( ω )
γ with ˜y
( τ ) = γ y
( τ ) ˜y = y − m y
y (t ) = operatorial representation
C (z )
A(z )
e (t )
Let , it is said ARMAX the steady
state solution of
e (t ) ∼ W N (0, λ
2 )
y (t ) = a 1
(t − 1 ) + … + a m
y (t − m) AR(m) part
= c 0
e (t ) + … + c n
e (t − n) MA(n) part
= b 0
u (t − k) + … + b p
u (t − k − p) X(k,p) part
y (t ) =
B (z )
A(z )
u (t − d ) +
C (z )
A(z )
e (t )
y
( ω ) = | W (z ) |
2 Γ e
( ω )
It can also be represented as the output of a
digital filter fed by e (t )
Covariance and spectrum bring the same amount of
information
y
( ω ) =
+∞
τ =−∞
γ y
( τ )e
j ωτ = ℱ{ γ y
( τ )}
γ y
( τ ) =
π
− π
y
( ω )e
j ωτ
d ω
ARMA (m,n)
Linear combination of its m past elements +
linear combination of n past elements of
It is well defined if the MA inf equivalent to
the AR part is well defined
e (t )
Theorem of ARMA representation
But this representation is not unique, there are
four sources of ambiguity in defining an
ARMA process
Optimal linear prediction
Assumption of linear optimal prediction
a.s. and zeroes <1 are enough, but the other
conditions of the canonical representation
make computations much easier
ARMAX prediction
Model Identification
PEM identification
PEM identification criterion
Empirical variance of the prediction error
estimates the prediction error over the whole
experiment and is minimized
Theorem of asymptotic PEM cost
The dependence on the realization is lost and
the curves corresponding to different
realizations shrink a single asymptotic curve
ARMA optimal linear prediction
PE V = E
[
(y (t ) − ŷ (t | t − k))
2
]
= E [E (z )e (t )]
ŷ (t + k | t ) =
F (z )
C (z )
y (t ) +
B (z )E (z )
C (z )
u (t + k − d )
J
N
( ϑ ) =
N
N
i= 1
y (i ) − y (i | i − 1, ϑ )
2
ϑ = arg min
ϑ ∈Θ
J
N
( ϑ )
Let be an SSP with rational spectral density, there
exists (i) with suitable mean and variance, and
(ii) rational function such that
y (t )
ξ ∼ W N
W (z )
y (t ) = W (z ) ξ (t )
Let
- is the
canonical representation of the process
- has all zeroes within the unitary circle
y (t ) = W (z )e (t ), e (t ) ∼ W N (0, λ
2 )
y (t ) = W (z )e (t ), e (t ) ∼ W N (0, λ
2
)
W (z )
Theorem of spectral factorization
Let be an SSP with rational spectral density, then, there exists a unique white noise process with
suitable mean and variance and a unique rational transfer function such that
And,
- are monic (coefficients of the maximum degree terms = 1)
- have null relative degree
- are coprime (they have no common factors)
- The absolute value of the poles and the zeroes of is less than or equal to 1 ,
when all the four conditions are satisfied : is canonical representation of
y (t ) ξ (t )
W (z ) y (t ) = W (z ) ξ (t )
C (z ), A(z )
C (z ), A(z )
C (z ), A(z )
W (z )
| poles, zeroes | ≤ 1
⇒ y (t ) = W (z ) ξ (t ) y (t )
W (z ) =
C (z )
A(z )
= E (z ) + F (z )z
−k
C (z )
A(z )
y (t + k) = E (z )e (t + k) + F (z )z
−k
C (z )
A(z )
e (t + k)
ŷ (t + k | t ) =
F (z )
A(z )
e (t ) =
F (z )
C (z )
y (t )
Also, by letting we can
state that
J
N
( ϑ , s)
N→∞ ¯ J ( ϑ ) = E [ ϵ (t , ϑ )
2
]
Δ = { ϑ * :
J ( ϑ ) ≤
J ( ϑ ), ∀ θ }
ϑ N
(s)
N→∞
Minimization of PEM identification criterion and search of the optimal parameters
- AR-ARX analytical : LS squares equation and formula
- ARMA-ARMAX numerical : recursive equation and formula
LS normal equations
LS formula
PEM takes model’s order for granted, but they are
parameters and have to be chosen in such a way to
optimally identify the data generation system
Idea: minimize the cost function
Model validation
Simple minimization of cost function lead to
overfitting (minimum error on data points but
no generalization capability), so we resort to
Akaike’s Information criterion
Based on statistical evaluations
The first term penalizes increase in degree
ϑ
(m)
I
= arg min
ϑ ∈Θ M
N
I
i= 1
N
I
ϵ (i , ϑ ) )
2
J
V
ϑ
(m)
I
N
V
N V
i= 1
ϵ (i ,
ϑ
(m)
I
2
LS squares
Can be computed only if the matrix in non
singular (invertible)
- if it is,
- If it is not, there exist solutions, and the
is a degenerate paraboloid with a
valley of identical solutions
i. Model is redundant
ii. Not enough data points
∃! ϑ * :
J ( ϑ *) = min
ϑ
J ( ϑ )
1
J ( ϑ )
Final prediction error
Obtained by enforcing a probabilistic criteria
the term penalizes the increase of order
N + m
N − m
FPE (m ,
ϑ
(m)
N
N + m
N − m
J (
ϑ
(m)
N
y (t ) = (a 1 z
− 1
−m )y (t ) + (b 0
− 1
−p+ 1 )u (t − d ) + e (t ) =
= ϑ
T φ + e (t ), with
ϑ = [a 1 … a m b 0 … b p ]
T
φ = [y (t − 1 )…y (t − m)u (t − d )…u (t − d − p)]
T
ŷ (t + k | t ) = ϑ
T φ = φ
T ϑ
J
N
( ϑ ) = E
[
N
N
t= 1
y (t ) − φ
T
ϑ )
]
d J N
( ϑ )
d ϑ
ϑ =
̂ ϑ
N
N
t= 1
(− φ )(y (t ) − φ
T
ϑ ) = 0
d
2
J N
( ϑ )
d
2 ϑ
ϑ =
̂ ϑ
N
t= 1
φ (t ) φ (t )
T
ϑ =
N
t= 1
φ (t )y (t )
ϑ * =
N
t= 1
φ (t ) φ (t )
T
− 1 N
t= 1
φ (t )y (t )
Numerical recursive computation of ϑ *
A I K (m ,
ϑ
(m)
N
2 m
n
(
J
N
ϑ
(m)
N
ARMA and ARX models, lead to an identification
criterion which is not quadratic with respect to
Solved with a iterative method : quasi newton’s method
θ
J N
( ϑ ) =
1
N
N
∑
i= 1
(y (t ) − ŷ (t | t − 1 ))
2
1
N
N
∑
i= 1
(
A(z )
C (z )
y (t ) +
B (z )
C (z )
u (t − d )
)
2
Cross validation
i. Identification on dataset
ii. Validation on dataset
Identification and validation on two different
datasets
