Time Domain Analysis - Time Series Analysis - Lecture Notes, Study notes of Mathematical Statistics

Download Time Domain Analysis - Time Series Analysis - Lecture Notes and more Study notes Mathematical Statistics in PDF only on Docsity!

STATISTICS 479/

TIME SERIES ANALYSIS PART II

Doug Wiens April 12, 2005

Contents

Part II

Time Domain Analysis

1. Lecture 5

• Suppose {Xt} is (weakly) stationary (and non- deterministic - Xt is not a non-random function of Xt− 1 , Xt− 2 , ..., i.e. it cannot be predicted ex- actly from the past). Write Xt − μ as X˙t. Then (Wold’s Representation Theorem) we can repre- sent X˙t as

X^ ˙t = wt + ψ 1 wt− 1 + ψ 2 wt− 2 + ...

X^ ∞ k=

ψk wt−k (with ψ 0 = 1) (1)

and

X^ ∞ k=

ψ^2 k < ∞.

Salient feature: Linear function of past and present (not future) disturbances. Interpretation: con- vergence in mean square; i.e.

E

⎡ ⎢⎣

⎛ ⎝ (^) X˙t − X^ K k=

ψk wt−k

⎞ ⎠

2 ⎤ ⎥⎦ → 0 as K → ∞.

The converse holds: Assume {Xt} linear; then

(i) E [Xt] = μ + E

h X˙t

i

= μ +

X^ ∞ k=

ψk E

£ wt−k

¤ = μ.

(ii) COV [Xt , Xt+m] = E[ X˙t X˙t+m]

= E

⎡ ⎣ X^ ∞ k=

ψk wt−k

X^ ∞ l=

ψl wt+m−l

⎤ ⎦

X^ ∞ k=

X^ ∞ l=

ψk ψl E

£ wt−k wt+m−l

¤

X^ ∞ k=

X^ ∞ l=

ψk ψl

n σ w^2 I (l = m + k)

o

= σ^2 w

X^ ∞ k=

ψk ψk+m.

(In particular, V AR [Xt] = σ^2 w^ P∞ k=0 ψ^2 k < ∞.) Thus

Stationarity ⇔ Linearity.

• Backshift operator:

B (Xt) = Xt− 1 , B^2 (Xt) = B ◦ B (Xt) = B (Xt− 1 ) = Xt− 2 , etc. Then {Xt} linear ⇒ X˙t = ψ(B)wt for the characteristic polynomial

ψ(B) = 1 + ψ 1 B + ψ 2 B^2 + .... This is not really a polynomial, but if it is, i.e. ψk = 0 for k > q, we say {Xt} is a moving average series of order q, written MA(q). We usually write ψk = −θk. Then

X^ ˙t = wt−θ 1 wt− 1 −θ 2 wt− 2 −...−θq wt−q = θ(B)wt for θ(B) = 1 − θ 1 B − ... − θq B q^ , the MA(q) characteristic polynomial.

• The above convention, with θ(B) = 1 − θ 1 B − ... − θq B q, is as in ASTSA, and is consistent with an earlier version of the text and many other texts.

— When P∞ k=1 |φk|^ <^ ∞^ we say the series is absolutely summable. The importance of ab- solute summability is that such series can be re-arranged - they can be summed in any or- der. In contrast, P∞ k=

(−1)k+ k = ln 2^ ≈

. P69, but the series is not absolutely summable: ∞ k=

1 k =^ ∞.^ The original series can be re- arranged to give just about anything; for in- stance μ 1 +^1 3

¶

μ 1 5

+^1

¶

• · · ·, in which two positive terms are always followed by a negative one, converges to something > .8.

• When is a stationary (i.e. linear) process invert- ible? Let {Xt} be linear, so X˙t = ψ(B)wt and P∞ k=1 ψ 2 k <^ ∞.^ Suppose it is invertible. Then φ(B) X˙t = wt; thus φ(B)ψ(B)wt = wt and φ(B)ψ(B) = 1. Thus φ(B) = 1/ψ(B), i.e. 1/ψ(B) has a power series expansion with absolutely summable coefficients. This makes ψ(B) quite special.

— Example: MA(1); ψ(B) = 1 − θB for some θ. Then if invertible we must have 1 /ψ(B) = 1 + θB + θ^2 B^2 + ... =

X^ ∞ j=

θ j^ B j; AND

1 + |θ| +

¯¯ ¯θ^2

¯¯ ¯ + ... < ∞; this last point holds iff |θ| < 1. Note that the root of θ(B) = 0 is B = 1/θ, and then |θ| < 1 ⇔ |B| > 1, i.e. the MA(1) process with θ(B) = 1 − θB is invertible iff the root of θ(B) = 0 satisfies |B| > 1.

— In general, a linear process Xt = ψ(B)wt is in- vertible iff all roots of the characteristic equa- tion ψ(B) = 0 satisfy |B| > 1 (complex modulus), i.e. they “lie outside the unit circle in the com- plex plane”.

— The modulus of a complex number z = a + ib

is |z| =

q a^2 + b^2 (like the norm of a vector with coordinates (a, b)).

1. Lecture 6

• Review of previous lecture. Assume now for sim- plicity that the mean is zero:

— Linear process: Xt = ψ(B)wt, ψ(B) = 1 + ψ 1 B + ψ 2 B^2 + ... with P∞ k=0 ψ k^2 < ∞. Then

γ(m) = σ^2 w

X^ ∞ k=

ψk ψk+m.

— Linear + “ψk = 0 for k > q”: MA(q) process, γ(m) = 0 for m > q. Characteristic polyno- mial written as θ(B) = 1 − θ 1 B − θ 2 B^2 − ... − θq B q^.

— Invertible process: φ(B)Xt = wt , φ(B) = 1 − φ 1 B − φ 2 B^2 − ... with P k |φk|^ <^ ∞. Note this is really X˙t; a non-zero mean can be accommodated as follows: wt = φ(B) X˙t = X˙t − φ 1 X˙t− 1 − ... = (Xt −^ μ)^ −^ φ 1 (Xt− 1 −^ μ)^ −^ ... = {Xt − φ 1 Xt− 1 − ...} − μ { 1 − φ 1 − φ 2 − ...} = φ(B)Xt − α,

if α = μφ(1).

— Invertible + “φj = 0 for j > p”: AR(p) process.

— Wold’s Theorem: Stationary ⇔ Linear.

— A stationary process is invertible iff all roots of ψ(B) = 0 lie outside the unit circle. Thus an MA(q) is stationary (linear), not necessarily invertible.

— An invertible process is stationary iff all roots of φ(B) = 0 lie outside the unit circle. Thus an AR(p) is invertible, not necessarily station- ary.

• Example: MA(2).

Xt = wt − θ 1 wt− 1 − θ 2 wt− 2 , θ(B) = 1 − θ 1 B − θ 2 B^2. If θ 12 + 4θ 2 < 0 (so both roots are complex), then invertibility requires |θ 2 | < 1. Suppose this is

• ARMA models are defined in operator notation by φ(B)Xt = θ(B)wt; if φ(B) is an AR(p) char- acteristic polynomial and θ(B) an MA(q), we say {Xt} is an ARMA(p,q) process. It is station- ary (linear, causal) if Xt = ψ(B)wt for a series ψ(z) = P (^) ψ k z^ k,^ |z|^ ≤^ 1, with square summable coefficients. Then the coefficients ψk are deter- mined from θ(z)/φ(z) = ψ(z). It can be shown that ψ(z) has the required properties only if all zeros of φ(z) lie outside the unit circle. Simi- larly an ARMA(p,q) is invertible only if all zeros of θ(z) lie outside the unit circle. We also require that the polynomials have no common factors.
• Example: (Example 2.6 in text)

Xt =. 4 Xt− 1 +. 45 Xt− 2 + wt + wt− 1 +. 25 wt− 2 ⇒

³ 1 −. 4 B −. 45 B^2

´ Xt =

³ 1 + B +. 25 B^2

´ wt ⇒ (1 −. 9 B) (1 +. 5 B) Xt = (1 +. 5 B) (1 +. 5 B) wt ⇒ (1 −. 9 B) Xt = (1 +. 5 B) wt.

Thus series is both stationary and invertible. It is ARMA(1,1), not ARMA(2,2) as it initially ap- peared. Students should verify that the above can be continued as

Xt =

⎡ ⎣ X^ ∞ j=

(.9)j^ B j^ · (1 +. 5 B)

⎤ ⎦ (^) wt

h 1 + (.9 + .5)B + ... + (.9)j−^1 (.9 + .5)B j^ + ...

i wt = ψ(B)wt where ψ(z) = P^ ψk z k^ and ψ 0 = 1, ψk = 1.4(.9)k−^1.

• Box-Jenkins methodology:
  1. Determine the theoretical ACF (and PACF, to be defined) for these and other classes of time series models. Use the sample ACF/PACF to match the data to a possible model (MA(q), AR(p), etc.).

Figure 2.1. Sample ACF of simulated MA(3) series.

Figure 2.2. Sample ACF of simulated MA(3) series.

• ACF of an AR(p) process: Let j ≥ 0; assume process is stationary. Then

wt = Xt −

X^ p i=

φi Xt−i

⇒ COV

⎡ ⎣Xt − X^ p i=

φi Xt−i , Xt−j

⎤ ⎦

= COV

h wt , Xt−j

i

⇒ γ(j) −

X^ p i=

φi γ(j − i) = COV

h wt , Xt−j

i .

Under the stationarity condition, Xt−j is a linear combination wt−j + ψ 1 wt−j− 1 + ψ 2 wt−j− 2 + .. with

COV

h wt , Xt−j

i = COV

" wt , wt−j + ψ 1 wt−j− 1 +ψ 2 wt−j− 2 + ..

= σ w^2 I(j = 0), thus

γ(j) −

X^ p i=

φi γ(j − i) =

( σ w^2 , j = 0, 0 j > 0. These are the “Yule-Walker” equations to be solved to obtain γ(j) for j ≥ 0, then γ(−j) = γ(j).