Download Understanding Autocorrelation Functions: A Comprehensive Guide and more Exams Advanced Education in PDF only on Docsity!
The Autocorrelation Function (ACF)
The j-th order autocorrelation function is defined as - correct answer rho(j) = gamma(j)/gamma(0) = ([E(X(t)-mew)E(X(t-j)-mew)])/(E[(X(t) - mew)^2]) = (Cov(X(t),X(t-j))/Var(X(t) for j = 0,1,2,... with -1 <= rho(j) <= 1 The plot of rho(j) against j = 0,1,2,... is called - correct answer the correlogram For a stationary ergodic process, the ACF can be estimated by the sample moments as follows - correct answer y bar = (1/T) * Summation from t=1 to T of y(t) sigma hat^2 = (1/T) * Summation from t=1 to T of (y(t) - y bar)^ for each j: r(j) = (Summation from t=j+1 to T of (y(t)-y bar)(y(t-j)-y bar))/(Summation from t=1 to T of (y(t)-y bar)^2) Sample variance of r(j): Var(r(j)) = T^-1 for j= Var(r(j)) = T^-1 * (1+2*summation from k=1 to j-1 of (r(k)^2)) for j > 1 Testing for autocorrelation - correct answer Also known as testing the null hypothesis that autocorrelations are significantly different from zero H(0): r(j) = 0 for j = 1,2,3,...,k and k < T **Can be done with Box and Pierce (1970) or Ljung-Box (1978)
Autocorrelation - correct answer In a longitudinal design, the correlation of one variable with itself, measured at two different times. Box-Pierce (1970) statistic - correct answer Q = T * Summation from j=1 to k of r(j)^ Q ~ X^2(k) under H(0) Ljung-Box (1978) statistic - correct answer Q = T(T+2) * Summation from j=1 to k of r(j)^2 / (T - j) Q ~ X^2(k) under H(0) Box-Pierce and Ljung-Box in R - correct answer Examining null hypothesis of independence in a given time series ('portmanteau tests') Box.test(x, lag=1, type=c("Box-Pierce","Ljung-Box") fitdf=0) x = numeric vector or univariate time series lag = statistic based on lag autocorrelation coefficients type = test to be performed: partial matching used fitdf = number of degrees of freedom to be subtracted if x is series of residuals **Sometimes applied to the residuals from an ARMA (p, q) fit, in which case the references suggest a better approximation to the null-hypothesis distribution is obtained by setting fitdf = p+q, provided lag > fitdf --> Gives value = a list with class "htest" and following components: -statistic = value of test statistic
Moving average processes - correct answer 1. MA(1)-process
- MA(q)-process
- MA(infinity)-process MA(1)-process - correct answer Y(t) = mew + theta(1)Epsilon(t-1) + Epsilon(t) with {Epsilon(t)} = Gaussian White Noise Checking for stationarity of MA(1)-process - correct answer E(Y(t)) = mew + theta(1)E(Epsilon(t-1)) + E(Epsilon(t)) = mew inverted A t gamma(0) = Var(Y(t)) = E[(Y(t)-mew)^2] = E[(theta(1)Epsilon(t-1)Epsilon(t) + Epsilon(t)^2] = theta(1)^2sigma^2 + 0 + sigma^ = (1 + theta(1)^2)sigma^2 inverted A t Autocovariance of an MA(1) - correct answer gamma(1) = Cov(Y(t),Y(t-1)) = E[(Y(t)- mew)(Y(t-1)-mew)] = E[(theta(1)Epsilon(t-1) + Epsilon(t))(theta(1)Epsilon(t-2) + Epsilon(t-1)] = E[theta(1)^2Epsilon(t-1)Epsilon(t-2) + theta(1)Epsilon(t-1)^2 + theta(1)Epsilon(t)Epsilon(t-2) + Epsilon(t)Epsilon(t-1)] = 0 + theta(1)sigma^2 + 0 + 0 = theta(1)sigma^2, inverted A t
The higher order covariances of MA(1) are all - correct answer zero gamma(j) = Cov(Y(t), Y(t-j)) = 0, inverted A j > 1 -> {Y(t)} of MA(1) is (covariance) stationary The MA(1)-process is stationary and - correct answer ergodic *Ergodic based on: Summation from j=0 to infinity |gamma(j)| = (1+theta(1)^2)sigma^2 + |theta(1)| *sigma^2 < infinity The autocorrelations for the MA(1)-process are given by - correct answer rho(j) = gamma(j) / gamma(0) for j = 0,1,2,... **Therefore rho(0) = 1 (always) and for the MA(1)-process we have: rho(1) = theta(1) / (1 + theta(1)^2) with rho(1) > 0 for theta(1) > 0 and rho(1) < 0 for theta(1) < 0 *For j > 1 : gamma(j) = 0 and rho(j) = 0 --> thus autocorrelations are useful to identify the process MA(q)-process - correct answer Y(t) = mew + theta(0)Epsilon(t-1) + ... + theta(q)Epsilon(t-1) **Normally with theta(0) = 1
- AR(p)-process AR(1)-process - correct answer Y(t) = c + phiY(t-1) + Epsilon(t) Given that a first-order linear difference equation is given by: Y(t) = phiY(t-1) + w(t) for the AR(1)-process we have: w(t) = c + Epsilon(t) Epsilon(t) is a stochastic process, therefore the AR(1)-process is a first-order stochastic linear difference equation Showing that AR(1)-process is only stationary and ergodic if |phi| < 1 - correct answer Y(t) can be written as: phi^(t+1)Y(-1) + phi^tw(0) + ... + phi^2w(t-2) + phiw(t-1) + w(t) where phi^t = dynamic multiplier --> the effects of past Epsilon only die out for |phi| < 1 and under this condition the difference equation is stable The AR(1)-process can be written as - correct answer Y(t) = (c + Epsilon(t)) + phi(c
- Epsilon(t-1)) + phi^2(c + Epsilon(t-2)) + phi^3(c + Epsilon(t-3)) + ... = c(1 + phi + phi^2 + phi^3 + ...) + (Epsilon(t) + phiEpsilon(t-1) + phi^2*Epsilon(t-2) + ...)
= 1 / 1 - phi, if |phi| < 1 + MA(infinity)-process Y(t) = mew + Epsilon(t) + phiEpsilon(t-1) + phi^2(Epsilon(t-2)) + ... with E(Y(t) = mew = c / 1 - phi Applying AR(1)-process |phi| < 1 condition to stationarity and ergodicity of an MA(infinity)-process - correct answer Summation from j=0 to infinity of |psi(j)| = Summation from j=0 to infinity of |phi^j| = 1 / (1 - |phi|) < infinity if |phi| < 1 Applying AR(1)-process |phi| < 1 condition to variance of an MA(infinity)-process - correct answer gamma(0) = E[(Y(t) - mew)^2] = E[(Epsilon(t) + phiEpsilon(t-1) + phi^2Epsilon(t-2) + ...)^2] = (1 + phi^2 + phi^4 + phi^6 + ...)sigma^ = 1/(1-phi^2)sigma^2 (if |phi| < 1) Applying AR(1)-process |phi| < 1 condition to autocovariance - correct answer gamma(j) = E[(Y(t) - mew)(Y(t-j) - mew)] = E[(Epsilon(t) + phiEpsilon(t-1) + phi^2Epsilon(t-2) + ...)(Epsilon(t-j) + phiEpsilon(t-j-1) + phi^2Epsilon(t-j-2) + ...)] -> gamma(1) = (phi + phi^3 + phi^5 + ...)sigma^ = phi(1 + phi^2 + phi^4 + ...)sigma^ = (phi / (1 - phi^2))sigma^
AR(2)-process written with lag operator - correct answer (1 - phi(1)L - phi(2)L^2)Y(t) = c + Epsilon(t) With factorization: (1 - lambda(1)L)(1-lambda(2)L)Y(t) = c + Epsilon(t) Only stable if eigenvalues lambda(1) and lambda(2) of F matrix (phi(1),phi(2),1,0) [solutions lambda(1) and lambda(2) of characteristic polynomial lambda^2 - phi(1)lambda - phi(2) = 0] lie inside unit circle (i.e., are less than 1 in modulus for complex numbers) Can also be checked by seeing if solutions z(1) and z(2) of lag polynomial 1- phi(1)z-phi(2)z^2=0 le outside unit circle (i.e., are greater than 1 in modulus) As AR(2)-process is a second-order stochastic linear difference equation, those conditions must be fulfilled for the AR(2)-process to be stationary Expression for (1 - phi(1)L - phi(2)L^2)^-1 so that the AR(2)-process can be written as an MA(infinity)-process - correct answer Y(t) = (1-phi(1)L-phi(2)L^2)^- 1 c + (1-phi(1)L - phi(2)L^2)^-1Epsilon(t) where: (1-phi(1)L - phi(2)L^2)^-1 = (1-lambda(2)L)^-1(1-lambda(1)L)^- = (1+lambda(2)L + lambda(2)^2L^ +...)(1+lambda(1)L+lambda(1)^2L^2+...) = 1+psi(1)L+psi(2)L^2+... =phi(L)
with phi(1) = lambda(1) + lambda(2) phi(2) = lambda(1)^2 + lambda(2)^2 + lambda(1) * lambda(2) MA(infinity)-representation of the AR(2)-process is given by - correct answer Y(t) = c/(1-phi(1)-phi(2) + Epsilon(t) + psi(1)Epsilon(t-1) + psi(2)Epsilon(t-2)+... with E(Y(t)) = mew = c/(1-phi(1)-phi(2)) and psi(j) = c(1)lambda(1)^j + c(2)lambda(2)^j where c(1) + c(2) = 1 Therefore, MA representation of AR(2)-process can be written shortly as: Y(t) = mew + psi(L)Epsilon(t) Showing that the autocovariances follow the same second-order difference equation as the process for Y(t) - correct answer Substituting c=mew(1-phi(1)-phi(2)), we get Y(t) = mew(1-phi(1)-phi(2)) + phi(1)Y(t-1) + phi(2)Y(t-2) + Epsilon(t) Y(t) - mew = psi(1)(Y(t-1)-mew)+psi(2)(Y(t-2)-mew)+Epsilon(t) Multiplying by (Y(t-j)-mew) and taking expectations results in: E[(Y(t)-mew)(Y(t-j)-mew)] = psi(1)E[(Y(t-1)-mew)(Y(t-j)-mew)] + psi(2)E[(Y(t-2)- mew)(Y(t-j)-mew)]+E[Epsilon(t)(Y(t-j)-mew)]
where (1 - phi(1)L - phi(2)L^2 - ... - phi(P)L^P)^- = (1 - lambda(P)L)^-1 ... (1-lambda(1)L)^- = (1 + lambda(P)L + lambda(P)^2L^2 + ...)...(1 + lambda(1)L + lambda(1)^2L^2 + ...) = 1 + psi(1)L + psi(2)*L^2 + ... = psi(L)
Also shown that psi(j) = c(1)lambda(1)^j + c(2)lambda(2)^j + ... + c(P)lambda(P)^j with Summation from i=1 to P of c(i) = 1 --> MA(infinity) of AR(p) given by: Y(t) = c/(1-phi(1)-...-phi(P)) + Epsilon(t) + psi(1)Epsilon(t-1) + psi(2)*Epsilon(t-2)
- ... with E(Y(t)) = mew = c/(1-psi(1)-...-psi(P))
Writing MA(infinity) representation of AR(p) shortly - correct answer Y(t) = mew + psiLEpsilon(t) **As for stationary AR(p) process Summation from j=0 to infinity of |psi(j)| < infinity the process is also ergodic Showing that the autocovariances of AR(p) follow same pth-order difference equation as the process for Y(t) - correct answer -Substituting c = mew(1 - phi(1)
- ... - phi(P)) in AR(p) process equation: Y(t) = mew(1 - phi(1) - ... - phi(P)) + phi(1)Y(t-1) + ... + phi(P)Y(t-p)+Epsilon(t) (Y(t) - mew) = phi(1)(Y(t-1) - mew) + ... + phi(P)(Y(t-p) - mew) + Epsilon(t) -Multiplying by (Y(t-j) - mew) and taking expectations results in: E[(Y(t) - mew)(Y(t-j)-mew)] = phi(1)E[(Y(t-1)-mew)(Y(t-j)-mew)]+...+phi(P)E[(Y(t- p)-mew)(Y(t-j)-mew)] + E[Epsilon(t)(Y(t-j)-mew)] -> gamma(j) = phi(1)gamma(j-1) + phi(2)gamma(j-2) + ... + phi(P)gamma(j-P) for j=1,2,... Calculating autocorrelations related to AR(p) - correct answer Divide gamma(j) by gamma(0): rho(j) = phi(1)rho(j-1) + ... + phi(P)rho(j-P) for j=1,2,... = Yule-Walker equations
-> gamma(j) = phi(1)gamma(j-1) + phi(2)gamma(j-2) + ... + phi(p)*gamma(j-p) for j > q **For j <= q, the MA part also affects the autocovariances, thus the autocovariances and the autocorrelations of the ARMA process have more complicated characteristics than those of an AR(p) or MA(q) process
