Model Fitting

Question 1

Model Selection -ρk and ACF

MA(q)

Answer 1

ρk = 0 for all k>0

-so the sticks drop to 0 after some lagq

Question 2

Model Selection - ρk

AR(p)

Answer 2

ρk = α1ρk-1 + … + αpρk-p

Question 3

Model Selection - ρk

ARMA(p,q)

Answer 3

-as for AR(q) except for the first q values

Question 4

Model Selection - ACF

ARIMA

Answer 4

ρk decays very very slowly and we need to apply differencing until ρk indicates AR, MA, ARMA

Question 5

Spotting AR Processes Lemma

Answer 5

-let y1,…,yp be roots of α(y) = 1 - α1y - … - αpy^p
-then for AR(p):
ρk = Σ cj/yj^k
-sum from j=1 to j=p
-for suitable constants c1,…,cp

Question 6

AR(1), ACF

Answer 6

ρk = α^k

• for α>0, exponential decay
• for α<0, exponential decay with alternating signs

Question 7

AR(P) p>1, ACF

Answer 7

ρk = c^k cos(φk)

• damped oscillations for some constants φ, c with |c|<1
• looks like sine/cos wave of sticks with peak amplitude decreasing exponentially

Question 8

Partial Autocorrelation Function

Definition

Answer 8

• for k=1,2,… (however many you think are necessary) use the Yule-Walker equations to fit an AR(k) model (using ρ1,…,ρk) obtaining coefficients αk1,…,αkk
• then αkk = lag-k partial autocorrelation function (pacf) of {Xt}

Question 9

AR(p) - PACF

Answer 9

-cuts off after lag p

Question 10

MA(q) - PACF

Answer 10

-exponential decay / damped oscillations

Question 11

ARMA(p,q) - PACF

Answer 11

-exponential decay / damped oscillations

Question 12

MA(q), ρk^ Distribution

Answer 12

-if X1,...,Xn is MA(q), then for k>q:
ρk^ ~ N(0, 1/n [1 + 2Σρi^²])
-sum from i=1 to i=q
=>
95% of sticks after lag q fall in range:
[-1.96 * sd , 1.96*sd]

Question 13

AR(p), αkk^ Distribution

Answer 13

-if X1,...,Xn is AR(p), then for k>p:
αkk^ ~ N(0, 1/n)
=>
95% of sticks after lag p fall in range:
[-1.96 * 1/√n , 1.96* 1/√n]

Question 14

Parameter Estimation - Non-Zero Mean

Answer 14

-so far we have assumed that stationary processes have zero mean
-we can cope with non-zero mean μ for {Xt} by modelling:
Yt = Xt - μ
-we estimate μ by μ^ = 1/n Σ Xt
-sum form t=1 to t=n
-subtract μ^ from observations to estimate {Yt} and proceed with fitting model to {Yt}

Question 15

Method of Moments

Definition

Answer 15

• choose parameters such that means and correlation for model and sample coincide
• for AR(p) models this gives the Yule-Walker equations

Question 16

Method of Moments

Steps

Answer 16

-can estimate σε² via method of moments assuming {Xt} is stationary
-write Xt in terms of the model, e.g. for AR(1):
Xt = α1Xt-1 + εt
-take variance of Xt
σx² = Var(Xt) = Var(α1Xt-1 + εt)
-and write in terms of σε²
-rewrite in terms of σε² for an estimate of σε²
-in the estimate, σx² can be replaced by the sample variance

Question 17

Least Square Estimation

Definition

Answer 17

-choose parameters to minimise the residual sum of squares

Question 18

Least Squares Estimation

Steps

Answer 18

-rearrange model to write εt = …
-calculate the residual sum of squares:
S(α) = Σ εt²
-find α to minimise S(α) using ∂S(α)/∂α = 0
-can then estimate σε² using the sample variance equation:
σε²^ = 1/(n-1) Σ εt²
-sum form t=1 to t=n

Question 19

Likelihood

Definition

Answer 19

L(α; X) = f(X; α)

Question 20

Maximum Likelihood Estimation

Definition

Answer 20

-to use MLE, need to assume distribution for εt, then maximise L(α|X) = f(X; α)

Question 21

Invertibility

Definition

Answer 21

-the past influence decays exponentially so that more distant events have a smaller effect on the present than more recent past

Question 22

Invertibility for MA(1) Processes

Answer 22

-for an MA(1) process:
Xt = `β εt-1 + εt
-with εt ~ N(0,σε²)
-equivalently:
Xt = (1 + βB) εt
=> 
Xt = εt + βXt-1 - β²Xt-2 + β³Xt-3 + ....
-if |β|>1 the distant past has more influence than the recent
-so for invertibility we need |β|<1

Question 23

Invertibility for MA(q) Processes

Answer 23

-an MA(q) process is invertible if all roots of β(y) are outside the unit circle
-where:
β(y) = 1 + β1y + β2y² + …