L2 - Stationary Processes

Question 1

What is a stochastic process?

Answer 1

• Theta helps us decide whether the process is stationary or not (for an autoregressive process)

Question 2

What are the properties of stochastic processes and how can we re-write an AR process?

Answer 2

Question 3

What is the expected value and variance of a stochastic/AR process?

Answer 3

• WE assume the distribution of the error-term –> To give us the gaussian properties
  
  • Expected value
    
    • As the expected value of all error terms is 0, the expected value of x is 0
  • Variances
    
    • Dont include the E(X) term as it is 0
    • Variance second term –> covariance of two error terms is zero
    • as σ_E² is a constant we can take it outside the summation. As theta is between -1 and 1 the sum will converge. So we can use the sum of an infinite series to get (a/1-θ²) where a is the constant of theta (which is 1)
      
      • Giving us the answer we have
    • Why do we get θ² on the bottom–>
      
      • infinite sum series is –>(1/1-ratio) where ratio is θ²⁽ⁱ⁺¹⁾/θ²ⁱ which leads you to θ²

Question 4

What is the Covariance of a stochastic/AR process?

Answer 4

• Cov(x,y) = E(XY) - E(X)E(Y)
  
  • As E(x_t) for all value of t is 0, we just look at the expected value of E(x_t,x_t-1)
• We are left with only the squares as the expected value of any cross products of the error is equal to 0
• Dividing through by theta in the 4th line gives use the equation for the covariance between x and its most recent past value.

Question 5

What are the special notations we need to remember about stochastic/AR processes?

Answer 5

Question 6

Example of first-order autoregression processes?

Answer 6

• These are the Autocorrelation (AC) functions
  
  • Each step i, is 0.7ⁱ

Question 7

What is a general definition of Stationarity?

Answer 7

• Not possible to test for ergodicity
  
  • Cant wait 1000 - infinite years to test it –>have to rely on weaker properties of stationarity
  • imagine it like the number of observations in a sample
    
    • as with CLT you need a large sample set to determine processes moments but in the case of time series data we are limited by the dimension of time

Question 8

What is a Strictly Stationary Stochastic process?

Answer 8

DIFFERENCE BETWEEN STRICTLY AND WEAKLY STATIONARY –> this bottom part is weakly stationary??

• Think of time like an infinitely long lock and at each point of time there is an infinitely long combination of outcomes but only one realised over time
• m –> moment
• Covariances are a function of the time-shift or lag k only
  
  • so the distance between two points (or lock digits) can affect the covariance of each realisation but that is NOT affected by starting at a future time but with the same distance between two points.
• ACF –> the length and strength of the processes ‘memory’

Question 9

What is Weak or Covariance Stationary Processes?

Answer 9

• If the error term follows a normal distribution we can say that weak stationarity is equivalent to strict stationarity
  
  • None of the moments depend on time!

Question 10

What is a weakly dependent time series?

Answer 10

Question 11

What is the Wold Decomposition?

Answer 11

• Mu = deterministic and psi = stochastic

Question 12

How can we represent a first-order autoregressive process AR(1)?

Answer 12

• (1-θL)^-1
  
  • We know that 1/1-R where R is a ratio that corresponds to the sum of a geometric series
  • That is why we can convert it into 1 + θL + θ²L²….
    *

Question 13

How do you calculate the Sample autocorrelation coefficient?

Answer 13

• Still, the sample covariance divided by the sample variance
  *

Question 14

What are partial autocorrelations?

Answer 14

• can find the autocorrelation between x_t and some lag k of x_t without losing the effect of all lags in the middle

Question 15

Example of generating Partial Autocorrelations?

Answer 15

• Gamma = Autocovariances
  
  • Gamma(0) –> Variance
• rho –> auto correlations
• theta –> partial auto correlation?

Question 16

What are the Yule-Walker Equations?

Answer 16

• MM –> Method of moments

Question 17

Examples of ACF and PACF?

Answer 17

• inside confidence bands –> cannot reject the null the autocorrelations are equally to 0 - rewrite this
  
  • AR(2)
    
    • Exponential decay in ACF
      PACF 2 significant PAC

Question 18

Example of estimating PAC from Yule-Walker equations

Answer 18

• End numbers are different as PAC accounts for lag effect

Question 19

What is the Q-statistics for Partial Autocorrelations?

Answer 19

Question 20

What does a MA(1) process look like and what are its respective moments?

Answer 20

• V(X) –> end up with the sum of the variances and the covariances of any white noise is 0
• MA series is stationary because E(X),V(X) and covariance don’t depend on time
• Weakly dependent –> autocorrelation of order 1 is different to zero, but quickly moves to 0 after the first lag so yes it is!

Question 21

What do the Yule-Walker equations look like for MA(1) processes?

Answer 21

• First-order MA –> have partial autocorrelation that do no cut off as fast as AR(1) processes, but their AUTOCORRELATION cut off
  
  • AC cuts off at the point of identification of the order of the equations

Question 22

Example of MA Process?

Answer 22

• AC cuts off after 1 lag
• +VE coefficient on the lag of error
  
  • Oscillating PAC
• -VE coefficient on lag of error
  
  • Exponential decline in PAC

Question 23

How do you move from a MA(1) to an infinite autoregressive process?

Answer 23

Question 24

What is the general formula and characteristics of an AR(p)?

Answer 24

• Theta being between -1 and 1 is no longer the stationarity condition –> need to look at the roots of the characteristic equation
  
  • Either the roots are less than 1 in absolute value or the solution to the characteristic equals are all greater than 1 in absolute value

Question 25

What is the general formula and characteristics of an MA(q)?

Answer 25

* moving average function is always stationary by definition * E(X), VAR(X) and COV(X) are all not a function of time? * complex roots work the same was as AR roots

Question 26

Examples of AR(p) and MA(q) correlograms?

Answer 26

* AR(2) * AC ==\> exponential decrease * PAC ==\> two clear PACs that are statistically significant * MA(2) * Ac ==\> 2 statistically clear autocorrelations * PAC ==\> smooth decease in oscillating function

Question 27

Example of using the characteristic and invertibility conditions for AR(p) and MA(q) processes?

Answer 27

* statiationarity condition for finding roots of AR(p) * sub in for L * get it into the form L - solutions for each bracket --\> as the equation needs to equal zero * rearrange until we get it into the characteristic equation form --\> coefficient on L will therefore be the roots * **EQUALLY, YOU CAN JUST SUB IN FOR L AND FIND THE SOLUTION WHICH IS A HELL OF A LOT EASIER!** * MA(q) process is always stationary so just need to check if we can invert it into an infinite order auto regressive process

Question 28

Example of MA(2) process with real and complex roots?

Answer 28

* Complex root * AC ==\> 2 significant autocorrelations (MA(2)) * PAC ==\> Smooth decrease in PAC (potentially a sine wave that's oscillating) --\> MA(2) * Real roots * AC ==\> two significant autocorrelations * PAC ==\> smooth exponential decrease in the lags

Question 29

What is the equations and characteristics of an ARMA(1,1) process?

Answer 29

* can change to an MA(∞) process by isolating x_t on one side * Can only do this if theta is lower than one in absolute value * if invertible we can create an AR(∞) process by isolating the error term ε_t on one side * For each case, they can be converted to an infinite sum of a geometric series in a similar way to before (watch out for the coefficients!!) * Although MA processes are always stationary we cannot able this directly to ARMA processes due to the AR component

Question 30

Example of an ARMA(1,1) and AR(1) process correlograms?

Answer 30

Not always clear what the ARMA process will be * ARMA(1,1) * AC ==\> sharp decrease after 1st autocorrelation then smooth decline (AR(1))? * PAC ==\> substantial decrease from 1st PAC to 2nd and 3rd but decreases in an oscillating sine wave shape similar to a MA process (MA(1)) * AR(1) * AC ==\> smooth decrease in the ACF * PAC ==\> sharp cut off after first PAC

Question 31

What are the equations and characteristics of an ARMA(p,q) process?

Answer 31

* characteristic equation * Get all the x's on one side and the errors on the other ==\> just like you would for for AR(p) and MA(q) processes

Question 32

Summary of the rules we can use to identify the data generating process from is ACF and PACF?

Answer 32

Question 33

What happens to the ARMA(p,q) process if its mean is not assumed to be 0?

Answer 33

* still deal with the process in the same fashion as we would have done before

Question 34

What is the Box-Jenkins approach to fitting ARMA models?

Answer 34

1. Identification 1. Examine the sample correlogram to try to figure out the data generating process 2. estimation 1. Try to fit a model to the data 3. diagnostic check 1. examine correlogram of the residuals 1. Is the coefficient on the lag significant? 1. Should have no significant AC or PAC 2. If it does, it may mean that there is more to our original model 4. choosing between alternative specifications 1. Box-Jenkins philosophy --\> select the most parsimonious model i.e. the one with the fewest parameters

Question 35

Examples of correlograms from real-world time series?

Answer 35

When fitting an ARMA to data we try to match the sample correlogram to the correlogram of a known theoretical ARMA process --\> important part of the estimation process of the Box -Jenkins approach 1. AR(1) 1. AC ==\> exponential decline 2. PAC ==\> 1 significant 2. Unit Root 1. AC ==\> small decrease/ linear decline in AR 2. PAC ==\> 1 significant 3. AR(2) w/ real roots 1. AC ==\> slow decline of ACF 2. PAC ==\> 2 significant 4. AR(2) 2/ complex roots 1. AC ==\> gradual decline in ACF but as a sine wave 2. PAC ==\> 2 significant

Question 36

What else can we use alongside the Box-Jenkins approach to help us decide on the specification of the data generation process?

Answer 36

* Always pick the specification that returns the lowest AIC and BIC * Then pick the one that is the most parsimonious (with preference to the model selected under the BIC criterion)

Question 37

What does an impulse response look like for an ARMA process?

Answer 37

* Can understand the impulse response of a shock in the error term for a AR(1) process if we transform it into an infinite MA process * IS the opposite for an MA process? * after the first shock the value of x_t is just the value of p^t * The left graph shows how the impact on x decays overtime but the right one shows how the affect accumulate on each other

Question 38

How can we forecast one period ahead with AR(I)MA processes?

Answer 38

Question 39

How can we forecast k periods ahead with AR(I)MA processes?

Answer 39

* Variance is higher for the k periods model over the one period one as it is obviously harder to accurately forecast a value in 10 years rather than just one year * From the variance, we can calculate confidence intervals for our predicted variable (where see = standard error = standard deviation).

Question 40

Example of forecasting using an AR(1) process?

Answer 40

* blue line is calculated from the last know x value we have in our sample

Question 41

How can we assess the performance of a forecasting model?

Answer 41

Question 42

What are the different types of forecasts we can do?

Answer 42

* Static ==\> input the most recent value observed (usually used for one period) * Dynamic ==\> use our forecasted values to predict future values