Introduction and Stationary Processes

Question 1

Trend

Word Definition

Answer 1

-the trend of a time series is a slow change in mean level

Question 2

Trend

Formula Definition

Answer 2

-consider the model:
Xt = μ(t) + εt
-where
->μ(t) is a deterministic trend as a function of t
->εt is the random fluctuation about the trend at time t, E(εt)=0

Question 3

Trend

Examples

Answer 3

μ(t) = α + βt
μ(t) = α + βexp(γt)
μ(t) = α + β log(t) ...

Question 4

Fitting a Linear Trend

Answer 4

• suppose we have observed values X1,…,Xn at times t1,…,tn and we believe a linear trend is appropriate
• we can estimate α and β by linear regression

Question 5

Trend and Summary Statistics

Answer 5

-if a trend is present in the data, the usual summary statistics (mean, variance etc.) will not be very useful

Question 6

Trend and Residuals

Answer 6

-after a trend is fitted, the residuals:
Yti = Xti - μ^/9ti) can be analysed further
-e.g by estimating Var(Yti)

Question 7

Trend and Forercasting

Answer 7

• we can predict future behaviour of X by considering μ^(t) for t>tn
• once we have analysed the residuals we can improve the forecast bu considering μ^(t) + Y^t for t>tn

Question 8

Trend and Forecast Error

Answer 8

-the forecast error depends on the variance of our estimates
-e.g. if εt are i.i.d. N(0,σ²) then:
Var(μ^(t)) =
σ² [1/n + (t-t_)²/Σ(ti-t_)²]
-sum from i=1 to i=n

Question 9

Seasonal Effects

Word Defintion

Answer 9

-seasonal effects are periodic components of a time series which repeat with a fixed frequency

Question 10

Seasonal Effects

Formula Definition

Answer 10

-revise the model to be:
Xt = μ(t) + s(t) + εt
-where
->μ(t) = trend
->s(t) is a seasonal effect with s(t)=s(t+a) ∀t where a is the period length
->εt = residuals/fluctuations

Question 11

Seasonal Effects

Finding s

Answer 11

-if we assume t=0,1,2,… and a=n∈ℕ
-to find s we have to estimate s1,s2,…,sn-1
-can write
Xt = μ(t) + s0δt,0 + … + sn-1δt,n-1 + εt
-where
δt,i = {1 if t is in season i, 0 else}
-parameters s0,…,sn-1 can be estimated using linear regression

Question 12

Steps of Time Series Analysis

Answer 12

• we start the analysis of a time series by identifying the trend (if any) in the model Xt=μ(t)+εt
• analyse the residuals Yt=Xt-μ^(t) further
• any results about the {Yt} need to be converted back by adding μ^(t)
• e.g. a forecast for Xt is given by the forecast for Yt^ + μ^(t)
• after the trend is removed, we removed the periodic components ( if any ) in the model Yt=s(t)+εt
• we then analyse Zt=Yt-s^(T) further
• convert the results about the {Zt} into results about {Yt} by adding s^

Question 13

Time Series as a Stochastic Process

Answer 13

• to get a model of a time series, we consider Xt to be random variables
• the collection {Xt} is called a stochastic process

Question 14

Lag

Definition

Answer 14

-time difference between two time points/observations

lag = t - s

Question 15

Stochastic Process

Mean

Answer 15

μ(t) = E(Xt)

Question 16

Stochastic Process

Variance

Answer 16

σ²(t) = Var(Xt)

Question 17

Stochastic Process

Auto-Covariance

Answer 17

γ(s,t) = cov(Xs,Xt) = E[(Xs-μ(s)) (Xt-μ(t))]

Question 18

Stochastic Process

Auto-Correlation

Answer 18

ρ(s,t) = cor(Xt,Xs) = γ(s,t) / √[σ²(s)σ²(t)]

Question 19

Stationarity

Definition

Answer 19

-the stochastic process {Xt} is stationary if {X1,…,Xn} and {X1+k,…,Xn+k} have the same distribution for all k,n∈N

Question 20

Stationarity

Results

Answer 20

• if {Xt} is stationary, then Xk has the same distribution as X1 for all k∈N, in particular if the first two moments are finite then μ(t) = E(Xt) = μ and σ²(t) = Var(Xt) = σ² for all t so a process with trend or seasonality cannot be stationary
• for stationary processes, sometimes it is useful to consider t∈Z instead of t∈N

Question 21

Weak Stationarity

Definition

Answer 21

• a stochastic process {Xt} is weakly stationary or second order stationary if:
  1) μ(t) is constant, i.e. μ(t) = μ for all t
  2) γ(s,t) only depends on the times t and s via the lag (t-s) i.e. γ(s,t)=γ(t-s)

Question 22

Weak Stationarity

Results (auto-covariance)

Answer 22

γo = cov(Xt,Xt) = var(Xt) = σ², for all t
γk = cov(Xt,Xt+k) = cov(Xt+k,Xt) = γ-k

Question 23

Are stationary processes weakly stationary?

Answer 23

-every stationary process is also weakly stationary since for all t:
μ(t) = E(Xt) = E(X1)
γ(t,t+k) = cov(Xt, Xt+k) = cov(X1,X1+k)
-i.e. both μ and γ are independent of t

Question 24

Weak Stationarity

Results (auto-correlation)

Answer 24

ρ(s,t) = γ(s,t)/√[σ²(s)σ²(t)] = γ(t-s) / √[γoγo] = γ(t-s)/γo = ρ(t-s)
ρ(t,t) = ρo = 1

Question 25

Autocorrelation Function (acf) and Correlogram
Definitions

Answer 25

• the quantity ρk is called the lag k autocorrelation; ρk as a function of k is the autocorrelation function (acf)
• a plot of ρk vs k is a correlogram
• strictly speaking it only makes sense to compute the acf for weakly stationary processes but it can be used to check if a process is stationary or not

Question 26

Estimation of Covariance

Answer 26

-normally we estimate covariance using:
cov(X,Y) = 1/n Σ (Xi-X_)(Yi-Y_)
-where the sum is from i=1 to n
-for a time series we typically have only one realisation X1,X2,…,Xn so can’t use this methof

Question 27

Estimation of μ^

Answer 27

-if {Xt} is weakly stationary
μ^ = 1/n Σ Xi ≈ μ(t)
-sum from i=1 to i=n

Question 28

Estimation of γ^k

Answer 28

-if {Xt} is weakly stationary
γ^k = 1/(n-k) Σ (Xi - μ^)(Xi+k - μ^) ≈ γk
-sum from i=1 to i=n

Question 29

Estimation of ρ^k

Answer 29

-if {Xt} is weakly stationary

ρ^k = γ^k/γ^o