Introduction and Stationary Processes Flashcards
fitting a trend, seasonal effects, stationary processes
Trend
Word Definition
-the trend of a time series is a slow change in mean level
Trend
Formula Definition
-consider the model:
Xt = μ(t) + εt
-where
->μ(t) is a deterministic trend as a function of t
->εt is the random fluctuation about the trend at time t, E(εt)=0
Trend
Examples
μ(t) = α + βt μ(t) = α + βexp(γt) μ(t) = α + β log(t) ...
Fitting a Linear Trend
- suppose we have observed values X1,…,Xn at times t1,…,tn and we believe a linear trend is appropriate
- we can estimate α and β by linear regression
Trend and Summary Statistics
-if a trend is present in the data, the usual summary statistics (mean, variance etc.) will not be very useful
Trend and Residuals
-after a trend is fitted, the residuals:
Yti = Xti - μ^/9ti) can be analysed further
-e.g by estimating Var(Yti)
Trend and Forercasting
- we can predict future behaviour of X by considering μ^(t) for t>tn
- once we have analysed the residuals we can improve the forecast bu considering μ^(t) + Y^t for t>tn
Trend and Forecast Error
-the forecast error depends on the variance of our estimates
-e.g. if εt are i.i.d. N(0,σ²) then:
Var(μ^(t)) =
σ² [1/n + (t-t_)²/Σ(ti-t_)²]
-sum from i=1 to i=n
Seasonal Effects
Word Defintion
-seasonal effects are periodic components of a time series which repeat with a fixed frequency
Seasonal Effects
Formula Definition
-revise the model to be: Xt = μ(t) + s(t) + εt -where ->μ(t) = trend ->s(t) is a seasonal effect with s(t)=s(t+a) ∀t where a is the period length ->εt = residuals/fluctuations
Seasonal Effects
Finding s
-if we assume t=0,1,2,… and a=n∈ℕ
-to find s we have to estimate s1,s2,…,sn-1
-can write
Xt = μ(t) + s0δt,0 + … + sn-1δt,n-1 + εt
-where
δt,i = {1 if t is in season i, 0 else}
-parameters s0,…,sn-1 can be estimated using linear regression
Steps of Time Series Analysis
- we start the analysis of a time series by identifying the trend (if any) in the model Xt=μ(t)+εt
- analyse the residuals Yt=Xt-μ^(t) further
- any results about the {Yt} need to be converted back by adding μ^(t)
- e.g. a forecast for Xt is given by the forecast for Yt^ + μ^(t)
- after the trend is removed, we removed the periodic components ( if any ) in the model Yt=s(t)+εt
- we then analyse Zt=Yt-s^(T) further
- convert the results about the {Zt} into results about {Yt} by adding s^
Time Series as a Stochastic Process
- to get a model of a time series, we consider Xt to be random variables
- the collection {Xt} is called a stochastic process
Lag
Definition
-time difference between two time points/observations
lag = t - s
Stochastic Process
Mean
μ(t) = E(Xt)
Stochastic Process
Variance
σ²(t) = Var(Xt)
Stochastic Process
Auto-Covariance
γ(s,t) = cov(Xs,Xt) = E[(Xs-μ(s)) (Xt-μ(t))]
Stochastic Process
Auto-Correlation
ρ(s,t) = cor(Xt,Xs) = γ(s,t) / √[σ²(s)σ²(t)]
Stationarity
Definition
-the stochastic process {Xt} is stationary if {X1,…,Xn} and {X1+k,…,Xn+k} have the same distribution for all k,n∈N
Stationarity
Results
- if {Xt} is stationary, then Xk has the same distribution as X1 for all k∈N, in particular if the first two moments are finite then μ(t) = E(Xt) = μ and σ²(t) = Var(Xt) = σ² for all t so a process with trend or seasonality cannot be stationary
- for stationary processes, sometimes it is useful to consider t∈Z instead of t∈N
Weak Stationarity
Definition
- a stochastic process {Xt} is weakly stationary or second order stationary if:
1) μ(t) is constant, i.e. μ(t) = μ for all t
2) γ(s,t) only depends on the times t and s via the lag (t-s) i.e. γ(s,t)=γ(t-s)
Weak Stationarity
Results (auto-covariance)
γo = cov(Xt,Xt) = var(Xt) = σ², for all t γk = cov(Xt,Xt+k) = cov(Xt+k,Xt) = γ-k
Are stationary processes weakly stationary?
-every stationary process is also weakly stationary since for all t:
μ(t) = E(Xt) = E(X1)
γ(t,t+k) = cov(Xt, Xt+k) = cov(X1,X1+k)
-i.e. both μ and γ are independent of t
Weak Stationarity
Results (auto-correlation)
ρ(s,t) = γ(s,t)/√[σ²(s)σ²(t)] = γ(t-s) / √[γoγo] = γ(t-s)/γo = ρ(t-s) ρ(t,t) = ρo = 1
Autocorrelation Function (acf) and Correlogram Definitions
- the quantity ρk is called the lag k autocorrelation; ρk as a function of k is the autocorrelation function (acf)
- a plot of ρk vs k is a correlogram
- strictly speaking it only makes sense to compute the acf for weakly stationary processes but it can be used to check if a process is stationary or not
Estimation of Covariance
-normally we estimate covariance using:
cov(X,Y) = 1/n Σ (Xi-X_)(Yi-Y_)
-where the sum is from i=1 to n
-for a time series we typically have only one realisation X1,X2,…,Xn so can’t use this methof
Estimation of μ^
-if {Xt} is weakly stationary
μ^ = 1/n Σ Xi ≈ μ(t)
-sum from i=1 to i=n
Estimation of γ^k
-if {Xt} is weakly stationary
γ^k = 1/(n-k) Σ (Xi - μ^)(Xi+k - μ^) ≈ γk
-sum from i=1 to i=n
Estimation of ρ^k
-if {Xt} is weakly stationary
ρ^k = γ^k/γ^o
