Learning with time series

Question 1

Problem definition

Answer 1

Given x(1), x(2), …, x(N) generated by a SSP x(t,s)
Compute
- the mean
- the covariance function
- the spectral density

Question 2

def correctness of a sample estimator

Answer 2

A sample estimator Q^N of a quantity Q is correct if

E[Q^N] = Q

Question 3

def consistency of a sample estimator

Answer 3

A sample estimator Q^N of a quantity Q is consistent if

E[(Q^N - E[Q^N])^2] - > 0 as N - > +inf

Question 4

Sample mean estimator

Answer 4

Given x(1), x(2), …, x(N) generated by a SSP x(t,s)
the mean m = E[x(t,s)] is estimated as
m^N = 1/N * sum(t=1,N) x(t)

This estimator is correct, while its consistency depends on the process

Question 5

Sample covariance function estimator

Answer 5

Given x(1), x(2), …, x(N) generated by a SSP x(t,s) with zero mean, the covariance function
γ(τ) = E[x(t) * x(t-τ)] , |τ| < +inf
is estimated as
γ^N(τ) = 1/(N-τ) * sum(t=1, N-τ) x(t) * x(t+τ) , 0 < = τ < = N-1

or

γ^N’(τ) = 1/(N-|τ|) * sum(t=1, N-|τ|) x(t) * x(t+τ) , |τ| < = N-1

This estimator is correct and consistent

Question 6

Sample spectral density

Answer 6

Given x(1), x(2), …, x(N) generated by a SSP x(t,s) with zero mean, the spectrum
Γ(ω)
is estimated as

Γ^N(ω) = sum(τ=-(N-1), (N-1)) γ^N(τ) * e^-jωτ

where two approximations are present, the first given by the sum that does not contain infinite terms, the second given by the estimate of the covariance function.

This estimator is asymptotically correct

Question 7

Problems of the sample spectral density

Answer 7

1) It is not consistent - > high variance

2) high computational effort, since the sample covariance function need to be computed first

Question 8

Solution to the computational problem

Answer 8

Computation via FFT

Γ^N’(ω) = sum(τ=-(N-1), (N-1)) γ^N’‘(τ) * e^-jωτ

where
γ^N’‘(τ) = 1/N * sum(t=1, N-|τ|) x(t) * x(t+τ) , |τ| < = N-1

• > it can be proven that
  Γ^N’(ω) = 1/N * | sum(t=1,N) x(t) * e^-jωt |^2
  that is the discrete FT of x(t), that can be computed efficiently by the FFT algorithm.

Question 9

Solution to the consistency problem

Answer 9

• Data-set is divided into 4 parts
• Γ^N/4(i) is the estimator of the spectrum using the i-th part of the data, i=1,2,3,4
• Γ^^N(ω) = 1/4 * sum(i=1,4) Γ^N/4(i)(ω)
• it can be proven that the variance of this estimator is reduced by a factor of 4

Question 10

Data pre-processing for time series: use

Answer 10

• The estimators are valid if data are taken from a SSP
• In case of trend and seasonality in the data, the process is not stationary

In these cases:
- estimate of trend and seasonality v^(t)
- removal of trend and seasonality
x^(t)ssp = x(t) - v^(t)
- estimations using the stationary SP
- add of trend and seasonality to the estimated mean

Assumption: x(t) = x(t)ssp + v(t)
where v(t) is a deterministic signal

Question 11

Trend estimation and removal

Answer 11

• Linear trend: v(t) = k*t + q
  k^ and q^ estimate of a linear regression problem
• > x^(t)ssp = x(t) - k^*t - q^
• Same approach can be extended to polynomial trends, considering:
  v(t) = kt + q + at^2 + b*t^3 + …

Question 12

Seasonality estimation and removal

Answer 12

x(t) = x(t)ssp + s(t)
where
s(t) = s(t+kT), with k app Z and T known period of the seasonality

s^(t) = 1/M * sum(h=0,M-1) x(t+hT)
with t = 1,2,…,T
and MT < = N

• > x^(t)ssp = x(t) - s^(t)