Markov Chains in Continuous Time Flashcards
Give the definition of a Markov chain in continuous time
Let (X(t) : t ∈ [o, infinity ) ) be a continuous time stochastic process with countable state space S.
It is called a continuous time Markov chain if it is a Markov process, i.e satisfies the Markov property:
P(X(tn) = x | X(t1) = x1,…, X(t_n-1) = X_(n-1)) = P(X(tn) = x | X(t_n-1) = X_(n-1))
Give the Chapman-Kolmogorov equations in continuous time
P(t) = P(u) * P(t-u)
Define a poisson process
- for any time points t0 = 0 < t1 0, the increment X(t + Δt) - X(t) has the poisson distribution with parameter λΔt
- X(0) = 0
Define a pure jump process
A process Y = (Y(t) : t ∈ R) is continuous time with Y(t) = Xn for Tn ≤ t < T_(n+1) is called a pure jump process
Give the n-th holding time of a process Y
Hn = T_(n+1) - Tn
How do you retrieve the transition probability matrix of the embedded Markov chain X from the generator matrix for Y?
λi = - gii pij = gij / λi pii = 0 by assumption
What should the row sum of a generator matrix equal?
zero
Give the Kolmogorov forward equations
sum(k ∈ S) pik(t) * gkj
Give the Kolmogorov backward equations
sum(k ∈ S) gik * pkj(t)
When is Y positive recurrent and irreducible?
When the embedded Markov chain X is
How do you find the stationary distribution for Y?
πG = 0
Write the balance equations for Y
sum(i≠j) πi * gij = -πj * gjj
When does Y have a unique stationary distribution?
When X is irreducible and positive recurrent
Give the load (traffic intensity) of a M/M/c system
ρ = λ / μ
When does an M/M/1 queue have a stationary distribution?
λ < μ
