Module 3 Flashcards
What is module 3 all about
Continuous time markov chains.
Stationary transition probabilities if
P[ X(t+s) = j | X(s) = i ] = P(t) Ie X(t+s) is independent of s
v,i is used to rep
the TRANSITION RATE out of i
V,i is the parameter for what
The Exp(V) which models Ti, time spent in a state.
q,ij is the
instant transition rate from i to j
q,ij expressed differentially is
d/dt of P,ij(t)
P,ij is the
prob that if we are in i, and a transition occurs, it will be into j
q,ij in terms of P and v
q,ij=v,i P,ij
Ie prob of leaving, times prob of leaving to j
v,i in terms of q,ij
∑q,ij. for i≠j Ie sum of all ways of moving out
q,ii =
-v,i
∑P,ij =
1 (its gotta go somewhere)
For cts MCs, for limiting probs to exist..
All states must communicate.
MC is +ve recurrent
Limiting prob P,j =
lim t -> inf
P,ij(t)
The balance equation
v,j * P,j = ∑ q,kj * P,k
LHS is rate of leaving j times proportion of time spend in j.
RHS is rate of going of k->j, times proportion of time in k.
Embedded MC is when we
Just look at the states the cts MC was in, so it becomes disc. time.
Birth and death processes grow with rate
λ
B and D processes shrink at rate
µ
B and D process, v=
λ+µ
B and D process, Pi,i+1
λ / (λ+µ)
B and D process, Pi,i-1
µ/ (λ+µ)
P(Tb < Td) next birth occurs before next death =
λ / (λ+µ)
Capital Vi represents (B and D process)
Expexted time to get to i+1 from i
