Module 3

Question 1

What is module 3 all about

Answer 1

Continuous time markov chains.

Question 2

Stationary transition probabilities if

Answer 2

P[ X(t+s) = j | X(s) = i ] = P(t)
Ie X(t+s) is independent of s

Question 3

v,i is used to rep

Answer 3

the TRANSITION RATE out of i

Question 4

V,i is the parameter for what

Answer 4

The Exp(V) which models Ti, time spent in a state.

Question 5

q,ij is the

Answer 5

instant transition rate from i to j

Question 6

q,ij expressed differentially is

Answer 6

d/dt of P,ij(t)

Question 7

P,ij is the

Answer 7

prob that if we are in i, and a transition occurs, it will be into j

Question 8

q,ij in terms of P and v

Answer 8

q,ij=v,i P,ij

Ie prob of leaving, times prob of leaving to j

Question 9

v,i in terms of q,ij

Answer 9

∑q,ij. for i≠j Ie sum of all ways of moving out

Question 10

q,ii =

Answer 10

-v,i

Question 11

∑P,ij =

Answer 11

1 (its gotta go somewhere)

Question 12

For cts MCs, for limiting probs to exist..

Answer 12

All states must communicate.

MC is +ve recurrent

Question 13

Limiting prob P,j =

Answer 13

lim t -> inf

P,ij(t)

Question 14

The balance equation

Answer 14

v,j * P,j = ∑ q,kj * P,k
LHS is rate of leaving j times proportion of time spend in j.
RHS is rate of going of k->j, times proportion of time in k.

Question 15

Embedded MC is when we

Answer 15

Just look at the states the cts MC was in, so it becomes disc. time.

Question 16

Birth and death processes grow with rate

Answer 16

λ

Question 17

B and D processes shrink at rate

Answer 17

µ

Question 18

B and D process, v=

Answer 18

λ+µ

Question 19

B and D process, Pi,i+1

Answer 19

λ / (λ+µ)

Question 20

B and D process, Pi,i-1

Answer 20

µ/ (λ+µ)

Question 21

P(Tb < Td) next birth occurs before next death =

Answer 21

λ / (λ+µ)

Question 22

Capital Vi represents (B and D process)

Answer 22

Expexted time to get to i+1 from i