Module 1

Question 1

T represents

Answer 1

the index set. Normally time

Question 2

X(t) is the

Answer 2

state at time t

Question 3

S represents the

Answer 3

State space

Question 4

Sample realisation means

Answer 4

one particular assignment of possible values

Question 5

Independent increments means

Answer 5

X(A+b) - X(b) is independant of X(s) where s<=b.

Basically the next increment is independent on where we are or where we’ve been.

Question 6

Stationary Increments

Answer 6

X(t2 + b) - X(t1 +b) has the same distribution as X(t2) - X(t1)

Basically the increments in all common time gaps will be identically distributed.

Question 7

Markov Property means

Answer 7

Only the most recent state has an influence on the present.

Question 8

Markov chain is when…

Answer 8

We have a markov process with discrete index set.

Question 9

Homogenious markov chain means…

Answer 9

Transition probabilities (and matrix) remain constant.

Question 10

Absorbing state

Answer 10

We’re trapped. We will come back to the current state next step with prob =1

Question 11

Accessable

Answer 11

When we can get from i to j. Ie transition prob is >0

Question 12

Communicate

Answer 12

if we can go i j back and forth

Question 13

Irreducible matrix meaning

Answer 13

MC has only one class. All states communicate.

Question 14

Recurrent class

Answer 14

We will return eventually with prob =1

return infinite times

Question 15

Transient class/state

Answer 15

We might return eventually, but with prob < 1

return limited # times

Question 16

Period =1 also called

Answer 16

Aperiodic

Question 17

Positive reccurance if;

Answer 17

1. State i is recurrent

2. Expected time to return i (after starting there) is FINITE.

Question 18

Ergodic =

Answer 18

Positive recurrent + Aperiodic

Question 19

Limiting probabilties only relevant for

Answer 19

Irreducable and ergodic MCs.

Question 20

Limiting prob is

Answer 20

Limit as n goes to infinity of;

Prob of going from i to j after n steps.

Question 21

Stationary probability =

Answer 21

P(Xn = j) = π(j) for all n. Ie chance of being in state j is constant for all n.

Question 22

Mean time between visits =

Answer 22

m = 1 / π(j)

Question 23

Period, d(i)

Answer 23

Greatest common divisor of the n’s that satisfy:

Pr(going from i to i in n steps) > 0

Question 24

Stationary prob is basically

Answer 24

long run proportion spend in a state.

Question 25

To find stationary probs…

Answer 25

Solve π=πP

Question 26

To find mean time in transient states

Answer 26

Solve S=(I-Pt)^-1