Markov Chains Flashcards
Discrete stochastic process
Set of random variables
Characterise the state of a system
Discrete points in time
Markovian property
Memoryless property
Past doesnβt matter
Completely ergodic
limit exists AND rows are identical
Unichain and Aperiodic
Steady State exists
States Accessible?
probability going from one to another is greater than 0
States communicate?
State i accessible from state j and state j accessible from state i
In the same class
State reflexive?
State communicates with itself
State symmetric?
If state i communicates with state j, then state j communicates with state i
State transitive?
if state i communicates with state j, and state j communicates with state k, then state i communicates with state k
Transient state
probability of returning to a state once it has left is <1 (or may never return)
Recurrent state
if and only if the state is not transient
Definitely return
Absorbing state
Special type of recurrent
Probability of being at a state is 1
Wont leave
Unichain
Markov chain consisting of a single recurrent class Transient classes transition into this class
Multichain
More than one recurrent class
Periodic State
t steps to return > 1
Aperiodic State
t = o or cannot be found
i.e recurrent state with t = 1 or πππ > 0
