Exam Definitions

Question 1

Define the term generator matrix for a time-homogeneous, continuous-time Markov chain in terms of transition probability functions

Answer 1

G = (gij), i,j∈S where gij = p’ij (t) | t=0

Question 2

Define the term irreducible Markov chain

Answer 2

The full state space of the Markov chain is a communication class

Question 3

Define the term positive recurrent Markov chain

Answer 3

Every state of a Markov chain is positive recurrent. A state is called positive recurrent if its ever-visiting probability is
one and its mean visiting-time is finite

Question 4

Define the term aperiodic Markov chain

Answer 4

Every state of the chain is aperiodic. A state i is
called aperiodic if Ai = {n : pii^n > 0} has the greatest common divisor 1, where pii^n is the n-step transition probability