Chapter 3

Question 1

Markov process

Answer 1

Any process satisfying the Markov property

Question 2

Markov chain

Answer 2

A Markov process in discrete time and with a discrete state space.

Question 3

Irreducibility

Answer 3

A Markov chain is said to be irriducible if any state j can be reached from any other state i.

Question 4

Period of a state

Answer 4

State i is said to be periodic with period d > 1 if a return to i is possible only in a number of steps that is a multiple of d.

Question 5

Aperiod state

Answer 5

One that is not periodic.

Question 6

State 4 assumptions made by the Markov Jump Chain

Answer 6

The holding time in each state is exponentially distributed

The transition intensities from each state are not time-dependent.

The parameter of this distribution varies only by state i, so that the distribution is independent of anything that happened prior to the arrival in current state i.

The destination of the jump on leaving state i is independent of holding time, and of anything that happened prior to the current arrival in state i.