Chapter 5

Question 1

Define a countable set

Answer 1

In mathematics, a set is countable if either it is finite or it can be made in one to one correspondence with the set of natural numbers

Question 2

When is a process a countable markov chain?

Answer 2

1. it satisfies the Markov property,
2. the state space X is countable: usually
   N = {0, 1, 2, 3, . . . }, sometimes we will consider
   Z

Question 3

What does it mean that we consider homogeneous chains?

Answer 3

Time homogeneous, means probability that Xn =j given X n-1 = i is p(i,j) meaning the time is not important to the probabilities.

Question 4

What is a transient class

Answer 4

if every state is visited only a finite number of
times

Question 5

What is a recurrent class

Answer 5

if every state is visited an infinite number of
times

Question 6

What does an irreducible chain mean

Answer 6

A Markov chain in which every state can be reached from every other state is called an irreducible Markov chain

Question 7

How can we deduce that an irreducible markov chain is transient

Answer 7

An irreducible Markov chain is transient if and only if the expected number of returns to a state is finite

Question 8

When there is a limit π(j) = limn→∞ pn(i, j) that does not
depend on i what are the two possible cases

Answer 8

The stationary distirbution is zero - the chain is null recurrent

OR

The sum of all Pi(j) is 1 forming a stationary distirbution and the chain is called positive recrruent

Question 9

What does Null recurrent and Positive recurrent mean

Answer 9

Null - no stationary distribution
Positive - Stationary Distribution

Question 10

Meaning pf Pn(I,j)

Answer 10

Probability of in n steps going from state i to state j

Question 11

For what values of p does an asymmetric random walk in N with reflecting boundary (cause its countable only one boundary) have a stationary distribution?

Answer 11

When p > 1/2 it is possible to find α, β such that π
is a probability measure. Otherwise, it is not possible.

Question 12

In branching processes define Xt

Answer 12

Xt is the number of individuals in the population

Question 13

What are the assumptions of the branching process

Answer 13

Individuals reproduce independently and the probability of having h offspring is Ph

Question 14

In the branching processes define Yi

Answer 14

Number of offspring produced by an individual

Question 15

What does Pk Denote

Answer 15

Probability Y=K

Question 16

In the branching process what is the state space

Answer 16

Natural numbers

Question 17

In the branching process what is Probability Xt goes from state 0 to state 0

Answer 17

1 state 0 is abosorbing as it denotes extinction

Question 18

What does mew denote in branching process

Answer 18

Expectation of Y - expected number of offspring for an individual

Question 19

What is the definition of A(K)

Answer 19

The probability of extinction a(k) is the probability that the
population dies out starting from X0 = k

Question 20

Under what conditions of mew does the population die out

Answer 20

If mew is less than 1 the population will die out. If mew is more than one we have to work out the probability that eventually the population will die out as it is possible but more unlikely.

Question 21

How do we find a

Answer 21

Let πY (a) = E(a^Y ), then the probability of extinction a(1) of
the chain is the smallest positive root of the equation
πY (a) = a. Also A=1 Is always a root.

Question 22

What is the condition we a=E[a^Y]

Answer 22

Mew greater than 1

Question 23

Define a Markov process

Answer 23

Continuous time and discrete state space (finite or countable).

We say that {Xt , t ∈ R+} is a Markov process if
P(Xt+s = j|Xr , 0 ≤ r ≤ t) = P(Xt+s = j|Xt ).
When this probability does not depend on t, we say that the
process is homogeneous.

Question 24

Why is a poisson process a markov process?

Answer 24

Let {Xt } be a Poisson process with intensity λ. Then we know
that: Xt+s − Xt is independent on the past values Xr for r ≤ s
and that
Xt+s − Xt ∼ P(λs)

Question 25

What are the row sums in the infinitesimal generator function

Answer 25

0

Question 26

How do you interpret the N X N A matrix

Answer 26

α(i, j) denotes rate at which process jumps from state i to state j
α(i) denotes the overall rate at which the process jumps form state i to any other state

Question 27

Given given X0 = i, what can one say about the time spent in time i before the first
jump T to another state

Answer 27

Its exponentially distributed: Exp(α(i)),

Question 28

What the expected value of time spent before the first jump of states in poisson process

Answer 28

1/alpha(i)

Question 29

given X0 = i, the probability that the next state is j not equal to i

Answer 29

P(XT = j) = α(i, j)/ α(i)

Question 30

What is T representing in poisson process

Answer 30

Time before the first jump

Question 31

Does a poisson process have an invariant distribution

Answer 31

No - the process infinitely grows

Question 32

Define lamda n and mew n in the birth and death rate processes

Answer 32

lamda n is the birth rate (rate at which population goes from n to n+1)
Mew n is the death rate (rate at which population goes from n to n-1)

Question 33

Define A matrix for a birth and death rate process

Answer 33

We assume that the infinitesimal generator A of {Xt } is given
by α(n, n + 1) = λn, α(n, n − 1) = μn so α(n) = λn + μn

Question 34

Define the poisson process in terms of births and deaths

Answer 34

it is a birth and death process with λn = λ
and μn = 0

Question 35

Define the population model

Answer 35

t is the size of the population at time
t. Each individual produces a new individual at rate λ and dies at rate μ. Then {Yt } is a birth and death process with rates
λn = nλ and μn = nμ

Question 36

How do we test if a birth and death process has an invariant distirbution

Answer 36

If q is finite then the birth and death process with intensities lamda n and mew n has an invariant distribution

Question 37

For a birth and death process with constant birth and death rate under what condition does the stationary distribution exist?

Answer 37

So the limiting distribution π exists only when λ < μ