Chapter 5 Flashcards
Define a countable set
In mathematics, a set is countable if either it is finite or it can be made in one to one correspondence with the set of natural numbers
When is a process a countable markov chain?
- it satisfies the Markov property,
- the state space X is countable: usually
N = {0, 1, 2, 3, . . . }, sometimes we will consider
Z
What does it mean that we consider homogeneous chains?
Time homogeneous, means probability that Xn =j given X n-1 = i is p(i,j) meaning the time is not important to the probabilities.
What is a transient class
if every state is visited only a finite number of
times
What is a recurrent class
if every state is visited an infinite number of
times
What does an irreducible chain mean
A Markov chain in which every state can be reached from every other state is called an irreducible Markov chain
How can we deduce that an irreducible markov chain is transient
An irreducible Markov chain is transient if and only if the expected number of returns to a state is finite
When there is a limit π(j) = limn→∞ pn(i, j) that does not
depend on i what are the two possible cases
The stationary distirbution is zero - the chain is null recurrent
OR
The sum of all Pi(j) is 1 forming a stationary distirbution and the chain is called positive recrruent
What does Null recurrent and Positive recurrent mean
Null - no stationary distribution
Positive - Stationary Distribution
Meaning pf Pn(I,j)
Probability of in n steps going from state i to state j
For what values of p does an asymmetric random walk in N with reflecting boundary (cause its countable only one boundary) have a stationary distribution?
When p > 1/2 it is possible to find α, β such that π
is a probability measure. Otherwise, it is not possible.
In branching processes define Xt
Xt is the number of individuals in the population
What are the assumptions of the branching process
Individuals reproduce independently and the probability of having h offspring is Ph
In the branching processes define Yi
Number of offspring produced by an individual
What does Pk Denote
Probability Y=K
In the branching process what is the state space
Natural numbers
In the branching process what is Probability Xt goes from state 0 to state 0
1 state 0 is abosorbing as it denotes extinction
What does mew denote in branching process
Expectation of Y - expected number of offspring for an individual
What is the definition of A(K)
The probability of extinction a(k) is the probability that the
population dies out starting from X0 = k
Under what conditions of mew does the population die out
If mew is less than 1 the population will die out. If mew is more than one we have to work out the probability that eventually the population will die out as it is possible but more unlikely.
How do we find a
Let πY (a) = E(a^Y ), then the probability of extinction a(1) of
the chain is the smallest positive root of the equation
πY (a) = a. Also A=1 Is always a root.
What is the condition we a=E[a^Y]
Mew greater than 1
Define a Markov process
Continuous time and discrete state space (finite or countable).
We say that {Xt , t ∈ R+} is a Markov process if
P(Xt+s = j|Xr , 0 ≤ r ≤ t) = P(Xt+s = j|Xt ).
When this probability does not depend on t, we say that the
process is homogeneous.
Why is a poisson process a markov process?
Let {Xt } be a Poisson process with intensity λ. Then we know
that: Xt+s − Xt is independent on the past values Xr for r ≤ s
and that
Xt+s − Xt ∼ P(λs)
