14 stochastic processes in continuous time Flashcards
Stochastic processes in continuous time:intro
we consider two simple continuous-time stochastic (Markov) processes: first the simple death process, and then the simple birth process. These will pave the way to the
study of continuous-time birth-and-death processes in the next chapter.
The SIMPLE death process: assumptions
“linear death process”
some probabilities
S(t)
The death process is a Markov process in continuous time in which the number of individuals in a population can only decrease in time.
Assume: initially N cells in the pop
cells may die only, new ones not made
Sample paths of death process Xᵢ decreases with t step pattern
each cell independently has a constant death rateµ > 0
if a cell is alive at time t, probability it dies in the
interval (t, t + ∆t) is µ∆t as ∆t → 0.
probability tat the cell survives in the interval is 1 − µ∆t
S(t)~ P(a cell, alive at t = 0, survives to time t)
then
S(t + ∆t) = S(t)(1 − µ∆t),
[S(t + ∆t) -S(t)]/∆t = −µS(t), with S(0) = 1
which in the continuous time limit∆t → 0 gives
**dS(t)/dt = −µS(t), with S(0) = 1 **
yielding S(t) = exp(−µt)
the time that the cell dies is a continuous RV with exponential distribution an mean 1/µ
The death process:
probability distribution
pₖ(t)
Xₜ ~#cells alive at time t
probability distribution {pₖ(t)}ₖ₌₀ ᴺ
pₖ(t) = P{Xₜ=k}
X₀ =N
p_N(0)=1 and p_{k>N}(0)=0 cells can only die and ~never exceed N
ALways reduction in #: sample paths like steps on stairs
The death process:
SIMPLE DEATH PROCESS
deriving ODE
each cell independently constant death rate µ > 0
*If a cell is alive at time t the probability that it dies in the interval (t, t + ∆t) is µ∆t as ∆t → 0
*probability that the cell survives in the interval (t, t + ∆t) is 1 − µ∆t
*S(t) probabilty a cell alive at t=0 survives to time t then S(t + ∆t) = S(t)(1 − µ∆t)
**
or (S(t + ∆t) − S(t))/∆t
= −µS(t), with S(0) = 1
**
which in the continuous-time limit ∆t → 0 gives
dS(t)/dt = −µS(t) with S(0) = 1, yielding S(t) = exp(−µt)
** time that the cell dies is a continuous random variable with an exponential distribution and mean 1/µ**
The death process:
looking at whole population of cells
probability that n cells survive up to time t
p_N (t) = exp(−Nµt)
comes from n products
p_1(t) = P {one cell survives to time t} = S(t) = exp(−µt)
P{1 cell is dead at time t} = 1 − S(t)
= 1 − p1(t) = 1 − exp(−µt)
The death process:
looking at whole population of cells
With X_0 = N, the probability that one cell has died and N − 1 are still alive at time t is
pN−1(t) =
Nexp−(N−1)µt) (1 − exp(−µt))
N ways to pick one cell for death
The death process:
looking at whole population of cells
With X_0 = N, the probability that one cell has died and N − 2 are still alive at time t is
p_{N−2}(t) =
N(N − 1)/2
** exp(−(N−2)µt)** (1 − exp(−µt)^2
The death process:
looking at whole population of cells
Assuming that X_0 = N, the probability that j ≤ N cells have died and N − j are
still alive at time t is
pN−j (t) = N!/j!(N − j)! x
exp(−(N−j)µt) (1 − exp(−µt))^j
We can say that the number of cells alive at time t has a binomial distribution Xt ∼Bin(N, q) with parameters N and q ≡ e−µt,
clearly probability conservation:
sum j=0 to N P_{N-j} = 1
The death process:
looking at whole population of cells
extinction probability
setting j = N that the probability that N cells
have died at time t is
p_0(t) = (1 − e−µt)^N
which is the extinction probability at time t in the death process with N initial cells (X0 = N).
Clearly limt→∞ p_0(t) = 1 simply says that in the death process the eventual extinction of the
population is certain.
An alternative way to find the pk(t) is by using the differential equations
The death process:
looking at whole population of cells
An alternative way to find the pk(t) is by using the differential equations
for 0 ≤ k < N,
dpₖ(t)/dt = -kµpₖ(t) + (k+1)µpₖ₊₁(t)
loss term-gain term (master
equation)
and
dp_N/dt=
-Nµp_N (t),
with p_k=N (0) = 0 and p_N (0) = 1
master
equation
The death process:
looking at whole population of cells
for 0 ≤ k < N,
dpₖ(t)/dt = -kµpₖ(t) + (k+1)µpₖ₊₁(t)
loss term-gain term
The loss term −kµpₖ says that a population with k cells (state k) can be left at rate kµ at which one of the k cells dies (leading to a state
k −1).
This is balanced by the gain term (k + 1)µpₖ₊₁ saying that the state k can be reached from state k + 1 after the death of a cell at rate (k + 1)µ.
Since we are dealing with a death process the number of cells can never exceed N, and hence
there is only a loss term contributing to dp_N/dt
death process
dp_N/dt=
-Nµp_N (t),
can be solved to obtain
p_N(t)=exp(-Nµt)
using k=N-1 obtain dp_{N-1}/dt solved to get p_{N-1}(t)
repeatedly
we could focus on mean behaviour instead
x(t)=E(X_t) = sum k=1 to infinity of [kpₖ(t)]
dx(t)/dt =sum k=1 to N of
[ k dpₖ(t)/dt]
=
=−µx(t)
since x(0)=E(X_0)=N we have
x(t)=Nexp(−µt)
death process:
extinction time
extinction times
probability of extinction at time t
Let T be the extinction time, defined as
T = {first time that X_t is 0}.
T is a continuous RV on the state space S=[0,∞] s.t t T ≤ t if
and only if X_t = 0
The probability of extinction at time t is: p_0(t) = (1−exp(−µt))ᴺ
cumulative distribution function of T:
P {T ≤ t} = P {X_t = 0}
= p_0(t) = (1 − e(−µt))ᴺ
says how probability to go extinct up to time t accumulates
thus
P {T ≤ 0} = p_0(0) = 0,
P{T ∈ [0,∞)} = p_0(∞) = 1,
the probability that
the population goes extinct between t_1 and t_2 ≥ t1 is
P{t1 ≤ T ≤ t2} = P{T ≤ t_2} − P{T ≤ t_1} = p_0(t_2) − p_0(t_1)
death process
Thus, by definition, the probability density of the extinction time, T, is
ρ(t) =dP {T ≤ t}/dt
dp₀(t)/dt
= µNe{⁻µᵗ}(1 − exp(−µt))ᴺ⁻¹
This clearly implies
P{T≤ t} = p₀(t)
= ∫₀ᵗ ρ(τ )dτ
and P {t₁ ≤ T ≤ t₂} =
∫ₜ_₁ ᵗ-² ρ(t)dt =
p₀ (t₂)−p₀(t₁)
graph of cumulative distribution p₀(t) upper bound at 1
simple death process calculating the mean extinction time
The mean extinction time can be computed from the density function
ρ(t) =dP {T ≤ t}/dt
=
dp₀(t)/dt
= µNe{⁻µᵗ}(1 − exp(−µt))ᴺ⁻¹
E(T)=
∫₀∞ tρ(t)dt =
µN ∫₀∞ t exp(−µt) (1− exp(−µt) ᴺ⁻¹ dt
by integrating by parts: compute µE(T) for the first few N: µE(T) grows logarithmically with N
E(T) ≈1/µ ln N when N → ∞
t against E(T) graph peaks then decreases
intuitive way of simple death process mean extinction time
Extinction from transition of a state with N cells to N-1 cells with µ_N = Nµ followed by transitions to N-1, N-2,,.. with rate µ_(N-1)= (N-1)µ
The time to jump from N to N −1 is an exponentially-distributed random variable with mean1/(Nµ)
N-1 to N-2 ….mean 1/{(N-1)µ}
….
Jump from X_t=2 to X_t=1 …. mean 1/(2µ)
X_t=1 to X_t=0 ….mean 1/µ
E(T)= 1/µ+ 1/2µ+….+ 1/Nµ
= (1/µ)(1+1/2+1/3+…+1/N)
E(T) =approx 1/µ(γ + ln N),
(limit as n tends to infinity of sum 1+1/2+..+1/n is γ + ln N, where γ = 0.577 . . . is the Euler Mascheroni constant)
confirms that the mean extinction time grows logarithmically with N.
“simple birth process” or the “Yule process”.
continuous-time stochastic process in which each individual (cell) of a population can reproduce without ever dying
In the birth process,** each cell alive at time t has a probability β∆t of dividing into two cells
before t + ∆t, as ∆t → 0**.
Assume each cell, independently, has a constant birth rate, β > 0.
cells do not die
sample path upward stairs
“simple birth process” or the “Yule process”.
pₙ(t) = P {Xₜ = n},
MASTER EQUATION derivation
Let X_t be the number of cells alive at time t, and the probability that there are n cells in the population at t be pₙ(t) = P {Xₜ = n},
assuming that there are initially X₀= n₀ > 0 cells,
Since X₀= n₀>0 and cells cant die pₙ(t)=0 for all t ≥ 0 if n< n₀
each of the n₀ can divide with a rate β the state is left with rate βn₀:
dp_n₀(t)/dt= −βn₀ p_ₙ₀(t)
which with the initial condition p_(n_0) (0)=1 gives
p_(n_0)(t) = exp(−βn₀t)
n₀ + 1 cells: any reproduce at rate β to new state n₀ + 2 (loss term −β(n₀+1)p ₙ_₀₊₁ for rate of change of pₙ-₀₊₁
n₀ cells: one reproduce at rate β to new state n₀ + 1 (gain term β(n₀)p ₙ_₀ for rate of change of pₙ-₀₊₁
With p_ₙ-₀(t) =exp(−βn₀t) we have thus
dpₙ-₀₊₁(t)/dt
= −β(n₀+1)pₙ-₀₊₁(t)+βn₀pₙ-₀(t)
= −β(n₀+1)pₙ-₀₊₁+1(t) + βn₀exp(−βn₀t)
with pₙ-₀₊₁(0)=0
,
“simple birth process” or the “Yule process”.
pₙ(t) = P {Xₜ = n},
MASTER EQUATION
With p_ₙ-₀(t) =exp(−βn₀t) we have thus
dpₙ-₀₊₁(t)/dt
= −β(n₀+1)pₙ-₀₊₁(t)+βn₀pₙ-₀(t)
= −β(n₀+1)pₙ-₀₊₁+1(t) + βn₀exp(−βn₀t)
with pₙ-₀₊₁(0)=0
Equation for the rate of change of p_n when n>n_0
dp_n/dt
= −βnpₙ(t) + β(n−1)pₙ₋₁(t)
loss term + gain term
with pₙ(0) = 0,
“simple birth process” or the “Yule process”.
pₙ(t) = P {Xₜ = n},
MASTER EQUATION
dp_n/dt
= −βnpₙ(t) + β(n−1)pₙ₋₁(t)
loss term + gain term
with pₙ(0) = 0,
The linear ordinary differential equation for p_{n_0+1) can readily be solved using standard
methods:
p_(n_0+1)(t) =
n₀( exp(−βn₀t) − exp(−β(n₀+1)t)
= n₀exp(−n₀βt) (1 − exp(−βt))
which has a max:
t_max =
(1/β) log(1 + (1/n₀))
graph of p_{n_0+1} (t)
peaks at t_max decreases as t goes to infinity
we could continue and use to solve ODE for n_0+2 etc but instead,…
“simple birth process” or the “Yule process”.
PGF
∂G(z, t)/∂t
pgf: p_k(t) ≡ 0 for k < n_0
sum from k=n_0 to infinity only needed p_k(t)*z^k
∂G(z, t)/∂t=−βz(1 − z)∂G(z, t)/∂z
, with G(z, 0) = z^{n_0}
The solution of the pde
G(z,t) =
[(zexp(-βt))/(1-z+zexp(-βt))]^n_0
G(1, t) = 1 (probability conservation
G(0, t) =p_0(t) = 0 when n_0 > 0
∂G(z, t)/∂t
satisfies….
birth process?
∂G(z, t)/∂t=−βz(1 − z)∂G(z, t)/∂z
, with G(z, 0) = z^{n_0}
“simple birth process” or the “Yule process”.
PGF compare
G(z,t) =
[(zexp(-βt))/(1-z+zexp(-βt))]^n_0
with powers
zexp(−βt)[1+z^2(1- exp(−βt))+z^3(1-exp(−βt))^2+…]
case n_0=1
1/(1-x)= 1 +x + x^2+…. when |x|<1
since |z|≤ 1
|z(1- exp(−βt)|<1
G(z,t) = zexp(−βt)[1+z(1- exp(−βt))+z^2(1-exp(−βt))^2+…]
p_1(t)z + p_2(t) z^2 +….
“simple birth process” or the “Yule process”.
Comparing each term with the same power of z in the last two lines, we readily find
Comparing each term with the same power of z in the last two lines, we readily find
p1(t) = e^{−βt}
p2(t) = exp(−βt)(1 − exp(−βt)), p3(t) = exp(−βt)(1 − exp(−βt))^2
and more generally, when
n_0 = 1, we have
p_n(t) = exp(−βt)[(1 − exp(−βt))]ⁿ−¹
for n ≥ 1
)
which is a geometric distribution
“simple birth process” or the “Yule process”.
Comparing each term with the same power of z in the last two lines, we readily find
G(z, t)
G(z,t)=
z^n_0 exp(-n_0βt)) [(1+z(1-exp(-βt)) + z^2(1-exp(-βt))^2 +…..]^n_0
= p_(n_0)(t)z^{n_0} + p_(n_0+1)(t)z^(n_0+1)+,,,,,,
thus we recover
p_n_0(t) = exp(–n_0βt)
and
p_(n_0+1)(t)=
n_0exp(-n_0βt)(1- exp(-βt))
.
In general, we have
p_k(t)
=
(k-1)
(k-n_0) exp(-n_0βt) (1- exp(-βt))^(k-n_0)
for k ≥ n_0
,
“negative binomial distribution”
p_k(t)=0 for k<n_0
p_0(t)=0 pop extinction impossible in birth process!
Appendix A: Cumulative distribution and probability density
functions (reminder & summary)
cumulative distribution function P {T ≤ t} : S → [0, 1] of a continuous random variable T (e.g. the extinction time) on the state space S describes how probabilities accumulate
P {T ≤ t} = P {T ∈ [0, t]}
ρ(t) ≡dP {T ≤ t}/dt
is the probability density function of T.
In the case of the extinction time T for a non-zero initial population, S = [0,∞) and
P {T ≤ t} = p0(t) = integral_[0,t] ρ(τ )dτ.
Therefore, the probability that extinction occurs between t1 and t2 ≥ t1 is
P {t1 ≤ T ≤ t2} = P {T ≤ t2} − P {T ≤ t1} = p0(t2) − p0(t1) = integral_[t_1,t_2] ρ(τ )dτ,
with P {T = 0} = 0 and P {T ∈ S} = 1.
We can use the probability density function to
compute the mean of T^n
(nth moment of T) for n ≥ 0 according to the general formula:
E(T^n) = integral_S τ^n ρ(τ )dτ
mean extinction time in the death process is thus IE(T) = integral_[0,∞] tρ(t)dt =integral_[0,∞] t dp_0/dt .dt
Appendix B: integrating factor method to solve first-order differential equations (reminder)
dp_{n_0+1}(t)/dt
= −β(n0 + 1)pn0+1(t) + βn0e
−βn0t
is a first-order linear differential equation of the form
dy/dt + P(t)y = Q(t) with y(0) = 0,
IF I(t) = exp (integral P(t).dt)
p_{n_0+1}(t) =
βn_0*
[([exp(−βn_0t)]/β + K exp(−β(n0+1)t)]
,
where the constant K is found from the initial condition: p_{n_0+1}(0) = 0 = βn_0(β^−1 + K) ⇒
K = −β^{−1}
yielding p_{n_0+1}(t) =
n_0[exp(−βn_0t) − exp(−β(n_0+1)t]
