10 Continuous Markov Processes and Diffusion Processes Flashcards
Here: Markov processes that are continuous in time with continuous sample paths =>
with time and X(t) = x ∈ (−∞, ∞) or [a,b] where range state space S continuous
We focus on single-variate process whose state space is one dimensional
at each time only one RV
A continuous MP X(t) is defined by its probability density function pdf p(x,t) and transition pdf (tpdf) p(y,s;x,t) with s>t
The pdf is the function whose integral over a domain give the probability that X(t) is in that domain:
Prob {X(t) = x ∈ [a_1, a_2]} =
integral_[a_q,a_2] p(x, t) dx
probability conservation means over S this equals 1
Assuming time homogeneity (only time difference matters for transitions), the
tpdf p(y, s; x, t)
is the probability density of the transition from state (x,t) to state (y,s),
There if often time homogeneity=> transition depends only on time difference s-t and in this case
p(y, s; x, t) = p(y, x, s − t)
Characterisation of time evolution of via its pdf follows from the Chapman Kolmogorov equation
(CKE) that now reads
p(x, t; x_0, t_0) = integral_ -∞,∞
p(x, t; x_1, t_1)p(x_1, t_1; x0, t0) dx_1,
for t_0 < t_1 < t,
p(x, t) = integral_ -∞,∞
p(x, t; x1, t1)p(x1, t1) dx1, for t0 < t1 < t
when the initial state X(0) is known
With pdf and tpdf
=> moments and jump moments of X(t):
or assuming S = (−∞, ∞)
kth (“raw”) moment =>
E [X^k(t)]
=integral_ -∞,∞
x^k p(x, t)dx
kth jump moment
∆X(t) = X(t + ∆t) − X(t)
giving
lim_∆t→0 (1/∆t) E(∆X(t))^k|X(t) = x)
= lim_∆t→0 (1/∆t) integral_(-∞,∞)
(y − x)^k p(y, t + ∆t; x, t)dy
diffusion process
Diffusion processes is a class of continuous Markov processes similar to the Brownian motion, which
is the continuum version of the random walk (Chapter 7). Diffusion processes are defined by 2 coefficients: the first (drift) and second (diffusion) jump moments
Useful to revisit the 1D random walk (Chapter 7)
p(x, t)∆x = Prob{X(t) ∈ [x, x + ∆x]} =>
p(x, t+∆t) =
q p(x − ∆x, t) + (1 − q) p(x + ∆x, t) | {z }
trans. to x from x − ∆x with prob. q
+
trans. to x from x + ∆x with prob. 1 − q
Taylor expansion in ∆x
p(x ± ∆x, t) = p(x, t) + (+−∆x) ∂p(x, t)/∂x
+ (∆x)^2/2 ∂^2p(x, t)/∂x^2
Substituting on the RHS, rearranging, dividing by and taking the limit ∆t → 0
Convection-diffusion equation:
∂p/∂t = −c (∂p/∂x) + D ∂^2p/∂x^2
with drift coeff
c=
= lim_∆t→0,∆x→0 (2q − 1)∆x/∆t
,diffusion coefficient
D = lim_∆t→0,∆x→0 (1/2)(∆x)^2/∆t
When c=0, there is no drift/convection
one-dimensional diffusion equation:
∂p/∂t = D ∂^2p/∂x^2
symmetric, no drift occurs
If initially the distribution is concentrated at x=0, i.e. [Dirac delta],
p(x, 0) = δ(x)
the pdf is the
fundamental solution of the 1D diffusion equation:
it is a Gaussian of mean zero and variance 2Dt
p(x, t) =
(1/√4πDt )exp(−x^2/4Dt)
it is a Gaussian of mean zero and variance 2Dt
integral_ -∞,∞ p(x, t)dx = 1
Similarly, for 1D convection-diffusion equation with initially ,
p(x, 0) = δ(x)
the pdf is
a Gaussian of mean ct and variance 2Dt:
E(X(t)) = ct,
var(X(t)) = 2Dt
⇒ Std(X(t)) = √var = √2Dt
p(x, t) =
(1/√4πDt )exp(−(x-ct)^2/4Dt)
General solution of the 1D diffusion equation, e.g., obtained by using Fourier transform (Appendix F)
mean also varies in time
this occurs due to drift
in general
sol of 1D diffusion can be obtained
If initially p(x,0)=f(x) given when c=0, the general solution of of the 1D diffusion equation is a convolution of f(x) and the Green function
p(x, t) = integral_{-∞,∞}
f(y) exp(−(x−y)^2/(4Dt))/√4πDt dy
we can use the 1D fundamental solution of BM with initial condition
because as linear we use the superposition method
The Brownian motion (with/without drift) is an
important example of diffusion process
By analogy with the Brownian motion, a generic diffusion process X(t) is defined by its first 2 jump moments, with ∆X(t) = X(t + ∆t) − X(t)
:
Drift coef.:
a(x, t) = lim∆t→0
(1/∆t) integral{-∞,∞}
(y−x) p(y, t+∆t; x, t)dy =
lim_∆t→0 (1/∆t)E (∆X(t)|X(t) = x)
diffusion coeff is b(x)/2
b(x, t) = lim∆t→0
(1/∆t) integral{-∞,∞}
(y − x)^2 p(y, t + ∆t; x, t)dy =
lim_∆t→0 (1/∆t) E(∆X(t))2|X(t) = x
and
lim_∆t→0 (1/∆t) integral{-∞,∞}(y − x)^n p(y, t + ∆t; x, t)dy =
lim_∆t→0 (1/∆t)
E ([∆X(t)]n and |X(t) = x) = 0 for n > 2
a and b meanings
a(x) is the “expected jump rate” or “first jump moment”, like c in the biased Brownian motion
b(x) is the “expected square jump rate” or “second jump moment”, like 4D in the Brownian motion If the diffusion process is time homogeneous (as in our course) a(x) and b(x) depend on x but not on t
a and b play role of C and D in BM
depend only on x and not on t
if i give you a process
you define a and b by limit of expectations
THEN
we can write the generators of the Fokker-Planck equations
