1. Stochastic Calculus

Question 1

What are properties of Markov chain?

Answer 1

• A stochastic process and the future values are only dependent on the current values
• has several states together called “State Space”

Question 2

What is n-step transition probability?

Answer 2

The probability from i => j after n times
P(X_{t+n} = j|X_t = i) = (P^n)_{ij}

Question 3

What is Vector Probability Distribution and example?

Answer 3

1. Let say we have distribution of 2 states at t = 0, x = [p1, p2] (horizontal vector)
2. We have transition matrix P
3. At t = t, the horizontal vector becomes x * P^t

Question 4

What is Trajectory probability and example?

Answer 4

Is the probability of a specific state trajectory
1. Let say we have state space (X1, X2, X3) with known probability
2. At t = 0, we have probability of (X1, X2 and X3) = (a, b, c)
3. For a specific trajectory X1 - X3 - X1 - X2, we have:
trajectory probability = a * p(X1X3) * p(X3X1) * P(X1X2)

Question 5

1. What is Absorbing state?
2. What is Absorbing Markov chain?

Answer 5

1. State j is called absorbing if p_{jj} = 1
2. Markov chain is called absorbing if:
   - Has at least 1 absorbing state
   - From ALL non-absorbing state, ALWAYS can move to AT LEAST 01 absorbing state

Question 6

1. What is periodic state?
2. What is aperiodic state?
3. What is this checked for?

Answer 6

1. Process to check if a state is periodic or not:
2. At t = 0: check all p_{ii} of each state, if p_{ii} > 0 => aperiodic
3. At t = 0: if any p_{ii} = 0 => do P^2 and P^3. If P^2{ii} != 0 and P^3{ii} != 0 => common denominator of t = {2,3} = 1 => aperiodic
4. If t = 2 and t = 4 that P^2{ii} and P^4{ii} !=0 => common denominator = 2 => periodic
5. For how long this state is back to this state?

Question 7

1. What is transient state?
2. What is recurrent state?

Answer 7

Transient: tạm thời
Recurrent: phản hồi
1. Transient: state i is transient if only exists state j that i => j, but j cannot => i (tồn tại ít nhất state j mà i chỉ cho đi nhưng không nhận lại)
2. Recurrent: the remaining

Question 8

1. What is accessible?
2. What is communication class?
3. What is a closed class?

Answer 8

1. We say i and j is accessible if and only if after moving some states, j will return to i
2. Communication class is a group of states that they can access to each other. Communication class is called a subchain
3. A closed class is a class which cannot be accessible to other class

Question 9

1. What is a irreducible Markov chain?
2. What is reducible Markov chain?

Answer 9

1. A irreducible Markov chain is if the chain has ONLY 01 communication class
2. Otherwise

Question 10

1. What is Hitting Probability?
2. How to calculate Hitting Probability?

Answer 10

1. Hitting probability is the probability that first time a given process hit a subset of a Markov Chain?
2. Let say a state space S = {1, 2, 3, 4} and a subset A = {1, 2}
3. Let H = {h1A, h2A, h3A, h3A} be the hitting probability vector that
   hiA = pij * hjA, in which j = {1, 2, 3, 4}
   h1A = h2A = 1
4. If there is numerous solution, hiA = 0

Question 11

1. What is Expected Hitting Time?
2. How to calculate Expected Hitting Time?

Answer 11

1. Expected Hitting Time is the time that a process hit A
2. Similar to Hitting Time, T = 1 + hiA
   for {1,2}, T1 = T2 = 0

Question 12

How to calculate Condition number?
1. Row
2. Column

Answer 12

1. Take absolute value of each element
2. K = ||A|| *||A^-1||
3. ||A|| = max(sum of each row)
4. similarly, ||A^-1||
5. Multiply together

Question 13

What are ways to calculate Det and name them?

Answer 13

1. 2x2: ad - bc
2. 3x3: triangle, sub-factor, LU decomposition
3. 4x4: subfactor

Question 14

Steps to LU decomposition?

Answer 14

1. Find U and elementary matrix E
2. For each step of finding U, have corresponding E
3. E = E1 E2 U => U = E1^-1 * E2^-1 * U
4. E1^-1 * E2^-1 = L

Question 15

How to find eigenvalues and eigenvectors?

Answer 15

1. Eigenvalues
   det(A - lamba I) = 0
2. Eigenvectors
   For each lambda, substitute to the equation (A-lamdba I)x = 0

Question 16

What is orthogonal matrix?

Answer 16

AA^-1 = A^-1A = I

Question 17

Ways to do Fourier transform to solve PDE?

Question 18

PDF and MGF?

Question 19

First order ODE?

Answer 19

1. Homogenous
   dy/dx - A*y = 0
   => dy/y = Adx
   => ln(y) = Ax + C
2. Non-homogenous
   dy/dx + p(x)y = Q(x)
   => P(x) = -integral(p(x))
   v’ = Q(x)/e^P(x)) => v
   => y = v*e^P(x) + C e^P(x)

Question 20

Second order ODE

Answer 20

1. Homodengous
   Ad^y/dx^2 + Bdy/dx - Cy = 0
   - Characteristic equation: a^2 + bx + c = 0
   + If 2 distinct solutions: y = Me^y1 + Ne^y2
   + if y1 = y2: y = Me^y + Nye^y
   + if y = alpha +- i*beta: y = e^{alpha x} (Mcos(beta y) + Nsin(beta y))

Question 21

Step to find adverse of the matrix?

Answer 21

1. 2x2: a<=>d. Change signs for b,c
   2: 3x3: 1/det * (det(sub))

Question 22

How to do Fourier transform?

Question 23

How to sketch contour of q(x,y) on (x,y) plane?

Answer 23

1. Let say we have 2 eigenvectors A1, A2 to draw new plane
2. We got 2 eigenvalues to find q(x,y) lambda 1 and lambda 2, lambda 1 * x^2 + lambda 2 * y^2
3. Draw eclip: Transform to eclip equation: x^2/a^2 + y^2/b^2, then on x: 2a, y: 2b and draw

Question 24

Fourier series formula?