Continuous Time Markov

Question 1

What type of processes are modelled by continuous-time
discrete-state Markov processes? What is the fundamental
quantity to be computed?

Answer 1

In Continuous time Markov, we are interest in determining the state of a homogenous, memoryless process at a certain time t. The fundamental quantatiy to be computed is the unconditional time-dependant state probability vector P(t)

Question 2

What are the assumptions of this
model? What are the properties of the transition probabilities?

Answer 2

Constant transition rates over time, mututally exclusive states.
Transitions occur in a stochastic process over time following an exponential distributiion

Question 3

What are the assumptions of this
model? What are the properties of the transition probabilities?

Answer 3

Constant transition rates over time, mututally exclusive states.
Transitions occur in a stochastic process over time following an exponential distribution

Question 4

How can you compute the transition rate matrix from the
continuous-time transition probability matrix? What are the
properties of the transition rate matrix?

Answer 4

We can compute the transition rate matrix from the probability of state transition aij per incremental unit of change dt.
The Continous time transition probability matrix rows always add up to one.

Question 5

How can you compute the unconditional state probability vector
P[X(n)=j] from the knowledge of the transition rate matrix?

Answer 5

The fundamental matrix equation dP(t)/dt =P(t) * A is a System of n linear differential equations. We can use the Laplace transformation to solve the equation. Transforming back into the time-domain we receive an expression as a a solution for P(t).

Question 6

Derive the fundamental equation for homogeneous continuoustime
discrete-state Markov processes.

Answer 6

We start with P(t + dt) = P(t) * A
- substract P0 from both sides and divide by dt
yields with lim dt->0
P0 is just the first state. by doing it for all states, we get the complete matrix
dP/dt = P(t) * A

Write this down a few times in

Question 7

Express the general solution to the fundamental matrix
equation using the Laplace transform method.

Answer 7

dP/dt = P(t) * A
s * P(s) - C = P(s) * A
Do not forget the starting condition.

now we isolate P(s) on one side of the equation. Calculate and then transform back!

Question 8

One component fails with rate λ and can be repaired with rate μ by
only one repairman. Plot the Markov diagram and write transition rate
matrix. Solve the fundamental equation for the continuous-time
Markov process and show the equation for the unavailability of the
component.

Answer 8

Write solution on paper.

Question 9

How do we calculate the inverse of a matrix?

Answer 9

[a b ; c d]
swap the positions of a and d, put negatives in front of b and c, and divide everything by ad−bc .

Question 10

What are the Markov diagram and the transition rate probability for a
system with N identical components and N repairmen? What are the
Markov diagram and the transition rate probability if only one
repairman is available? Discuss the differences between the two
cases.

Answer 10

If only one repairman is available we can only go one state back at a time.

Question 11

How can you compute the steady-state probabilities in continuoustime
discrete-state Markov processes.

Answer 11

We can calculate the steady state probability by using simply taking the laplace transform of P(t) and calculating P(s = 0) this is equivilent to calculating the limit of the Integral ∫ P(t) dt t-> infinity.

Question 12

What do the steady-state probabilities in continoustime discrete-state Markov processes represent?

Answer 12

They represent the asymptote of the state probability vector for t>inf.
This is the probability that a system is found in a certain state for all states at a time after the system has settled (reached equilibrium)

Question 13

Define the system failure intensity for a general system modelled by
continuous-time discrete-state Markov processes. How can you
compute it?

Answer 13

The system failure intensity describes the rate at which components fail. We can determine it by adding all the sproducts of state Probability ∑ Pi(t) * λi (lambda being the failure rate.)

Question 14

Define the system repair intensity for a general system
modelled by continuous-time discrete-state Markov processes.
How can you compute it?

Answer 14

System repair intensity is the rate of component recovery from failed state to repaired state. “How often do we have to dispatch the repair crew?”

It is calculated with the state probability vector for (partially) failed state * repait rate µ:

∑ Pif(t) * µi

Question 15

Compute the failure and repair intensities for a system
consisting of one component and one repairman by using
continuous-time discrete-state Markov processes

Answer 15

Do on paper

Question 16

How can you compute the average time of occupancy of a
state in continuous-time discrete-state Markov processes.
What is the probability distribution that describe this time?
Why?

Answer 16

The average time of occupancy of a given state is simply the inverse of the sum of the transition rates leaving the states.
The occupancy is exponentially distributed, because expected future occupancy is always the same no matter how long the system is in that state.
This is in accordance with the Markov criterion, that the process is memoryless.

Question 17

How can you generally compute the availability of any system
using continuous-time discrete-state Markov processes?

Answer 17

For the point-availability we calculate the the state propability vector P(t) and substract all the state probabilites of failed states from one.

Question 18

How can you generally compute the reliability of any system in which repair transitions cannot occur using continuous-time discrete-state Markov processes? Use an exemplary system to discuss the steps of your answer.

Answer 18

In this case we can simply calculate the point availability as it is equal to the reliability of the system.

Question 19

How can you generally compute the reliability of any
system in which repair transitions can occur using
continuous-time discrete-state Markov processes? Use an
exemplary system to discuss the steps of your answer.

Answer 19

To compute the reliability, we simply remove the failed states from the transition matrix A. By doing this we achieve that the system can no longer recover from faiure states. The success states become transient.

Question 20

Compute the reliability of a system of two identical
components (failure rate λ) connected in parallel logic and
two repairmen (repair rate μ) using continuous-time
discrete-state Markov processes

Question 21

Compute the mean time to failure (MTTF) of a system of
two identical components (failure rate λ) connected in
parallel logic and two repairmen (repair rate μ) using
continuous-time discrete-state Markov processes.

Question 22

Compute the reliability of a system of two identical
components (failure rate λ) connected in series logic and
two repairmen (repair rate μ) using continuous-time
discrete-state Markov processes.