Lecture 3

Question 1

How to calculate the average costs per week when dealing with a Markov decision process?

Answer 1

Question 2

What is a stationary policy?

Answer 2

Every time you are in state i you choose action a

Question 3

What is the Markov property and Time Homogeneity?

Answer 3

Question 4

What is the definition of discounted costs or discounted rewards?

Answer 4

Question 5

When is a stationary policy optimal (in case of minimization)?

Answer 5

Question 6

What follows for any stationary policy if the Markov property is satisfied? V.. = c(i,…)

Answer 6

Question 7

How follows that the minimal discounted costs are finite?

Answer 7

Question 8

What are the optimality equations?

Answer 8

Question 9

What does solving the optimality equations look like? (Do not solve)

Answer 9

Question 10

When is a stationary policy just as good? When is a stationary policy better?

Answer 10

Question 11

What is the definition of an long term average cost per time user problem?

Answer 11

Question 12

What is the definition for an optimal long term policy?

Answer 12

Question 13

When is a policy optimal for discounted and long term costs?

Answer 13