Decision Theory

Question 1

Explain why utility functions are a strictly more expressive preference model than the goals used in deterministic planning

Answer 1

They are more expressive because the solution depends on the agent’s preference towards certain worlds; there is more than one solution. Whereas, the goals used in deterministic planning does not care about the preference as long as a solution is found

Question 2

What is the difference between random variable and decision variables

Answer 2

Random variables are variables we can not control and are determined by the world.
Eg. accident, weather etc…

Decision variables are variables we can choose to happen in the world.
Eg. The robot wears pads, takes the longer route etc..

Question 3

In words, what is the agent’s expected utility for making a single decision?

Answer 3

The agent’s expected utility for making a single decision depends on the utility value of that decision and the probability of that world where the decision is made.

Question 4

How is a sequential decision problem different from a single decision problem? Why does it make sense that in the “single decision” framework, we can still talk about problems ( such as the example given in the slides) where there is more than one decision variable.

Answer 4

A single decision problem is when agents know what actions they can carry out, the effect of each action is described as a probability distribution over outcomes and an agent’s preferences are expressed by utilities of outcomes. We can still talk about problems where there is more than one decision variable by combining one or more decisions and treat them as a single macro decision to be made before acting. Such as (WearPads, Whichway).

A sequential decision problem is when at each stage, the actions are available to the agent, information is or will be available to the agent when it has to act, the effects of each action and the desirability of these effects are available. In other words, the agent has to take actions without knowing what the future brings.

In short 
Repeat: 
1. Make Observations
2. Decide on an action
3. Carry out the action

Question 5

In words, define a policy for a sequential decision problem

Answer 5

A policy specifies what an agent should do under each circumstance.
The agent makes an observation of the world.
The agent makes a decision depending on what is has observed
The agent executes the decision.

Question 6

In words, what makes a policy “optimal” in a sequential decision problem

Answer 6

The higher the expected utility value in a sequential decision problem the more “optimal” it will be.

Question 7

How is a decision variable D eliminated when VE is applied to a sequential decision problem..?

Answer 7

To eliminate that decision node, it chooses the values for the decision that result in the maximum utility, creating a new factor on the remaining variables and a decision function for the decision variable being eliminated…