CCC

Question 1

What are the properties of DEC-POMDPs

Answer 1

Combines elements of game theory and POMDPs
NEXP-Complete
Agents can benefit from communication
Optimal solution balance cost of communicating w/ cost of not communicating
Some algorithms, heuristics, applications are known

Question 2

What is a DEC-POMDP

Answer 2

A POMDP with a set of agents taking actions simultaneously

Question 3

What is inverse reinforcement learning?

Answer 3

An agent uses the environment and induced behavior to infer a reward function

Question 4

Briefly describe Maximum Likelihood Inverse RL (MLIRL)

Answer 4

Guess rewards -> Compute policy -> Measure Pr(D|Pi) -> Gradient on R -> Guess rewards

Question 5

What is the general case (multinomial) probability of an action being optimal in policy shaping?

Answer 5

Pr(a|d_a) = C^(delta_a)/[C^(delta_a) + (1-C)^(delta_a)]

Question 6

What is the only one case (multi-binomial) probability of an action being optimal in policy shaping?

Answer 6

P(a|d_a) ~= C^(delta_a) * (1-C)^[Sum_(j!=s) delta_j]

Question 7

How do you represent trajectories as an MDP?

Answer 7

States: partial sequences
Actions: story actions
Model: player model
rewards: author evaluations