Class 9

Question 1

supervised learning

Answer 1

learning by which an agent passively learns by observing example input/output pairs provided by a teacher

Question 2

reinforcement learning

Answer 2

type of learning where an agent interacts with the world and periodically receives rewards, goal is to maximize expected sum of rewards

Question 3

markov decision process

Answer 3

when a specific action in a specific environment have provided sufficient rewards in the past these are the actions the machine should continue to take in the future

Question 4

model based reinforcement learning

Answer 4

type of learning that uses a transition model of the environment to help interpret reward signals and make decisions about how to act

Question 5

utility function

Answer 5

something that calculates a sum of rewards from a certain state onward

Question 6

model free reinforcement learning

Answer 6

type of learning where the agent neither knowns nor learns a transition model for the environment

Question 7

action utility learning

Answer 7

type of learning when the agent learns a quality function which calculates a reward in a state if a specific action is taken – the agent searches for action with the highest Q value

Question 8

reflex agent

Answer 8

agent that performs a policy search when it directly maps states to actions

Question 9

passive reinforcement learning

Answer 9

type of learning where the agent’s policy is fixed and the task is to learn the utilities of states

Question 10

passive learning agent

Answer 10

agent with a fixed policy to determine its actions and which tries to learn a utility function, does not know state transition model or reward function

Question 11

direct utility estimation

Answer 11

estimated total reward from that state onward

Question 12

adaptive dynamic programming agent

Answer 12

agent that takes advantage of the constraints among the utilities of states by learning the transition model that connects them and solving the corresponding markov decision process

Question 13

prioritized sweeping

Answer 13

heuristic that prefers to make adjustments to states whose likely successors have just undergone a large adjustment on their own utility estimations

Question 14

absorbing state

Answer 14

when an agent can perform no actions that can have any effect and no rewards are received

Question 15

q learning

Answer 15

type of learning that learns an action utility function instead of a utility function, off policy learning algorithm

Question 16

bayesian reinforcement learning

Answer 16

type of learning that looks at prior probabilities over hypotheses about what the true model is, agents of this type of learning don’t explore as much as they should because they get caught on prior probabilities

Question 17

sarsa

Answer 17

state, action, reward, state, action – close to q learning but updates with the q value of the action that is actually taken, on policy learning algorithm

Question 18

evaluation function

Answer 18

compact measure of desirability for potentially vast state spaces

Question 19

function approximation

Answer 19

process of constructing a compact approximation of the true utility function of q function

Question 20

joint state space

Answer 20

foundation of HRL, each state is composed of a physical state s and a machine state m

Question 21

policy search

Answer 21

keep tweaking the policy as long as its performance improves, then stop

Question 22

stochastic policy

Answer 22

specifies the probability of selecting an action a in a state s

Question 23

apprenticeship learning

Answer 23

field that studies the process of learning how to behave well given observations of expert behavior

Question 24

imitation learning

Answer 24

applying supervised learning to the observed action state pairs to learn a policy

Question 25

inverse reinforcement learning

Answer 25

learning rewards by observing a policy rather than learning a policy by observing rewards

Question 26

boltzmann rationality

Answer 26

to allow for mistakes by the expert we have to assume that rewards exist

Question 27

feature mapping

Answer 27

algorithm that assumes the reward function can be written as a weighted linear combination of features

Question 28

feature expectation

Answer 28

expected discounted value of the feature f when a policy is executed