Agents

Question 1

Agents

Answer 1

Anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators

Question 2

Percepts

Answer 2

Inputs obtained through any sensor (e.g. camera) which the agent perceives the environment through

Question 3

Actuators

Answer 3

Tools that the agent uses to act upon the environment

Question 4

Abstract description of an agent, external

Answer 4

The agent function maps from percept histories to actions: [f: P* -> A]

Question 5

Concrete implementation of an agent, internal

Answer 5

The agent program runs on physical architecture to produce f : agent = architecture + program

Question 6

Rational Agent

Answer 6

For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.

Question 7

Rationality at any moment depends on:

Answer 7

• the performance measure that defines the criterion of success
• the agents prior knowledge of the environment
• the actions that the agent can perform
• the agents percept sequence to date

Question 8

Exploration as it relates to Rationality

Answer 8

to do with actions in order to modify the future percepts so that the agent would not do any uninformed action

Question 9

Learning as it relates to Rationality

Answer 9

Adapting to changing environment.

A successful agent computes its agent function in 3 periods:

1) when it is designed by its designer
2) when it decides its next action
3) when it learns from experiences to decide how to modify its behavior (improve the rules)

Question 10

Autonomy as it relates to Rationality

Answer 10

it should learn what it can to compensate for partial or incorrect prior knowledge, relies more on its percepts than on the prior knowledge of its designer

Question 11

Task Environment

Answer 11

The problem to which the rational agent is a solution. Environment type largely determines agent design.

Question 12

PEAS

Answer 12

Performance Measure
Environment
Actuators
Sensors

Question 13

Environment types:

Fully Observable

Answer 13

(vs. partially observable)

An agent’s sensors give it access to the complete state of the environment at each point in time

Question 14

Environment types:

Deterministic

Answer 14

(vs. stochastic)

The next state of the environment is completely determined by the current state and the action executed by the agent

Question 15

Environment types:

Strategic

Answer 15

The environment is deterministic except for the actions of other agents

Question 16

Environment types:

Episodic

Answer 16

(vs. sequential)
The agents experience is divided into atomic “episodes” (each episode consists of the agent perceiving and then performing a single action), and the choice of action in each episode depends only on the episode itself

Question 17

Environment types:

Static

Answer 17

(vs. dynamic)

The environment is unchanged while an agent is deliberating

Question 18

Environment types:

Semi-Dynamic

Answer 18

The environment itself does not change with the passage of time, but the agent’s performance score does

Question 19

Environment types:

Discrete

Answer 19

(vs. Continuous)

A limited number of distinct, clearly defined percepts and actions

Question 20

Environment Types:

Single agent

Answer 20

(vs. multi-agent)

An agent operating by itself in an environment

Question 21

An agent is completely specified by…

Answer 21

… the agent function mapping percept sequences to actions.

Question 22

Percept sequence summarizes….

Answer 22

…the history of what the agent has perceived.

Question 23

An agent program is …

Answer 23

… an implementation or execution of the agent function.

• The agent function is realized by an agent program executing on the physical architecture.
• We aim to find an optimal agent program for the rational agent function.

Question 24

The four basic agent types, in order of increasing generality:

Answer 24

Simple Reflex Agents

Model-based Reflex Agents

Goal-based Reflex Agents

Utility-Based Agents

These can all be turned into Learning Agents

Question 25

Simple Reflex Agents

Answer 25

• Selects actions on the basis of the current percept, ignoring the rest of the percept history
• Some input has an established connection in the agent program to an action: such a connection is called a “condition- action rule” AKA if-then rules (if car in front breaking, then initiate breaks)
• Simple, but limited intelligence
• Will work only if correct decision can be made on the basis of just the current percept (only if the environment is fully observable)
• Infinite Loops often unavoidable

Question 26

Model-based Reflex Agents

Answer 26

• Agent maintains some internal state that depends on percept history, reflects at least some of the unobserved aspects of the current state.
• Requires ‘Transition Model’: effects of agents actions, and how the world evolves independently from the agent
• Requires ‘Sensor Model’: information about how the state of the world is reflected in the agents percepts
• ## Seldom can determine current state of a partially observable environment exactly - represents agent’s “best guess”. Uncertainty unavoidable.

Question 27

Goal-based Agent

Answer 27

• Keeps track of world state, as well as set of goals it is trying to achieve, and chooses an action that will (eventually) lead to the achievement of its goals
• Search and Planning AI subfields deal with finding action sequences that achieve an agent’s goal
• Flexible, because the knowledge that supports its decisions is represented explicitly and can be modified.

Question 28

Utility-Based Agent

Answer 28

• Uses a model of the world, along with a utility function that measures its preferences among states of the world.
• Then it chooses the action that leads to the best expected utility, where expected utility is computed by averaging over the all possible outcomes, weighted by the probability of the outcome.
• Utility refers to the quality of being useful
• An agent’s utility function is essentially an internalization of a performance measure
• Complicated to design - has to keep track of environment, and choosing the utility-maximizing function is a difficult task

Question 29

Learning Agents

Answer 29

• A process of modification of each component of the agent to bring the components into closer agreement with the available feedback information, thereby improving the overall performance of the agent.
• Any type of agent (model-based, goal-based, utility-based) can be built as a learning agent (or not).
• Advantage, allows agent to operate in initially unknown environments and to become more competent than its initial knowledge alone might allow

4 components:

• Performance Element
• Learning Element
• Critic
• Problem Generator
• (outside component) Performance Standard

Question 30

Performance Element

Answer 30

Responsible for selecting external actions. What is considered the entire agent in other models: it takes in percepts and decides on actions.

Question 31

Learning Element

Answer 31

Responsible for making improvements, takes feedback from ‘critic’ on how the agent is doing and determines how the performance element could be modified to do better.

Design of Learning Element depends on design of performance element.

Question 32

Critic

Answer 32

• Tells the Learning element how well the agent is doing with respect to a fixed performance standard.
• Precepts themselves provide no indication of the agent’s success.

Question 33

Problem Generator

Answer 33

• Responsible for suggesting actions that will lead to new and informative experiences
• Suggests exploratory actions to the agent, some exploration action that could do some perhaps suboptimal results in short term, but would help discover much better actions in the long run.
• Might identify parts of the model that are in need of improvement and suggest experiments

Question 34

Performance Standard

Answer 34

• Distinguishes part of the incoming percept as a reward or penalty that provides direct feedback on the quality of the agent’s behavior
• It is important that the performance standard be fixed. Conceptually, think of it as being outside the agent altogether, because the agent must not modify it to fit its own behavior.
• Precepts themselves provide no indication of the agent’s success.